CRO+MA Lab Codeon

Overview

I collaborated with Phd students on researching how programmers seeking remote code help

I participated in evaluating the old Codeon experience, redesigning Codeon user flow and interfaces, conducting user testings, and writing paper for CHI conference.


Codeon is an Atom package developed for programmers to seek for remote expert programming help in an asynchronous manner

The main goal for this project was to streamline the help seeking experience, and create the proper environment for research purposes.


This was a 4-month research project



Context Of This Research

What is Codeon?

As an Atom package, Codeon is developed for programmers to request remote expert programming help in an asynchronous manner.The general claim we made was that Codeon could help programmers seek/ receive assistance from crowdsourcing workers in an async, scalable fashion, so that they could use their time more efficiently, with less interruptions and could multitask.

An earlier paper published by the CRO+MA Lab team showed the advantages of using asynchronous help requesting approaches in programming. It also discovered that audio requests, context capture, inline comments, etc. could make this process more effective. The initial version of Codeon was built based on those findings. But we found that some design issues could impeded the user experience of Codeon, which further influenced our observations of programmers' asynchronous help seeking behaviors.


Previous findings...

status
  • "Request status is not clear."

Because the help seeking is remote, showing system & user status is how we can form trust.

system feeback and notification
  • "Feedback and notifications are not always noticeable."

We need to make the communication smooth, but not interruptive while coding.

associations
  • "Associations between code and context are not easy to make."

Utilize gestalt principles (color and proximity) can better the situation.

readability and legibity
  • "I can't read!"

We can improve the color contrast, free space, typographies, and font of Codeon!

modalities
  • "Modalities of communication are inbalanced and a lot of switch."

We can integrate and we can hide things away.

version control
  • "I want my code back."

User control over the content they create should be of high value.



Personas

Codeon Persona
Codeon Persona
Codeon Persona

We narrowed down our research targets to inexperienced programmers, e.g. students

We decided that Codeon was not for everyone at its initial phases,and we narrowed down to inexperienced programmers for phase one due to testing constraints.



The Design

I designed the user flow and interfaces for better experience, and also for research

This part was what I struggled most with, as even though this project was meant for crowdsourcing, at this stage we were only testing people's asynchronous help seeking behaviors, not the algorithm of how to pair help requesters and helpers.

The final user flow I came up with was simple to be expained in the paper and clear enough to address all our claims and hypothesis. It includes flow for both the help requester and the helper.

User flow diagram requester
User flow diagram helper

I prototyped the user interfaces using Adobe Illustrator and Justinmind

I had 3 iterations of the user interfaces. The initial design was very complex and refered as the "aviation-style". Then, by combing through the user flow and our research claims and hypothesis, I simplified the interfaces.

Though the final system interface does not fully implement my design due to time constraints. My core design recommendations, including instant/salient feedback, clear status, color coding code changes, etc. were implemented in Codeon for our user studies.

The following screens shows how the interface evolved.

Codeon Make-a-request Interface V1
Codeon Make-a-request Interface V2
Codeon Make-a-request Interface V3


User Testing

I prepared for the user testing, and went through the whole research process from beginning to end

With the upgraded Codeon, we created a new study plan that addressed our hypothesis and project goals, while reducing factors that could possibly weaken the validation of our research. Key behaviors we observed in the studies were interruptions, task resumptions, lag, multitask, etc. We also interviewed all participants about their experience of using Codeon.


The following image shows how we compared two potential flows for the requester

User flow diagram comparison

The following image shows how we planned to analyze our testing results

User study analysis


The Outcome

User testing results indicated that the Codeon experience was imporved

Watch a short video of how Codeon works.


I learned...

"Design depends on the project goal - when the goal is too broad, it is necessary to narrow it down, as this is the only way to achieve it. Project goal, together with concrete personas, is fundamental for all design decisions."


Future Recommendations

  • Clear project goal before design.
  • Clarify target user and use the personas to guide your design.
  • Avoid 'Aviation Design'.