User Research Reflections

This is a collection of my thoughts on various user research methods and skills developed during MHCI+D Spring 2014 Evaluation Studio.

During the ten week course, we'll develop both broad research skills that are applicable to various methods as well get exposure to key methods to expand our design research toolkit. 

For more detailed process documentation and reflections on each technique, visit my HCI+D Research Journal.

Observations - An exercise in watching and listening, not interpreting

Sample of my observation field notes

Sample of my observation field notes

The first of these broad research skills we examine this past week was our observation skills, which is so foundational to getting the most of various user research methods.

As human beings, who have built up strategies for interpreting and creating meaning to the observations we make everyday, it takes a level of skill and awareness to manage the interplay between pure observation and interpretation. In our everyday lives our interpretations of the world are linked so closely with our observations that they can sometimes be difficult to separate.

As researchers, we must be aware of our biases and tendencies, and as much as we can, resist the urge to prescribe meaning to our observations. Rather, we should note our assumptions and make "thick descriptions" that remove the social assumptions we encounter while observing and documenting and replace them with rich descriptions of the context in which observed behaviors take place. 

Frameworks - A tool to fight "Analysis Paralysis"

Collecting raw data from primary and secondary research produces a wealth of information, but how do you create meaning and discover useful insights for design? 

Here’s where we get to the fun part, the analysis and synthesis phases of the research process. During this phase, making the data visible and tangible and being able to manipulate the data quickly become extremely helpful. That’s why we all love post-its and whiteboards! They make it easy for us to externalize our thoughts and begin to draw relationships between the data. It also keeps your design team on the same page. It also helps to create permanence, so that we can look at where we are and how we got there. Conversations flow and change, but without something tangible, what’s discussed is fleeting and often lost. 

Frameworks are also your friend because they help you to structure your analysis in a meaningful way.

STEEP Analysis

STEEP Analysis

Initial results from STEEP Analysis

Initial results from STEEP Analysis

Thematic Map

Thematic Map

SWOT Analysis  Resulting Design Question: How can we create more value in the store experience to offset the convenience of online and mobile shopping? 

SWOT Analysis

Resulting Design Question: How can we create more value in the store experience to offset the convenience of online and mobile shopping? 

Interviews - Conversations with a Purpose

Interviews are essentially conversations that have a specific purpose-to discover, to learn, or to clarify. Usually interviews are conducted in the front end of the product development cycle. However, good interviewing techniques are critical in many UX research methods used throughout the process.

Now let’s get into the nitty gritty when it comes to the logistics of actual conducting an interview. From failed experiences and practice this is what I’ve learned so far. 

  • Video & sound quality matter. Make sure you have a decent quality video camera. A tripod is great for keeping the camera steady and out of the way. It’s also recommended to record separate audio using an external mic or other audio recording device. This way you have a back up in case the sound quality captured on the video camera isn’t up to par. My smartphone audio recorder has worked well for this. Keep in mind that snippets of your interview might be crucial evidence to support your research finding that you will share with various stakeholders.

  • Framing the shot. Make sure you have framed your shot to capture what you want. In most cases, it’s best to frame the shot around the participants face, so you can clearly see their facial expressions during the interview. You might also want to include enough of his/her upper body to capture hand gestures, which can be quite expressive and convey things that are not captured in the audio. Having a bit of background to show context may also help to legitimize your interview and keeps it authentic. 

  • Balancing structure and thoroughness. The art of interviewing is being able to balance the conversation and keep it natural while also getting the information you need. Find a way to create a system that works for you. Work on transitions between topics. Ask about specific stories or examples, and when disconnects arise, dig deeper and note them as they might provide valuable insights. 

  • Location, location, location. Think about where you will be conducting your interview. What is the setting? Is it public or private? Is it in context? Think about how the answers to these questions will affect your interview. 

  • Do a trial run with your equipment. Test your equipment and do a trial run with either a friend, colleague, or an additional participant. Sometime, you won’t realize that anything is wrong unless you actually set up and go through a pilot interview. One of the worst, yet preventable, things that can happen is setting up and conducting a fabulous, insightful interview just to discover that your memory card capacity was low and only captured 2 minutes of a 20 minute interview! And yes, I’m speaking from experience. Check and double check. Some things will be out of your control, but there are things that you can do to minimize the risk of losing your data. 

Card Sorts - Not Just a Method, but a Platform for Research

Card sorts can be reshaped in a variety of ways to meet various research needs. Here are just a few ways that card sorts can be used. 

  • At the front end, you can use a think aloud protocol to begin to understand the mental model users might have of your system or how they might approach a task.
  • You can resort and recapture card sorts when users are given specific scenarios to understand how user behaviors might change based on varying context or situations.
  • Card sorts can be done using images, objects, photographs, or text. And items on the cards can be user-generated or predefined. 
  • Card sorting activities can be embedded into existing interviews. 
  • They can be done in-person or online using various tools. 
  • They can be used an evaluative way to understand navigation or categories for a website. 

Pretty much their uses are endless. You can mix and match, customize, or create a brand new flavor of card sorting for your own research needs. 

Ethnography... is it a method, practice, approach, or deliverable?

Yes. Ethnography is all of the above. In user research we are most concerned with applied ethnography, also know as field studies. Common methods that fall into this category include but are not limited to contextual inquiry, diary studies, shop-alongs, etc. 

Applied ethnography is using ethnographic approaches such as observing and describing a group/culture, specific behavior, values, ritual, etc. to understand and inform stakeholders, design products, services, systems, and organizations. Ethnographic methods are ideal when understanding an experience in context is critical. For example, when looking for new opportunities, gaps, or pain points in an existing experience. 

It’s all about context! Applying these methods in-context is key.

However, ethnography come with it’s own set of challenges. Ethnographic methods can be time and labor intensive. At times, it may be difficult to gain access to what you want to observe. And it is only as good as your analysis and synthesis. 

That being said, remember, frameworks are your friends, and allow you to structure information until you see or learn something new. Trust the framework process. The light will eventually shine through the chaos and ambiguity. And those aha moments are golden! The expertise is in figuring out how to get the answers you need, not in knowing the answers. 

Now some practical tips:

  • Planning and preparation are key. Have a research plan that outlines the research questions you are trying to answer and the specific activities involved. Like most things in life, the more you plan, the more you will get out of your study. 
  • Make detailed observations. Write down your assumptions and note when you observe behaviors and actions that challenge them. 
  • During the analysis and synthesis phase, make your findings visible. Look for patterns and surprises in the data. Separate observations from meaning and interpretation. 
  • Common deliverable outputs for these types of studies are created to enable empathy with users and to identify opportunities. They can take the form of journey or experience maps, work models, opportunity maps, etc.