As more and more companies become aware of user experience (later referred to as UX), more teams are being tasked with conducting user research to validate the user experience objectives and test the design assumptions. We’ve already written several pieces by now on web usability and accessibility, so make sure you’ll look into that first, as it covers some major basic terminology and principles that go hand in hand with the user experience — the subject is too broad and deserves multiple features and you surely need to familiarize yourself with the basics before you embark on the full-fledged user experience research.
What Is User Experience Research, When and Why Do It?
User Experience research is the process of discovering the behaviors, motivations, and needs of the users through observations, surveys, sample tasks, analysis, and gathering of feedback on the use of your company’s services or products.
Why would you research the behavior of your users? Well, in fact, UX research helps to understand the experience of users with new websites, apps, products, services, prototypes; discover, optimize, and evaluate that experience; fix the mistakes; make informed decisions about any changes you’re about to implement to your design; and finally perfect your product.
User research can be done at any moment in the design cycle; however, you can choose which method to use when. Moreover, conducting research at every stage of the design process keeps product-development efforts on the right track and in sync with the real user needs.
So the rule of thumb is to do most of the user research very early in the project, then check on your previous analysis, and conduct new supplementary research at every other stage.
Project Stages in the Design Cycle
Each project is inherently different, hence — stages will vary according to the time constraints of your project, system maturity, type of product/service you design, and other top concerns you wish to address. Nevertheless, usually, the design cycle embraces the following four major stages where you conduct research in order to discover, explore, test, and listen to your users’ needs and concerns. It’s obvious that the design process starts from the discovery: interviewing users and stakeholders and finding common ground between what users need and business owners want.
The end of one stage is the beginning of the next — this time — after discovery, comes exploration, where designers perform multiple analyses, build prototypes, come up with users’ personas, and even write stories. At the test stage, it’s time for extensive usability and accessibility testing, which is later superseded with feedback and bug reviews, surveys, and so on. What we have just described above are the research methods that corresponded to each individual design stage. Let’s now outline them further below and address some of those methods in detail in later sections.
Discover
The discovery stage is critical because, during this stage, you’ll find out if it really makes sense to do the project at all. The most important part of the discovery and its ultimate goal is to validate and discard assumptions; bring the researched data to the team to make final assessments; and arrive at the necessary conclusions.
The following research methods are recommended during this stage:
Field study is a research method that’s conducted in a user’s natural habitat (meaning their preferred context and location). There are many types of field research that mainly differ in terms of conducting a study (for example, the amount of interaction between participants and researchers), among those are:
- Flexible user tests in the field which combine usability testing and adaptive interviews (where you change questions as you learn more about your users);
- Customer visits where you research potential customers of your application (product, service) in their particular business context like industry;
- Direct observation involves watching over users’ behavior in a given situation (for example, observing staff or shoppers at the mall);
- Ethnographic research allows designers to gain insights into mental models and the social situation of a particular ethnic group;
- Contextual inquiry combines many other appropriate field research studies and activities.
User interviews are another valuable method of exploratory research; however, they do have a lot of shortcomings. The main problem with user interviews is that you’re making people remember past events or speculate on the future, but the human memory is fallible, and people usually end up making stories and trying to rationalize their behavior. Moreover, users tend to have no idea how they are going to use future product or technology, thus rendering most of those interviews useless. Having said that, the best way to conduct interviews is to ask your users questions while they use your product or feature or shortly after.
Use interviews if you want to explore users’ general attitude toward a specific issue or problem, or combine interviews with other research methods that can give you an in-depth rather than superficial view on a given situation.
Diary study is the research into users’ daily logs of activities as they occur, which helps designers receive contextual insights about real-time user behavior, activities, needs and define user experience feature requirements. Diary study can range from a few days to a few months or longer, during which participants are asked to keep a diary and log specific information about activities that are being studied. Diary research is specifically useful when trying to understand users’ habits, usage scenarios, attitudes, motivations, changes in behavior, customer journeys, etc. One of the biggest challenges in conducting diary studies is motivating the users, most of the time the money incentive would work, but even then, it’s best to spread the incentive in installments to keep users going and interested in receiving the full sum.
Stakeholders, sales, support, and training staff interviews are performed to understand business requirements, constraints, problems that users relate to sales or support teams. Listening to support and sales calls is crucially important for understanding issues users go through while using a feature, the vocabulary they use to describe their problems, etc.
Competitive testing is gathering and evaluating data on what works well or poorly on other sites of similar nature. There are different types of competitive evaluations, such as holistic comparison based on overall site-usability metrics, or conversely — more focused research dedicated to evaluating and studying specific features, content, or design elements. Competitive analysis can be done in the form of expert reviews, with one or few experts reviewing competitive websites, or competitive usability testing, where users complete a set of tasks using two or more competing websites.
Explore
During the exploration stage in the design process, designers compare different features against competitors, perform design reviews, build users’ personas, show stakeholders the prepared user journeys, explore design possibilities, obtain feedback on early-stage tasks, and test paper and then, interactive prototypes. Let’s briefly define each research method below.
Building personas is based on a user-centered design approach, where designers try to understand the end-user, their behaviors, motivations, attitudes, needs, and goals. Persona is, therefore, a fictional yet realistic description of a target user; an archetype rather than an actual living human. The descriptions of personas should be thorough and detailed including such characteristics as age, gender, occupation, behaviors, and other important aspects of an imaginary user’s life that might affect their use of a designed product. Personas should be concrete without abstractions or generalizations in order to be successful representations of real people. Building up the personas usually happens after filed studies and interviews where you gather typical characteristics of users and then synthesize those by arriving at definitive conclusions.
These are the key characteristics that you need to include in your personas’ descriptions:
- Name
- Gender
- Age
- Picture
- Tagline describing what they do in life
- Experience level in using your product, service
- How they would interact with your product (context: job, home; desktop, mobile; etc.)
- Primary goals while using your product (speed, accuracy, thoroughness)
- Any other quotes that can sum up their attitude
Another important method is journey mapping, i.e. visualizations used for understanding customer needs and pain points as people interact with the product or service. Journey maps typically have three zones: focus and context for the journey (zone A), an area that depicts the user experience (zone B), and insights or additional commentary derived from analyzing the journey (zone C). So, zone A defines the user and the context where they are examined; zone B is the heart of the map that’s divided into chuckable phases of the journey and includes actions, thoughts, emotional experience of the user; zone C focuses on the discovered opportunities and ownership, which outlines who in your team is in charge of that aspect of a customer journey.
Other methods that comprise the exploration stage are paper prototypes and interactive prototypes. Whatever prototyping method you choose, a user interface prototype is always a hypothesis – a design solution that you create to solve a specific design problem. Prototypes vary in size and functionality, depending on the goals of the testing, resources available, tools used to create prototypes, and more. Thus, they can be single-page versus multiple-page with menus, screens, click targets; realistic and detailed versus hand-sketched on paper, interactive versus static. To learn more about prototypes and recognize real examples, watch the following videos:
Example paper prototype # 1
UX prototyping tutorial # 2
Digital Prototyping # 3
Another valuable method to use during the exploration stage is card sorting. Card sorting is a design exercise used to evaluate the information architecture of a website, during which the participants organize the topics into categories that make sense to them. This design exercise is extremely helpful in discovering how to structure your site, decide what to include on the home page, how to label categories and navigation. There are two types of card sorting: open and closed. In the open card sort, participants organize the topics into groups and then name those groups in a way that they feel accurately describes the content. In the closed card sort, participants sort topics within your site into predefined categories.
Test
Testing is used for checking if your design system works during the development stage and beyond. The ultimate goal of every test is to make sure everything works well for the end-users. There are different research methods that are used during this stage, among those are: performing qualitative usability testing, scrutinizing and evaluating diary studies, conducting audit training classes, talking with users online and offline, analyzing user-forum posts, and performing benchmark testing.
So what’s the difference between qualitative and quantitative usability testing? Obviously, quantitative data is numeric with primarily statistical analysis, where you can measure an average success rate for a given task (for example, 30 people clicked on the blue button as opposed to 15 who clicked on the red). The research subjects here are your representative sample of a larger population. Qualitative data is not numeric, it’s either a narrative or a description. In this case, analysis involves extracting information from the data (for example, a qualitative report might include information on how a tester attempted their way through the website or performed a certain set of tasks). Qualitative data gives you an opportunity to find out why users had trouble navigating your site as opposed to how many of them had the said trouble.
UX benchmarking is yet another UX research method that allows you to test how your website or app is progressing over time and where it exactly fails or falls short compared to competitors. To conduct a benchmarking study you’ll have to set up a plan, write an interview script, where questions need to be focused on the most important and/or basic tasks of your website or app, then you’ll need to pick up the participants for your study ranging from 25 to 30 users, and since it’s benchmarking against something, then you’ll need to do the same study over time, and benchmark your progress against the previous results. There are many metrics you can track in those studies, among them are task success (where 0 is fail and 1 is pass), time on task, number of errors, single ease question rate (measures the participant’s perceived ease of a task), subjective mental effort question rate (measures how mentally difficult the task was), confidence rate (measures how users were confident that they completed the task successfully), and so on.
Listen
Listening throughout research helps to understand the existing problems as well as look for new issues that users might experience when using your product/service/website/app. During the Listen stages of your user experience research conduct surveys, monitor analytics and metrics to discover trends or gauge your progress, analyze search queries, collect frequently asked questions, attend and communicate with people at conferences, give talks, prepare and explain demos to potential users, and so on. During this stage, it’s essentially important to pay attention to user sentiment, communicate future directions where you consult the users on changes rather than straightforwardly inform them about the changes made, recruit more people for future research and testing (since the user experience research is not something you can give up after your product’s launched, but rather something you do iteratively many times over).
Conclusion
We’ve tried to describe every possible known UX research method that can help you enhance the user experience of your product, analyze and work on your mistakes. It’s absolutely imperative that you understand your users and their needs for every project you embark on very early in the process, this way you’re almost always guaranteed to succeed in your endeavors.
