When a company embarks on a new digital project, the pressure to launch fast and to constantly innovate frequently sends products to market prematurely. The rush to market can lead to these products being untested, increasing risks to their return on investment.
Many early assumptions in product plans (e.g., budget, timeframes, and ROI) are formulaic and need to be validated prior to product launch. Much as companies conduct market research for a physical product, the same should apply for digital products, commonly known as user experience (UX) research. Unfortunately, many companies are not experienced in conducting UX research, which is impacting revenue and reputation.
Challenges and Misconceptions of UX Research
The challenges many companies face include a lack of knowledge about the various user experience research methodologies. Other common misconceptions are that UX research processes are long, costly, and yield scattered information that cannot be easily synthesized.
Different UX research methods can mitigate business risks and create products and features that provide real value to customers and organizations. The chart below illustrates where the methods we describe fit within a Research Method matrix featuring Behavioral x Attitudinal and Qualitative x Quantitative dimensions.
Before Testing
When designing experiments and tests, sample sizes and demographic selection are important; otherwise results tend to be nonrepresentative, irrelevant, or unmanageable and inconclusive. Before engaging in testing, make sure these parameters are properly set.
Additionally, having a team of qualified professionals that is able to gather and interpret results, develop insights, and determine whether concepts can be combined or eliminated is invaluable. In the absence of a qualified team, the wrong conclusions might be drawn—setting up a product or feature for failure.
Some UX Research Types
There are multiple UX Research methods; 8 types of research provide clear, actionable data and mitigate risk.
- Surveys
Surveys are a non-resource-intensive method, yielding results quickly. They can be used to measure and categorize attitudes or collect self-reported data that can reveal important issues to address. - Card Sorting
When considering how to structure the flow of a product, the card sorting method is invaluable. By presenting categories and features cards, and asking users to group like items to workflows, card sorting can provide insights about users’ categorization of information, thus helping determine the best information architecture for a product.
- Pretotype Testing
A pretotype is a simulation of a new product’s experience to test its appeal and usage without spending much time or money. Pretotypes are designed to test concepts and essentially answer the question: Is there market interest for this product?
Pretotypes should offer the absolute minimum features and screens that can validate assumptions and determine a product’s probability of success.
This concept testing method involves a user researcher observing individuals, selected from the intended audience, directly interacting with the pretotype and finding out if users would be inclined to use or purchase it.
The data yielded allows for the de-risking of projects by validating demand and determining and prioritizing features based on actual interactions with its intended audience, not assumptions or models of customer preference. - Eye Tracking
A fast and relatively inexpensive research method, eye tracking is useful to understand how users interact with a design, essentially revealing which screen areas attract focus and attention and which do not.
There are specialized companies which offer eye track subject recruiting and testing. The technique can be useful to guide the location and prominence of design elements on a page, which might significantly impact ROI.
- A/B Testing
A fast and easily iterative method, A/B testing allows for clear-cut results as it involves presenting a variation to a site or marketing campaign design – while keeping all other elements constant – and exposing one version to half of the audience and another to the other half, to determine which variation performs better.
In that way the behavior of users can be measured and tests iterated to refine the user experience, thus increasing a product’s chances of success upon launch. - Field Tests: Quantitative Studies
Quantitative tests can provide easily understood information that requires little interpretation, as they feature predetermined outcomes; for instance, whether a UI element is clicked, a task is completed or success is achieved within a timeframe or not.
They can also contain multiple-choice questions, producing how many/ how much answers that can whittle down alternatives quickly, helping fast iteration.
When done correctly, this unmoderated testing method can provide a clear sense of which features to include in the first release, which can shorten launch timelines significantly—reducing costs and freeing up development resources. Quantitative studies can also reveal issues prior to development, generating more accurate budgets and schedules, directly impacting ROI. - Field Tests: Qualitative Studies
Qualitative field studies yield data about behaviors by directly observing how users get the information they need and complete tasks, and provide answers about why or how a behavior is shown.
It is important to highlight the role of a qualified research team to gather and interpret results, develop insights, and determine whether concepts can be combined or eliminated. A qualified team can then capture the outcomes of the interpretation of test results and document them as lessons learned for future cycles.
Despite the time and cost of qualitative analysis, it can produce a comprehensive picture of the product in action, providing the context to course-correct before the very costly development phase.
- Testing Beyond UX Research
After development and usability testing there is still another round of tests that products must undergo to reduce risk before they reach the market.
This includes testing the product against any regulatory and legal standards, damage testing and/or compatibility testing with systems and devices.
Capturing Value Post-launch
After a product or feature has had exposure to the market, assessing its impact on customer satisfaction provides additional value. Creating this feedback loop is an opening for continuous improvement and harnesses additional value from research.
Despite financial and market pressures, it is useful to think of product/ feature launches not as a race, but as a marathon of continuous improvement that can be hedged by engagement in user research.
Orion’s award-winning Experience Design team can help you conduct effective UX research and partner with you to reimagine a more intuitive, human-centered and accessible experience. Learn about Orion’s Experience Design services.
Silvia Grossmann is a strategic thinker with more than a decade of XD experience. Silvia recently designed a new global interface for a Fortune 100 company with a presence in 160 countries, as well as the online learning interface for the world-renowned ArtCenter College of Design. She holds a Master’s degree from the University of Southern California.
