Generative vs Evaluative Research Methods: A Breakdown

Generative vs evaluative research methods are two techniques for conducting user research when managing your product.

Both UX research methods are different but provide valuable insights to help your product development process.

TL;DR

• Generative research explores and understands users’ needs, motivations, and behaviors to inspire new product ideas and concepts.
• Some of the benefits of generative research are understanding how users experience your product, how much they know about it, and generating new ideas.
• It’s easy to conduct generative research using concept testing, in-app surveys, user journey mapping, focus groups, and diary studies.
• You can improve user onboarding with generative research by conducting interviews, usability testing, and user journey mapping to uncover issues.
• Decide which new feature to work on by using in-app surveys with open-ended questions.
• Use generative research methods to determine what content types users favor in knowledge bases.
• You can use exit interviews and surveys for generative research to improve customer churn by uncovering issues.
• Use generative research methods for usability testing to plan and gather insights into new UI concepts and interactions to identify areas of confusion.
• Evaluative research assesses existing products to identify usability issues and areas for improvement.
• You can conduct evaluative research by usability testing, A/B testing, multivariate testing, and clicking testing.
• Evaluative research is perfect for analyzing feature usage, identifying which features are most and least used, and where users may drop off.
• You can assess customer loyalty with evaluative research using NPS surveys to determine how likely users would recommend your product.
• Evaluative research helps with conversion funnel analysis by evaluating what is causing drop-off.
• Use evaluative research for competitor benchmarking by conducting market research to understand where your product excels or falls short.
• Generative research explores new ideas at a product’s inception, while evaluative research assesses existing concepts for effectiveness.
• Userpilot is a great tool for performing generative and evaluative research. It offers NPS surveys, feature tags and heatmaps, path analysis, and A/B testing. Book a demo to find out more.

What is generative research?

Generative research is a user research approach that delves into understanding users to unearth opportunities for innovation and address real-world problems.

Often termed “discovery” or “exploratory” research, its overarching aim is to amass critical information about how users behave, their decisions, and the environments in which they operate.

This research is invaluable because it allows businesses to discover unique innovation avenues by venturing beyond pre-existing assumptions.

Why is generative research important?

Generative research serves as an invaluable tool in the product development and design process process. But why exactly is it so crucial?

• Understanding User Experience: Every product’s value lies in its user reception. Generative research delves into user interactions and feelings about a product, offering a comprehensive view of their experience. Harnessing this knowledge allows businesses to align offerings with user sentiments, bolstering satisfaction and loyalty.
• Gauging User Awareness: Generative research identifies knowledge gaps, pinpointing areas needing user education or targeted marketing. By discerning user blind spots, companies can communicate optimally, ensuring clarity without overwhelming or underserving.
• Futuristic Ideation: Generative research reveals users’ hopes for future products. Leveraging these insights, businesses can innovate to fulfill and surpass user aspirations.

How do you conduct generative research?

You can conduct generative discovery research using a variety of techniques. Here’s a breakdown of methods:

• User Interviews: One of the most direct ways to understand user needs. These structured or semi-structured dialogues allow you to dive deep into user experiences, motivations, and pain points.
• In-app Surveys: Provide real-time user feedback as they interact with a platform. These surveys can garner specific insights on particular features or interactions.
• Concept Testing: Offers a glimpse into the potential reception of a new feature. Concept testing involves presenting users with a preliminary version to gather feedback. It helps validate assumptions, refine ideas, and ensure alignment with user expectations.
• User Journey Mapping: This visual representation charts a user’s interactions with a product or service over time and across different touchpoints. By mapping out the user’s journey, researchers can identify moments of friction, delight, confusion, or any other emotion, ultimately pinpointing areas of improvement or innovation.
• Diary Studies: Participants keep a diary of their interactions with a specific feature over an extended period. Diary studies offer longitudinal insights that might not be evident in shorter-term studies, highlighting how user behaviors and perceptions evolve.
• Focus Groups: Are moderated discussions that explore user reactions, perceptions, and ideas regarding a product or concept. The group dynamic can stimulate diverse discussions, leading to a multi-faceted understanding of user perspectives.

Examples of when you can use a generative research method in SaaS

Certain situations exist when you should conduct generative research methods within your SaaS. Here are some examples.

User onboarding optimization

If you observe a significant drop-off rate during the onboarding process, it signals something might be amiss. Rather than making assumptions or relying on mere quantitative data, generative research provides qualitative insights into the heart of the issue.

By conducting user interviews, you can delve directly into the user’s mindset, understanding their pain points, confusion, or any challenges they might face during onboarding.

Usability testing observes users navigating the onboarding and can provide real-time insights into where they stumble, hesitate, or get frustrated.

Lastly, when focused on onboarding, journey mapping paints a comprehensive picture of the user’s experience. It showcases moments of clarity and confusion, offering a blueprint of the processes to be optimized.

New feature prioritization

Generative research can help you decide which feature to prioritize rather than relying on internal brainstorming or competitor analysis. It provides an understanding from the user’s perspective, ensuring you make the right decision.

Surveys or customer feedback forms are quintessential tools in this process. They allow you to tap into the user’s perspective and help quantify the importance of various features.

Instead of casting a wide net, such tools enable you to hone in on what truly matters to your users.

Make sure to incorporate open-ended questions to ensure the feedback is as genuine and unfiltered as possible.

Knowledge base content improvement

The knowledge base is the primary touchpoint for users seeking guidance, answers, and deeper understanding. Ensuring that the knowledge base resonates with user needs and preferences is paramount.

Generative research methods have become a valuable ally for SaaS apps that lean heavily on content.

Surveys query users about their experiences and preferences so you can gather actionable insights about what works and what doesn’t in your current content strategy.

User interviews further elevate the depth of these insights. They provide a platform for open dialogue, allowing users to share detailed feedback or specific pain points.

Customer churn analysis

Addressing high churn rates requires understanding their reasons, not just the numbers.

Generative research delves into user perceptions and issues, aiding in retaining subscribers. Exit interviews pinpoint reasons for churn and improvement areas.

Meanwhile, surveys reach a broader departing user base, allowing them to share feedback and provide diverse insights.

Usability testing

Usability testing reveals how users navigate and perceive new features. Observing their interactions highlights potentially confusing areas.

While inherently evaluative, generative research guides the process, helping craft a focused test plan informed by prior user behavior understanding.

What is evaluative research?

Evaluative research is a user research method to assess how well a product or service aligns with its users’ needs, desires, and goals. It’s the bridge between conceptualization and realization, between the design and how it’s perceived.

Evaluative research is interwoven throughout the product’s development lifecycle. From the preliminary phases, where rough sketches or prototypes are assessed, to more mature stages, where a minimum viable product is examined through surveys or first-click testing, evaluative research offers continuous feedback.

Why is the evaluative research method important?

Evaluative research helps to understand how your creations fare in the real world, offering a beacon of clarity and direction. Here’s why it’s of utmost importance:

• Assessing User Needs Alignment: Evaluative research definitively answers whether your product or feature addresses user needs. Through usability testing, surveys, or feedback sessions, it assesses whether the product or feature genuinely resonates with the user’s requirements, pain points, and expectations.
• Measuring Success Against Intent: Evaluative research is a benchmarking tool’s pivotal role. It helps determine if the product or feature is being used as envisioned and if it’s achieving its intended goals.
• Gathering Holistic Feedback: Evaluative research offers a platform for users to voice their opinions, concerns, and suggestions regarding the complete product journey. This invaluable feedback offers insights about specific pain points and potential enhancement, innovation, or expansion areas.

How do you conduct evaluative research?

While there are numerous evaluative research techniques, let’s delve into some of the most common evaluation research methods.

• Usability Testing: Participants matching your user profile navigate your product, performing set tasks. Their actions and verbalized thoughts are observed, pinpointing usability pain points.
• A/B Testing: Users see one of two slightly varied versions of a product or webpage. After a period, data reveals which version better achieves the desired outcomes.
• Multivariate Testing: Different elements on a webpage are simultaneously varied, creating numerous combinations. A segment of users sees a specific version, and the performance of each variety is analyzed.
• Click Testing: Users see a static webpage image and click where they anticipate actions, with tools like Userpilot tracking interactions with tagged elements, thus revealing design misalignments.

Examples of when you can use an evaluative research method in SaaS

Understanding user interactions and product efficacy is paramount in the SaaS world. Here are some examples of when evaluative research is right for a SaaS.

Feature usage analysis

By meticulously analyzing feature usage, you can discern which features resonate with users and which don’t.

Furthermore, this analysis illuminates potential pain points or hurdles, highlighting areas where users might be disengaging or facing challenges.

Assessing customer loyalty with NPS surveys

NPS (Net Promoter Score) surveys serve as an effective tool for understanding user loyalty by gauging how inclined users are to advocate for your application to peers.

The survey feedback provides invaluable insights into areas of improvement, ensuring you maintain a loyal user base and foster positive word-of-mouth referrals.

Conversion funnel analysis

You can examine your customer’s conversion funnels to pinpoint stages where users seem to disengage or abandon crucial processes.

It’s not just about identifying these drop-offs but delving deeper to understand their root causes. This insight lets you make informed improvements, ensuring a smoother, more intuitive user journey.

Competitor benchmarking

You can juxtapose your application against rival products in the market through comprehensive benchmarking studies. This comparison illuminates areas where your product shines and those where it may lag.

This knowledge ensures you match and surpass industry standards and user expectations.

What are the main differences between generative and evaluative research methods

Generative and evaluative research both serve pivotal roles in understanding user needs and enhancing product design, but their approaches differ significantly.

Generative research is centered on the birth of fresh and innovative ideas and concepts, making it pivotal during the early stages of product development. It aims to uncover new opportunities and avenues for innovation.

In contrast, evaluative research meticulously assesses the effectiveness of these conceived ideas and concepts, providing feedback on their actual impact on users. It’s instrumental in ensuring a product or feature resonates with its intended audience.

What’s the best research method: Generative or evaluative?

Choosing between generative vs evaluative research isn’t a matter of superiority but appropriateness for the task at hand. Your decision largely hinges on your research objectives and the specific challenge of the product you’re addressing.

When the objective is to dive deep and uncover user aspirations, needs, and desires, generative research emerges as the ideal choice.

On the other hand, evaluative research steps in when you’re poised with a product and need to measure how it aligns with user expectations. It provides actionable insights into a product’s usability and the overall user experience.

However, to truly harness the power of research, blending generative and evaluative methods often proves to be the golden ticket.

How to perform generative and evaluative research with Userpilot

Userpilot is a product growth software that can help you research and analyze your users without relying on technical expertise. Some of the features that can help so it’s not generative vs evaluative research, but both, are:

• NPS and microsurveys: Launch surveys that fit your brand feel to gather data on customer loyalty.
• Feature tags and heatmaps: Understand what features are most and least used and what might be causing customer issues leading them to drop off.
• Path analysis: Help you to understand where users are dropping off or not completing key processes in their journey.
• A/B testing and multivariate testing: Present different versions of elements of your product to see which ones perform the best.

Conclusion

Regarding user research methods, it doesn’t have to be generative vs evaluative research. The best approach is to combine both to ensure you get the best outputs.

Want to get started with generative and evaluative research? Get a Userpilot Demo and see how you can start conducting generative research.