Concept testing is an integral part of designing UX that not only satisfies user needs but delights them.

This article introduces various concept tests, shows you how to conduct them, and discusses ways to collect user feedback to gain actionable insights.

Let’s dive in!

What is concept testing?

Concept testing is a market research method used to evaluate the chances of success of a new product idea before it’s fully developed and launched.

It involves collecting feedback from target users via surveys, interviews, and focus groups to evaluate their perceptions as well as quantitative insights through product analytics.

Concept testing UX vs. usability testing

Usability testing and concept testing focus on different aspects of the product development process.

Concept testing is conducted early on in the design process, often before a fully functional prototype is available. The aim is to evaluate whether the basic product concept aligns with users’ needs and expectations.

On the other hand, usability testing is conducted later in the process, when a working prototype or the actual product is available. It assesses how easy it is for users to navigate the product, complete tasks, and whether they can intuitively understand and use the product’s features.

What are the benefits of conducting concept testing?

Concept testing should be an integral part of product development for a number of reasons. Let’s check them out.

Validation from your target audience

Concept testing allows SaaS businesses to validate early-stage ideas through the prism of customer needs and preferences.

This makes the product development process more customer-centric. It helps teams achieve the product-market fit by creating a product that addresses genuine customer needs and pain points.

Early detection of potential issues with the product idea

Customer feedback at the early stages of product development allows teams to identify potential problems or concerns related to the product idea.

For example, it can help you uncover some wrong assumptions that were made at the ideation stage.

By doing so, teams are able to gradually refine the product concept during the subsequent iterations.

Reduced risk of product failure

Concept testing allows teams to mitigate risks involved in product development and avert product failure.

By scrutinizing the product concept early on, teams are able to determine if the product is likely to be successful before committing any resources to its development.

The insights from concept testing can help teams to either pivot in the right direction or kill the idea altogether and concentrate their efforts on more worthy initiatives.

When to conduct a concept test?

The idea behind concept testing is to evaluate the idea before any time or money is spent on its development. That’s why the process should take place before you start building the product – during the initial product discovery phase.

However, it’s not a one-off exercise but rather a continuous process. Ideally, every single feature you decide to add to your product should be tested before you develop and launch it.

Who should be involved in the concept testing process?

The process of concept testing should involve colleagues from teams across the organization.

This includes the product team, marketing team, as well as UX designers and developers. As all of them bring unique perspectives and expertise to the table, it makes it easier to identify potential issues and find the best ways to tailor the product to satisfy customer needs and wants.

Of course, it goes without saying that you should always involve customers in concept testing. Without their feedback, you won’t have the insights you need to iterate on the initial product idea.

The different types of concept tests

There are 4 main types of concept tests that you may want to consider. The key differences between them are the scale and their objectives.

Monadic testing

Monadic concept testing involves the evaluation of a single concept or product by a participant.

The aim of this method is to obtain in-depth feedback without influencing the participant with other concepts or products. It’s particularly beneficial in situations where you want to understand how users perceive and feel about a product in isolation, without any comparative distractions.

This kind of concept testing is very resource-intensive because it requires individual interviews with participants.

Sequential monadic testing

In sequential testing, you present several concepts or products to participants in a sequence – each participant evaluates multiple concepts.

This approach provides comparative data, which makes it easier to identify the best-performing concept.

However, sequential testing results can be skewed by order and sequence biases, where the order of presentation can influence the evaluation or how well they’re remembered. What’s more, it might be more time-consuming and complex than other methods.

Comparative testing

A comparative concept test, also known as a preference test, involves presenting two or more concepts to participants at the same time and asking them for feedback.

This method is highly effective in determining which concept is better and can help you make a decision between a few well-formed concepts.

It’s possible to carry out comparative concept tests at scale online. However, if you choose to conduct it this way, bear in mind you will not always get in-depth qualitative feedback.

Protomonadic testing

Protomonadic testing combines monadic and comparative testing. Participants first evaluate a concept in isolation (monadic phase) before comparing it with other concepts (comparative phase).

This provides both detailed insights into the strengths and weaknesses of each concept individually and comparative data on how they stack up against each other.

Despite its advantages, protomonadic testing can be fairly complex to administer.

How to carry out effective concept testing?

The concept testing process consists of 5 main steps.

You set your goals and decide on the kind of test, prepare the test and questions to ask, recruit the right participants, collect and analyze the data, and act on the insights.

Let’s dive into each of them in a bit more detail.

1. Define concept testing goals

Defining clear goals at the beginning of the concept testing process is essential for obtaining valuable insights. Without a clear focus, you won’t be able to choose the right kind of test, ask meaningful questions, or choose the right participants.

For example, let’s imagine your goal is to increase demo bookings from your website and you want to conduct a concept test to choose the best CTA button microcopy. For this, you would choose a comparative test.

If, on the other hand, you want to validate a new feature idea, a monadic test would be more suitable.

If you need help choosing goals, why not use a goal-setting framework, like OKR or SMART? By doing so, you will ensure that your goals are clear and actionable. It will also help you choose the right tools to measure progress.

2. Prepare the concept test and the questions to ask

Once you have set your goals, it’s time to prepare for the actual test by choosing the right medium to conduct it and writing the questions that will enable you to gather the insights you need.

Surveys for gathering user feedback at scale

In-app surveys are an excellent way to collect customer feedback from a lot of participants.

Their main advantage is that they’re easy to design, and you can trigger them for specific user segments. However, to do so, you need a product as a launchpad for the tests, so in-app surveys are more suitable later in the development lifecycle.

To get the best of your surveys, use them to collect both quantitative and qualitative data. Start with a closed-ended question, for example, ‘On a scale of 1-5, how useful would you find this feature?’.

Follow this up with an open-ended question, for example, ‘How would this feature help you achieve your goals’ or ‘How would you improve this feature?’.

Make the qualitative questions optional so that users still submit the survey even if they don’t have the time to provide detailed feedback – or don’t know what to say.

Interviews for collecting deeper insights

Interviews and focus groups could be an excellent source of in-depth qualitative feedback.

They allow you to follow up on the initial participant responses to better understand user perceptions. This is important because your participants may not be able to express their ideas clearly to start with.

This makes them a great choice for monadic, sequential, and protomonadic concept tests.

Their downside is that they’re time-consuming and expensive to conduct. And if you don’t make the effort to prepare them and think through the questions to ask, you could end up wasting a lot of time.

A/B testing for gathering quantitative data in-app

A/B testing is normally used in the later stages of product development, where you have a working product in place. That’s because you need a platform to launch the two versions you’re testing and track their performance.

Is this your situation? If so, you could use them for comparative concept testing.

They’re a great tool when you’re trying to choose between two different designs, for example, menu designs or microcopy. Their main advantage is that you can use it to gather reliable quantitative data to validate your idea.

Its main downside, though, is that it offers no qualitative insights. In fact, users don’t even realize they’re part of a test.

3. Recruit the right test participants

Choosing the right participants for the tests is essential to gather the right insights.

First of all, you need to ensure that the concept you’re testing is relevant to their circumstances. If your users don’t have any use for a feature you’re testing, they won’t be able to offer anything of value.

Secondly, make sure to choose a representative sample of your target audience. While getting responses from your power users or early adopters might be easier, they only contribute a small percentage of your revenue. To get a product-market fit, your concept needs to appeal to the majority of users.

How do you recruit your test participants?

If you have a product in place, leverage your existing user base. Use your user segmentation tool, like Userpilot, to pick the right users and approach them via in-app messages or email.

If not, tools like Hotjar give you access to a pool of potential testers across different niches and can help you manage the testing process.

4. Conduct the concept test and analyze the responses

Having conducted the test, it’s time to analyze the data.

If you use a dedicated tool for testing, there’s a big chance it will do the quantitative analysis for you. If not, make sure to test the data for statistical significance to ensure the insights are valid.

Qualitative data analysis could be more challenging. Your feedback tool may allow you to tag responses for easier analysis or even use AI to identify patterns in user responses. If not, you can export them into a spreadsheet and then hack a generic AI tool to analyze it for you.

Iterate and repeat the test if needed

If your test reveals that the concept has no traction with your users whatsoever, kill it straight away. A failure before you commit any money or mental energy to feature development is not really a failure.

If concept testing shows that you’re on the right track, but the initial idea needs tweaking, iterate on the concept and conduct additional testing.

This iterative process will help you fine-tune the concept until it fully aligns with customer preferences and expectations.

Common mistakes when conducting concept testing

What mistakes should you avoid when concept testing?

Here are some of the most common ones:

• Lack of clear research goals – if you don’t have a clear objective, you won’t achieve anything worthwhile.
• Recruiting wrong testers – for example, relying on the most engaged or vocal users who are not representative of the user population and skew your perception of the demand for the product or feature.
• Asking wrong questions – questions that are not detailed enough or poorly worded, like double-barrel questions, will confuse participants and not provide valid data.
• Poor test design – choosing the wrong testing method or testing too many concepts at once can distort the results.
• Biased result analysis – people who are closely involved in product development and test design aren’t always the best ones to interpret the results because they may be biased, so it’s a good idea to involve other stakeholders in the analysis.

Conclusion

Concept testing enables product teams to validate product, feature, and design ideas before investing any money or time into their development.

Apart from smarter resource allocation, it enables the product to achieve the product-market fit in less time and, in extreme situations – avoid its complete failure.

If you want to see how Userpilot can help you conduct your concept test, book the demo!