Concept Testing Examples and Methods for SaaS

What is concept testing and how can it help accelerate your product-led growth?

In this article, we’ll go over some concept testing examples from some of the best B2B and B2C brands to help you get started with your strategy.

TL;DR

• Concept testing evaluates an idea before it’s introduced to the market to gauge its potential success.
• Concept testing helps to build customer relationships, understand customer needs, and gather market research data to drive decision-making.
• There are four different concept tests: Monadic testing, sequential monadic testing, comparative testing, and protomonadic testing.
• First, you must define what you’re testing and ensure you need concept testing, not fake door testing.

How concept testing can help:

• You can gather feedback about a new UI design with a survey showing an image of the wireframe design.
• To ensure they’re suitable, you can ask users for feedback on new website designs before they launch.
• Validate new features by asking users if they helped to achieve their goals.
• You can ask users for feedback on pricing to find the sweet spot between the most revenue and the highest amount users would pay.
• Get feedback on different messaging and ads to find which resonates.

How brands use concept testing:

• Todoist conducted 30-minute interviews with users to validate a new feature.
• To improve its blog strategy, Better Proposals asked its subscriber base which blog types they read the most so they could focus on creating more.
• Tesla announced an undesigned vehicle and let the number of people who put down a deposit give them confidence the vehicle would sell.
• Airbnb failed to concept test its logo and faced ridicule due to its design.

Userpilot can help gather user feedback with in-app surveys, use feature tagging and heatmaps to validate survey responses and conduct A/B experiments. Book a demo and get started!

What is concept testing?

Concept testing is a market research method used before launching a product or service. It’s where you present an idea or prototype to your target audience to gauge their interest, understanding, and potential acceptance using feedback collection methods.

By engaging in concept testing, you can gather feedback, refine your product idea further, and increase the likelihood of its success in the market.

Benefits of concept testing

Embarking on the journey of concept testing isn’t just about validating an idea and conducting market research; it’s about forging a deeper relationship with your audience and making informed decisions.

Let’s delve into the key benefits of concept tests:

• Build Lasting Customer Relationships: When you actively involve your customers in the development phase, it’s not just about getting feedback. It’s about showing them their opinions matter. You’re refining your product and building trust by working closely with them.
• Understand Customer Needs and Optimize the Product Development Process: Concept testing lets you pinpoint exactly what features resonate with your audience. Instead of shooting in the dark, you can tailor your product development process to align with your customer’s needs, ensuring every feature you add or modify fits their needs.
• Gather Real Data to Drive Decision-Making: It’s tempting to assume your new feature or product will be an instant hit. However, assumptions can be costly. Through concept testing, you gather tangible data from real users.

Different product concept testing methods

When diving into product concept testing, it’s essential to recognize while various methods are available, surveys remain a predominant tool. Let’s explore some of the most common product concept testing methods:

Monadic testing

In a monadic test, you’ll break down your target audience into multiple groups.

The unique aspect here is that each group gets exposed to only one concept. This approach allows you to analyze a single concept in-depth.

Since each group focuses solely on one idea, the feedback is usually short, highly targeted, and free from comparative biases.

Sequential monadic testing

For a more comprehensive view, sequential monadic testing might be your go-to.

Here, you divide your target audience into groups, presenting each group with all the concepts. To ensure fairness and avoid research bias, you randomize the order in which these concepts are shown.

Then, respondents answer the same follow-up questions, granting you deeper insights into each idea’s strengths and weaknesses.

Comparative testing

Comparative testing gives you clarity on which concept stands out the most.

You show respondents two or more concepts simultaneously. They then compare these concepts by rating them or selecting the one they deem best.

Protomonadic testing

For those seeking in-depth analysis and comparative insights, protomonadic testing is the answer.

This method starts with a sequential monadic test, where respondents evaluate multiple concepts. Following this, you ask them to choose their preferred concept in a comparison test.

This two-step approach provides detailed feedback on each concept and validates the results from the sequential monadic phase.

How to create an effective concept test?

Concept testing is a powerful tool, but like any tool, its effectiveness depends on how you wield it. To ensure you’re on the right track, consider the following steps:

Define what you need to test

Before diving in, take a moment to reflect. Not every idea or feature requires rigorous testing.

Ask yourself: Do you have a concrete idea you’re eager to validate? Can you create a tangible prototype, or is it something you can gauge through a survey?

If neither applies, there are better approaches than concept tests.

Do you need a concept test?

Sometimes, the goal isn’t to validate an entire concept but to gauge interest in a specific feature.

A ‘fake door test‘ might be more appropriate in such cases. This test presents users with a hypothetical feature to see if it piques their interest.

Remember, the essence of a practical concept test lies in ensuring you use it for its intended purpose.

Define the type of concept test for your use case

Once you’re clear on what you’re testing and why, it’s time to select the right method. Here are some examples to guide your choice:

• Monadic Testing: Best suited when you have a single concept and want to understand its strengths and weaknesses in isolation.
• Sequential Monadic Testing: Ideal when you wish to compare multiple concepts but want to avoid direct comparison biases.
• Comparative Testing: Opt for this when you need clear, direct feedback on which concept stands out the most among a set.
• Protomonadic Testing: A hybrid approach for in-depth insights and comparative feedback.

Concept testing use cases

Here are several use cases in concept testing that can help guide you when making decisions.

UI wireframes concept testing survey

A wireframe is a blueprint or sketch of a new web page or app, serving as a foundational step in the user interface (UI) design process. Testing these wireframes is paramount because it allows you to validate your vision early on.

After initial stages like logo testing, wireframe testing aids product development by providing insights into user preferences and potential design pitfalls.

This early feedback is invaluable, as it can save time, resources, and potential redesign efforts later.

Website concept survey testing

Website concept testing systematically evaluates a new website design or layout before its full development or launch. At its core, it ensures the website aligns with user expectations, needs, and preferences.

Why is this essential?

A website is often the first touchpoint for potential customers. Ensuring it resonates with them can significantly impact user engagement, retention, and conversion rates.

The process of website concept testing can vary based on the specific goals and needs of the project. However, two popular methods stand out: Comparative Testing and Protomonadic Testing.

By employing these testing methods and using a concept testing survey, you can better understand user preferences, ensuring the final website design is user-friendly and aligned with brand objectives.

New feature concept testing

New feature concept testing evaluates and validates a potential new feature before its implementation.

Introducing a new feature is both an opportunity and a risk. You need to ensure the feature functions as intended and resonates with the target audience, ultimately driving user engagement and satisfaction.

One of the most effective ways to test a new feature is through monadic testing. This method involves presenting a single feature concept to a segment of your user base, allowing for in-depth feedback on that specific feature.

You can pinpoint which features resonate with which user groups by segmenting users and testing features individually.

This targeted approach ensures that you allocate resources effectively and develop features with a clear understanding of their potential impact on different user segments.

Price testing

Price testing is evaluating different price points for a product or service to pinpoint the optimal price. With pricing and offer testing, you determine at what price you will make the most revenue and what users are willing to pay.

It’s a delicate balance between communicating value, affecting brand perception, and influencing a customer’s decision-making process.

To conduct price testing, first, establish your objectives. Understand whether you aim to maximize profit, increase market share, or achieve another specific goal. Then, select an appropriate testing method.

After determining a range of prices, you can present these different price points to a segment of your target audience. You can do this through surveys, A/B testing, or in a controlled environment.

Once you conduct the test, gather feedback and analyze how the price points impact purchase intent, perceived value, and other relevant metrics.

Based on the insights gathered, refine the price and implement it in the market if necessary.

Messaging and ad testing

Messaging and ad testing evaluates and validates different advertising messages, visuals, and formats to determine which resonates most effectively with your target audience.

Ensuring your message stands out and connects is paramount to driving engagement, fostering brand loyalty, and leading conversions.

The messaging and ad testing process typically begins with creating multiple ad variations. These variations can differ in messaging, visuals, call-to-action, or any combination. You then present these variations to a segment of your target audience.

Feedback from the audience provides insights into which ads are most effective. Metrics such as click-through rates, engagement levels, and conversion rates can offer quantitative data, while qualitative feedback can shed light on the reasons behind these numbers.

B2B Concept testing examples to inspire you

In the B2B landscape, concept testing plays a pivotal role in ensuring products and services align with the unique needs of businesses.

In this section, we’ll delve into examples of concept testing, shedding light on their approaches and the outcomes they achieved.

Todoist

Todoist, a leading to-do list software, identified a strong demand for a “Boards” feature among their user base. Recognizing the significance of this feature, the team was determined to ensure its debut would seamlessly fit within its product and meet user expectations.

The primary goal of their concept testing was to identify the essential features that would serve as a solid foundation for future iterations and resonate with their users. The team engaged with users who had previously shown interest in the Boards feature to gather the necessary insights.

They conducted in-depth interviews with 17 of these users from their global audience.

The insights from this concept testing proved invaluable. They clarified the most relevant usage patterns and allowed the team to prioritize specific user requests. With this data in hand, Todoist confidently introduced the Boards feature, which was met with enthusiasm by their user community.

Better Proposals

Better Proposals, a prominent online proposal-making software, faced a dilemma with its blog content strategy. The team was oscillating between two distinct content types: one focused on SEO-driven keywords and the other rooted in their hands-on experience with sales, marketing, and product development. While the SEO-centric content attracted new visitors via organic search, the experiential content retained their existing readership.

The question was: Which content type should they prioritize for their blog?

To address this, the Better Proposals team decided to turn to their most valuable resource: their subscriber base. They conducted a survey, posing questions to discover which content users preferred.

The feedback from the survey revealed an evenly split preference among readers for both content types. The team continued crafting SEO-focused articles and content based on their firsthand experiences, aiming to cater to the diverse interests of their readers and harness the strengths of both content strategies.

B2C examples of concept testing

While B2B companies often have a more niche audience and specific needs, B2C companies cater directly to consumers, making their target market vast and diverse. This broad audience presents its own set of challenges and opportunities. Concept testing becomes crucial to ensure that products resonate with the general public. In this section, we’ll delve into real-world examples from giants in the B2C sector.

Tesla

In 2016, Tesla embarked on an unconventional journey of purchase intent testing. Instead of presenting a finished product, they announced the Model 3, a car still in the conceptual phase. To gauge interest and intent, they offered potential customers the opportunity to place a $1,000 deposit for this yet-to-be-designed vehicle.

The response was overwhelming. Approximately 400,000 individuals expressed their intent to purchase by placing deposits. This response provided Tesla with a clear indication of the market’s appetite for the Model 3 and furnished it with the necessary funds to develop.

Airbnb

Concept testing, even for seemingly minor elements like logos, is vital. A logo, after all, is often the first thing people associate with a brand. It should be distinctive and memorable, but clarity is paramount to prevent unintended interpretations.

Airbnb’s 2014 new logo launch is a cautionary tale in this regard. Their redesigned logo, a creative rendition of the letter ‘A,’ was intended to symbolize unity and a sense of belonging. However, the public’s reception was mixed. Many Twitter users pointed out its striking resemblance to the logo of Automation Anywhere. Others humorously noted its likeness to certain anatomical features. The design was said to be inspired by the idea that a great logo should be simple enough to be drawn in the sand with one’s toe.

Despite the initial confusion and jests, both Automation Anywhere and Airbnb addressed the similarity, with Airbnb ultimately retaining the design. They persisted with the logo because of its distinctiveness and because it resonated with the brand’s ethos. While the unconventional design might have been the subject of internet humor, it was undeniably memorable and aligned with the brand’s values.

Airbnb’s experience underscores the importance of thorough visual concept and testing, especially before significant brand changes. While they managed to navigate the situation successfully, it’s a reminder for other brands to rigorously test new logos and visual concepts before a major launch.

Tinder

Choosing the right name for a product or service is more than just a creative endeavor; it’s a crucial step in brand identity and positioning. Name testing ensures that the chosen name resonates with the target audience, is distinct from competitors, and avoids potential confusion with other products or services.

Tinder’s branding journey offers a compelling case in point. The dating app, now synonymous with its fiery logo and the concept of ‘matches,’ was initially christened ‘Matchbox.’ While playful and suggestive of finding the right match, the name lacked distinctiveness and risked being conflated with other unrelated products or services.

Recognizing the potential pitfalls, Tinder opted for a name that was not only catchy but also unmistakably unique. The term ‘Tinder’ refers to ‘dry, flammable material used for lighting a fire,’ making it hard to confuse with another app and perfectly symbolizing its mission of igniting connections.

This evolution underscores the importance of rigorous name testing. By ensuring a name is clear, memorable, and aligned with brand values, companies can set the stage for lasting brand recognition and success.

Tools for running concept tests

Concept testing is pivotal in product development and marketing; the right tools can amplify its impact. This section will highlight leading tools like Userpilot, Hotjar, and Qualtrics, each offering specialized capabilities for various testing needs.

Userpilot for product and concept development tests

Userpilot offers a platform tailored for SaaS (Software-as-a-Service) companies looking to drive product growth. It offers a suite of features perfect for conducting product and concept development tests.

Some of the features that can help are:

• In-app surveys: Customize or choose surveys from templates to get feedback directly in-app.

• Feature taggings and heatmaps: Implement feature tagging and heatmaps to validate your survey responses.

• A/B testing: Use A/B testing to implement the different types of concept testing methods.

Hotjar for landing page testing

Hotjar is a comprehensive analytics and feedback tool that provides insights into website user behavior. With its suite of visual tools, including heatmaps and session recordings, Hotjar allows you to understand how users interact with their landing pages.

By analyzing where users click, how far they scroll, and what elements retain their attention, businesses can refine their landing pages for optimal engagement and conversion.

Qualtrics for concept testing reports

Qualtrics is a leading experience management platform that empowers businesses to gather, analyze, and act on feedback across various touchpoints.

When it comes to concept testing, Qualtrics stands out with its robust reporting functionality. Users can design surveys to test new concepts, and once data is collected, the platform offers in-depth analytics tools to dissect responses. Visual dashboards, trend analysis, and segmentation tools allow businesses to understand user perceptions and preferences in detail.

With Qualtrics, companies can transform raw feedback into actionable insights, ensuring that new concepts are refined and optimized based on real-world data.

Conclusion

Concept testing is more than just a step in product development; it’s a strategic approach to understanding and meeting user needs.

By leveraging tools and methodologies, you can run an effective concept testing survey to refine your offerings and ensure they resonate with their target audience.

Want to get started with concept testing? Book a demo and see how you can optimize your product for maximum user satisfaction.