How UX Researchers Can 4X Their Usability Test Response Rates With Userpilot
Usability testing is an invaluable resource for UX researchers…but only if you’re able to recruit participants in the first place. This is a problem that our own UX researcher at Userpilot, Lisa, faced when she tried recruiting participants the traditional way.
But when things don’t work out, we are reminded of the importance of eating your own dog food, aka using your own product to solve the problem you set out to address for your customers.
Learn how Lisa (and you) can quadruple your UX research response rates with Userpilot.
Overview of case study for usability tests with Userpilot
- Challenge: Lisa, our UX researcher, found it difficult to recruit participants for usability tests via email since B2B users are busy individuals with cluttered inboxes. This challenge was critical because it could lead to inaccurate product decisions, delays in development, and missed opportunities in a competitive industry.
- Solution: Lisa realized she needed a channel with less distraction and more engagement, leading her to choose our own product for inviting users. Using Userpilot’s survey functionality, Lisa created an interview invite survey and triggered it to the right user segment inside Userpilot.
- Results: In only a few days, Lisa was able to recruit 19 participants when she only expected to speak to 5 users. Hence, achieving four times better results than expected by inviting users inside the app.
- If you, too, are looking to streamline your user research process, there is no tool better than Userpilot. It enables you to collect customer feedback in-app, analyze user behavior using multiple reports, and conduct product experiments. Book a demo to learn more.
Challenge: Recruiting usability test participants
After leaving Microsoft for Userpilot, Lisa set out to conduct usability test interviews to learn how users utilized our popular customer segmentation feature. She began to recruit participants by emailing them but much to Lisa’s dismay, all she got was crickets.
This was not a problem in Microsoft because when you have millions of B2C users, it’s far easier to recruit participants with a $100 voucher incentive. There are even websites created specifically for enlisting user interview participants, cutting the efforts of researchers in half.
However, as Lisa experienced, this was not the case in the B2B space. B2B users, especially on the executive level, are busy individuals with even busier inboxes filled with spam, cold emails, meeting invites, and whatnot.
This left Lisa with a challenge because when you’re working in a competitive industry and a high-growth startup, you can’t afford to lose time getting users on board with your UX research efforts.
What’s the business impact of this challenge for SaaS teams?
Being unable to recruit participants for usability testing is a problem no UX researcher would want to face.
Usability tests are used to validate design decisions, monitor ease of use, and identify areas for improvement. But if you’re unable to gather a decent pool of participants, you bear the risk of your decisions being inaccurate, which may lead to costly changes post-launch.
But even before that, difficulty in enlisting participants can lead to delays in conducting usability tests, which can prolong product development timelines. Not only will this impact time-to-market, but it will also result in missed opportunities if your competitors beat you to the finish line.
Solution: Finding a different channel to recruit test participants
This delay in recruiting participants made one thing clear – there was a need for a better channel, one with fewer distractions and greater engagement. For Lisa, the solution became evident: leveraging our own product, i.e., Userpilot, as the channel.
Using our product to recruit interview participants was a logical choice. Unlike other channels with spam messages or advertisements, our app offered a focused environment where participants’ attention wasn’t divided. And, since our product was already integral to their work, it served as a familiar channel for engagement.
Lisa then used Userpilot’s in-app survey functionality to create a close-ended survey in mere minutes to recruit participants.
The best part? Lisa was able to send this interview invite to the right user segment – those users who had already previously used the segmentation feature.
She didn’t have to use another tool for this purpose. This was done no-code, right inside Userpilot.
Results: 4X more response rates for in-app invitations
So was Lisa’s decision to recruit participants in-app any fruitful?
In just a few days, Lisa was able to recruit 19 participants when she only wanted to speak to 5 people. She witnessed 4x better results than what she expected by inviting users inside the app (using Userpilot, of course).
This high response rate is not just limited to interview invite surveys. The team also launched a feature feedback survey and got 55 quality responses in just 2 days.
How to perform user research with Userpilot?
If you want to conduct user research seamlessly like Lisa, Userpilot is the solution you need. Here are a few features that will streamline your UX research efforts.
Collect customer feedback through surveys
We’ve already touched upon Userpilot’s survey functionality but that was just the tip of the iceberg. There are several survey templates you can customize for different purposes: market research, product-market fit, NPS – you name it!
You can trigger these surveys for relevant segments and choose where and when to display them. If you have a diverse audience, you can localize these surveys by translating them into different languages.
And if you still want to send email surveys (even though we proved why you shouldn’t), you can integrate Userpilot with HubSpot and Salesforce to send emails to highly targeted audiences.
Analyze customer behavior data over time
Analyzing user behavior can provide valuable insights, especially if some users struggle to articulate their feedback effectively.
Here are a few analytics features Userpilot offers that you can use and for which purposes:
- Funnel analysis – to identify conversion and friction points across the customer journey.
- Trend analysis – to monitor how user behavior patterns vary over time.
- Cohort retention analysis – to track retention rates of similar user cohorts.
- Path analysis – to discover the shortest path to value.
- Feature heatmaps – to identify features of high and low engagement.
- Custom event tracking – to track multiple user actions as if they were one.
- Analytics dashboards – to monitor important metrics and reports in visual dashboards.
Conduct A/B testing for product experiments
With A/B testing in Userpilot, you can ascertain how product changes impact user behavior and preferences. You can compare different variants of in-app flows to determine which version performs better and for which customer segment.
You can also conduct multivariate tests to compare more than two in-app flows.
Conclusion
This example showed us that there is no better channel for collecting customer feedback than in-app as it offers a less cluttered environment and enables you to reach your most active users.
If you want to streamline your feedback efforts in-app, there is no better tool than Userpilot. Book a demo to see it in action.
Try Userpilot and Take Your Feedback Collection to the Next Level
