User feedback is the single most valuable resource you have available when you’re trying to grow a SaaS.

But implementing feedback indiscriminately can do you more harm than good.

Why?

Too much user feedback can present problems:

  • What if the feedback is pulling you in all directions?
  • If acting on everything your users tell you would overstretch your resources?
  • If you can’t decide what you need to do about it?

On the other hand: when you receive feedback (even if it’s negative) you’re able to improve faster than you would have done without it.

Feedback drives you to prioritize the things people don’t like or struggle with and to emphasize what they love.

That’s a lesson we’ve taken to heart at Userpilot.

So how to analyze user feedback (even from multiple sources) and use it well to grow your product?

That’s what we’ll show you in this blog.

Looking for ways to proactively collect and categorize user feedback? Talk to our Product Expert at Userpilot and try our AI-powered feedback collection and categorization tool out!

TLDR? Here are the headline takeaways:

User feedback comes in all shapes and sizes: chat logs, support tickets, social media, in-app, email, public comments, reviews, even analytics and more.

You need different methods and tools for collecting (i) in-app feedback from users, (ii) out of app feedback from users and (iii) feedback from the public.

Always sort feedback into categories: (i) whether it’s solicited or unsolicited, (ii) the types of the user (role, use case, company account vs. individual, plan) giving it, (iii) the types of issue being raised, (iv) the severity of the issue, and (v) whether it’s positive or negative feedback.

(i) Collate and analyze the feedback, (ii) identify solutions, then (iii) prioritize and (iv) share them. But be careful to (v) consider whether you’re using feedback in the right way.

We take a look at (i) Userpilot, (ii) Drift, (iii) Refiner, (iv) Canny and (v) ProdPad.

Negative feedback needs special treatment, but with all types of feedback: (i) be ready to dig until you find something actionable; and (ii) always keep your users informed of what you’re doing about their feedback.

Sources of User Feedback

Users, prospective users, ex-users and even people who’ve never used your service can all provide valuable feedback.

The trouble is, it can come from anywhere. Such as:chat logs, support tickets, social media, in-app, email, public comments, reviews, even analytics and more.

You know user feedback when you see it – but how to categorize it and proactively collect it?

How to Collect User Feedback

We covered this in depth in our last blog.

In a nutshell:

In-App Feedback from Users

  • Collect feedback at the end of a session, an onboarding flow or on completion of a task, when every aspect of the experience is fresh in the user’s mind
  • Ask for feedback when users navigate away from a task before completing it
Qualaroo microsurvey

Source: qualaroo.com

  • Ask near to the end of a free trial period
  • Get feedback when a problem has been resolved

Out of App Feedback from Users

  • Email newsletter subscribers to find out whey they are not converting to users
  • Email inactive users to find out why they are disengaged
  • Email outgoing users to find out why they are leaving
  • Acknowledge problems raised by users on social media and public forums on the same channel, but ask them to deal with you further in private (email, phone, chat)

Out of App Feedback from the Public

  • Use social listening tools like Brand24, Awario and SentiOne to track mentions of your company and your product across public sources
Awario

Source: awario.com

How to Categorize User Feedback

These are the most important lines along which to categorize the feedback you receive.

– Categorize Feedback by Solicited/Unsolicited

Feedback is solicited or prompted when you ask for it. For example, this CSAT microsurvey from Google solicits feedback about a particular experience.

User feedback: Google Hangouts

Source: google.com

This is very different from feedback that somebody has provided completely spontaneously. When you ask for feedback:

  • You control the question and when it is asked. The example above appears immediately after a Hangouts call when the experience is fresh in the user’s mind. This helps to get an honest and accurate response.
  • You only get feedback on issues you ask about. Unsolicited feedback has the advantage of being able to bring problems you weren’t aware of to your attention.

It’s important to note that:

  • Unsolicited feedback tends towards extreme positions. Anyone who comments on a SaaS online without being asked will generally love it or hate it. Extreme reactions are far less likely when dealing with solicited feedback, because the majority of people are “fine”.
  • Unsolicited feedback is rarer. If you don’t distinguish it from solicited feedback, you are likely to end up undervaluing its lessons.

Bridge the gap between these types of feedback by asking follow-up questions to structured, solicited feedback that allows users to respond in their own words.

Here’s how this is done with Userpilot NPS surveys.

Userpilot NPS survey builder

Looking for easy-to-use NPS surveys? Talk to our product specialists now!

Also – if you want to learn more about using feedback to improve your product adoption – check out our Product Adoption School:

 

– Categorize Feedback by User Type

Different users have different needs and priorities. Break these down and detail them when you work out your User Journey.

When a user offers feedback on a subject that’s relevant to where they are (or are heading) on the User Journey, pay particular attention:

  • New Users – Look for anything preventing/promoting Activation; issues relating to Time To First Value; difficulties with Primary Onboarding, etc.
  • Trusted Long-Term Users – Loyal customers often have much insight into use cases, familiarity with features and Knowledge Base material. Or they may be calling for new features or failing to get continually high levels of value.
  • Power Users – Although power user feedback can lead you astray, power users who understand and advocate for your service can give valuable help in designing your future roadmap.

Plus:

  • Inactive Users – Look for details about why they are not getting value from your service (analytics are particularly valuable here – these people tend to be quiet), and what would lure them back.
  • High-Paying Users – You’re in business to make money, right? So naturally, you should give greater weight in general to the feedback your top payers provide.
  • Exiting Users – Ask users who are leaving why they left! Seth Banks of Cashboard explains here why he decided to make exit surveys compulsory.

– Categorize Feedback by Issue Type

You must also categorize user feedback by the kind of issue/problem/solution that is being sought.

The number of types will depend on:

  • Volume of feedback received
  • Variety of features and use cases your service supports
  • Resources you have available

For most SaaS businesses, issue types will include:

  • Software bugs
  • Usability issues
  • User knowledge/success problems with educational solutions
  • New feature requests
  • Product insight
  • Pricing and billing issues

The right category may not always be obvious. For example, as our case study explains, Kommunicate.io received loads of requests for features that were already present.

So these comments were really user knowledge/success problems with educational solutions.

– Categorize Feedback by Severity

You must also categorize feedback by the seriousness of the consequences it involves.

  • Feedback relating to matters with urgent, severe consequences (privacy, security, payment errors) needs immediate attention and clear, rapid-response policies in place
  • Feedback that relates to systemic problems (eg bugs that potentially affect all users) need urgent attention

When setting up a user feedback management system, be sure to include measures that alert you to these kinds of issues immediately.

– Categorise Feedback by Sentiment

Sometimes it’s very easy to tell if the feedback is good or bad. This Slack example asks for nothing else!

Sentiment - good or bad?

Source: slack.com

When users comment in their own words, it’s not always so easy. Some tools, like Brand24 use language processing to help classify sentiment, picking out “positive” and “negative” words in comments.

Sentiment analysis with Brand24

Source: brand24.com

Don’t forget the importance of positive feedback. When users are telling you that, you’re doing the right thing:

  • Do more of the stuff they like!
  • Ask them for more information – if they’re well-disposed towards you, they may be happy to talk

How to Implement User Feedback in Your Product Roadmap

Step 1: Collate and Analyze

A five-star review, an NPS score of 6, a 40-minute support chat and a smiley face – it’s not always easy to boil all of these down into common terms.

That’s why you categorize the feedback.

Most people will start with a big spreadsheet, with entries looking something like this:

Categorizing user feedback

Source: intercom.com

Once you have all the data in one place and presented in a consistent format, you can analyze it for patterns in:

  • Issues that keep coming up
  • Issues that are associated with particular cohorts of users
  • Issues that are raised by users who display certain in-app behavior patterns, as revealed by your analytics
  • Issues experienced by users at different stages of the user journey, etc

It should hardly need to be said, but this will probably require you to read the detailed feedback in depth and interpret it.

Remember, users won’t always use the jargon you’re familiar with and they will have gaps in their knowledge.

Step 2: Find A Solution

  1. What would it take to solve the issue raised?
  2. How important is solving it to the user?
  3. If the data you have doesn’t answer these questions, go back to the user for more.
  4. Once you know what it would take to solve the issue, work out how much effort it needs.

When you need to find out more about a particular issue:

  • Set up in-app microsurveys at relevant points, to get feedback from users immediately after they take the related action
  • Email other users with longer surveys to explore their experiences
  • Write a blog or social media post and invite comment

The more users’ experiences you can collate around an issue, the greater the likelihood of:

  1. Choosing the right solution for the problem
  2. Choosing the right problems to work on

Step 3: Prioritize Solutions

A simple matrix like this can help sort out solutions. In general, you would order them:

  1. High user demand (lots of users or high user importance)/simple to build – Maximum impact for minimum outlay
  2. High demand/complex to build – High impact, but a lot of investment or effort
  3. Low demand/simple – Easy wins but of little importance to the business
  4. Low demand/complex – Bottom of the list…
Eisenhower matrix

Source: outcry.com

Within each segment, prioritize solutions by relative frequency of the issue being mentioned in feedback.

This helps counteract the natural bias most people have towards the last comment they read!

Step 4: Share Across the Team

As Product Manager, your point of view may not incorporate every relevant factor. Perhaps your Dev Team is working on a back-end upgrade that you’re unaware of?

In putting your product roadmap together in this way, make sure you get feedback from all affected teams.

Implement or Ignore?

Henry Ford wasn’t a big fan of user feedback. He famously said:

If I had asked people what they wanted, they would have said faster horses.

Sometimes that’s true in SaaS. If you have a clear vision behind your product roadmap and you know where you want to go, listening to users could lead you astray if those users don’t get the vision.

Andrea Saez of ProdPad told us:

A product manager’s job is to solve problems, not to build things based on popularity contests. People don’t know what they want, they just know they have problems.

So should you be incorporating user feedback into your product roadmap at all?

The answer is: Yes, but you need to use the right feedback in the right way.

As Andrea explains:

Feedback doesn’t get prioritized. Feedback is used to validate the product backlog, which then gets prioritized. Feedback is there to give you insights as to how you might solve a problem, ideas are there to help you understand what problems to solve, and initiatives help you hypothesize as to the opportunities you have to address these problems, based on your current objectives.

Tools for Making the Most of User Feedback

In this section, we’ll highlight some of our favorite tools that will help you manage feedback.

Userpilot

Userpilot lets you integrate microsurveys (including NPS surveys) into user onboarding experiences without writing a line of code.

These are vital for getting that contextual, immediate feedback.

And soon it will feature NLP-powered sentiment analysis.

Userpilot microsurvey

Book a free call with one of our product experts to find out more now!

Drift

This is the tool we use at Userpilot for managing the chat functionality within our Help Widget. As we explained at the very start, these kinds of interaction provide really important information about what users value.

Drift

Source: drift.com

Refiner

Refiner has been built for collecting and analyzing user feedback in-app. What we like about it is that it supports more types of question than virtually any competitor.

Refiner

Source: refiner.io

Canny

Had enough of spreadsheets? Canny is a complete platform for collecting and analyzing user feedback and integrating it into your product roadmap.

Canny

Source: canny.io

Canny is really easy to use, it integrates with many other tools (important when you’re bringing data from multiple sources together) and it offers a 14-day free trial.

ProdPad

Like the “big spreadsheet” a lot of people stick their product roadmaps into very simple tools like Trello.

When that’s no longer enough, you’ll need a proper product roadmap tool – and ProdPad does the job.

ProdPad

Source: prodpad.com

It’s great for collecting user and staff feedback and co-ordinating everything into a single, shared product backlog. They even offer a free trial!

Following Up on User Feedback

Our last blog dealt in detail with:

Here are two extra lessons to add:

#1 Dig Deep

If user feedback isn’t giving you enough information to act on (eg an NPS score of 5 followed by the comment “bad value for money”), dig deeper.

7 out of 10 people who leave bad reviews change their minds after the company responds – often simply listening and providing a solution is enough to win a detractor over.

Plus, the solution you arrive at may help you avoid other users experiencing the problem in the future.

This applies to good feedback as well. Always try to find out enough to act on.

Find out more when you get feedback

#2 Keep Users Informed

Everyone knows how important it is to close the loop on customer service issues, but it’s true of feature requests (for example) as well.

Proactive communication helps to engage users more and more deeply with your product. Even if you’re telling someone that you’re not going to act on their suggestion, they’ll usually appreciate the courtesy.


If you need a reminder of the takeaways, check out the TLDR.

About the Author: Emilia Korczynska is Head of Marketing at Userpilot. 

Emilia Korczynska

 

 

 

 

 

 

Stay in the loop with our email course: