How to Conduct Retention Research

Have you ever received this question from stakeholders: “Why are people churning?”

The first time I heard this question, I was immediately invested. I wanted to find out all the reasons people had left our app. I wanted to unlock this magical realm of information for our teams that I thought would solve our churn problems.

And then, I tried to recruit churned users. I tried to recruit people who were annoyed with us, didn’t want to hear from us, were pissed about something we did, removed themselves from our product universe, and, quite frankly, disliked us.

Spoiler: It didn’t work well.

I spent months trying to find people to talk to. Most of my recruitment efforts were rebuffed. No one wanted to talk to me about why they left. No one wanted to take my churn survey, even for a pretty solid incentive.

I repeated this process a few times with little success, and when I was finally accepting defeat, I decided to go about it in a slightly different way. 

What if we focused on retention instead of churn?

What’s retention?

Retention is all about keeping your customer base. It’s great to acquire new customers, but it is often (for most companies) really expensive. And, if you are acquiring new customers, but they are leaving, then you are burning through quite a lot of money.

Retaining customers not only ensures a stable revenue stream but can also lead to brand advocacy and long-term growth. You can measure retention using metrics like:

• Customer lifetime value.
• Churn rate.
• Returning to your website.
• Opening emails repeatedly.
• Checking your product repeatedly in a given timeframe.

For this particular article, we’re going to focus on calculating and tracking retention rate.

So, when it comes down to it, retention looks at how many people continue to use your product over a period of time.

Where does user research fit into this?

Retention research dramatically impacts organizations by revealing why customers stay or leave, influencing revenue, lifetime value, and acquisition costs.

The key advantage of user research is its focus on outcomes beyond metrics. Approaching projects solely with goals like “increase retention to 70%” emphasizes business outcomes while neglecting user outcomes. Users won’t stay simply because we want to retain them.

Users stay with products because of the positive experience the product creates for them. This positive influence can come in the form of:

• Helping them achieve a goal.
• Making a task more efficient.
• Alleviating a recurring pain point.
• Making their day-to-day life easier.
• Enabling them to do a difficult task more effectively.
• Meeting an unmet need that they can’t get met elsewhere.

A product serves a purpose for people. By clearly understanding the purpose, goals, needs, and pain points, we can start to understand how we might help people and encourage them to stay. User research is the key to developing that understanding.

Read the full article here. 

 

Components of retention

Retention isn’t as straightforward as other metrics, and I’ve seen projects quickly spiral out of scope. There are many confounding variables that affect retention, and looking too broadly often leads to vague, unhelpful insights. 

In this article, I’ll break down the key components to help you focus your retention analysis: broad enough to be meaningful, but narrow enough to be actionable.

1. Usage and timing

The usage of the product is one of the most critical concepts in retention. It will help you better understand your particular customers and the habits/behaviors you’re trying to encourage within your product. 

Here are some different ways to think about usage:

• One-time (or one-off) use versus repeat use: For instance, if we look at a subscription service, such as Netflix, versus someone purchasing a new TV, the amount of time between the purchases varies considerably. Knowing the frequency at which people are coming to you for conversions can help you measure retention more properly.
• Shifting milestones/goals: For example, if you subscribe to an app that helps you run faster, your goals might shift over time. You might go from learning to run a 5k to 10k, to more. Taking into account shifting milestones is incredibly important with retention and making sure you are focused on understanding these shifting needs.
• Lifestyle or seasonal context: Some products/services are only applicable for certain seasons or lifestyle events. If you’re a travel company that’s focused on summer family vacations, looking at a quarterly retention rate probably won’t be helpful and might lead to inaccurate conclusions. However, looking at a more seasonal or annual retention rate would make more sense.
• Long product lifecycle: Some products/services expand over an extended timeline, sometimes up to two years. It is really difficult to understand retention over a two-year period, so if you are faced with a long product lifecycle, it is really important to break up the experience, try to understand the different phases people go through, and measure retention within each phase.

2. User types

In addition to thinking about usage and timing, the other component to look closely at is your user types. 

When we think about retention, we can often lump users together into this mindset of “retaining more people”. 

However, we need to look more closely at which users we are trying to understand better because this will hugely help with scoping the research project to a manageable degree:

• New users: Are completely new to your product and haven’t yet engaged much. They might be in an onboarding phase, or they might have downloaded an app but not used it yet. To retain these users, you must pay close attention to your onboarding process.
• First-time users: Have completed something within your product/service (e.g., a conversion or done a particularly important behavior), but have yet to complete that action again.
• Repeat users: Repeat users are those who have used your product/service multiple times. Learn how to acquire and keep repeat customers.
• Multiple users: This is a product/service that has multiple users collaborating on a particular platform. When you have multiple users, it is important to think through which users you want to focus on (first), while still appreciating there is an ecosystem. For multi-user products/services, you typically have several stages of research to cover the most frequent/revenue-driving users.

Of course, for your organization, you might have personas, archetypes, or segments that go deeper than these types. 

I recommend using those when thinking through the exact users you want to target to understand more deeply. If you are struggling with this, I always recommend trying to understand the audience that will bring you the highest degree of revenue.

3. Stages of a product

A lot of data points to the fact that users usually churn within the first three months. During that time, they usually go through several phases within a product/service. 

For organizations with a much longer product lifecycle, these phases may be extended quite considerably, but they are still important phases to be clear on.

I recommend thinking through the phases and investigating which ones seem to have the lowest retention (or highest degree of churn). This can help you further focus your project on the “worst” part of the user experience. Let’s expand upon those phases:

• Onboarding: Not every product/service has an onboarding process, but you can think of this as the first touch-point that users have with your product/service. In this stage, they have completed the first step to using your product/service, and you are trying to get them what they need as smoothly as possible.
• Activation: Activation happens when users take the next step after onboarding and begin to see the potential impact of your product in their lives. This might include visiting other pages or spending a certain amount of time on your product. 
• Habitual use or adjacent use: The last stage I’ll mention is turning people into habitual users of the product. This might not be entirely applicable to every organization and might come in the form of adjacent usage. For example, most people won’t habitually purchase a mattress. However, if the experience has been good for them, they might make adjacent purchases, such as pillows, sheets, or other accessories. 

Defining a retained user at your organization

The next important thing to do, once you have a clearer idea of your usage, user type, and stage, is to define what a retained user actually looks like at your organization.

For example, Amazon looks at retention based on multiple completed purchases. Instagram might look at the number of things you post over a given time period, or the amount of time you spend on the app.

It’s important to define what a retained user is so you can understand what a retained user isn’t. You can ask yourself:

• What behaviors do they have to do?
• What is the frequency of behaviors?
• How long do they have to spend using my product/service?
• What is the timeframe they need to do this in?

The best thing you can do at this point is to take a look at the data of returning customers and understand what they are doing in a given time period. This can help you properly define a retained user.

How to set up a retention research project

We’ve covered a lot of complicated concepts within this article already. Retention is tough! 

There’s a reason why a lot of people don’t do retention research; it can get away from you so quickly, and you can end up with a lot of disjointed information that doesn’t end up being helpful.

So, how do we bring this all together into a cohesive, focused research project?

With a lot of planning and intention. So, let’s go through the steps to approach and set up a retention research project.

Step 1: Defining retention and business metrics for the project

Before we even delve into the research and segmentation, it is incredibly important to understand what retention means to you for this particular project. This is kind of like the success criteria portion of a research plan: What are you trying to achieve via this project? It’s okay if this looks a bit more business-focused.

For example, if we’re a flower subscription service, we’d want to define what we want retention to look like. It could be something like:

• Increasing the average subscription duration from three months to six months.
• Increasing the number of people who subscribe to larger or multiple bouquets after three months.

Basically, with this project, we want to have people stay with us for longer and, ideally, increase their customer lifetime value through larger basket sizes (either multiple or more expensive bouquets).

Therefore, our business metrics would look at:

• Increase customer lifetime value over six months.
• Increase retention rate from three to six months.

Step 2: Get clear on the gaps in knowledge and user outcome

Now that we have decided on the business outcome, which is increasing customer lifetime value and the retention rate, it’s time to think about the user outcome. You can ask your team questions like:

• What is it that the user is trying to achieve?
• What need is/isn’t being fulfilled?
• What is their purpose in using the product?
• What pain points/frustrations does the product help/hinder?
• How have you helped/hindered them in achieving their goals?

This step not only reminds you that you need to have a focus on the user, but it also highlights huge gaps in knowledge. If the team doesn’t know the users’ goals, needs, or pain points, then there might not be much of a clear retention strategy. 

Understanding the teams’ gaps in knowledge when it comes to user needs and goals will help you narrow the scope of the project.

For example, let’s say that our flower subscription service is here to ensure you always have fresh and beautiful flowers without having to go and get them. We know we are helping in the following areas:

• Making the task of getting fresh flowers more efficient.
• Introducing people to new flowers they’ve never heard of and educating them.
• Creating nice bouquets so they don’t have to.

However, we’re still unclear about some of our users’ additional pain points and unmet needs, which then become our focus for research.

Now we ask the question to get at the user outcome: “How will someone’s life improve if we deliver what the person needs that alleviates their pain points?”

At this point, you can hypothesize, and those hypotheses are something you can explore and evaluate in the research sessions:

• Having flowers ready for a party or visitors without having to think about it.
• Making someone’s day better and brighter.
• Allowing making oneself feel loved and special through self-care.
• Saving money on larger or multiple bouquets.
• Ability to get all your flowers from one place.

Step 3: Segment via user types, time/usage, and phase

Now that we have a clear picture of what we’re trying to understand more when it comes to a business and user context, it’s time to reduce the scope of our project by picking the best people to talk to via segmentation.

You can have several research phases planned here, so you don’t only have to do one retention project focused on a particular group. But, each part of the project should be focused so you don’t get lost trying to get “all the retention information from every user.”

In this step, it’s time to look at those user types, time/product usage, and the phase to focus on. You can use the previous information you gathered above to help you narrow down who you should talk to first.

For instance, we are seeing a drop-off at three months with our flower subscription, and we want to increase the average to six months, so we want to focus on repeat users that are at the three-month mark.

We also indicated that we want to increase each customer’s lifetime value regarding basket size, so it would be interesting to speak to people who purchase multiple bouquets or larger bouquets after a three-month period.

So, with this, we aren’t trying to impact completely new users who are in the onboarding phase, or only first-time buyers who are still in the activation phase. Rather, we are looking for repeat users who are at the three-month mark and those who have upgraded to larger/multiple bouquets.

As you can see, this reduces the scope substantially and gives us a much clearer picture of who to speak to when it comes to the research phase.

If you’re having trouble narrowing the scope, go back to steps one and two and try to drill down as far as you can. If not, you risk talking to too wide a range of people at the wrong times, which might lead to unactionable data.

Step 4: Create a user research plan

Based on the above information, it’s finally time to create your research plan. Within this research plan, you will highlight the information you have already gathered and then move on to choosing your approach.

Of course, this is a huge generalization, but when it comes to retention, you are trying to deeply understand people’s needs, goals, and pain points. When we satisfy these needs, goals, and pain points, we create a product that people tend to come back to and use.

With this in mind, our goals for our flower subscription project might look like:

• Discover pain points or frustrations that have occurred within the first three months of the subscription.
• Uncover any unmet needs of subscribers within the first three months of their subscription.
• Identify why people decide to upgrade to larger/multiple bouquets around the three months.

Using these goals to decide on an approach, we would be looking at conducting 1×1 interviews with this audience, as those interviews would allow us to deeply understand the above points. 

To be honest, the typical approach to retention research is 1×1 interviews because it allows for deep understanding and context-building, which then gives you recommendations to move the team forward.

For this, you could also follow up with a survey to help you prioritize the insights, such as the top three pain points to work on first, or the top three unmet needs to address.

An example retention research project

At a travel company, our high customer acquisition costs weren’t balanced by adequate retention, draining our finances. After failing to recruit churned users for research, I focused on retention.

Working with my product team, we analyzed data and discovered a key insight: customers who purchased three trips were significantly more likely to continue buying and had higher lifetime value.

Our retention goals became:

• Increase repeat orders, especially beyond the three-order threshold.
• Boost customer lifetime value.
• Drive more platform searches, which correlated with purchases.

I identified knowledge gaps and ran a team workshop to address them. While we understood users wanted easy ticket purchasing, we lacked insight into their specific pain points.

We hypothesized five key user outcomes:

• Prevent users from having to go to multiple websites to compare travel prices and ensure they are getting the best price for their trip.
• Provide clear and detailed information on what will happen if there are any delays, cancellations, or problems with the trip and very straightforward instructions of who to contact in case of a problem with the trip.
• Make it easy to book multi-destination tickets for people looking to travel through several different cities during one trip.
• Make it simple to customize the type of booking people are typically looking for (e.g., train-traveler) so that they get the most relevant results higher in their search.
• Create a seamless app-to-web experience so that people can easily pick up previous searches or check out via the web if necessary.

Two key audience segments:

1. Customers with fewer than three purchases.
2. Customers with more than three purchases.

We focused on leisure travelers within these segments as they drove our revenue. Later, we discovered that customers booking three trips within six months had the highest lifetime value: information we would have included in our initial recruitment criteria had we known it then.

With this in mind, I created a user research plan to make it super clear what we were trying to accomplish within this research project. I no longer have that research plan (SAVE YOUR WORK), but I will give an overview of what it looked like.

Goals:

• Understand the pain points people have encountered in the timeframe before booking three trips with us.
• Identify perceptions behind why people booked more than three trips with us.
• Uncover unmet needs of users who haven’t yet booked three trips, as well as those who have booked more than three trips.

Approach:

• 1×1 generative research interviews: I took these interviews away from the product and into the complexities of planning trips from end to end, including their needs, goals, and pain points. I used the TEDW framework to ensure open-ended questions.
• Usability testing: I first conducted qualitative usability testing, observing trip booking behavior, revealing issues with filters, trip linking, and favorites. After implementing improvements, I benchmarked the experience using quantitative metrics (task time, success rates, SEQ, and SUS scores). This identified struggles with basic tasks, leading to critical usability fixes that improved our platform.

This study revealed where we failed to meet customer needs. By adding price comparison, eco-friendly options, and trip sharing, we significantly increased repeat orders, customer lifetime value, and overall retention rates.

Read the full article here.