Detecting dropouts in the user journey and hypothesizing effective solutions
Lesson #3
Detecting dropouts in the user journey and hypothesizing effective solutions
In this lesson, we’ll be focusing on how you can use product usage analytics to detect where users stray off from your mapped journey.
Additionally, we’ll be looking to understand why these dropouts happen.
This will help you to hypothesize effective solutions that you can test.
Detecting Dropouts in the Path to Activation
One of the most alarming issues that any software business faces is having users drop out on their path to the activation event(s).
Failing to address this issue is guaranteed to have an abysmal effect on conversion rates.
Naturally, the best place to start is by actually knowing whether the problem actually exists. And, if it does, what is the magnitude of it?
Luckily, by now you should have defined the activation set of events and mapped them into your analytics tool.
Now it’s time to do some quantitative analysis!
For example, let’s say you’ve had 50 new signups in the last 60 days. If only 5 of those have activated, then your activation would be 10%.
At this stage, your actual activation % doesn’t really matter. The goal is to use it as a compass that can help you know if the onboarding process is working or not.
You’ll also want to keep track of the activation metric constantly, and make sure you keep an eye on how it changes over time.
Knowing your activation metric is one thing, but understanding where users are actually dropping out is another thing.
If your activation metric is very simple, that is, only one event, then you already know where the issue happens.
For activation metrics that consist of a series of different events, you’ll need to take a deeper look to know exactly where the dropout is happening.
For that, you might want to run a funnel analysis.
Using your analytics tool, simply create a funnel that consists of all your activation events. It will be easy, then, to see the point of the funnel where users struggle to make it through.
At Userpilot, for example, our activation events consist of building a product experience, installing the JavaScript code and finally deploying at least one experience live.
Doing a funnel analysis might show, for example, the following:
As we can see above, 80% of new signups successfully build a product experience. However, only 20% of them install the JavaScript.
Now luckily, almost all users who installed the JavaScript go ahead and deploy at least one experience live.
One can easily deduct from the above funnel that the dropout happens in installing the JavaScript code.
Using this method, you can easily detect where the dropout is happening.
But why is it happening? Well, that’s the next question.
Understanding Why Users Are Dropping Out
In the last section, we were able to find our activation % as well as pinpoint where the dropouts are happening.
It’s now time to investigate the why.
The truth is, it’s very hard to figure out the “why” using just quantitative data.
Therefore, it’s almost always true that you have to depend on qualitative data at this stage of analysis.
Let’s take a look at some of the approaches you might want to try.
Hold user interviews
Conducting one on one interviews with existing users is by far the best approach to understand the ‘why’ behind dropouts.
Try to clearly identify the moments of frustration that the user has experienced during their path to the activation event(s).
Most importantly, understand why they happened.
Was it a UX issue? A technical difficulty? Were the next steps not clearly defined? Or did your initial Aha! Moment not give them enough motivation to continue?
Whatever the reason may be, this approach will give you the first clue on why things went wrong.
You should also take this chance to collect feedback from users on how the process could be improved.
Run usability tests
Usability tests can help your team understand in-app user behavior in real time.
This can be particularly helpful if dropouts are caused by UX issues.
Watch session playbacks
Software like FullStory can also help you understand your product’s usability through session playbacks.
Use this to analyze how new signups are navigating around the product, and try to detect why certain tasks are not being completed.
Survey users & collect feedback
Email and in-app survey polls can also be helpful when collecting qualitative data about why users are dropping out.
This technique is especially useful when you’re trying to collect feedback from a large number of users.
Hypothesizing Effective Solutions to Improve Activation
By now, you have mapped the user journey to activation and understood the underlying reasons of why users are dropping out.
It’s now time to hypothesize some adjustments to the onboarding experience that can improve activation.
Of course, every situation is different. It’s up to you and your team to find the right hypothesis based on the problems you are facing.
Nevertheless, here are some approaches that can help you make the right adjustment to the onboarding experience.
Create a laser-focused first run onboarding experience
The first-run experience is by far the most important one. Make sure it’s laser-focused by giving fewer choices to the user and focusing solely on driving that first Aha! moment.
Don’t try to showcase all the features of your product. This will confuse your users.
Instead, focus on driving pure value by paving the way for that Aha! moment you identified earlier. This will increase the likelihood of activation.
Trigger contextual hints and tips
Avoid general product tours that trigger without any real context.
Instead, focus on helping users when they actually need it. Trigger UI patterns such as tooltips when users visit a certain view, or when they successfully complete a certain action.
Utilize empty states
Some tabs in your product are in an empty state (zero data) by default.
A blank page, though, isn’t a very welcoming one for new users.
You can look at this as a chance to drive actions. Instead of leaving them empty, call for an action using a button or a UI pattern such as a modal.
In this example from Sketch Cloud, the user has navigated to “My Documents” only to find there isn’t anything there.
Sketch have used some copy to fill the space and explain what the user should do to start using this functionality.
You could also try pre-filling the page with relevant demo data to make things more intuitive for your customers.
Analyzing Secondary Feature Adoption
In lesson 2, we talked in depth about mapping the path to adoption; that is, the path to adopting all the persona-relevant secondary features.
In particular, we used correlation analysis to map each secondary feature and when it should be used.
While failing to adopt secondary features is not technically a dropout, it’s well worth noting that it can have a negative effect in the long run.
Analyzing secondary feature adoption is a relatively easy task using your analytics tool.
Simply ask your developers to pass the custom events indicating whether a certain feature has been used or not, and then segment your user base based on that.
This way you can see which features are most popular as well as track the adoption % for each specific feature.
Your correlation analysis from lesson 2 should have given you at least some sense of why and how these features are used.
You can also use qualitative data, such as customer interviews, to try and learn more about the use cases for such features.
You can then use this information to hypothesize solutions on how to drive adoption for each of these secondary features.
As a general rule of thumb, the best way to increase feature adoption is by triggering product experiences that push for them in the right context.
Lesson 3 Key Points
- Calculate your current activation%; this will be your compass moving forward.
- Use your analytics tool to find exactly where dropouts happen in the path to activation.
- User interviews, usability tests and surveys are qualitative data that can help you understand why the dropouts are happening.
- Hypothesize solutions based on both qualitative and quantitative data.
- Track secondary feature adoption, and try to push users to adopt them at the right context.
Next Steps
In the last couple of lessons, you have learned how to properly map the user journey to adoption as well as detect problems associated with it.
By now, you must have come up with a few hypotheses on how you’re going to improve the user onboarding process.
In the next lesson, we’ll dive deeper on how to choose and integrate the right UI patterns in your onboarding experience, and ultimately test your hypothesis.