A Beginner’s Guide To Saas Product Analytics

Without product analytics, how do you know how to move the needle with your product growth?

If you’re only beginning your adventure with product analytics, looking at all the usage data may seem overwhelming at first glance. But making the effort to understand the fundamentals will allow you to make significant improvements to all your growth metrics faster.

This post will help you wrap your head around the main concepts in product analytics, and how to implement them to grow your product!

Let’s start with a brief introduction:

What is Product Analytics?

Product analytics is the process by which you collect, inspect and analyze data about users’ interactions with your product.

By tracking, recording, and attributing users’ every action, you can build up an incredibly detailed picture of:

• Who your users are
• How they are navigating around your product
• What tasks do they struggle with?
• Points at which they experience friction
• Which features are most used and which are under-used
• How different groups of users behave and achieve value, etc, etc.

In the broadest terms, product analytics makes it possible for you to assess the impact of the digital experiences you have built into your product.

Is Google Analytics is a SaaS Product Analytics Tool?

No.

If you’ve ever worked anywhere near marketing, you’ll have come across Google Analytics.

If you’re interested in getting the full benefits that product analytics has to offer, you cannot rely solely on Google Analytics.

Why?

Because Google Analytics is a marketing analytics toolkit that is geared solely towards the first part of the user journey – that is, where visitors come from.

This blog from Semetic has a nice illustration of this point:

Source: mixpanel.com

The way a user gets to a website is infinitely less complex than the ways a user can interact with a product.

Although it is hackable to a certain extent with Tag Manager, the fundamental point remains: Google Analytics relies on anonymized traffic rather than event-based tracking that is linked to a single user ID.

Without that persisting user ID, there’s no way to gain insight into how individuals behave.

Are Product Metrics the same as Product Analytics?

No.

Product analytics involves finding patterns in product usage data that can be used to inform business decision-making.

Product metrics (or KPIs) are measurements of progress towards pre-agreed business goals as we’ll discuss a little later on.

Analytics is one of the most important ways for identifying and validating ways of improving product metrics, as we’ll discuss in the next section.

Why is Product Analytics important?

51% of people will never return to a company they’ve had a bad experience with.

In the SaaS world, the overwhelming majority of your users’ interactions with your company take place through your product.

So, understanding what they do within your product is critically important for:

• improving new user Acquisition
• improving user Activation
• maximizing and increasing Revenue
• improving user Retention
• encouraging current users to refer you to new users, etc, etc.

As Peter Drucker famously put it:

What gets measured gets managed.

All the metrics that product managers need to care about depend on user behavior – on the things they do and don’t do. On how they feel about the product. And on the value they get out of it.

What Roles Use Product Analytics?

But it’s not just product managers who should care about product analytics:

• UX designers need to understand how users navigate around products and interact with UI elements
• Developers need to understand what bugs users are experiencing and what improvements should be prioritized
• Marketers need to understand which channels and campaigns are bringing in the most profitable users
• Finance teams need to understand revenue trends and costs for business forecasting

Not only that, it’s essential in businesses of all sizes. Even startups can make great use of in-app analytics to adjust and refine their products to ensure Product-Market Fit.

Why is Product Analytics More Effective than Other Feedback Sources?

We’ve produced a long post on this topic previously, but here are the takeaways:

✔ Objectivity

Product analytics is objective. Most surveys and interviews allow subjectivity to creep in (the exception being very short microsurveys, like the NPS functionality provided with Userpilot). People frequently misremember what they did, why they did it and what happened next. Analytics doesn’t.

What people do is a much better indicator of how they feel and what they will do next than what they say – particularly when behavior patterns are aggregated across large numbers of users.

✔ Specificity

Analytics also allows you to take an extremely granular, specific view. It enables you to zoom in on particular issues – adoption of a particular feature, abandonment rates of a particular task, onboarding flow completions – and diagnose what is happening.

Most analytics tools also allow you to segment your audience – by demographic features, by cohort, by user journey stage, by payment tier, etc.

By breaking your user base down into segments, you can tailor individual solutions to their problems and messaging that addresses their needs directly.

✔ Efficiency

Analytics is also much more efficient that surveys and other feedback sources. Surveys take a long time to set up and carry out. The response rate is often low. Interviews take even longer, and are particularly prone to subjectivity risks.

But installing an analytics tool on your SaaS is usually just a matter of dropping in a line of code, and then managing the data through a user-friendly GUI.

That’s not to say that feedback from sources other than analytics is without value. Check out this blog on getting the best out of user feedback to find out more.

However, as the latest State of Product Analytics report showed, the more data-literate and data-driven a product team is, the more likely product analytics is to be their main source of user insights.

Source: mixpanel.com

What KPIs are used in Product Analytics? How to use Product Analytics for your SaaS

Better to ask: how do I set the right KPIs for my SaaS?

The best product analytics tools will let you track and measure almost any kind of in-app event. Most will let you define your own custom events or combine activities into virtual event groups.

But not everything can that be measured matters!

So follow these rules:

Product Analytics KPIs 101a: Articulate Your Business Goals

What is your business trying to achieve and how does the product serve that?

• If it generates revenue, then revenue should definitely be a KPI!
• If the product is a freemium marketing tool aimed at upselling users to a paid alternative, the number of free trials and conversions from free to paid should be tracked
• If your product has a high Customer Acquisition Cost (CAC) that you only recoup gradually through subscription fees, then churn and retention rates are going to be of huge importance when it comes to making a profit

Bear in mind, there are likely to be a lot of stakeholders in your business outside the product team who deserve to be consulted at this stage of the KPI setting process.

Product Analytics KPIs 101b: Make the Goals Quantitative

One highly effective way of deriving KPIs is via “SMART” goals.

“SMART” stands for :

• Specific – Tied to a particular activity or goal
• Measurable – Progress towards achieving it can be measured
• Attainable – There’s a realistic prospect of getting there
• Relevant – The goal makes a meaningful difference to your business
• Time-Limited – There’s a deadline to when it should be achieved by

Source: Fitsmallbusiness.com

A SMART goal for a SaaS could be to increase the number of users who have paid more than their CAC by 150% within one year.

Product analytics is all about numbers – it’s first and foremost a quantitative activity.

Next, translate your SMART goals into numbers – in terms of the changes in behavioral patterns that will be needed to realise them.

How?

1. Benchmark where you are now – for example, perhaps you have 500 profitable paying users
2. From the SMART goal, derive a quantitative target – in this case, getting another 750 users to the same position within 12 months

A quantitative KPI doesn’t have to be an absolute number. The authors of “Lean Analytics” recommend using ratios and ranges instead.

Product Analytics KPIs 101c: Hypothesize and Test

How do we get from where we are to where we want to be?

Come up with explanatory hypotheses and put them to the test.

For example:

1. “At our current rate of new user acquisition, lowering the CAC by 10% would see the point at which users become profitable drop from 4 to 3 months”
2. “Trial users who do not use the product more than once in their first week are significantly less likely to become paying users”
3. “Users who have adopted more than one feature are significantly less likely to churn in their first 6 months than those who have adopted only one”

• 1 could be tested by taking steps to lower CAC, then analyzing the retention of the cohorts acquired under different marketing campaigns.
• 2 could be tested by split testing a more intensive primary onboarding workflow for new users against your existing arrangements.
• 3 could be tested by building in-app experiences to alert certain segments of users to the most relevant features.

Your analytics will tell you whether each hypothesis is valid.

Looking to build in-app onboarding experiences? Talk to Userpilot’s Product Expert and get started today.

Product Analytics KPIs 101d: Drop the Vanity Metrics

Too many businesses take an off-the-peg approach to their KPIs – selecting them not because of the difference they make, but because everyone else uses them.

Even worse than this is depending on “vanity metrics” – figures that look good but don’t really mean anything.

As Roman Pichler suggests here, for example, “app downloads” is often just a vanity metric (especially when the app is free). What matters far more is repeated log-ins, time on-site, onboarding flows completed etc.

Of course, whether a metric is a vanity metric depends on what moves the needle for your SaaS – which takes us back to the start of this section!

The Best Product Analytics Tools for SaaS

Which tool is best will depend on what your SaaS does and what you need to know about your users.

There are loads of different tools available serving both particular niches and the sector as a whole. Here, we’ll look at five that serve a variety of different needs.

Product Analytics Tool #1: Userpilot

In addition to giving you the power to build and deliver contextual in-app experiences for your users, Userpilot provides product analytics to track key metrics around:

• New User Activation in Onboarding
• Feature adoption
• Educating and upskilling existing users
• Self-serve and helpdesk usage
• User sentiment

So Userpilot’s analytics capabilities are aligned with the needs of growth managers, product managers and product marketers in SaaS. The insights that can be extracted will also help UX designers and web developers.

Once the Userpilot javascript code is installed on your app, the click data can be collected through feature tags on the user interface. So there’s no need to code, making it intuitive, simple to use and modify and to provide comprehensible reports to colleagues. Alternatively, custom events and user attributes can be still set to Userpilot.

Userpilot also allows you to split test experiences between or within different segments.

Tracking the analytics on these tests gives you data-driven evidence on how to improve your KPIs, as we explained in the last section.

At the other end of the scale, it does not currently include powerful visualization tools for viewing actual user journey progress versus your mapped-out ideal journey.

Want to find out more about what you can do with Userpilot analytics? Talk to one of our Product Experts today and see it in action!

Product Analytics Tool #2: Heap

Source: heap.io

Heap is a standalone, comprehensive product analytics suite.

The single best thing about it is that once it’s installed in your app, Heap tracks everything all users do.

There’s no configuration or definition of “events” to track required. From day one, it’s all there.

This is incredibly helpful because some of the other tools on this list – Mixpanel, for example – need users to carry out extensive setup work to determine what they want to monitor. Sure, you can add new events as you go along in those tools – but data is only collected from the time you defined the event.

Heap also enables:

• User and account-level tracking
• Combining tracked data into your own custom “virtual events” for monitoring
• Analysis along multiple axes and behavioral segments

Source: heap.io

But it’s not cheap. The free package is fairly restricted in usage terms, and paid plans start from $1,000 per month.

Product Analytics Tool #3: FullStory

Like Heap, FullStory captures all user interactions with your app without configuration.

Source: fullstory.com

FullStory even tracks and measures “rage clicks” – when a frustrated user goes to town on their mouse. Who says analytics has to be dull!

Combined with the analytics is Fullstory’s primary feature: session recordings. When you can get these two features working in tandem, it’s incredibly effective for extracting insights.

The quantitative data you collect across multiple users can be cross-referenced with the qualitative experience of what real users actually did on-screen. This makes understanding why things happen in the order they do so much easier to comprehend than by data analysis alone.

Source: fullstory.com

FullStory also has AI and machine learning features that help you extract insights from your data.

What’s the catch? Well, pricing is on application only…

Product Analytics Tool #4: Mixpanel

Mixpanel is probably the most powerful product analytics tool on this list, but it’s also the most user-unfriendly.

Source: mixpanel.com

While Mixpanel has a vast array of capabilities for tracking and collating user data – particularly real-time data where it excels – it’s hard to implement.

• Any event you want to track has to be defined before it captured
• Setting up reports that make the most of data collected is tricky and time-consuming
• Analyzing the data you have collected requires expert skills

Strangely, one thing Mixpanel doesn’t support is account-level tracking (unless, of course, you can figure out how to add all users from one company into a custom segment).

If you’re a big company with in-house data and analytics expertise, or if you really need to dig deep into the minutiae of a long and complex user journey, Mixpanel could be the best option for you.

Source: mixpanel.com

Beginners should be wary of the cheap price tag (the Growth package is just $17 per month). This is a tool that offers the most to enterprise users.

Product Analytics Tool #5: Iteratively

Source: iterative.ly

Iteratively is doing something different from the other tools on this list.

It allows you to combine all your other sources of data in one place – into a “single source of truth”. As well as working with tools on this list, such as Mixpanel and Heap, Iteratively also integrates with Facebook Analytics, Google Analytics, Segment, Intercom and other apps beyond the scope of product analytics.

This is incredibly useful:

• For decluttering your data and trimming it down to a manageable, actionable level
• For ensuring that all teams are aligned on the KPIs and metrics they are looking at, no matter what tools they prefer

If you’re a data-literate business grappling with the sheer amount of information your analytics packages are providing, $120 per month spent on Iteratively could be a great investment.

How to Extract Insights from Your Product Analytics Tools

The biggest problem in product analytics is rarely – if ever – data capture.

It’s interpretation: the process of figuring out what the data is telling you to do in order to improve your metrics.

Whatever tool you choose, make sure you configure these dashboards:

✔ Cohort Analysis

Source: ChartMogul

Tracking the behavior of groups of users who started using your product at different times (”cohorts”) will show how changes you make to your product or to onboarding affects churn, retention and value achieved over time.

✔ Funnel Analysis

Source: Wikipedia

Plot out an ideal user journey in terms of in-app events, and track where users are dropping out because they are missing the necessary activities.

✔ Feature Adoption

Source: Userpilot

Which features drive the most user value? Are they being used and seen enough to give the best experience? By tracking feature use analytics, you can develop and provide a product that gives users better and better experiences.

Can Only Data Scientists and Experts Use Product Analytics?

Absolutely not.

They may well use it better than a beginner, but the range of easy-to-use tools available today means that anyone who is data literate, open-minded, and thinks in a scientific way can improve their product’s performance with just a little effort!

That’s all for now. We hope you found it useful and that you’re now well on your way to getting more out of your product analytics!