When we closed our last project, our team was so proud. The design was fabulous. The product owners were happy because they saw their plans coming alive. The ideas were tested and researched in a pleasant way. The last round of user tests showed the prototype was seamless and easy to use.
Yes, but… what happens next? When you launch, how will you know if the design is successful? How will you know if your users are engaged?
The answer is out there, and it lies in data. Metrics help turn data into digestible information, which can help drawing conclusions and making decisions. Defining the appropriate ones is not an easy task. In fact, it can be really tough. So, in order to define the right metrics for your app, let’s go through the following steps:
- What are metrics? What’s the difference between “metrics” and “UX metrics”?
- Why should you use UX metrics?
- When should you measure your design?
- How can you define the right metrics?
Who should define these metrics?
What are metrics?
Metrics are standards of measurement assessing a website’s/application’s efficiency, performance, progress, or quality. It provides much more insight into the real problems you’re trying to solve.
What can metrics do?
- They can tell you exactly what users are doing, but they can’t tell you why they’re doing it or how to make them stop.
- They can tell you whether your design is better or worse than another (through A/B testing).
Metrics & UX metrics
UX metrics are one type of metrics. They represent a product’s user experience, which is hard to quantify, but some useful frameworks can help.
Many big brands use UX metrics to improve the user experience of their products (Google, Airbnb, YouTube). One from Youtube which measures users’ engagement is “Average number of minutes spent watching videos/user/day”.
Why bother with UX metrics?
The most straightforward answer is better decision making. The key is to figure out what pieces will improve learning and decision making. Without an endless budget for design, it’s important to know where a product suffers the most.
When to measure a design?
- At the product launch. Before giving the app to users, prepare it to receive the data. It is the very best time to define a measurement plan with UX metrics. First-time users can give crucial data. Don’t let this important information slip from your hands.
- Before planned product improvement … A product on the market should already have metrics set up from the very beginning. If not, solve it with some planning. Before the next design sprint, define the UX metrics in advance and implement them in the app, then present user insights for the whole design team. It will be much easier to decide what needs redesigning.
- … And after the redesign. Before implementing any changes to your site, gather enough data going one or two months back. Measure and compare the designs to see if it generated the expected result.
How to define UX metrics?
If there is one thing to remember from this article, then it’s this:
You need a plan.
Know what to look for. Just wiring Google Analytics to your site is not enough. That can do more harm than good. Not knowing exactly what to looking for can get you lost in data. Even worse, you may come to the wrong conclusion, which will lead to the wrong decision.
Doesn’t sound good, does it?
So, before getting into the technical details, take a step back. Look at the big picture, and spend the time and resources to make a measurement plan.
There are a few frameworks which can help define the metrics to measure product user experience. Kerry Rodden, Hilary Hutchinson and Xin Fu from Google’s research team designed the one which we use at UX studio: the HEART. It’s extremely useful!
The HEART framework
1st step: Choose categories
There are five categories to brainstorm about from the user’s perspective:
- Happiness – Measures of attitudes, often collected via survey
- Engagement – Level of involvement
- Adoption – Gaining new users of a product or a feature
- Retention – Existing user return rate
- Task Success – Efficiency, effectiveness, and error rate
First, define just one or two things that are really important for the product. The fewer actionable metrics, the better. For example, let’s choose “Engagement” for the product YouTube for this exercise. After choosing a category, there is a three-step process to follow:
Goal > Signal > Metrics
2nd step: Define a goal
Start with the goals! In my experience, it can be really hard to start on an abstract level. In this step, define the “big picture”. What does engagement mean from a user’s perspective.
The goal of “engagement” for the product YouTube is “for users to enjoy the videos they watch and discover more videos.”
3rd step: Break the goal down to signals
Only after goals are defined can they be broken down to signal. How do we know we have reached the goal? Define signals which will answer this question.
The signal for “engagement” will be “the amount of time users will spend watching videos.”
4th step: Convert signal to metrics
Signals can be transformed into metrics that can be measured in the product.
The metric of the signal will be “average number of minutes spent watching videos/user/day.”
Following these steps (Goals > Signals > Metrics) ensures getting actionable metrics, information that will guide you in the right direction.
Once you have a plan, you can start the dirty work and apply your metrics to your product. There are tons of analytics tools which can help. The most popular ones include Google Analytics, Kissmetrics, Mixpanel, Crashlytics, Firebase, and Hotjar.
Who should define these metrics?
Defining the right metrics to measure your design is a team sport. The right result requires several different types of knowledge. You need people who understand…
- Design, the infrastructure of your product (UX/UI designer);
- Users, target audience and user behavior in general (UX Researcher);
- Business goals, the product itself, and its business goals (Product Owner);
- Technology, the technical side, who know its limitations (Developer);
- Metrics & Analytics, the possibilities of the tool to be used to set up the measurements. (Google Analytics expert).
How do you measure your product? Do you measure user experience? Have you used any other framework which did or did not work for you? We’d be more than happy to hear your opinion on this topic.