Sunday, November 16, 2008

Defining actionable & business-driven KPI’s – a practical methodology

Measure your online performances!Key performance Indicator (KPI) is probably one of the most over-abused terms in Web Analytics. How often metrics & graphs are put in a dashboard and labelled as Key performance indicators? Too often if you ask me. It is a very common mistake that many Web analysts do (and yes, I’ve done that too :-)).

When business users are not familiar with the online world, whether they put as many metrics, data & graphs as they can in a report that can fill a wall or they think that online success is measured just based on total number of visitors, time spent on site or total of page views. In the end, the results fail at delivering actionable insights or any clue about their level of success.

To measure your success you need business-driven key performance indicators (KPI’s). These are pre-requisite to move from “reporting” to “actionable analytics”. But what is actually a KPI?

Definition & properties of a good Key Performance Indicator
Get actionable insights with business-driven KPIsThere are plenty of definitions of what is a KPI – here’s the one I usually use (can’t remember from where I got it):

« KPIs are quantifiable measures that help decision makers define and measure progress toward business goals. KPI metrics translate complex measures into a simple indicator that allows decision makers to assess the current situation and act quickly. »

What does such definition tell us? To qualify as a KPI, a measure should:
  • Link to the business objectives – not just site centric. 
  • Be meaningful for the business stakeholders – it should connect to their business world.
  • Provide context – not just raw figures but relative figures measured & compared over time.
  • Be well defined & easy to understand – no ambiguous interpretation.
  • Create expectations - allow to set targets
  • Drive actions!
If your metric does not fulfil these conditions then it is not a KPI (see also very similar definition from Dennis Mortensen – Director of Data Insights @ Yahoo! ).

Keep in mind that KPI’s are metrics but not all metrics are KPI’s. It doesn’t mean that you will limit your measurement to KPI’s. You will need also to identify contributing metrics (i.e. metrics that will help you in understanding any variation of your KPI’s).

“Ok, I need business-driven KPIs but how can I get these from my business goals?”

Get a methodology!
The Big book of KPIs from Eric T. PetersonMany books on Web Analytics talk about the importance and the concept of business-driven KPI’s but most of the time these don’t provide a step-by-step practical approach. Some books provide list of KPI’s for main site categories – there is even a “Big Book of KPI’s” (from Eric T. Peterson). But I don’t believe that defining appropriate KPI’s is about making a selection in a pre-defined list - even if it is a long one.

Every business is different. Every site is different. And in many case, websites can fit in different categories. What you need are tailored KPI’s that will really suit your particular organizational goals – even complex ones.

So how to get these? Common sense and good analytical skills can do it but the best way is to have a practical methodology i.e. formal processes & framework to guide you and the business actors in defining your golden metrics.
 
The great news is that there is a good methodology out there that is documented & freely available thanks to Vincent Kermorgant, Senior Web analyst at Nokia. You can download the PDF here. Highly recommended reading!

Evaluating your online success - The Nokia way 
Nokia vision on Web AnalyticsNokia is part of these companies who seem to have developed a strong Web Analytics culture. They even have a very nice creative artwork to tell how they view customer analytics. Cool!

The methodology is quite simple, intuitive and it is applicable to a wide range of businesses. Based on my own experience, it works very well with different types of business, whatever is the audience level. 

The document covers the full process, from preparation (actors identification) to implementation and maintenance. In this post, I will focus on the KPI’s definition phase.

Identify Major goals
Major goals are the raison d’être of your website. Basically, it consists in answering one simple question: “why do you have a website?”.

The results should be simple business oriented sentences like “attract maximum relevant traffic”, “increase awareness about the company & what we do”, “increase knowledge about products”, “generate revenues through sales of services/products” or even “deliver optimal customer experience”. 

The major goals are typically defined by the business actors – they are the ones who know why they are spending the money. 

Identify micro objectives
From major goals to realizationsMajor goals are usually too vague to be translated in KPI’s as such. You need to drilldown into details and break down each major goal in smaller goals i.e. what it takes to fulfil the major goal. Basically, it consists in answering the following question: “What is the site expecting the users to do?”

For example, if the major goal is to attract maximum relevant traffic then possible micro goals can be:
- Get maximum traffic from Organic search
- Get maximum traffic from SEA
- Get maximum traffic from banner campaigns
- …

A major goal can have one or more micro goals. Here again, business actors should lead the definition phase with your support.

Define realizations
The third step is to define “realizations” i.e. what actions the users need to do on the site to fulfil a micro goal.

For example, if the micro goal is "Get maximum traffic from Organic search", possible realizations can be:
- User searches for a term related to your business and results is in top 5
- User clicks the results and visits the site

Group micro objectives & realization per main categories
Every site is (or should be) designed to fulfill 4 essential goals: acquire, engage, convert and retain. Take your list of micro goals (and related realizations) and assign one of the 4 essential goals.
The four essential goals of a website: acquire, engage, convert and retain
Examples:
- Get maximum traffic from organic search (= Acquisition)
- Get users to read product description (= Engagement)
- Get users to purchase products (= Conversion)
- Get users to come back (= Retention)

In the end, you should have KPI’s in all 4 areas (acquisition, engagement, conversion & retention). If it is not the case then it is very likely that some KPI’s are missing and corrective work is needed. 

Derive KPI’s
Once you have the realizations, it should be straightforward to turn it into the corresponding measurement i.e. KPI.

For the previous example, a possible KPI can be the “% of users coming from SEO who searched for a business related term”.

It is here that Web Analytics experience is the most valuable. Be creative, for each tentative KPI, list action points. If you can’t find actions that can influence the KPI then chances are that it is NOT a KPI (i.e. not actionable).

In the end, your KPI’s list will be probably composed of “industry standard” KPI’s (like the ones you can find in the Big Book of KPI’s) and your site-specific ones (the ones you can’t find in a book). From there you can aggregate KPI’s in composite KPI’s (per main category) to give a more high level view. Typically these KPI’s will be the ones that top management will want to see.

(Note that it is just the definition phase – at this stage you should not care yet about practical implementation. It is like any functional analysis – focus on what is required before thinking about the implementation. This is addressed after the definition phase.)

Conclusions
The Nokia methodology is very complete and practical – it even explains how you should organize your definition workshops, how to group KPI’s in dashboard and more. It comes also with many examples.

More important, it is not just another theoretical methodology. It really works. As a practioner, I manage to apply it in different cases with very good results. Not only we identified real KPI’s but on the business side, they feel involved and find themselves with measures that relates to their work.

You do not have necessarily to apply every single aspect as described. It is like a cooking recipe: first time, try to follow the book as much as possible and then according to the results, adapt it to your taste, skills and organization. Hey, you are the chef!  

All in all, it is a highly recommended reading. It may inspire you.

And you, do you have a methodology? How do you get actionable KPI’s? Any recommended reading you want to share?

Related sources:

Notes on the Nokia methodology: The methodology is the result of more than 5 years of work and it is already the 2nd version (released in 2008). The latest version was presented at this year eMetrics summits in Stockholm and Washington DC.

3 comments:

  1. Nice post Michael. I saw Vincent present this at eMetrics Stockholm last year.

    If you get a chance I would love to get your feedback on the KPI approach I use in the Advanced Web Metrics book - www.advanced-web-metrics.com

    The book is built around Google Analytics but the KPI chapter is tool agnostic.

    I start with OKRs (Objectives & Key Results). This is the business langauge of the organisation. Then I "translate" these into KPIs i.e. the analyst and marketers language.

    Essentially its very similar...

    Best regards, Brian

    ReplyDelete
  2. Brian,

    Thanks for the comment. I would be interest to have a look at your methodology. Any chance you can get me an electronic version of that specific chapter? :-)

    Most methodologies I have seen are based on same principle: start from business definitions and translate these to appropriate online metrics. They differ in how you get there.

    What I really like with the Nokia one, it is its simplicity while being quite structured and easy to apply to different contexts. It is also easily understandable by any business users even those who are not familiar at all with the Web world.

    Cheers,

    Michaël

    ReplyDelete
  3. Hello Michael - did I ever get back to you on this?

    Apologies if not, though I would like to touch base with you again

    ReplyDelete

Feedback Form