Friday, July 1, 2011

Web Analytics – plan, do, check and... ACT!

One of the critical success factors for building a successful online analytics culture is having processes in place. But why is it so critical? Are all processes made equal? In this post I would to share some of my experience on processes and to talk about the approach I use. Maybe it will inspire some of you.

Building an online measurement culture is easy...
It usually starts with nothing in place until someone on the business side starts requesting data. This usually happens after the project has be launched (otherwise where would be the "fun"). Unfortunately, it is likely that nothing or very little is implemented.

At this point, a sound reaction should be to put in place some process to make sure measurements are correctly implemented. Because implementation is more a technical aspect, you may be lucky to have someone in IT helping you define such process. After all IT is very familiar with processes - they love processes (note: I started doing Web Analytics in IT :-)).

The process should ensure that whenever there is a new project, someone gets in charge of defining business objectives, translating these into measurement requirements and coordinating the implementation.The process would make sure that measurements are tested, rolled out and reports are set-up before being delivered to the business. No more projects are rolled out with proper measurements in place. Mission accomplished! Hurray!

Getting there may take you from few weeks to a year. I guess that most of us can live with that. So what’s the big deal about it?

...but building an analytics culture is another story.
The problem is that at this stage, you just have an online measurement process in place. What about the analysis and – most important – the action part? Aren’t these the steps for which we, web analysts, strive for?

The challenge is to make analysis and decision-making systematic – not something you do just once here and there (as it is too often the case). You need a process that includes these activities as well. Before you start inventing your own process from scratch, it worth having a look at some existing successful approaches used in other areas.

Some Web analytics experts refer to process excellence and Six Sigma for creating a data-driven culture. In my case, I looked into another approach that is part of the culture of my previous employer: the Plan Do Check Act (PDCA) management process.

PLAN-DO-CHECK-ACT (PDCA)
Originally created by DR. W. Edwards Demming, PDCA is “a framework that provides a methodical approach to problem solving and continuous improvement. It is an iterative cycle composed of four main phases:

1.PLAN: It is the preparation phase where you:
  • Grasp the situation i.e. identify your objectives (your ideal situation) & stakeholders, understand your current environment and conditions
  • Set your targets based on your objectives – ideally in a SMART ones
  • Define your “implementation” plan (tasks, timing, costs, roles...)
  • Communicate and share your plan with all involved parties and stakeholders

 2. DO: It is the implementation phase where you
  • Implement the plan
  • Monitor progress and adjust the plan if needed (environment or requirement change, issues...)
  • Communicate status and adjustments  with all involved stakeholders

    3. CHECK: It is the review phase where you
    • Evaluate the results against the initial objectives
    • Evaluate the process i.e. identify what went well and what needs to be improved
    • Communicate the results and share success but also failure factors to avoid making same mistakes
    • Get agreement on the next steps to be taken in the Act phase

    4. ACT: It is the action phase where you
    • Address identified issues by determining their causes and by applying countermeasures. This will be the start of a plan phase of a new cycle.
    • Standardize what is working well and that can be repeated
    • Look if you can improve existing process and standards
    • Communicate decisions, new standards and improvements to be made
    PDCA is shared by many manufacturing companies and especially by Toyota where it is one of the pillars of the “Toyota Way”. PDCA is how Toyota applies Kaizen principle within its organization in all areas – from manufacturing plants to dealerships.

    This process can be applied to any type of initiative – from small to large - and by any type of business function. So why not Web analytics?

    PDCA and Web Analytics
    Web analytics is supposed to be a continuous activity aiming at constantly bringing improvements to your online initiatives and business. Improvements do not have to be “big” ones that occur once in a Web analyst lifetime. Instead it is better to look for small enhancements that - put together - will increase the global results on the long run. That's the principle behind Kaizen applied to Web Analytics.

    The challenge is to implement such principle in practice, to make it something systematic and that can be repeated. I try to achieve this by adapting and applying PDCA in a Web analytics context. This approach has been mentioned in John Lowett’s recent white paper “Building a culture of measurements” (John has very good sources ;-)) but let me give you a bit more information on the approach I am personally using:

    PLAN:  too often we tend to measure too many metrics (the more the better) but not the right ones - mainly because of a lack of preparation. Therefore the plan phase is crucial and it involves the following activities:
    • Understand the business objectives of the online initiative and from there, define the corresponding KPI’s and metrics. For such work, you can use a methodology like the Nokia methodology or Avinash’s Web measurement framework (works very well!)
    • Identify the required data sources & tools (“multiplicity" rules) and ideally set/review targets for each KPI’s
    • Plan not only the measurement implementation but also the checkpoints – already set dates where results will be reviewed and analysed (very important!). Once you get consensus with all stakeholders, communicate your plan, KPI's and plan!

    DO:  In this phase, measurement requirements are analysed and implemented, data is collected, reports are set-up and delivered to the stakeholders. Other key activities in this phase are:
    • Supervise the progress according to the plan, coordinate the different activities between involved parties (business, IT, agencies, vendors...) and adjust plan if needed
    • Keep stakeholders updated on a regular basis.

    CHECK: It’s here that all your efforts will make you move from doing “reporting” to doing “analysis”. You will transform your fancy Excel dashboards (built in the DO phase) into business insights and you will also look at the project execution. So, you will
    • Analyse the results against the objectives & targets and then identify area of improvements and successes.
    • Evaluate the process – identify process issues and recommendations for improvements (i.e. if you have to do it again, what will you change?)
    • Communicate the results  - all the results, good and bad ones!

    ACT: All the previous work makes no sense if it does not lead to actions. You know what needs to be improved so let’s do it.
    • Identify potential root causes and countermeasures (i.e. the actions!!!!)
    • Define an action plan – agree on the issues you will address in the next cycle. You may not address all in one go so focus on issues with biggest impacts and on quick wins...
    • Improve your process where it can be improved, document success stories, lessons learned and share them within the organization
    • Communicate your action plan to the involved parties.
    Go! Take action!

    At the end of the act phase, you have all the input to start a new cycle - i.e. plan you actions, implement them, check them and again, take action...

    The Power of PDCA
    In my opinion the PDCA approach have several key strengths and there is no reason why Web Analytics could not benefit from them.
    1. It is very simple - It is very basic in its principle and it can be adapted to any kind of business and organization, whatever their size.Here's an example in the context of Paid Search.
    2. It is a very powerful approach that has made its proofs – just look at what Toyota achieved.
    3. It is continuous – it doesn’t stop at the end of the last phase but it is an on-going process
    4. It is iterative – you can start with a very basic process, with a simple scope and enrich it at your own pace, according to your maturity, resources... By beginning simply, it will make it more likely to be adopted – less reluctance to change.
    5. It is flexible - it can be applied to small, short projects or big complex ones. A cycle can last a week or be spread over a year. Depending on the complexity, a full cycle may be split in smaller cycles.


    Be on your guard!
    So that's the approach I am using to instill an online analytics culture - maybe it can work for you. Anyway, a good way to get started with your Web analytics process is to see if there is already a well-accepted process in place in your company. If yes, try to adapt it for Web analytics. Reusing an existing process will give you more chances to have it adopted as people will be familiar with the main concepts behind it.

    If there is nothing you can possibly reuse then look for inspiration in proven process, be it PDCA, Six-Sigma or others. But whatever you decide, don't forget that having a process is not a nice-to-have, it's a must.

    Finally, it takes time to have a new process adopted, to change people habits but it can go away very quickly. Bad habits are hard to kill. You will have to be the process guardian, pushing it until it becomes not a process anymore but a way of thinking. If you reach that point then you will have made a huge step towards creating a data-driven decision making culture.

    Sounds very theoretical? Well, in a next post I will illustrate the PDCA approach with some practical examples. In the meantime, I would be really curious to have your feedback on this approach or share your own. What do you think? What process / approach do you use? What are according to you the biggest hurdles in implementing an analytical process? Feel free to leave your comments.


    Related posts & resources

    4 comments:

    1. Great article Michael! I completely agree that the PDCA approach has application across industries and disciplines within specialties. I don't believe there's an area of digital marketing that couldn't benefit. Any activity that generates data that's captured can realize benefits.

      ReplyDelete
    2. Hi Eric. Thanks for taking the time to leave your comment. You are absolutely right.

      I can tell you that PDCA works fine for digital marketing in general as far I could see when at Toyota.

      While doing some more researches for this post, I found an example of PDCA applied to SEA - a post from end of 2008. It illustrate the principle applied to another digital area:
      Making Paid Search Profitable Using PDCA Process and Standardization

      (I will update the post and add it to related resources)

      Cheers,

      Michael

      ReplyDelete
    3. Hi Michael.

      I agree PDCA is a great process and it absolutely should be something that is simple and iterative and applied to analytics.

      The problem I see (not with PDCA, but with organisations), is that they quite often begin to implement this or a similar plan, but it only operates for one rotation and it does not become a fully cyclical process.

      It's more like Plan, Do, Check, Review, Act a Bit, Do Something Else - where the Review stage is intended to examine successes and failures but looks back rather than forwards and the Act a Bit stage is a point where impetus is lost. Then everyone does something else because this was seen as a "project" and not a "system" - leaving an analyst on their own with a leviathan.

      For analytics to become fully integrated within an organisation, PDCA or any other process is vital and it MUST be reviewed and repeated, not just tried once.

      It's not an easy thing to do if there isn't already a process culture in the company - but it's worth it in the end, not just for the sake of analytics, but for scalability within a company/department, for training, for trouble-shooting and so on.

      ReplyDelete
    4. @Emer: You are absolutely right. A lot of time is spent on the Plan, the DO and some CHECK. But when it comes to ACT... Ah!

      And I second your point about the fact often WA process is limited to one operation, seen as a "project" (this was especially true when I was on the IT side). That's where the challenge lies - to keep pushing until it gets in otherwise not matter you use one of the best process on Earth, it will simply fail.

      Even if it is part of the company culture, there is no guarantee for adoption if you are not behind it to make it happen, rotation after rotation, pushing hard. And I am talking about experience here :-)

      Process is indeed necessary for the different purposes you mentioned - for repeatability, for scalability, for consistency (everyone works the same way) but also for continuity (i.e. when people leave the boat). So sad sometimes to see how many companies just don't get it.

      Thanks for commenting.

      ReplyDelete

    Feedback Form