Wednesday, June 23, 2010

A Journey Into Web Analytics (Part II): The challenges

Scream of terror - web analytics is so hard!
[After an introduction post on the business value of Web Analytics, the second part of this series covers some of the major challenges & obstacles that await you on the long journey to successful online analytics.]

“Web Analytic is hard”
Web Analytics holds attractive promises for businesses. After reading part I, it certainly sounds like online analytics is the magic silver-bullet that most businesses are dreaming for. Then why so many companies are failing in seizing such opportunities? A simple and common answer is that “Web Analytics is “hard, damn hard”!

Ok, let’s be honest with ourselves: most people would certainly say about their job that it is hard. But for sure leveraging Web Analytics value isn’t certainly easy – not as easy as many people think or as most vendors used to claim it. Creating an online analytics culture is quite challenging as there are many difficulties and obstacles to tackle on the way.  And these are not always lying where you think they are (ooh, the vicious ones!). Many companies fail because they underestimate the challenges behind successful Web Analytics.

I have a Web Analytics tool so…
Usually, the first challenge that comes to mind is the technology and the tools. I think too many people believe that Web analytics is just about deploying a great tool, getting online content tagged properly and that’s it. So how can that be so difficult? After all, nowadays most businesses have a Web analytics solution implemented on their websites. For example, last year I had a look at European automotive sites and only 4% had no recognized analytics solution. So does that prove that the other 96% are doing what is defined as “Web Analytics”?

Web analytics use by automotive websites in Europe, 2009 vs.2008
No. Having a Web analytics tool just proves that you are doing online measurements and possibly reporting but not that you are doing analysis (i.e. turning data into insights) or that you are taking actions. The purpose of Web analytics is about understanding and optimizing online usage!

Moreover, having one tool (or two) whether it is Google Analytics, WebTrends, Omniture or Analyser Nx is not enough as these are just measuring quantitative information know as the "What" (=what happened on your site) and the "When" (= when it happened). But what about the qualitative information - the “Why” (= why people came to your site) and the “How” (= how do they feel about it)?  Usually measuring qualitative information will require different and specific tools.

Finally, in many cases, the online channel is just a part of a (much) larger business picture. Online data can not sit alone on their side – they fit in a lager context (see further). Therefore to be really effective, online data has to be integrated with other data. That means integrating your Web analytics tool with other systems (such as CRM, other databases…). Oooh, system integration! That is where things usually get dirty. Now, we are talking about challenges! :-)

Too much data
Ok, let’s assume you have overcome the technical challenge. Now you are likely to face another typical challenge of Web analytics: having way too much data – more than any data geek can handle. The great thing with the Web is that it is probably the most measurable media we have so far but it is also a problem. Indeed, it is very easy to get overwhelmed by the huge quantity of data that can be collected.

The measurement possibilities are so enormous that it is tempting to succumb to let’s-measure-every-single-click frenzy and to turn any Web analytics tool into a data-report-puking machine.

The problem is that the insightful information is usually a very small portion of this mountain of data. But this is the one you need, the specific data that will bring real insights and that will make you take the right actions. So finding this tiny piece of information is often like finding a hairpin in a haystack.

For that you need the right people and that is the next challenge…

Lack of “staffing”
Another typical problem with Web Analytics is that many limit its scope to a tool and technology. Too often, the focus is on allocating budget for implementing tagging, configuring the tool and the thousands of reports that come with it. But when it comes to allocating people: nothing, nada, nicht, que dalle!

Ok, now is the right time to slip in some intelligent expression like “Owning the best hammer doesn’t make you a good carpenter”. Just replace the words “hammer” by “web analytics tools” and “carpenter” by “analyst”. Amen. Web analytics tools are just er… tools. They don’t analyse nor interpret the data, they don’t make recommendations, yet (but who knows, I am sure people like Joseph Carrabis have probably their own idea on that topic).

This is the job of (Web) analysts to put the data in context, to grab its complexity and turn it into business intelligence. And, this is not a job you can assign to first person you will find. It is not something that can be done one hour a day. Don't get me wrong here, I am not picking on Avinash’s great book, “Web Analytics an hour a day” (highly recommended reading by the way) – I just want to say that analytics is not a side task, it is a real new job that requires specific skills and experience (more about this in Part IV).

Finding qualified Web analyst is not easy

And even if you decide to assign dedicated persons, you will need to find these. And you will not find them at the corner of the street. Experienced analysts are highly demanded and come for a price. What about hiring junior or freshly graduated Web analysts? The problem is that online analytics is not much taught in universities or high schools – apart for few exceptions (but it is slowly changing). Online analytics is more something that you learn by yourself, on the job. It may get easier in the future thanks to great projects such as Web Analytics Without Borders and Analytics Exchange that offer the opportunities to beginners, students to develop their skills and acquire experience on real projects.

Siloed organizations& the reluctance to change
But then even if you have good tool and a good analyst, it will not be a guarantee for success (at least it is a good start). You will need to face “organizational” challenges. It is very typical (and even almost unavoidable) that big companies are organized in a vertical way, with what are commonly called silos. Each department focusing on its own area: Internet marketing takes care of the website, media department handles advertising, sales department is responsible for lead & sales management… All these departments contribute to the whole business process and it should be the same for Web Analytics. Its scope shouldn’t be limited to the website (or online channels) and to the few departments that are managing it. Online data are part of a more global context: business processes.

What is it the point to measure online lead conversions and increase these if it is not linked with sales? I mean, a Internet marketer, may be doing a great job at using the website to double the number leads but if these don’t turn into sales because of their  poor quality, it makes no sense. Online data can prove to be useful for many departments inside the company: product design, sales/production planning, brand strategy, marketing intelligence... The problem is that there is often a lack of awareness and sharing. It is very likely that most departments are not aware about the existence of online knowledge and about possibilities.

Therefore to really leverage the value of online analytics, one will have to “break” the silos. I don’t mean re-organizing the whole company - that would be impossible :-) - but making people communicate and work more together, change people habits, change processes, change the culture. And if there is something difficult in big organization, it is change. Finding the right organisation is a challenge on its own – as who should own Web analytics? Where should it sit in the organization? Vertical vs. horizontal? Centralized vs. distributed?

Patience & perseverance, you will need!

Beware the Hippos
HiPPo, the Highest Paid Person Opinion will give you hard time - can be worse than a real hippo
The people you work for can be a serious challenge as well. Internet marketing managers had a quiet life – as long they could persuade people around that their job was brilliant. The boss could impose any idea because he/she was the boss and there was nothing to contradict it. Web Analytics means a possible end to this “state of grace”. Web Analytics is often a painful reality check -  “bye bye” judgement based on gut feelings or influence and welcome to facts & figures! And the Web analyst is likely to be the bad news messenger. So the persons that are supposed to support you may do it to a certain extent only – as long as it serves their interest. Many people prefer to live in ignorance (I call it the “ignorance is bliss” syndrome), they won’t say it out loud of course but in practice…

Your findings may go against the HiPPO’s - highest paid person opinions. How will you handle that? How will you make people accept something that may show they are not doing such a great job? You can’t just come in and throw your facts & conclusions at your boss face while saying victoriously “Ah! Ah, see how wrong you are!”. When it reaches a certain level, political aspects get in the way and you will have to deal with these with extreme caution.

And there are more…
Challenges don’t stop there. The media itself make it challenging. The Web is a (super) fast evolving media. First, there is what some call the “decentralization” of the Web, induced by social media. Content is not centralized anymore in a limited number of sources (typically your sites) but it is disseminated across multiple types of sources and platforms. Companies now have RSS feeds, blogs, YouTube channels, Facebook brand page, Twitter account… Brand content can be shared by consumers or embedded in other sites. Each source and platform need to measured, usually in different ways, leading to multiple data sources that you will have to put together in order to grasp the your full ecosystem.

Web can be accessed via more and more platforms - each being measured in a specific way

Secondly, there is also the multiplication of technical platforms as well. Until not so long ago, it was easy as the majority of people used their computer to surf on the Web but now they can access Internet via their mobile phone, their tablets (like the iPad for example), their TV, their game console or even from their car systems. The same person will use different platforms at different moments for different usage, bringing more challenges in terms of measurements and data reconciliation. For example, measuring mobile sites does not work exactly the same way as measuring websites.

The consequence is that online analytics is constantly evolving (and fast), setting new challenges. No time to rest…

Don’t despair!
The challenges covered here are just examples. I could keep on enumerating more but this post is already quite long (you can find more for example in last year Econsultancy’s Online Measurement and Strategy Report or in this great post from Avinash) but I guess you get the idea. Web analytics is certainly not that easy, it is not just a matter of having one tool implemented that provides you with tons of sexy reports. Technology is just one of the difficulties with other aspects like organization, company culture, people, expertise and others. Like in many other areas, whenever significant changes are required, you are up for a long and difficult battle. Even the constantly changing nature of the media itself adds its own set of difficulties.

But don’t get desperate – successful web analytics exists (well, I truly hope so – I keep repeating that to myself :-)). All challenges can be overcome. How so? Find out in Part III – “Critical factors for successful Web Analytics”


Your turn now: based on your experience, what are the toughest, trickiest, most vicious challenges you encountered on your Web analytics journey? I am curious to know so please share your experience.

Interesting posts on Web Analytics challenges:


  1. Bravo Michael,

    I am a freshly graduated web analyst and whole heartedly agree with your analysis of the state of web analytics. Especially with the gut check part. I have been a big reality check for things that were supposedly "doing well" on the web, only for me to come in and swizel a few numbers and tell them otherwise. It takes guts for someone new to come in and tell a manager that something is wrong and needs to be addressed. I take pride in what I do and hope I am learning fast enough to keep up!

  2. Howdy and thanks for the nod.
    Your post comes at an interesting time for me; I've been asked to submit an event for an analytics conference and my working title is "Analytics Schmanalytics, If it doesn't get me from A to B then what am I paying for?"
    That noted, you make many excellent points that echo our research - the tools are only a part of the problem-solution matrix. An analyst can have the worst tools, determine valid and substantiable truth but that work is for naught unless management is willing to accept it (whatever it may be). Likewise, an analyst can have the best tools, determine completely erroneous and unsubstantiable results but if those results are accepted (dare I suggest "needed") by management, they're used.
    To your point of (online) analytics not being taught...hmm...we're witnessing a decrease in critical thinking, situational awareness, situational analysis and cognitive readiness trainings in higher ed. Definitely a discomfiting situation, this. Some elements develop and mature with experience and time, true, and a fundamental training in the basics definitely hastens the process. All of these skills are (I believe) required in good analysts, period, regardless of the application of their analytics focus.

    Again, thanks for the nod.

  3. @Kyle: Thanks for your feedback. Indeed, it can feel like an awkward situation for a newcomer to start questioning the manager work. It requires a lot of communication skills & diplomacy. As I said, one has to be cautious: of course you should never hide or distort the truth but it needs to be presented in the right way :-) Wish you all the best in your journey.

  4. @Joseph: Thank you for taking the time to read my post and more important to add some valuable feedback and comments. I really appreciate.

    Web Analytics is often reduced to a one or two facets problem while it is much more complex (like many problems in organization). There are multiple factors to consider - I plan to cover these in Part III.

    Regarding education - your point of view is very interesting - as mine is actually limited. I assume this based on what initiatives I have heard about like UBC courses or hearing people I know giving "lectures" or presentations at university courses. Of course, it still remains limited but it is a start I guess.

    The decrease of "critical thinking" you mentioned sounds a bit not-very-encouraging and it somehow echoes what Stephanel Hamel mentioned in his latest post.

    Regarding the skills you mention, can't agree more - it gave me some more food for thought for the last coming part. Thx.

    BTW - love your next conference title! :-)

  5. Hi Michael,

    Great post. I wonder if you had a chance to read cult of analytics? You can get a third of it free here;

    When I read your post it reminded me of why I wrote it in the first place. Your point about Hippo's are spot on. In my work I've found that without their buy in its impossible to build an effective culture so getting them on board is a critical part of the process - as with any change management program.

    In order to get their buy in I've found that speaking their language, showing them how you can help them achieve their goals and providing monetized wins works best. Get them to be your champion internally and the hardest part of your battle is over.

    The WAA is doing a lot around education and the programs you mentioned are positive. I will also be developing a knowledge test for the WAA in the coming months in which any WAA member can participate. More details at my blog.

    Anyhow great series of posts this.

  6. I would suggest give Clicktale a try. I’ve been using it for 2 months and it is neat to watch what your users do, I learned a lot.

  7. @Steve: Unfortunately I haven't read yet your book - but it is on my "must-read" list. Just need to find the time.

    I fully agree that top management buy-in is key (more in coming part) and you raise a very important point: communication language. Top management doesn't speak same language as we, analysts or marketers, do. One has to leave "conversion rate", "bounce rates" and all the usual terms at the door and use a different perspective. Monetization is probably the best way - an executive don't care you will increase conversion by 75% or that check-out abandon will be decreased by 50%. But if you tell them that it will increase revenues or decrease costs by xxxx $/EUR, it will certainly ring more bells :-)

    Thanks for your contribution.

  8. Great article. Have seen a lot of group think going on in online marketing operations led by the HIPPOS and they often just get frustrated with understanding the complexity of web analytics implementations. Also have experienced a lot of analytics structure and implementation errors (some made or allowed to be made by large consulting companies) that have massive data impact much later on.

    Global businesses sometimes struggle due to decentralised control of SEM and online which can cause big challenges for keeping web analytics consistent.

    The web analyst also needs to be a quasi programmer to understand stuff like regex, JS implications, tag management, redirects, mod_rewrites, testing, parameter retaining. So agree it is like a hybrid role. Throw in the world of apps and games and you add even more complexity.

    On the plus side there are some great innovations occurring in web analytics - inpage analytics, real time, social analytics, attribution, so it is getting more interesting.


Feedback Form