Friday, December 11, 2009

A Web Analytics challenge: the “Ignorance is bliss” syndrome!

The 3 monkeys feigning ignorance - like your stakeholders?
The biggest challenges with Web Analytics adoption are rarely the technology or the tools. Earlier this year, Econsultancy’s Online Measurement and Strategy Report listed top 11 barriers to effective online measurement. Technology was in last position. So where are the challenges hiding? They are in the organization, in the resources and…in the people & their entrenched mind-sets!

Challenge may even come also from where you would not expect it – from your business stakeholders – those for whom you directly work for! Are your audience prepared to face the truth? To become accountable? Or do they prefer to feign ignorance like the 3 monkeys(1)? After all what you don’t know can not hurt you, does it?

“Welcome to the real world!”
Imagine the following situation. An online marketer launches a new campaign microsite – a costly flashy one with HD videos, 3D CGI’s and spectacular design. The marketer is so proud of her/his creation, he/she shows it all around the place and especially to her/his boss. Everyone is impressed, the microsite even wins a web design award and the online marketer is more than happy – he is delighted! Now, how do you think she/he will react if you – the web analyst – come in with figures & facts that demonstrate that actually, the microsite is total disaster (from a business perspective)?

It is actually like parents: they are so proud of their babies who are the cutest & smartest in the world (here also I am talking from experience :-)). If you have friends who are parents and then you demonstrate them in a scientific & objective way that actually their kids are plain stupid and ugly, I am pretty sure they will not invite you for dinner anymore.

For business stakeholders, Web analytics is a (often painful) reality check in the beginning. It is saying “bye bye” to judgement based on gut feelings (welcome facts & figures!). Web Analytics is like taking the red pill in the Matrix movie – it shows them how deep the rabbit-hole goes.

Web Analytics - your online marketer red pill?
In such perspective, do the business stakeholders really want to know? The answer sounds obvious but do not underestimate this aspect. In the poem “Ode on a Distant Prospect of Eton College” (1742), Thomas Grays said “Where ignorance is bliss, ‘tis folly to be wise!”. From there came the proverb “ignorance is bliss” i.e. lack of knowledge results in happiness.

You, as the Web Analytics, you will bring knowledge and potentially put an end to some people happiness. Hoooo you evil!

Don’t shoot the messenger!
In old uncivilized times, it was a kind of custom to kill messengers with bad news. I am really happy that such times are over as otherwise I would been killed... a lot! :-)

As a Web analyst, you need to be prepared to bring bad news – especially when Web Analytics is new to your organisation. People gut feelings about their sites are often wrong. So be careful when bringing your bad news in the beginning. If you don’t, there is a risk that your stakeholders will start fleeing away when seeing you – like if you’ve got the Mexican flu.

This reality check is a necessity – no way to avoid it and it will often hurt in early stages. But people have to accept bad news – they have to know! Otherwise how the hell can they improve?

Communications skills are key!
Communications skills are often mentioned as an important trait of good Web analyst. I fully agree. Good communications skills are needed, not just for turning complex data into actionable insights but also in knowing how to bring bad news in a good way (so you don’t get killed).

Be careful how you announce bad newsYou can’t just come in and throw the truth in your stakeholder face (and start laughing in an evil manner). Be positive & constructive - don’t come with your hands empty, show your values, emphasizes the positive aspects and most of all bring recommendations for improvements (look especially for quick wins, small but easy to implement). Show there is hope!

A personal experience example
Let me illustrate this by a personal example. Few years ago, one of our marketing departments was all ecstatic about a brand new Flash content, full of 3D CGI’s & animations (in a time where these were not so common). It looked really top-class! But being curious (another Web analyst trait :-)), I decided to analyse the data. The results were not what everyone expected. The report probably ended its short life in the shredder or got locked forever in a drawer (and the key got thrown in a lake)).

But together with the report, there were a series of recommendations - some were really basic ones. Some of these were applied on the similar project that followed. Guess what? Results really were much better. But we did not stop there, and in a true Kaizen way, we looked again at other smaller improvements to make the next one even better (and we did :-)). All in a sudden, the reporting became important and the marketing people started to ask for more because it helped them to demonstrate their progress and good performances.

The final words
As a Web analyst, you may be seen as the "bad" guy by many of your colleagues so be prepared to sometimes face resistance from where you would not necessarily expect it. Do not under estimate this “psychological” aspect. You will need to put your best communications skills to good use, turning yourself in a “good news” messenger. The best way to do this is to show your added-value by bringing actionable insights – not just figures. That is what makes you a good Web analyst!

Your turn now. Did you ever face such situations? How do your stakeholders react when you bring them bad news? What is your best strategy in such cases? Share your experience!

Related posts & resources:

(1) Note that in Japan, the concept of the 3 wise monkeys refers to being of good mind, speech & action. It is in the western world, the phrase is often used to refer to those who deal with impropriety by looking the other way, refusing to acknowledge it, or feigning ignorance (thanks Wikipedia). In my post, I used the western interpretation of course.

Tuesday, November 17, 2009

Automotive & Web Analytics in Europe - one year later

Update: Following comments & feedback received, the research results have been updated (25th of November). It mainly concerns Sophus3 eMetrics that managed to increase its share - as Peugeot and Citroen are actually using it as they core Web Analytics solution for their main markets. The chart and PDF document have been adapted accordingly.

Last year, I did my own little market research on the European automotive industry and Web Analytics usage, using the excellent tool, Web Analytics Solution Profiler (WASP). Being curious - I decided to do it again and see if there was any major change (revolution?) or specific trend in this industry. Last year 1st place was held by Sophus 3 eMetrics, a industry-specific solution. Is it still the case? Does the crisis made manufacturers to massively switch to the free Google Analytics (or Yahoo Web Analytics)? Are paid vendors left with just their eyes for crying? The answer is just below...

Research background
The context is (almost) the same as last year: the research covered the 5 major markets in Europe i.e. UK, Germany, France, Italy & Spain. It includes 32 brands (last year I forgot Skoda -shame on me!) present on the European soil including premium and luxury ones. That represents in total 135 websites. Note that some brands only have one global website (or local sites under same main .com domain name).

As said, I used WASP to detect "core" Web Analytics solution used on the websites - I didn't look at ad tracking, voice-of-customer nor behavioral targeting solutions. It may be possible that some manufacturers are using other tracking solutions, either own-made or tweaked ones that are not recognized by WASP.

Finally, I regrouped in the total counts solutions from same providers i.e. Omniture includes both Omniture SiteCatalyst and Omniture HBX, Google Analytics includes Google Analytis & Google Urchin...

The full results for 2009 research can be downloaded using the button below:

Download research detailed results
And the winner is...

Web Analytics usage by Automotive manufacturers in Europe 2009
  • Google Analytics took pole position, being present on almost half of the sites (45.9%). Its share progressed by more than +100%! But is it really a surprise? In many case, Google Analytics coexists with a major paid solutions. It is probably used to track specifically search engine traffic and paid search performances as it fully integrates with Google Adword. Yet, Google Analytics is also used main primary solution in 23% of all sites.
  • More surprising is the result for Sophus 3 eMetrics, the industry specific solution, that managed to strengthen its position & presence (30.7% of all sites, +10% vs. 2008). Not only it is resisting against traditional Web Analytics vendors and but it managed to get new customers (Peugeot & Citroen, all major markets) in spite of the crisis. (Updated on 25th of November)
  • Omniture (16.3% of all sites) took the last place on the podium away from WebTrends (14.1%), leading traditional paid vendors (Note: for more details about Omniture result vs. last year - see post comments). In most case it is used as a global Web Analytics solution (i.e. on all local sites) by big players like Audi, Volskwagen or Mercedes-Benz.
  • More and more manufacturers are using more than one solution on their site. The average is 1.25 solution per site vs. 1.1 last year.
  • Some paid vendors lost customers: WebTrends (Ferrari, BMW Germany(!?), Volvo (!?)) and Nedstat (Nissan replaced it by Omniture Site Catalyst).
  • The number of sites with no (known) Web Analytics solution found decreased - only 5 sites with no solution found (vs. 14 last year).
  • The paradox: luxury brands (Lotus, Ferrari, Maserati, Porsche) seems to not want to spend money on Web Analytics as all adopted Google Analytics.
  • The missing ones: Yahoo Web Analytics, Coremetrics & Unica were not found - same situation as last year.
Google Analytics penetration is quite impressive but not really a surprise. Still, it is not yet really replacing paid vendors but for how long? Especially knowing that Google Analytics is closing the gap with paid vendors. Just look at the latest features that were announced last month. On top of that, traditional paid Web Analytics vendors are battling against an industry specific solution, Sophus3 eMetrics that is dominating the paid market and that even increase its presence. (Updated on 25th of November)

However, automotive manufacturers are still licking their wounds from the crisis and may reconsider paying for services that they may get for a much lower cost. Future will tell...

As said last year, this research does not tell anything about the expertise and usage of Web Analytics by the different manufacturers. After all, "a fool with a tool remains a fool" :-).

So what do you think? What comment would you add? Please share your thoughts with the rest of us.

Note: if you spot any error (yes, this can happen :-)), please let me know and I will update the figures accordingly.

Related posts & resources:
Note on Sophus3:
Sophus3 provides two types of online measurement products to automotive manufacturers. the first one, eMetrics is full Web Analytics solution used to measure & track websites activity like traditional Web Analytics solutions. The second one, eDataXchange is a collaborative benchmarking project for the automotive industry. A majority of manufacturers are participating to the eDataXchange project that uses the same tagging technology as eMetrics - both based on TouchClarity script (now Omniture Test & target). However for the benchmarking project, only a subset of key pages are tagged. for more information, see Sophus3 website.

Wednesday, October 28, 2009

Google Trends for Websites - do you trust it?

Google Trends for Website - competitive intelligence for freeGoogle Trends for Websites is one of the free competitive intelligence services provided by Google. It is pretty easy and straightforward to use: just type in domain names of sites you want to compare and it gives you traffic trends over time (for the selected period) and some additional data. Unlike Compete free service, Google Trends for Websites offers worldwide coverage with the ability to segment results per country. Cool, isn't it?
Compare your site traffic vs. competitors
Simple but valuable
While simple, it is a very interesting service as you can compare traffic trends of your site vs. your competitors. You can learn a lot like the impact of competitor advertising campaigns (were they successful?), product launches, announcements... You also get the key search terms used by visitors and the sites they also visited (and find out who are your biggest rivals). Super-cool isn't it?

See what sites your visitors also visited
(Note: for competitive search terms analysis, I strongly recommend Google Insights for search, another cool free Google service. See the practical example I did earlier this year)

For me the main limitation is the fact that data are only available for sites with traffic above a certain threshold (Google gives no details on this limit). Practically, it means that it is only usable for big sites and big countries.

Still, even if the provided information is limited, it can be very valuable and, hell, it is FREE (and ones knows that it is important nowadays :-)).

For more examples or details on Google Trends for Websites, read Avisnash's excellent post on that topic.

"But are Google data reliable?"
This is the question that usually comes when I show Google Trends for Websites. For quite some times, Alexa is offering similar service but it has received a lot of critics regarding its accuracy and reliability. Alexa collects data mainly from the Alexa toolbar - not the most used toolbar to be honest. The consequence is that the sample is rather limited (from a size & geographic perspective) and therefore it is quite biased in my opinion.

For Google it is quite different. Google has access to a huuuuuuuge amount of information from a variety of sources, such as aggregated Google search data, aggregated anonymous Google toolbar /Google Analytics data, consumer panel data, and other third-party market research. As Google claims in Google Trends for Websites help section - the data is aggregated over millions of users. So that is how Google can get information on your website traffic - even if you don't measure your site activity with Google Analytics.

Ok sounds good but still is it reliable? To answer the question in a simple way, I decided to do a simple test: compare our own data against Google Trends for websites data. I took the traffic trends for 2009, for 2 markets with high traffic volume, no Google Analytics on it! I put everything in Excel, I adapted the scale to get as close as possible to the scale of the Google tool and this is what I get (without any data tweaking, I swear!):
Own data vs. Google Trends data - test 1wn data vs. Google Trends data - test 2
I think the result speaks for itself (a picture is always better than long explanations :-)). Spooky, isn't it?

Maybe I was lucky with my 2 examples but for me it is a simple way to show that Google data is certainly not bad quality and that somehow I can trust it. Therefore I usually propose it as a data source for benchmarking in appropriate cases - taking into accounts its limitations.

I would be really curious to know if anyone did similar comparisons and what was the outcome. Did it work out for you or Google Trends for Websites results were miles away from your own data? What do you think about this Google service? Like it or not? Please share your experience.

Cheers!

(A little side note: surprisingly Google Trends for Websites can give you trends for Yahoo.com or Bing.com but not for... Google.com (see here). Doh!)

Related posts & resources:

Tuesday, October 20, 2009

My little Web Analytics bookshelf

What a nice bookshelf
When it comes to advices on how to get started in Web analytics and or how to expand your Web Analytics skills, nothing beats practical experience. However theory is also important. Blogs from Web analytics expert are a good sources of inspiration and knowledge (check my little blogroll on the right side for few ones). And Web Analytics books. Yep, books that talks about Web Analytics.

Not so long ago, Web Analytics books were a very rare specie but now these are almost like rabbits. There are plenty of them and the number just keeps growing. Just in the course of this year, many famous Web analytics experts published their own book. Like if you don’t have written a book, you can not qualify as being a true Web Analytics expert (Shame on you then! :-))

Web Analytics books are really useful. Serious. Not just to look smart or to impress your boss. These are a highly valuable source of ideas, best-practices & knowledge, whether you are a beginner or an experimented practitioner. No need to read them all, one can be more than enough. And it can change the course of your career. Actually it did it for me.

So let me present you the few ones I read and few others I plan to read.

The ones I read
  • Web site measurement hacks by Eric T. Peterson“Web site Measurement Hacks” by Eric T. Peterson: This is not a new one but it was the first one I read – almost 4 years ago (gosh, time flies!). This book changed my professional life. Simple as that. It made me evolve from a reporting squirrel to analysis ninja rookie. I will never be grateful enough to the nice people who offered it to me (hi Aurélie, René & Siegert :-)). Personal memories aside, It may be bit old by now but I still think it is a very great book, well written by one of the most influential and experienced Web Analytics expert. Most of it is still relevant even after all those years. As its title says, it focuses more on the “measurements” part but it is where Web analytics starts, no? No one can pretend doing good WA without the right measurements. It is very practical – plenty of ideas that can be applied right away. And if you are a “code” geek – you will learn how to program your own Web tracking tool.
  • the Big Book of KPIs by Eric T. PetersonBig book of KPI’s” by Eric T. Peterson: Bought it just after reading the previous book. It is not a book that is meant for reading chapter by chapter but more to be used as a reference, a source of inspiration for finding KPI’s based on the type of site you are working on. Even if I believe that best KPI’s are not found in a book but the ones that you defined yourself (using a good methodology like the Nokia’s methodology) – there are few standard and universal KPI’s that you will find in this book. Still worth to check it out. (Remarks: it is only available in electronic format. There is also a French version (for free) thanks to Julien Coquet and other contributors who made a great job translating it.
  • Web Analytics an hour a day by Avinash Kaushik“Web Analytics an hour a day” by Avinash Kaushik: Do I really need to present it? If you don’t know Avinash Kaushik then start reading his blog, right now! His book is a must-buy for anyone having interest in Web Analytics whether you are a beginner, an experimented Web analyst, an online marketer, a marketing manager involved in the Web, an HiPPO... It covers lot of topics from the basics to more advanced ones (segmentation, processes, organization, framework…). It is written in Avinash’s unique style that makes it easy to read, funny and entertaining. You won’t get bored. For me it is constant source of inspiration as I keep diving in it regularly. I can’t wait for receiving my copy of Avinash’s 2nd book (see further). On top of that, you do a good action by buying this book as all proceeds goes to the Smile Train and Médecins Sans Frontières.
  • Actionable Web Analytics by Jason Burby and Shane Atchinson“Actionable Web Analytics: Using Data to make Smart Business Decisions” by Jason Burby & Shane Atchison: Another book I got offered (yes, I know such nice people :-)) and a pleasant surprise it was. In my opinion this book is less for beginners but more for those who want to go further in Web Analytics as it addresses more advanced topics like creating a “performance-driven" culture, KPI’s definition, organization (setting-up a team, selecting partners…) and more. But the most important, it provide methods to take “smart” decisions. How? By using a monetization model to prioritize your action. The idea behind the monetization process is really great and it makes this book a very recommended read.
The one I will read (as soon as it is delivered)
  • Web Analytics 2.0 by Avinash Kaushik"Web Analytics 2.0 – the art of online accountability & science of customer centricity” by Avinash Kaushik: Let’s just called it “Web Analytics 2.0” :-). Avinash Kaushik just released the follow-up of his first great book. Knowing the quality of his first book and his blog, one can imagine that the 2nd book will as good as the first one – if not better. According to the author, the book covers topic like the web analytics 2.0 model (I am a big fan of it!), multi channel analytics, social meda measurement, multichannel campaign attribution analysis, mobile analytics, and so much more. Can’t wait to get my copy (too bad I ordered it on Amazon UK and it won’t be available before early November. Aaaargh!)
The ones I want to read later
There are two books that are currently on my “books I wish I will find time to read them as I heard they are very interesting” list.
  • Multichannel Marketing Metrics by Akin Arikan“Multichannel Marketing – Metrics & methods for On & Offline Success” by Akin Arikan: For me Web analytics is only one piece of the puzzle and Web data should be integrated with other data (i.e. offline) – especially when you are working for a company where Web is just a channel or an activity among others. This book covers aspect like developing better marketing programs, optimizing online-offline advertising programs, selecting the right metrics for your specific needs, collecting measurements and reporting on customer behavior across channels & more. Why would I like to read this book? Because I heard a lot of good feedback on it and Arin’s blog is also one of the blogs I enjoy reading.
  • Cult of Analytics by Steve Jackson“Cult of analytics – driving online marketing strategies using Web Analytics” by Steve Jackson: For once, a book that doesn’t come from the US but from an European expert! This book is a detailed guide on how to build an analytics driven culture in your business or organization (well that's what the cover says :-)). The book is said to be very practical, full of examples and “tools” like scorecards, dashboards and more. Learning how to “do” Web Analytics is a first step. Making it a organization practice is one of the toughest challenges that comes next – and I am talking about experience here. So that is why this book on my wish-list.
The one that you should read if you speak French
  • Web+Analytics = Profits by Nicolas Malo and Jacques Warren"Web + Analytics = Profits" by Nicolas Malo & Jacques Warren: the first brand new complete guide on Web Analytics in French. It is set to become the reference book in the French Web Analytics community. Like Avinash's first book, it targets a large audience: whether you are an analyst, a manager, new to the field or experienced. The book explores step by step key concepts that anyone need to master to leverage the real value of Web Analytics. The two authors are really great and talented people (Congrats to both of you!). I will certainly add this book on my bookshelf (for once, it will be a change to read a book on Web Analytics in French).
So that's it for my personal bookshelf (and its future evolution). These are just a sample, there are plenty other great books out there, recent or older that worth reading.

I would be curious to know what would you recommend as a must read for a Web Analytics specialist? If you had to choose just one book – which one would it be?

All suggestions are welcome. I have a bookshelf to fill in. :)


Friday, September 25, 2009

Dear WebTrends, I need some REST!

REST logo
Early August 2009, it was quite difficult to not hear about WebTrends Analytics 9 launch. The new release generated a lot of buzz in the Web Analytics community – mainly about the new “revolutionary” user interface that “reinvented” Web Analytics (Cough! Cough!). Well, I had the opportunity to get my hands on the new baby and to test it. To be honest, what excited me the most was not the new UI (which is good I must admit) but the integration of REST URL’s, the latest WebTrends data extraction API.

What are REST URL’s?
REST acronym stands for Representational State Transfer – in case you want to know. It relates to a style of software architecture for distributed hypermedia systems such as the World Wide Web (thank you Wikipedia!). But let’s put aside the technical aspects. What is important is what you can do with REST URL’s: extract data from the Web analytics system in a easy and structured way!

WebTrends REST URLs generator
A REST URL will return a data set whether in XML, HTML or JSON format. One can not only retrieve report data but also profiles list, reports list or even report properties. WebTrends people are so kind that they even provide a REST URL’s generator that helps you to generate the URL’s you need.


The URL can be used in whatever application that support Web requests. First one that comes in mind is of course Excel - the most widely used spread-sheet application in the world. Just imagine that you can build self-refreshing dashboard & tables in a very easy way. You can find an example on WebTrends Developer Network site.

With a bit of programming expertise, you can use REST URL’s to feed other data systems. Or you can integrate data in a website like an Intranet site and display key web statistics to the whole company for example. Or develop a Flash desktop dashboard that retrieves & refreshes data automatically, giving direct access to key infos to your stakeholders in one double-click.

Well, you get the idea, REST URL’s offer a lot of possibilities to better communicate Web data!

Free the data! Communicate them!
Get data out into whatever format you want
With REST URL’s, Webtrends finally opened this 'closed box' that contains all your precious Web data. Analytics product is really good at what it is designed for: convert Web data logs into (plenty of) reports. But Webtrends Analytics like any other Web Analytics tools is NOT a communication tool. Period.

Data needs to be communicated to the people who need or use them in order to turn these into “intelligence”, “knowledge” or “insights” (pick your favourite one).

To quote Dennis R. Mortensen, “providing people a login and password to the analytics interface, is in most cases more harmful than good”. I can’t agree more with that! No one can expect business users, managers or executives to log in the tool, to start learning how to use it, to find their way in the enormous pile of reports, to figure out how to extract the data they need... And in the end, what they get is just data, still have to transform these into insights.

Web activities are often only a part of the whole picture. Being able to get your web data out of the 'closed box' allows you to integrate them into your reports/systems with other data sources.

Data must be freed from the analytics system so you can communicate them in most appropriate way according to your stakeholders. For that, openness is key. WebTrends REST URL’s are maybe not revolutionary (other tools also have very good API’s to extract data – just look at Google Analytics & Excellent analytics plug-in for Excel) but WebTrends certainly made a huge step in the right direction towards data openness. Finally!

I want it! I want it! Pleaaase!
Please, I am begging you!
Unfortunately for me, my company is a software customer, so no REST for us as it is currently only for On-demand customers. Boohoohoo (me crying)!

Such flexible API is terribly lacking in current WebTrends Software offering. Yes, the software version has the ODBC driver, which is good and useful. But it is showing limitations as it is not meant for retrieve huge amount of data. So when we generate differents (automated) dashboards on a monthly basis for more than 30 countries and two brands, I can tell that our server suffers!

Moreover, implementing good ODBC dashboards require some expert programming skills. In our case, our former Web Analytics agency had such expertise and did a very good job (thanks to you Guillaume! :-)). On top of that, the ODBC driver needs to be installed & configured on every client computer, format is not real SQL... All in all ODBC lacks the flexibility that REST URL’s offer.

So I am dying for the day when Analytics 9 will be released in Software version with, I hope, REST URL’s support. Please Mr. WebTrends make sure it will be there. It is so so important. Otherwise it would be a huge disappointment.

And don’t make us, software customers, wait to long, pleaaaase!

Related resources & references

Friday, September 11, 2009

Usability labs: a way to really understand your online customers!

Hi, I am a Web user. Called me stupidImagine the scene. You have what you think is a very simple. But when you look at the poor results you get from your favourite Web Analytics, it leaves you scratching your head and wondering: “How can our online customers failed to use it properly? Are our website users stupid?”

“I am a web user, call me stupid!”
Well, there are probably few of your users who may be stupid like in everywhere – but no, online users (i.e your customers) are not stupid! It is just that too often, we get it wrong!

Yes us, you know the super Web experts (web designers, online marketers, Web Analytics ninjas).Too often, we forgot one important fact: we are NOT our customers. We are not our Web users. The problem comes from the fact that our perspective is biased. We know all about our websites, their functionalities & tools. We designed these, we know their purpose, and how these are supposed to be used.

Our customers don’t. They have a radically different perspective and it is a very difficult exercise to put yourself in their shoes. For me, we should never forget a golden rule when trying to interpret the results of any online page or tool: "Never assume that other people surf they way we do"

“If users are not stupid then data must be wrong”
Let me share a personal experience. A while ago, we started a new major redesign of our website. In that context, we did some measurements to better understand how users navigated from our homepage to the product sections. One way was via a product overview page – that was supposed to help our customers choose a specific model and drive them to corresponding model section.

Marketing didn’t expect the results we got. The page was not performing well. The results were so surprising that first reaction was to question the data (“yeah – let’s blame the data”). But data were proven to be correct. So how was that possible? After all, the page was darn simple to use. At least that what we though. The answer came few months later…

Usability labs to the rescue!
Users under close observation during a usability labAs the new site design was in advanced stage, a usability lab was done to test the new layout. The great thing with usability labs is that not only users are filmed but they are also interviewed: they explain what they think about the page, how they understand it and what they expect from the page to do... It provides qualitative insights (hurray!).

The overview page was part of the content that remained unchanged – still it was tested and we finally got our answer. We actually saw it with our own eyes: users were definitely struggling with the page. While they entered the page with the right intent (i.e. get an overview of the full product range), they were puzzled by the way it was presented and how it behaved. They basically expected something else, they got confused and mostly frustrated in the end.
And what was a mystery for us suddenly became so obvious. Yes, they were absolutely right (customers are always right anyway :-))! How did we manage to not figure it before? Because we are not our customers.

Qualitative is a must-have because can it bring some lights on the numbers and the “why” that is missing. Without, one can spend weeks trying to figure out the reasons (and never find these).

D.I.Y. Usability labs
Usability labs are one way to get qualitative data. Sure, usability labs have some flaws and limitations but the great thing is that you can really see what people do with your site and understand what's in their mind.

“But a usability labs is hard to set-up, requires an expert company and damned expensive” you might say. It is true that usability labs are often used for major redesign or critical projects. These are often handled by expert companies that work for a price. Not something most of us can afford too often.

However for smaller projects, you don’t especially need the “big artillery”. So why not doing it yourself? It is not that hard to set-up your own usability lab. One laptop with a video camera & recording software, a moderator (ask your beloved web agency for help), a quiet place to do the tests and some incentives for the testers (like free Belgian chocolates or…beers! ;-)) and that’s it. It shouldn't cost you more than 1,000 or 2,000 EUR max. Of course, it may not have the rigorous & scientific approach that you get from an expert company nor the big powerpoint/100-page report but what you can learn can be very valuable.
The 5 things you need to run your own usability lab
Small details can turn a project into a failure. Doing light usability labs can help you uncover these and avoid painful results. Again, I am talking about experience here. For a recent project, we did a small usability lab with the help of our Web agency, all at a low cost. The feedback was not only very interesting but more important it helped us to discover a serious usability issue caused by a little detail. Project was a bit delayed in order to fix the issue but without this finding, we would have put live a project that would certainly have failed to achieve its goal.

Get qualitative insights!
All in all, the important points of this post are:
  • You are not your customer - never assume they surf the way you do.
  • Get a better understanding of your customer: complement your data with qualitative insights
  • Small usability labs can be done for a cheap price and still get you valuable insights that can make a real difference, from failure to success.
  • For bigger projects/major redesign, look for real professionals. It has a cost but often worth it.
And you, have you ever done usability labs? What do you think about these? Like or dislike? Don not hesitate to share your own experience and views on that topic.

Related resources:

Thursday, August 20, 2009

Happy birthday Kaizen-Analytics!

What a cute 1 year birthday cake, isn't it?
One year ago, I started this blog – going one level further in my Web Analytics passion (or “geekiness” if you prefer :-)). 37 posts later, it is time to blow the first candle. The first one of a long serie? Well, I hope so.

It is personally an “emotional” milestone for me. I can not hide that this experience has been really thrilling. Yes, really. Running this blog is very exciting & lot of fun (told you, I am a Web Analytics geek - no way I can hide it). But most important, it is rich in learning.

But it would not have been possible without you – yes you, behind your screen (or PDA, smartphone or whatever device). You, who spent time reading my posts, leaving comments or to forwarding/retweeting posts. You who added me to your blogroll or sent me encouraging emails. Maybe it is a bit cliché but I sincerely want to thank you all for your contribution – whether passive or active. This experience would make no sense without you.

And thank also to the 300+ (and steadily growing) faithful subscribers.

A quick look back…
This blog allowed me to share some of my personal experience, bringing in a practioner perspective that I think is sometimes missing in this world of Web Analytics gurus, experts and consultants.

It gave me the opportunity to extend my skills and expertise by testing and discovering things that I couldn’t not learn directly through my work (e.g. Google Analytics, 4Q Surveys, Twitter….). And that I could use for my work afterward. And how one can learn about blogs and social media without using these? After all the social media revolution is on the way. Better get ready!

Bit on the downside, I found out difficult to get “conversation” started – it took time before people really start commenting or interacting. And there were few ideas of post that couldn’t make it. Especially few ones relating more to Kaizen principles and processes. But the adventure is not over, so these will be for the coming year. Promised!

What’s next?
Well, I will do my best to keep up (or even improve) with current level of quality – I have still many ideas on my list – and with frequency. I have to admit that it has been a bit quiet over the two last months but I took time to enjoy the most important thing in my life – family. And you all know - children grow so fast… (Still, I am already preparing the future – see pic below ;-))
It is never too early to learn Web Analytics - especially with a book like that one

I will continue to share my experience, opinions & ideas from a practioner point of view but if you have any suggestion or wish, please let me know. These would more than welcome.

Again, thank you all for reading my blog.

Cheers!
Now let's have a piece of that cake!


And special thanks to my wife for supporting me in this initiative.

Monday, July 27, 2009

Web analytics– where should it sit in the organization: in business or in IT?

Where do you think that Web Analytics should sit in your organization?
In this post, I would like to talk about organizing Web Analytics in a company and the important question : who should own and lead Web Analytics competencies? where should these competencies sit in the organization? And what if Web Analytics all started from the IT side?

As a practitioner and based on my personal experience so far, my answer would be: not in IT.

But why do I believe that IT is not the best option for managing and leading Web Analytics competencies within an organization?

Web Analytics relates to business
Basically, Web Analytics is about improving business performances of online channels. This is done by understanding what are the business objectives, by defining appropriate “business-driven” KPI’s, by analyzing and interpreting the figures and taking necessary actions & business decisions.

But do not get me wrong, IT is key player as it is needed for all technical aspects such as hosting and maintenance of the Web Analytics infrastructure (if you are hosting your solution – not the most common set-up), definition of the technical/tagging specification based on business requirements, implementation of the measurement code in your CMS, websites and online applications. And if you integrate your Web data with other systems (BI, CRM...), IT is the one that defines and implements the required architecture & technology.

But in the end this is the business that drives the needs and that uses the data to gain insights. That is the business who takes decisions. That’s the business that normally defines the analytical strategy & long term vision (as it relates to what business will do).

Can IT have “business” mindset”?
Web Analytics requires a strong business mindsetAn important trait of a good Web analyst (or expert) is the “business acumen”:
“Good web analysts know the business and the current business climate. They understand the business purpose and can identify and focus on the business problems and disciplines apart from analytics. They have both a great sense of the data and the business.”
Basically a good Web analytics expert needs to have a strong “business mindset”.

Having “business mindset”, is it something that IT can have? I mean genuinely. I don’t want to be harsh on IT departments but let’s face it, in many cases, IT department mission is to implement technologies, tools, systems, to develop & deliver applications, to make sure everything is up & running and all.

I am talking about personal experience here as I am working on the IT side (no one is perfect :-)). I started as business analyst and I took over the requirements analysis, the reports creation and the coordination of measurements implementation when my company insourced Web Analytics. From there I stepped into the world of Web Analytics and I started developing competencies & expertise while working closely with business.

Explaining to IT why having Web Analytics competencies sometimes feels like this Dilbert cartoonAs we moved from Web reporting to Web Analytics, business demand grew – requiring to make Web Analytics competencies more structured and organized. But doing this in IT is far from being easy. You need to spend a lot more time in order to explain the purpose and necessity of Web Analytics (and why having a dedicated centre of expertise). Already a challenging task! And if you succeed, the next question you may get: “a large part of the competencies are purely business so why should we offer such expertise in IT?”.

Centralization is key!
The main advantage of IT is that it usually has a central view of the needs from the various departments – especially in large organizations where business departments are often structured in silos. I believe that Web Analytics expertise & competencies should be centralized in order that standardized solutions & technology areused by the different departments and to share best practices (all will reduce costs in a significant way while improving efficiency).

Web Analytics should be lead centrally by a team that supports the different business departments (who need and use the data) and that makes the link with the technical side (i.e. IT, vendors & external agencies). A team that would assist & coach business key users in how to better use Web Analytics. I quite like the “Hub’n’Spokes” model proposed by Eric T. Peterson. This is something similar that I wish to develop.

The Hub and Spokes organization model from Web Analytics Demystified

But building such central team in IT is difficult as it mixes business expertise and practices with technological expertise. In most of the examples I have seen from other large companies, Web Analytics competencies are regrouped in a dedicated team that sit on...the business side.

So it is no surprise that I feel like sitting on the wrong side. “So why don’t you move to the other side then?” you may ask. Well, it is easier said than done.

Nevertheless, I am doing my best to improve organization in order to better serve the company growing needs. But changing organization and mindset in a large company is quite a challenge and takes time. Lot of time.

So what is your experience? How is Web Analytics organized in your company? Centralized or decentralized? Do you know examples where IT ownership of analytics does work? I am very curious to hear your experience and it would be more than welcome.

Other resources:


Saturday, June 20, 2009

Successful Web Analytics: what does it take?

The road to successful analytics - the key values you need
Is Web Analytics hard? Well, I do not want to resurrect an old debate here but I must say that if Web Analytics may be not that hard, successful web analytics is.

We all know that Web Analytics is not just about having a tool (whether it is a free or a paid solution) and the right tags & key reports. You may have heard the famous Avinash Kaushik’s "90/10" rule: “For every 10$ you invest in your WA tool, you should invest 90$ in people". Or Eric Peterson’s "70/20/10" rule: “Web Analytics is 70% people, 20% processes & 10% tool”.

So people & process are essential for Web Analytics but is that all you need to be successful in your Web Analytics quest? No. I think you need some key “values” on top of these.

Key values you need to be successful
Recently, Jim Sterne - the Web Analytics Godfather - made a very good analogy when commenting the recent report "Online measurement and Strategy Report" from eConsultancy, comparing Web analytics to building a house:

"Web analytics reports are just lumber. It takes an architect, a designer, a builder and a lot of other skills to turn it into a house.”

On my side, I will use another analogy, comparing Web Analytics with a sport I like a lot – Formula One. Yes. Formula One! To win a race or even better to win the championship, a team (your company) needs more than having a very good car (the tool) and a very good driver (you). It needs other key ingredients to cross the finish line in first position.

The follow promo video from Toyota F1 summarizes well these key values.

  • Determination: Formula One is a very demanding discipline. So is Web analytics. In both cases, it takes a lot of patience and perseverance to get to the top. It takes a lot of time & efforts to make progress, to get all things correctly implemented, to get quality data, to get valuable insights, to make people act upon data, to establish a data-driven culture! It means also doing (and accepting) errors here and there. It means sometimes frustration, lot of frustration. You will need to get over it. Commitment all along the way and at every level is essential. Rome wasn't build in a day!
  • Continuous improvements (Kaizen): Every little detail matters in Formula 1. You need to constantly look for any small improvement that would translate in a slightly fastest lap time on the track – even if it is by 0.01 of a second. Because all improvements together make you faster and get you closer to success. Victory is not achieved overnight. Same for Web Analytics. Try to improve your processes, your work, your website step by step. Instead of looking for THE change that would rocket your results (and that you may never find), look for more modest, minor improvements that will be easier to achieve and that globally will lead you toward your goals. And if you do errors, impacts will be limited and you will learn from these. All in all, Kaizen approach is more likely to succeed than the big-leap approach.
  • Challenge: Formula One is constant challenge. It is a fast changing world with new technologies, new rules... and tough competition. It makes teams push their limits and skills further. It is kind of the same in our industry where Internet technologies, usage and businesses are evolving very fast - meaning new tools and new ways to measure these. New challenges! For example, look at social media or mobile. That is what I like in Web Analytics - the fact that it is a continuous challenge. If you manage to solve a problem, you can be sure new ones will come. There is (almost) no routine. If you don't like that then move along.
  • Teamwork: Victory in Formula 1 is the work of a whole team: designers, engineers, mechanicals, drivers, management team and more. A F1 driver alone can not make a team win. Same for Web Analytics – a Web analyst alone is not enough. It is a joint effort between business stakeholders (to define strategy and actions), IT team to host / support the tools (if hosted internally), developers (to implement tags), agencies (to advise & implement changes)… And you the Web Analytics expert as the key link between all these.
Formula One is TEAMWORK
  • Passion: this one is not in the promo video but I think that Web Analytics & Formula One share that one too: passion. I had the chance to meet many talented Web analytics experts and practioners – famous or not. All have one thing in common: passion for what they do. I think this is an important ingredient for success. Why? Because passion is contagious. Passion is a positive (winning?) attitude. So do not keep it for you – communicate your passion around. Get people on board! Get support!
So don’t think Web Analytics is one-man job – make sure your get the right people around you, working as a team and committed towards same final goal: success! Don’t be hasty and persevere. Be prepared! It is a long road to reach success. But once you get there, you will never forget the taste of victory.

The taste of victory! (Sebastian Vettel 1st victory, Monza 2008, Italy)

Related resources:

Friday, June 5, 2009

Toyota Europe launches the Toyota Range Experience!

I usually try not to do (too much) propaganda for the company where I work but this time – I can not refrain myself. I am so excited by this new project that went live today on the Toyota European site and that will be rolled out to Toyota national websites across Europe in the coming weeks.

Welcome to the Toyota Range Experience
Today we launched a new “tool” – the Toyota Range Experience, a rich full screen environment that allows you to explore Toyota European model line-up.

Explore Toyota European model line-up with Toyota Range Experience
The “tool” was developed by our European agency – Amaze – and the Toyota Europe Web developers team. The main goal the tool is to increase Toyota model line-up & product knowledge while providing an emotional experience to the user in a "passive & relaxing" way.

Why am I so excited about this?
There are several reasons why I am so thrilled about this projec. First from an experience point of view. It is emotional, it is dynamic and stylish. It creates a special atmosphere. It will (we hope) certainly gives a more modern image of our brand. We really hope our customers and website visitors will enjoy it.

Full screen video & high quality CGI will allow you to see Toyota models like you have never seen before!

Secondly, from a technical point of view (this is the engineer part of me :-)): the content is full screen flash video featuring high quality computer generated images (CGI). Lot of efforts were put to deliver best possible performances on most broadband connections in order to not spoil the fun. A though challenge.

Finally, from a Web Analytics point of view. This project is the starting point of new Web Analytics project that will integrate quantitative and qualitative data. It will allow us to get all the insights we need to measure if we are successful but also to improve & deliver the best online experience to our customers. Future will tell.

In the meantime, please have a look and I hope you will enjoy the…experience!
(and please let us know what you think if you get prompted :-))



Thursday, June 4, 2009

Web Analytics, Dutch women and Dennis' vision of tomorrows analytics

Web Analytics Congres 2009
Last Thursday, I spoke at the Web Analytics Congres in Amsterdam – presenting Toyota case together with WebTrends – one of the key sponsors of the event. In my part, I explained how we leverage Web Analytics in our pan-European context to drive actions and to continuously improve our business performances in true Kaizen way.

Me presenting Toyota case @ Web Analytics Congres 2009
I was really pleased and honoured to present at such event. And from what I heard or read in some posts (for example see "26 learning van Web Analytics Congres 2009") and tweets, it seems that the audience was listening and had some interest. If sharing my experience has given some ideas to anyone then my main objective is achieved!

Dutch women rule Web Analytics!
Wehkamp.nl presentationWhat impressed me was not only the size of the audience – around 200 attendees – but also the quality of the cases & presentations I saw (even if I did not see all of them). Web Analytics practices seem to be quite mature and well developed in the Netherlands. Web analytics is recognized and has its place in the organization of many companies like Bol or Essent that see the importance of Web Analytics. Others are regularly using A/B testing to successfully improve their website performances (from a business point of view, of course) like Wehkamp or ING.

But what surprised me the most was that in the morning sessions, five of the seven speakers were women! Who said Web Analytics was a male thing? :-)

Future of Web analytics will be on recommendation & automation
This year International Keynote was Dennis R. Mortensen, former COO of Indextools and now Director of Data Insights at Yahoo! It is always a pleasure to listen to Dennis – he’s a very good speaker with a very direct style. And he has always a few funny stories to tell. His presentation was about “Tomorrows Web Analytics Technology and Usage”.

Dennis developed thoughts expressed in a recent blog post. Today, there is still a lot of work to do to reach excellence regarding 3 main areas:

  • Data collection: Simple in theory but often a huge and complex challenge in real life.
  • Data reporting: Looking at some metrics from a WA interface is not reporting – "it is entertainment”. Data has to be reported and communicated in the most appropriate format according to your audience (and I agree on that – said the same thing during my case).
  • Getting insights from the data: Most of us still fail to get the insights we are looking for. This is, according to Dennis, one of the most difficult steps to achieve. One that requires to have a real data-driven culture
Dennis Mortensen sharing is vision of tomorrows Web analyticsBut how does Dennis see the future of Web Analytics tools – from a vendor perspective?

The problem today – Dennis pointed out – is that some companies are collecting too much data. It takes them too much time get recommendations & actions out of it. It prevents these companies to take fast & reliable actions. Being slow in today fierce world can sometimes mean to get killed by the competition. Therefore Dennis believes that focus will be on:

  • Recommendation: Based on all best practices that exist in the Web Analytics industry, it will be possible to use data modelling so tools will be able to not only report on data but to make recommendations. Recommendations on what you should do next in order to optimize your paid search campaigns, your landing pages, your online forms…
  • Automation: If tools can make recommendations, why not let them do the actions (through some advance API’s for example)? Automation would also include testing and taking appropriate actions based on the results. For me this sounds like “learning” or “intelligent” systems (it reminds me AI classes at university :-)).
My practioner perspective
Systems doing recommendations?! Well the idea is not crazy at all. Just have a look at what NextStage Analytics is preparing. This is where they are going. And it will not be in 5 years but in few months!

But, as a practioner, I must admit that I am a bit sceptical about the last point: automating actions. The barrier will not be the technology – I am sure they are plenty of genius people out there that will make that possible earlier than one may think. The barrier will be cultural. Today it is already difficult to have organizations to trust online data and to take even simple actions based on these. So I wonder how can one expect to see organizations to trust a system that would make changes to their sites? I think this is where the biggest challenge will lie. And it will take more time to go over it than to develop the required technology.

But that’s just my humble practioner opinion. What is your opinion on Dennis’ vision? I am curious to hear.

Finally, as speaker, I also received a small gift from the event organisers: a book called “Dutch Delight”. A book about Dutch cooking recipes of course – not a Red Light District guide. :-)
And a very intriguing book I must say.

I woud like to thank again WebTrends for offering me the opportunity to present and the Web Analytics Congres organizers for having me as a “guest” speaker.



Related resources:

Wednesday, May 27, 2009

Competitive intelligence for free: a practical example

Competitive intelligence is an important element of any ideal analytical framework. While measuring and benchmarking your online performances is a must-do, your website is part of an ecosystem. It is not just about you and the customers but you, the customers and competitors! So it is important to understand how you are performing against the competition.

Competitive intelligence is also usually one of the most difficult or expensive data to obtain. For a long time, it has been mainly limited to the Northern American market or to specific industries. Until Google introduced services like Google Trends, Google Trends for websites or Google Insights for Search. Basic competitive intelligence for all and for free!

Avinash Kaushik wrote a very good serie of posts on these services - definitely recommended reading (see related resources at the end of this post). Here is a practical example I made - inspired by a conversation I had with a friend about the Toyota Prius and the Honda Insight.

Toyota Prius vs. Honda Insight
Toyota Prius vs. Honda InsightNo need to present the Toyota Prius, the "most-famous-best-selling" hybrid car in the world. The Prius has contributed a lot to Toyota brand image and popular success in a context where the "green" factor has become critical for a majority of car buyers(1). In 2009, Toyota launched the 3rd generation Prius. The Toyota Prius had no real contender until recently when Honda launched a new hybrid model, the Honda Insight - a Prius look-alike for few thousands Euro's/dollars less. The fight has begun! (here's where you insert the "Rocky" movie theme :-)) Will the Honda Insight put an end to Prius supremacy? What is the interest in the new Honda? Does it equal or surpass interest in the Toyota hybrid?

Understanding customer interest
Interest can be measured based on related search volume. And who is best placed to provide you with insight about search? Our good ol' friend Google of course and in particular Google Insights for Search. It is really a great tool as you can easily get actionable data out of it and more important - some competitive intelligence. "Yes, you can!"

Google Insight for Search - comparing productsFor my example, I will take the French market (I working in Europe and I am French speaking :-)) and I will compare following terms "toyota prius" and "honda insight". Let's start to look at the last 12 months. Here is what Google Insight for Search returned:

First, let's have a look at the interest trends and see what we can learn from these:

Look at peaks during key events or following product launch
  • Trends clearly shows the impact of key motor shows on customer interest: Paris Motor show in October (with the reveal of the Honda Insight), Detroit in January (with the reveal of Prius 3rd generation and official launch of the Insight) and Geneva in March. Clear peaks are visible at these times and one can see how the event influenced search activity.
  • The graph also shows that interest in the Honda model increased over time to get closer to the level of interest in the Toyota Prius. Good for Honda. Not that good for Toyota.
  • The other thing this graph can show us is the impact of advertising or media campaigns: in April, the Honda Insight curve jumped as major advertising campaign started in France for the launch of the new hybrid car. It even surpassed temporarily interest in the Toyota Prius before settling down.
So you see how you can use interest trends to compare your products against the competition but also to see the impacts of external factors like advertising campaigns, press & media coverage or events.

Now, you can drilldown to look into more details when exactly peaks occured by using smaller time range (last month, last 90 days...). For example, we can see a rising peak in May for the Prius - so let's look at last month trends.
Drilldown to see exactly when peaks happenedSee that peak that started around the 18th-19th? Is it a coincidence that it happened after the official launch of the new Prius in Japan and after the coverage of the news in the French media (like this article)?

You get the idea and you see how it can be used to get insights from the interest trends. Want more? Want "actionable" insights? Good because there is more you can get from Google Insights for Search.

Finding out customer "carewords"
I will not spend time on the regional interest - up to you to see if it can be of any value for your business - and I will jump to the last section of the page: related terms & rising terms.

Related & rising search terms - Toyota Prius - France - last 12 monthsThis section is fantastic as you can see most popular related terms used by your customers - not the one you may think they should use :-) The rising terms give you indications about "hot" terms and trends.

(Tip: Only the top 10 terms are displayed on the screen but you can get more if you download all figures and terms in CSV format. All you need is a Google account and to log in)

Looking at a wide time range (e.g. last 12 months) gives you a snapshot of the main trends. In the above results, we can see that new Prius is part of the rising terms with terms like "Prius 2009", "Prius 3" and variants. Note also the presence of the "Honda Insight" in the rising terms. The presence of competitors in the list is an important information.

The great thing is that Google Insight for Search allows you to see the same list for the other terms. For "Honda Insight", this is what we get:

Related & rising search terms - Honda Insight - France - last 12 monthsHere we can see that people where also looking for prices and reviews ("essai" in French) on the new Honda hybrid. And here again, "Prius" related terms are among top related & rising terms - showing how these two models are competing each against the other.

What you should do with this information? Well, first thing is to check your search engine ranking for these "customer" words and then use these as input for SEO and/or SEA (if you can not achieve a good organic ranking). Adapt your content and adword copytext so they include these terms! You can also check competitors ranking for their own terms to see if they do a better / worse job than you do and possibly use this info to convince your boss about the necessity of improving organic ranking and paid search investments.

I also advise here again to drilldown to smaller time period (last 3 months, last month) to detect specific recent hot trends for search terms. However the inconvenient is that you may not get enough search volume and the result list may be quite short if not empty.

Conclusion
Free competitive tools are often underutilized while they can sometimes provide insightful information, depending on context. It is not because they are free that it means you can not do anything valuable with it. The problem is that most the time, people ignore that such competitive information exists and is available.

The main inconvenient of Google Insights for search is that it is not always usable. It will provide information only if there are significant search volumes. Also the level of details will vary according to the market/region you look at.

But still it worth the try - you never know what you could learn or discover. And it is free so what are you waiting for?

What do you think? Did I make sense? And you, how do you use such tool? Or others you may suggest?

Related resources:
Feedback Form