Web Analytics Tools: Where to Begin?

by Lynne on March 7, 2011

Choose the right tools.

Choose the right tools.

Frequently when user experience practitioners approach me wanting to learn more about web analytics, one of the first questions I am asked is “Which applications should I use?”. The answer here is very similar to the “Which metrics should I track?” question and one that UX folks should be very familiar with: It depends. Before you can begin to select the tools you want to jump in and learn it’s important to understand the different types of applications that make up the Web Analytics 2.0 tool set. Though there are roughly six or seven types of tools, the three that will be of the most practical use to UX practitioners and designers are Quantitative tools, Qualitative tools and Optimization tools.

Quantitative: The ‘What?’ Tools

Top picks: Google Analytics, Yahoo! Web Analytics, KISSmetrics, Omniture SiteCatalyst

Quantitative tools are the tools that tend to be used the most frequently and are likely the largest source of web analytics data you will have access to. These tools track and report on metrics such as pageviews, site visits, visitors, audience segments and the use of various website sections and pages. These applications are also commonly referred to as ‘Clickstream Analysis’ tools, as they enable you to view users’ paths through your site. They help you to understand where the traffic to your site or application is specifically coming from (Google Adwords campaigns? Facebook fan pages? Another of your corporate sites? Twitter? TV advertisements?), what people are doing once they get there and for how long (Checking out through the shopping cart? Downloading files? Searching for specific terms or keywords? Reading articles?) and where they tend to leave (What was the last specific page or link they touched?).

Quantitative tools are an excellent piece of diagnostic equipment. They enable you to not only see areas of concern and pain points within a site, but in many cases provide assistance in prioritizing these issues. They can also assist you with benchmarking your designs and measuring outcomes or results of your design work.

Qualitative: The ‘Why?’ Tools

Top picks: Survs, KISSinsights, 4Q by iPerceptions, Clicktale, UserTesting.com, Loop11.com

I can hear many of you saying “Wait! These are surveying tools! Usability testing tools! These aren’t web analytics tools!” but yes, they are in fact considered ‘web analytics tools’ by most web analysts. Many of you are actually already doing ‘web analytics’ without even meaning to!

Qualitative or ‘voice of customer’ tools help to provide the ‘yin’ to your quantitative data ‘yang’. While a quantitative tool such as Google Analytics may reveal that the second section of your sign up causes 77.3% users to leave without completing the process, a qualitative tool may provide deeper, more contextual insights as to why exactly this is occurring. Qualitative tools assist you with capturing and deciphering the ‘why’ aspects of user or customer behavior.

Optimization: The ‘Testing’ Tools

Top picks: Google Website Optimizer, Omniture Test & Target

While several of the tools we’ve classed as Qualitative can legitimately be called ‘testing’ tools, Optimization tools such as Google Website Optimizer actually enable you to perform much more intricate A/B or multivariate testing of your designs. What does this mean? ‘A/B testing’ refers to testing and comparing the performance of a version A of your design against a version B. This is the simplest form of experimenting or testing, and Optimization tools make it simple to both set up these tests and accurately monitor their outcomes. ‘Multivariate’ is just a fancy way of saying ‘more than two variables at once’: This is the testing of several versions or variations of your design against each other. Optimization tools manage this powerfully, enabling you to serve literally dozens or hundreds of variations of your design to some or all of your users. Optimization tools handle all of the background calculations and complexity and spit out accurate reporting on the designs or elements that performed best. They make iterating through various potential design elements fast, easy and relatively painless, provide you with clear data on outcomes and save you from having to worry endlessly over minor things such as the color or label on a button. With an Optimization tool in place you can experiment hands-free, enabling you to focus on other priorities.

How do I choose the right tool?

As every user experience project has its own process and is its own animal, you should select the tools that best fit the engagement. In some cases, you might be brought into an enterprise that has already implemented and is paying for Adobe (Omniture) suite products. In others there might not be any web analytics tools installed whatsoever. In the case of the former, educating yourself on the tools already in place can prove invaluable. In the latter, you want to make sure that you recommend installation of the appropriate tools and that you know how to use them. Budget certainly factors in, but what is interesting about the web analytics space is that some of the very best options (Google Analytics, Yahoo! Web Analytics, Google Website Optimizer) are actually free. You will likely need to utilize a mix of various types of tools as you progress through your research and design process: Quantitative tools to assist you with sussing out pain points and assigning priority to re-design or correction of them; Qualitative tools to show you what specifically within the  current designs are tripping up your users; Optimization tools to assist with testing your solutions and determining their ultimate value.

Over the coming few weeks we will be posting more in-depth reviews, tips and tricks for all of the applications listed above. Please let us know if there are any tools you love or would love to know more about!

{ 3 comments… read them below or add one }

Lucy Spence March 7, 2011 at 10:00 am

Great blog topic, and one that gets surprisingly little coverage. Looking forward to more posts in the future.

My one quibble is that in terms of top picks for optimisation tools you’ve gone to fairly extreme ends of the scale. Let’s face it, Test & Target is likely to be out of range for many people, especially for anyone who hasn’t already got Omniture implemented on their site, especially anyone who is getting started. One the other hand website optimizer is free but it also lacks some of the sophistication of other tools. How about a good mid range tool? Visual Website Optimizer or Optimizely are two that spring to mind….


Lynne March 7, 2011 at 4:17 pm

Thanks for your comment, Lucy! I definitely agree that GWO and Test & Target are two extreme ends of the spectrum. I often find clients either have no tools or really expensive ones already in place, so these are the tools I’ve ended up having the most experience with. I’ve looked at Optimizely but haven’t used it extensively yet–I will definitely take a look, and if you have any tips or feedback on how it’s worked for you I’d love to hear more about it.


Lucy Spence March 9, 2011 at 7:19 am

The best comparison of the two tools Optimizely/VWO I’ve found is a discussion by the founders:
It’s by far the most honest and open conversation on the merits of two products by two companies that I’ve seen. I’m investigating VWO at the moment and have been impressed so far.


Cancel reply

Leave a Comment

Previous post:

Next post: