What can your data really tell you?

6 12 2007

Online testing is a bit different from other marketing data. It uses live traffic to find out what works. Analytics is the same, measuring what’s occurring at the moment. So why is that important? Well you can infer all you want from surveys, usability studies and demographics, but in the end you can’t argue against what real users are doing.

Avinash Kaushik, a popular analytics blogger, summed up the juiciest bits of a presentation by Jim Novo at eMetrics. In it, Jim asked, “What data yields insights that can be actioned the most?” The answer:

Data pyramid

“[A]ctionability, relevance of insights that can be actioned decreased as you go down the slide”

He makes the point that the farther away you get from the top of the pyramid, the harder it is to accurately predict your users actions. Yet often times too much value is put into the bottom levels of the pyramid. Even when marketers do test and measure actual behavior, they go about it the wrong way because they stick to all this other data too much and end up testing things that are all alike, defeating the purpose of testing.

Think about it, can you really tell if a red button will work better than a blue button if all you have are demographics? There are places for all of these types of data, but there should not be a fear of actual behavioral data. Yes we are using live traffic, yes the data is driven by technology (online visitors, javascript, cookies, rather than people filling out a survey), but those numbers tell a story unlike any other data.

Make the most out of all types of your data, but don’t die by one or the other. Use what’s best for every situation, but realize that you will never know you are right until you test it out.


Getting Cultured in Testing

5 12 2007


“I only need to run my analytics for one month because it’s not going to be different next month.”

I think saying that would get any online marketer fired. So why do I encounter people that assume they only need to test pages once?

This is beginner level testing methodology. Every online business needs to have a testing culture. What do I mean by this? Go back to my example of web analytics. Analytics is continuous because you need to know how your pages are performing. Testing is the same way, once you stop testing, you stop the flow of information about what visitors like and dislike about your pages.

A testing culture is where you continuously test your pages and realize that anything you do or change, should be a candidate for testing.

Got a new web design? Do a split test to see if it actually performs. Want to update your banner? Test it! Your copywriter make a few product descriptions to choose from? Try them all out and see which works.

Even after you’re done with all of that, try to beat it.

There are times when a page can be left untested, but be aware of anything that may merit another test for a page. Everything from seasonality to changes in PPC ads could have a huge impact on your web site and therefore your previous test results might not be optimal anymore.

If your company is holding you back from testing, do what we recommend our customers. Get your feet wet by trying one or two tests and after you see the results start continual testing. Not only will you be able to optimize your pages more efficiently, but you’ll open up yourself to a lot of deeper testing techniques… that’s for another day though.