Multivariate testing: Let customer actions tell you what’s right

5 02 2008

This comic is both hilarious and insightful. Take a second and see if you’ve experienced something similar to this situation:

customerneedscomicthumb.jpg

I came across this before starting my work at Widemile, but stumbled upon it again while browsing Marianina’s Web Analytics Princess blog. Surprisingly, I found it extremely relevant to the multivariate testing I do at Widemile now.

The comic is a joke about how hard it is to do what customers want. Even when you have the luxury of a customer telling you everything they want, they don’t always do it correctly and no two people interpret their instructions the same. I like this comic because it shows how difficult it is to create good experiences and products for customers, no matter the amount of help you have.

Now, imagine how this applies to your website. Visitors want your product presented to them in a certain way with certain offers. Before testing, you’d listen to visitors with analytics, surveys and usability studies even. From here everyone would make their own “panel” of the comic; the creative team would have a go, the marketing team would revise it and the CEO would change a few words around. Eventually a page is made and your business puts it online, not knowing whether the page is what your visitors truly want.

Take that same situation and add multivariate testing. Now we get the same perspective that we have when looking at the full comic strip. In the previous situation, all you could see was your individual panel and your customers could only choose the one you gave them. With a multivariate test, different customers see different versions and the trends that show up in your data, reveal the “whole comic” and point you to which one should be the last panel… what the customer really needs.

If you look at the first panel and the last panel, it’s immediately obvious that they are different. Your customers can tell what they like, even if they can’t articulate it. Start listening to their true feelings by testing your pages.

For more of these comics, check out the official Project Cartoon site.

Advertisements