Multivariate testing: Let customer actions tell you what’s right

5 02 2008

This comic is both hilarious and insightful. Take a second and see if you’ve experienced something similar to this situation:

customerneedscomicthumb.jpg

I came across this before starting my work at Widemile, but stumbled upon it again while browsing Marianina’s Web Analytics Princess blog. Surprisingly, I found it extremely relevant to the multivariate testing I do at Widemile now.

The comic is a joke about how hard it is to do what customers want. Even when you have the luxury of a customer telling you everything they want, they don’t always do it correctly and no two people interpret their instructions the same. I like this comic because it shows how difficult it is to create good experiences and products for customers, no matter the amount of help you have.

Now, imagine how this applies to your website. Visitors want your product presented to them in a certain way with certain offers. Before testing, you’d listen to visitors with analytics, surveys and usability studies even. From here everyone would make their own “panel” of the comic; the creative team would have a go, the marketing team would revise it and the CEO would change a few words around. Eventually a page is made and your business puts it online, not knowing whether the page is what your visitors truly want.

Take that same situation and add multivariate testing. Now we get the same perspective that we have when looking at the full comic strip. In the previous situation, all you could see was your individual panel and your customers could only choose the one you gave them. With a multivariate test, different customers see different versions and the trends that show up in your data, reveal the “whole comic” and point you to which one should be the last panel… what the customer really needs.

If you look at the first panel and the last panel, it’s immediately obvious that they are different. Your customers can tell what they like, even if they can’t articulate it. Start listening to their true feelings by testing your pages.

For more of these comics, check out the official Project Cartoon site.


Actions

Information

One response

9 02 2008
Chris in Seattle

The tire swing cartoon has been around for many years. I’ve probably seen twenty different versions of it, but the message is the same: each step in the process of converting needs into requirements into product is subject to misunderstanding, and that’s true in any industry not just software.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s




%d bloggers like this: