Two types of tests (and I’m not talking MV or Split)

There are a lot of things to know to be a pro at testing, so I’ll cover some of the basics every week. Hopefully I’ll have a nice inventory of posts that can be used as a resource for you or anyone else to quickly learn the testing language and methods.

A good starting point to learning about testing methodology is to wrap your head around the idea of template and in-place tests.

A template test is a test using a new template in competition with the original template and is best done as a split test. A template test basically means you have all the same content but in different positions on the page. This is an example template:

If this is my original template and I am doing a template test, I would switch the items around, but have the same exact content (hero shot, headline, price, etc.) to make a new template like this:

These two templates would compete in a split test. A note though, even if I only move one thing, say I bring the Call-to-Action Button above the First Party Validation, it is still considered a template test.

However, if all I am doing is swapping out the content in those positions then it is an in-place test. Usually a multivariate test is an in-place test, trying out new headlines, images and text within the positions of the original content.

Typically you want to do a template test first to find the best positioning and then an in-place test to find the best messaging.

Continue on for more in depth examples

Test to success

Jupiter Research released a new report, Compelling Benefits of Multivariate Web Site Testing But Adoption Remains Low. I guess the title says it all but here are some charts I put together to summarize:

Note that this includes A/B testing, meaning only a fraction of marketers are doing multivariate testing.

 

 

The large majority of early adopters have received gains from testing, but only 68% of testing users have raised conversions. That number should be much higher!

But I guess if it was, I wouldn’t have as much to write about.

Why test?

I love web designers and creative people. They create beautiful things out of words and ideas we give them, yet still have to deal with people judging every little thing they put out into the world. Working on company websites is especially an ordeal. Everyone has an opinion, from CEO to marketer, and too often there is a struggle of who to listen to, but really who knows what will work?

When creative runs up against marketing, a gray area falls on who knows best. Do you go by experience, seniority or even… degree? In the worst case, the CEO is calling all the shots and everyone else is just nodding in agreement or can’t get the CEO to listen to their advice.

So why test? At its core, testing is about finding out what works best, without any of the politics. If I put up two versions of a page and one gets more sales than another, its hard to argue with that. My job ends up proving people right or wrong, judged by what real visitors are telling us.

Test to:

• Know the impact of redesigns (quantitative)
• Figure out what your customers are and are not looking for (qualitative)
• Find the truth (awesome!)
  

As a marketer, our goals are to speak to our audience and get them interested in our business. With testing, marketers have a way to directly listen to what visitors want, to find out their language. So start testing to start listening.

Make sure you come back for more, I will be writing more on reading messaging through testing. This is just the beginning!

Getting started

Being a newcomer to the small but growing online testing industry has put me in a unique position. Not only do I get to learn about completely fresh and exciting marketing technology, but also I have an opportunity to build upon this nascent industry. While I have only just started, already I am way ahead of most online marketers in terms of understanding online content optimization.

Recently, A/B, split and multivariate testing have become hot topics in online marketing, yet they are still far from commonplace. With that comes a lack of resources or even knowledge about testing, although the demand to know more is quickly rising. As I have found out, and hope to share with this blog, there is a lot more to testing than just trying new things out. Much like the many businesses that have been burned by bad PPC campaigns, there are businesses that have had bad experiences testing. However, I know for a fact that good testing methodology and technology drives results. I hope that I can help other marketers out there learn more about optimization and develop a culture of testing and optimization in their companies.

Don’t just change it, test it.