Get Certified in Landing Page Optimization

30 01 2008

After 4 months, I finally received certification via the *breath in* Marketing Experiments Certification Course on Landing Page Optimization – Subscription Path Track.

Marketing Experiments Logo

If you already follow Marketing Experiments, much of the material they put out for free is discussed in the class (although in greater depth.) Flint McClaughlin, who runs Marketing Experiments, knows testing and optimization very well, but the class could be stronger. Taking the class, training at Widemile and working with clients simultaneously has taught me a lot, very quickly, so as the class went on, I wasn’t learning as much. Those of you who don’t have the benefit of being surrounded by testing pros, probably will get a lot more out of it.

In addition, sometimes the number of conversions for their case studies are quite low, which leads me to question some of the testing numbers. I’m sure they got lifts, but their numbers are a little outrageous at times and, as my boss Frans brings up, seem to not account for seasonality.

Despite all that, I’d definitely recommend it to anyone who wants to get into or needs to learn how to make better landing pages. They offer many other certification classes and while I can’t really say how good the other classes are, I have a strong feeling that they are worthwhile too.

Regardless if you do or don’t want to take the class, you should take the time to learn their conversion index formula. It’s the overarching idea of the class and really helps you think in a systematic way about what should be improved on your pages and funnels. The conversion index formula is:

Marketing Experiments Conversion Index

C is the probability of conversion, so this formula deals with variables that cause visitors to convert or not convert. Here’s the quick rundown of each letter:

  • m – motivation of the visitor
  • v – your value proposition
  • i – incentive to convert
  • f – friction of the process
  • a – anxiety about converting

I don’t want to go into too much depth, but I will mention that my favorites are incentive and friction. They are together in the equation because they counteract each other. You use incentives to overcome the friction of the page. So offering your visitors a white paper helps them deal with giving up their name and e-mail address to you. Visitors know they are going to get a call or e-mail when they give you info, but you have to give them reasons to give it to you.

An example of this that most of you have probably experienced as an internet user is when you find a great deal on a badly designed website. Even if it’s tough to get through the checkout process (friction), you’re likely to finish it since it is a great deal (incentive).

One of things I’ve learned is that making website changes is rarely a streamlined and easy process. The best situation would obviously be to have good incentives and low friction, but you can’t always improve everything because of office politics, technical reasons, lack of resources or numerous other things. So by using this formula and keeping these things in mind, you have multiple ways to attack problems either offensively (increase incentives, value proposition, focus on user motivation) or defensively (decrease friction and anxiety).


Google Optimizer is slow (or Not all Multivariate Testing is the same)

28 01 2008


Without knowing it, people might assume that there’s only one method to multivariate testing. That it has been long figured out by math and statistic wizards. I have learned otherwise from Widemile’s personal math wizard, Chief Scientist, Vladimir Brayman.

(Just as a side note, he does not have a typical office. Rather than papers and folders strewn about, he has statistic and math books. Lucky for me though, he has a great skill at distilling all the goodness in those books and teaching me what I need to know, in a way I understand.)

Most recently, we discussed why Widemile’s technology trumps Google Optimizer.

Widemile vs Google


Having a strong creative team and testing experts ensures better results than giving a marketer a tool like Google Optimizer, that’s easy for most people to understand. But explaining how Widemile’s technology can test more, faster, is a little more complicated.

Let’s explore how Google’s testing works versus Widemile’s. Google Optimizer uses full factorial test design, meaning it creates a page for every combination of your tested page elements. So if you wanted to test 4 different hero shots, 4 buttons and 4 headlines, that would require 4*4*4=64 page combinations. The disadvantage of this method is that you need significant traffic for each of the 64 pages. Meaning you either need a lot of traffic or a lot of time; for most companies, they’ll need both.

To solve this, Widemile’s optimization platform use fractional factorial test design. This method tests only a small fraction of the total possible page combinations and uses statistical analysis to derive almost all of the same information that would be found in a full factorial test. This works because marginal information is gained in testing all 64 page combinations, while testing a few important combinations tell us nearly everything we need to know.

Google actually criticizes fractional factorial test design (look here where it says “A note about ‘fractional factorial testing'”), saying that it requires the same number of impressions, but can not derive the depth of conclusions that a full factorial design can. While true that full factorial squeezes out the most information, that is at a sacrifice of extending the test many times longer than with a fractional factorial test, all to learn the smallest influences.

Doing successive tests to find high influence items with fractional factorial testing will get much higher gains than getting every ounce of information out of one extremely long full factorial test. In addition, with a carefully designed fractional factorial test you can learn all the major influences and the interactions between elements on the page.

Fractional factorial test design gets you a completed test in weeks rather than months or years even, and because of that, you can test more than you would normally be able to in the same time frame. You can either test more in one larger test, or do many smaller successive tests.

Not to say that Google Optimizer isn’t a great tool, especially since it is free, but any company that spends thousands of dollars on SEM has a lot to gain by using technology that gets rapid results.

If you got any questions about this, let me know and I’ll try to answer them or get you an answer.

Doubling conversion rates: MarketingSherpa case study

23 01 2008
MarketingSherpa Logo

MarketingSherpa is giving us a lot of love recently with The Weather Channel case study and now one on Smartsheet. If you were too shy to sign up for our case study on the Widemile website, then please check out the one at MarketingSherpa.

I especially like the last 4 points in the story:

#1. Don’t sit pat on your conversion rates. Colacurcio didn’t know that her original 5%-7% was above the industry average going in and is glad she didn’t. “If I would have said, ‘My conversion rate is pretty good’ and done nothing, I would have totally missed the opportunity to double it.”

#2. Conducting just a few multivariate test and applying them to a greater number of landing pages works. Simply put, you don’t have to test each landing page individually. “Sometimes it seems overwhelming when you think of multivariate testing, but you can cut corners. There are a lot of things that are low-hanging fruit — things that can be applied across landing pages.”

#3. Redesigning shouldn’t stop with your team’s new ideas. “You really have to get your organization into the mindset: ‘We are testing. We are not just going to spit out the next five Web pages.’ ”

#4. Even if your higher-ups are impressed with the initial results, Colacurcio says, marketers should expect to face organizational barriers when they start their second round of testing.

Don’t forget to check out the before and after in the creative samples.

If none of that catches your eye, Janet Meiners at MarketingPilgrim wrote a great summary on the case study.  She also makes a great point that, “I’ve been in heated debates about the best course of action but testing works best – assuming you’re humble enough to go with the data over your ego.

Trust your data and watch your numbers build your ego.

Test nothing and get results

18 01 2008
Test Nothing

Now this is a blog about testing right? So why test nothing? Because nothing is powerful.

I don’t mean don’t test at all, I mean test having less on your page. Those awesome sub headlines your copywriter created? Or those testimonials? Your audience might not care or even look at them.

Best practices say to use trust logos, reviews, awards and a whole lot of other content, but you never know if it really helps. In fact, one of the biggest lifts ever at Widemile, came from removing everything but the core material on the page. One hero shot, one headline, one description and one button.

It was plain and simple.

To me and everyone else, it was an empty page filled with white space, but it converted at an extremely high rate compared to the previous page. Everyone at my office and at our client’s couldn’t believe it was the optimal page, but we couldn’t argue with the data.

While this might be an extreme example, it also is an example of the opportunities you may be missing if you don’t test turning off (hiding) elements on your page. So don’t just test something, test nothing too.

How multivariate testing can change your whole business

15 01 2008
Test Tube

My boss, Frans Keylard, taught me one lesson that exponentially increased my respect for the power of multivariate testing.

While on the outside multivariate testing is about finding the best version of a page, once you know how to test, it can do a whole lot more than that. The information you glean from multivariate testing can shift the whole direction for your product, service and business in general.

Multivariate testing does help you find good headlines, the right images and other content, but it also acts as a survey about your product/service that your visitors don’t even know they are taking.

For example, my company deals with a lot of companies selling to both business and home users. Traditionally, to figure out what was more popular, they would survey people asking, “Would you buy this product for your home or for your business?” They then would count up the responses, the highest being the best one to go with.

At Widemile, I accomplish the same thing using a multivariate test. I serve some visitors business messaging and others home messaging and they respond by buying or not buying. If the page with business messaging has more conversions, then that is the way to go, otherwise go with the home messaging.

(Better yet, if they are both significant in size, find ways to segment them and do more testing.)

In both situations, you’re asking a question and getting an answer. While multivariate testing asks the question less directly, it gets the most direct answer possible, a conversion, from the most direct audience possible, live traffic. This deals with the weakness inherent in surveys; people’s answers and actions don’t always match up.

I’m not saying multivariate testing replaces surveys in all situations, but you get real valuable and actionable information from testing.

It is like killing two birds with one stone, with one small bird (your landing page) and then a huge bird (your overall marketing and business plan.)

Some marketers already do this with their PPC and banner ads, seeing what people respond to and adjusting their overall marketing strategy to what works best. Multivariate testing is an extension of this, but it requires an actual conversion by the visitor.

Start taking surveys of your audience using multivariate testing. All you have to do is key in on a few messages that you think might work, try them out. You’ll learn how to improve your web pages and your business at the same time.

Questions? Comments? This is my favorite topic, so I encourage you to leave a note for me.

225% Lift MarketingSherpa Case Study

9 01 2008


One of our good clients The Weather Channel were invited by MarketingSherpa to do a case study on a landing page optimization campaign we did with them. This was a really interesting page with a lot of great learnings and obviously with that high of a lift, it was very successful. Check out the case study here and make sure you look at the creative samples.

Notify original and optimized

One of the biggest things was that the flash banner out performed the static banner, which best practices typically argue against using flash banners. Not only did it out perform the static banner, but it was one of the most influential items on the page.

Increase SEM spend or optimize your page?

4 01 2008

SEM vs Multivariate Testing

Most decision makers start multivariate testing because it makes sense in terms of ROI. While I may talk about helping marketers make good decisions about their pages, my real job is to make my clients look good by getting them a result that looks good online (a quality web page) and on paper (quality returns.)

The value of multivariate testing and landing page optimization lies in the fact that SEM is rapidly becoming more expensive as competition rises.

How much would you have to spend in PPC to get a sustainable 30% raise in conversions? To be sustainable, you would have to spend that much money month after month.

  • Sustainable lifts even after optimization ends

With testing, the increased conversion rate will typically be sustainable over a significant period of time. Pay to optimize once and you’ll receive that raise in conversions for free for months and even possibly years after you finish your first test. Of course the best way to go is continual testing to squeeze even more out of your pages, but the results from any testing come pretty quickly and make good business sense.

  • More efficient SEM spend

A bonus is the natural cycle that occurs from optimizing. Once your page is optimized, your advertising spend will be more efficient. This allows you to either spend the excess money towards other campaigns or feed it back into the optimized campaign to further drive conversions.

  • Large increases in conversion rate

Lastly, while this only happens occasionally, conversion rates often shoot through the roof after testing. We have more than doubled conversion rates for some pages, which typically is not possible even with dumping money into advertising.

If you have a solid SEM campaign but aren’t seeing results you want or once had, or if you just want to stay ahead of the competition, multivariate and split testing are the necessary next steps.

I work in this field for two reasons: 1. I love marketing and testing, 2. I know that in a few years, every business will be testing.

SEM will be around for a long time and must be done right, but it is not getting cheaper anytime soon and most businesses already participate in it. Testing is still nascent and probably where SEM was 3-4 years ago. Don’t get left behind, the future is in testing.