Big news…

18 03 2008

There is a big announcement today at Widemile and a new post by me. However you won’t see it here. Please head to http://testingblog.widemile.com to see the new site! To celebrate Widemile’s annoucement, I have moved to a widemile.com domain and have a new design.

If you subscribe to my feedburner RSS feed, I have automatically changed it to the new site, however if you use the wordpress.com RSS feed, you will continue getting this site’s feed which is now dead. Please update your feed to http://feeds.feedburner.com/billyblogwm.

I will no longer be updating this site, other than to point people to the new page. So please head there now!

New URL testingblog.widemile.com

Advertisements




3 posts on 3 topics

6 03 2008

Edit: I fixed all the links in this post.  Copy and pasting is getting the best of me!

I recently came across a few great posts that I enjoyed and wanted to pass onto you all. The first is from Tim Ash, who has written a great book on Landing Page Optimization. One of his more recent entries discusses how to write effective copy to increase conversions.

One of my favorite bloggers, Avinash Kaushik tells marketers to embarrass their managers in order to succeed at their campaigns. Testing tops that list of course, but his other techniques are great methods at “working the system.”

Lastly, Lenny de Rooy, wrote a guest post at SEO Scoop about 5 misconceptions of Google Web Optimizer. It goes slightly beyond just GWO itself and into testing methodology





How to get ideal test conditions (and results)

4 03 2008

A big mistake in testing is to overlook variables inside and outside of the test that impact results. In an ideal test, the only variables would be the ones you are testing on your page. That usually isn’t possible though, but as long as you account for them in your analysis, you will get correct and actionable information.

Sky image

If you test a seasonal page, then the optimal page you get for that season, probably won’t perform when the season ends. By not paying attention to those kind of variables, you are setting yourself up into thinking you’ve found the optimal page. The same type of mistake is made by grouping e-mail, print, SEM campaigns and event traffic, unless you know they react the same to your changes.

Even within segments, there might be more segments to uncover. Your only limitation should be traffic; don’t segment so granular that you can’t run a decent sized test in a decent amount of time.

One of my clients doesn’t get a lot of traffic, but the traffic he does get is very distinct. One converts in the single digits and the other converts in the teens. Although combining them would get me more data, it would be very confused data since they convert so differently.

A few things to look out for:

  • The ad or offers visitors see beforehand
  • Interactions between your factors (if you aren’t testing interactions)
  • Technical problems
  • Problems that occur before or after the tested page

A note about the last bullet, the problems can range from a technical problem to a problem with the overall funnel. If people get different experiences in the funnel that drastically impact whether they convert or not, it can add a noise to your test. Some examples are different checkout processes for registered and non-registered users or users being inelligible for service.

The purpose of testing is to find out if a certain element performs well under the conditions you provide. If you aren’t paying attention to all the conditions, then the results you derive will be incorrect without you knowing.