Every marketer’s New Year’s Resolution

27 12 2007

We marketers get to have a lot of fun. We have a variety of projects that we get to plan, develop and execute. Wrapping up every project is an accomplishment for us. Or is it?

New Years Fireworks

Unfortunately, many projects are started without tangible metrics to strive for other than finishing it on time and within the budget. One part of being a marketer, or any businessman or woman, is showing that what you do is valuable and has a solid number attached to it.

Because of that, every marketer should make a resolution to measure and test their campaigns rather than just finishing them.

Achievements like site redesigns and campaign launches should be celebrated, but they also should be effectively tracked and analyzed from beginning to end.

Jason Burby wrote a fantastic article at ClickZ about ringing in the new year by defining and executing based on goals for 2008. In it he describes how sometimes projects succeed in some measures but fail at others, so overall is it a success?

He even mentions multivariate testing as part of the solution: “One part of this new way of thinking is to ensure you have a simple, easy to use testing platform to try different things and to measure the impact of tests based on goals. If you haven’t invested in an A/B or multivariate tool, 2008 is the time.”

What I’ve found in doing optimization and working with marketers in web analytics is that defining success in every campaign is key to being a quality marketer. And the only way to know if you met those goals is to measure what you do and how much it drives those goals.

For web page testing, the goal typically is to raise conversion rates and so its easy to keep that in mind. However if I never changed conversion rate as my ultimate goal, I might miss out on other valuable goals. A great example is if there are two pages with different subscription levels for a product, then possibly the lower converting page has a higher lifetime ROI.

ROI Table

The great side benefit of having a goal and measuring what gets you there, is that you can learn from success and failure. Even when I test a page and get an improvement, I exam what factors were the worst, along with what did best. It tells me what scares away and attracts customers so that I can look out for those things in other parts of the page and overall funnel.

Never assume that changing or making a new version of anything is always better. Figure out what you want to improve, make the changes and see if you find that improvement. As long as you have a measurement for success and take the time to track your progress, you can learn from your campaign and in the end, get closer to reaching and beating your goals.


Multivariate testing: a quick primer

26 12 2007

New URL testingblog.widemile.com

Want to quickly get up to speed on multivariate testing? This post is designed to help you grasp the basics of multivariate testing so that you can get started talking and even doing your own tests. While its hard to design great tests, it’s easy to get good results with only a little education. I will definitely be teaching more about multivariate testing soon, but get this stuff down first!

Note: Since the industry is new, there isn’t consistency in much of the vocabulary of multivariate testing, so I will try to use generic terms.

Billy’s Multivariate Testing Primer

What is Multivariate Testing?: Testing multiple versions of a page to determine a set of elements* providing the highest conversion rate.

*Elements can be any content (text or image) on a page, typically hero shots, buttons, button text, headlines and text blocks. Sometimes it can refer to position/layout also.

What happens:

Multivariate testing diagram

  • Visitors come to a page and are shown a random version of the page
  • Conversions are tracked based on which page they saw
  • Once a statistically significant number of conversions is reached, analysis determines the elements on a page that create the highest conversion rate


  • If used correctly, it hones in on correct messaging direction and then exact messaging can be found using further testing
  • Allows for quicker testing of multiple elements than split or A/B testing
  • Analysis derived from live visitor data
    • Proves winning page elements to be better than others


  • Medium learning curve
    • Test results easily ruined by mistakes or poor methodology
  • Requires a minimum number of conversions
  • Code must be added to web page
  • Cookies and JavaScript must be enabled by visitors

Keep in mind:

  • Increasing the number of elements being tested increases the time needed for the test.
  • Number of conversions over time determines test size and length
    • The faster a page gets conversions, the shorter the test will be and/or the more things you can test
    • More conversions is better
    • A shorter time frame is better than a long time frame, but don’t go shorter than 2 weeks
  • Don’t test things that are too similar, look for different segments or messaging to pursue

Other concepts:

  • Using it with split testing:
    • Use split tests to determine the best layout with a template test. Test layout against layout with the same content (see this article for more advice.)
    • Afterwards, use multivariate testing to try out different messaging and refine it with continual tests.
    • If you want to try new layouts later on, go back to split testing.
  • Advanced: There are two types of multivariate tests: Full factorial and fractional/partial factorial
    • Full factorial means every version is shown. Meaning if you have 4 headlines and 4 buttons, that is 4×4=16 combinations, so 16 pages are used in the test.
    • Partial factorial is when only a portion of the total possible combinations are shown to visitors. This relies on statistical formulas and algorithms to determine the influence of the various factors since not every page combination is shown.
    • Full factorial takes a much longer time, so partial factorial allows for more testing in a quicker time frame.

There’s still a lot more to teach, even about some of the things mentioned here, but I hope this is enough to get you off the ground and really digging into testing. Google has their free tool if you want to try out multivariate testing.

Why always optimize landing pages?

19 12 2007

Sometimes people ask me why landing pages are such popular targets for optimization and testing. Why not optimize a home page? A product page?

Actually, we can optimize those kinds of pages, but almost all businesses come to us with a landing page that needs help. Beyond the demand for landing page testing though, is the fact that landing pages inherently are fit for testing. Let me explain.

Widemile LPO Landing Page
One of Widemile’s landing pages

The strength of multivariate and split testing is in pulling out the best page possible for what you want to do. With a homepage you have multiple things you want to do (e.g. show your products, company history, customer service, get people to spend X minutes browsing) and with a product page you are typically working with a CMS template for your whole site. Those factors complicate things a bit since you have to figure out what counts as improvement for the homepage and for the product page, you have to work with a CMS system and make changes that improve the majority of products pages using that template.

These things are not impossible to do or even difficult in many situations, but a landing page is usually totally independent of everything and has only one goal.

In a technical sense, a landing page is more simple to deal with. In a measurement sense you only need to improve one metric, the conversion. With a landing page, we don’t have to make copy and creative that works okay for all situations, we simply make copy and creative that is optimal for one situation.

This makes testing really fun and easy, since you can test and find out who your audience really is and what causes them to convert.

Now that I’ve answered this question, in the future I’ll move onto a more interesting post: Why you should optimize everything else too.

Around the Net in Search Marketing

18 12 2007

Welcome! I just got word that I was featured in MediaPost’s Around the Net in Search Marketing. If you aren’t sure where to start, check out my organized lesson guide.  Also, I have attempted to organize everything in appropriate tags and categories found on the right navigation bar.

I just started this blog a few weeks ago, so there’s going to be a lot of growth and new posts, so if you like what you see, subscribe via RSS or get e-mail updates.

In addition, if you have any questions, opinions, or suggestions on the site or my posts, feel free to leave a comment.

Case Study: Smartsheet.com

18 12 2007

I try to keep this blog non-company specific but my boss Frans Keylard, Director of Optimization, helped write up an informative case study of a client he worked on, Smartsheet.com. The case study is 4 pages long, covering the process and analysis of a multivariate test we did on one of Smartsheet.com’s landing pages. You can download it from Widemile.com, it requires a name and an e-mail to get it (sorry!)

Here’s my favorite image from the whole thing…

Smartsheet.com optimization results

5 rules for quality landing page design

17 12 2007

New URL testingblog.widemile.com

Landing pages are effective. When you want to accomplish one thing, there are few scenarios that can trump a quality landing page. The fact that they are the easiest page to test makes them even stronger. However, before you start testing, there are some best practices you can follow.

Here are five rules to keep in mind:

  1. Keep it focused: Force your page to have only one goal. If you are required to have two, then choose one as a priority and emphasize that one. Remove excess baggage like advertisements and navigation bars so that visitors have only two choices: convert or leave.

    Keep it focused bad example
    2 choices should have 2 ads and 2 pages

  2. Give a good second impression: People don’t see the landing page first, they see your text or banner ad, e-mail or even search result. Build relevancy between them. Use the same messaging from the first point of contact to your landing page. I try to use the same words or exact title from advertisement to landing page. Keep creative consistent too. One thing that is often overlooked are offers; if you put an offer on your page, put it on your ad and vice versa.

    Good second impression example
    Matching titles tell visitors they’re at the right place

  3. Target your biggest (middle) audience: There’s 3 types of visitors. Ones that’ll convert no matter what, ones that might convert and ones that are just looking. You only need to grab that middle audience. A landing page is not meant to please everyone, it is meant to drive conversions, meaning pleasing only those that will convert! For example, putting less information on a page will drive away people only looking to learn more, but help push along those that are looking to buy.

    Target the middle diagram
    Stop catering to window shoppers and get those looking to buy

  4. Stop talking about yourself: Customers come to your page to read about the product, not your entire company history. Talk about yourself to the extent that it will calm visitor’s fears about your legitimacy and quality, or else you’ll clutter the page and intimidate the visitor with blobs of text. Third party validation logos (BBB, Hacker Safe) and quotes from happy customers are often enough.

    Logos and quote example
    Logos and quotes talk faster than your own text

  5. Use a product shot: So a cheetah might be a great symbolic way to show how fast the computers you’re selling are, but really you should be showing your computer. Customers come in and will only spend a few seconds to see if they’re in the right place before hitting back and so you need to communicate what you’re selling fast. Why distract them with symbolic images, when your product is what you want them to buy? If you’re service oriented, then people probably are a good idea, but make sure they directly represent what you’re doing.

    Product shot
    Don’t have a tangible product? You can still have a product shot.

I could go on and on about landing pages (and I will!), but these are some good tips to get started. Expect more posts like this in the future, but leave a comment if you have anything to say.

Oh I have one caveat to all this, best practices work most of the time. Make your page better, but test it and prove that it performs.

5 tips to maximize your holiday campaign

13 12 2007

Gifts for the holidays

Ready for the holiday season? My boss, Frans Keylard (Director of Optimization), wrote up a 3 pager on “5 short tips to maximize your holiday campaign” (download PDF). I summarized it in my own words below, but check the PDF for more detail on how to get your conversions up during the holidays or any season!

Monthly sales increase as much as 20% for many online retailers in December and so having a good site now is more important than any other time. Optimization can drive up your sales beyond the seasonal increase in the short term and, in the long term, helps to add even more lifetime customers to your business. With that in mind, here’s 5 short tips to ramp up your site.

  1. Seasons Matter: Change your site to match the season, we call it “seasonal-tuning.” Don’t let your site look stale; you don’t want Halloween colors and images for Christmas time. Matching what your customers are actively seeking will really drive your campaigns to success. This may not always work though, so make sure to test what has worked previously against new seasonal images. Lastly, even seasonal images need testing, so try different messaging in each, e.g. product shots, giving the product to another, and people using the product.
  2. Know your existing customers: Reach out to existing customers, they are the most qualified audience and you should know them better than anyone else. Create offers that have a lot of perceived value to them. Also, use offers for people who don’t follow through the whole purchase process and try to pull them back in. They are interested but just need a reason to bite. Remember to know both what customers buy and why too. If you’re best selling chairs are shown in an office setting, when your customers are really home business users, then you might lose conversions. Use a home office image instead and get more relevancy to your items.
  3. Find a new audience: Seasonality doesn’t just go for your web pages, apply them to your PPC text and banners ads and you’ll get new visitors quickly. Think holidays, turkeys, gift-giving, family, anything that resonates with your audience and your products. Include some seasonal offers for good measure too. You can do SEO around these words too, but that takes future planning. A good way to setup seasonal SEO for next year is by looking at what PPC and SEO worked this year, so make sure you don’t lose that valuable information.
  4. Offers over branding: Don’t get too caught up in branding, just sell the product and offer. People are under pressure to buy nowand if your focus isn’t on helping them to do that, they’ll find someone else. You can build your brand later and have a reason to if they bought from you!
  5. Build seasonal landing pages and optimize them with multivariate testing: Building a seasonal landing page is easy to do and optimizing that page is a must during seasonal periods. We optimized an online recipe search over the summer and found the best summer imagery, but once winter came around we changed the type of food and got a 30% sustained lift over the best performing summer image.

Stores change their decorations every season, so make sure to do that with your online store front. Give people a reason to stay at your site, even if you think it is small, it could have a huge impact. Finally, never forget to test and tune your changes every season, every year. Good luck!