Real-World Education for Modern Marketers

Join Over 600,000 Marketing Professionals

Start here!
Text:  A A

Multivariate Testing, Part 1: An Introduction

by Eric Hansen  |  
April 22, 2008

As web marketers seek new ways to boost conversion rates and improve their visitors' site experience, interest in multivariate testing is on a feverish rise. But those unfamiliar with the techniques are often unclear about where to start, or how to ensure success.

In this article, I'll discuss the following:

  • A clear definition of what multivariate testing is and how it differs from another common type of testing: A/B testing.
  • How testing provides a foundation for continuous improvement of your Web marketing initiatives.
  • An overview of five common mistakes to avoid when planning and running tests.

What Is Multivariate Testing?

Common methods for running controlled experiments on Web sites range from simple A/B testing to sophisticated multivariate testing, also known as multivariable testing.

In A/B testing, one or more new versions of a page or single site element competes against an existing control version. For example, two versions of a headline might compete against an existing headline.

Multivariate testing, on the other hand, is like running many A/B tests concurrently, where there are multiple elements being tested at the same time. For example, two alternate product images, plus two alternate headlines, plus two alternate product copy text, for a total of 27 possible combinations (including the original control versions).

What's important to understand about multivariate testing is that it not only shows you which combination of elements generate more sales or pull more leads but also reveals which individual elements influence visitor behavioral vs. those that do not. For example, did variations in product image influence visitor behavior more, less, or the same as the copy?

Understanding how each site element causes visitors to interact with your site is the essence of a test-learn-repeat process that marketers can use to synthesize new ideas and continually improve their site's ability to achieve—and exceed—their marketing goals.

Sign up for free to read the full article.Read the Full Article

Membership is required to access the full version of this how-to marketing article ... don't worry though, it's FREE!


We will never sell or rent your email address to anyone. We value your privacy. (We hate spam as much as you do.) See our privacy policy.

Sign in with one of your preferred accounts below:


Eric Hansen is the president and founder of SiteSpect Inc. ( and architect of SiteSpect's multivariate testing and behavioral targeting platform. Reach Eric via

Rate this  

Overall rating

  • This has a 4 star rating
  • This has a 4 star rating
  • This has a 4 star rating
  • This has a 4 star rating
  • This has a 4 star rating
4 rating(s)

Add a Comment


  • by s Tue Jun 10, 2008 via web

    sorry but i did not get how it comes up to 27 combinations

  • by Eric Hansen Tue Jun 10, 2008 via web

    It's basically 3 * 3 * 3.

    We calculate that as follows:

    3 (control + two alternate product images)
    3 (control + two alternate headlines)
    3 (control + two alternate product copy text) =%3E 27 possible combinations.

  • by will nichols Wed Jun 11, 2008 via web

    You can use a really useful online tool that allows to determine any number of different combinations.

    The "Taguchi Orthogonal Array Selector"

  • by s Thu Jun 12, 2008 via web

    thanks - makes sense :)

MarketingProfs uses single
sign-on with Facebook, Twitter, Google and others to make subscribing and signing in easier for you. That's it, and nothing more! Rest assured that MarketingProfs: Your data is secure with MarketingProfs SocialSafe!