Marketing Profs

As web marketers seek new ways to boost conversion rates and improve their visitors' site experience, interest in multivariate testing is on a feverish rise. But those unfamiliar with the techniques are often unclear about where to start, or how to ensure success.

In this article, I'll discuss the following:

• A clear definition of what multivariate testing is and how it differs from another common type of testing: A/B testing.
• How testing provides a foundation for continuous improvement of your Web marketing initiatives.
• An overview of five common mistakes to avoid when planning and running tests.

What Is Multivariate Testing?

Common methods for running controlled experiments on Web sites range from simple A/B testing to sophisticated multivariate testing, also known as multivariable testing.

In A/B testing, one or more new versions of a page or single site element competes against an existing control version. For example, two versions of a headline might compete against an existing headline.

Multivariate testing, on the other hand, is like running many A/B tests concurrently, where there are multiple elements being tested at the same time. For example, two alternate product images, plus two alternate headlines, plus two alternate product copy text, for a total of 27 possible combinations (including the original control versions).

What's important to understand about multivariate testing is that it not only shows you which combination of elements generate more sales or pull more leads but also reveals which individual elements influence visitor behavioral vs. those that do not. For example, did variations in product image influence visitor behavior more, less, or the same as the copy?

Understanding how each site element causes visitors to interact with your site is the essence of a test-learn-repeat process that marketers can use to synthesize new ideas and continually improve their site's ability to achieve—and exceed—their marketing goals.