Split Testing Software - What You Need To Know.

You've probably heard that split testing elements of your website can improve your conversion rates. And that's true. But there are many kinds of split testing out there. And there are many ways to set up split tests on your website.

I can't discuss every feature of every testing tool out there, but I can give you a major overview of the different kinds of split testing software.

First, there's a difference between multivariate and single variable testing.

With single variable testing you just pick one element on your page to test. Take your headline, for example. If your current headline says "Are you having trouble with tooth decay?", you might want to see how "Do your kids have tooth decay?" works. In a single variable test, you just find a way to rotate each of those headlines on your page, and keep track of how many sales each generates.

Google offers a single variable split testing tool, but you have to have an Adwords account to use it.

Single variable testing can be slow. How would you like to test 10 times as much with the same amount of traffic? Well, you can. You use a method called multi-variable, or multivariate testing. This allows you to test several elemens on your page simultaneously.

There are further distinctions among multivariate split testing tools. There are hosted solutions, and solutions you install on your server. Typically a split testing program can run a little more quickly if it's on the same server as your webpage. Installed programs tend to be less expensive, too. You also will have complete control of your data. On the other hand, with a hosted solution, you don't have to do an installation.

Hosted solutions include Google's Website Optimizer, and Vertster. Installed solutions include the Split Test Accelerator and Kaizen Track.

Taguchi testing is only one kind of multivariate testing. There are also "full factorial" tools, and random independent rotators. When you run a Taguchi test, you run only a small fraction of the possible combinations of your factor levels. You might, for instance, run only 18 pages to test 4374 combinations. This is made possible by orthongonal arrays. This allows you to find the best combination with out running through all of them. You do have to be careful about your factors interacting negatively with each other, but if you take care as you design your test, you can avoid the bulk of the problem with these. The Split Test Accelerator, Vertster, and Kaizentrack are Taguchi testing tools.

When you run full factorial tests, as you would with Google's Website Optimizer, you create a few factors, and a few options for each factor, and the software cycles through each combination. So, if you have 5 factors with 2 options each, you would cycle through 32 different pages. FFT can be used like a Taguchi test for small test designs. Or, if you run a lot more traffic to it, you can deal even more effectively with interaction effects by just letting all the interactions (good and bad) play out. Note that you can't run as many combinations with full factorial tools as you can with a Taguchi too.

Some multivariate tools randomly serve the options for each factor independently from what is served for the other factors. This can lead to more flexible test design, but also to more interaction effects and other statistical noise, leading to longer tests.

Now that's just the overview. There are many other features to compare and questions to ask. Does a tester allow you to exclude bots and spiders? Does it allow real time access to data? What kind of reports does it show? What kind of information does it track for each visitor, and is the visitor-level data available for viewing. Etc.Dominic DeLong has used taguchi testing software for years to improve landing pages.