Jean-Marie Bonthous, Seamless Social
A/B testing, split testing, bucket testing, multivariate testing
A/B testing is a great way to improve the effectiveness of your marketing. A/B testing, also called split testing or bucket testing is a way to test marketing by comparing a baseline control with a variety of single-variable test samples, to improve response rates. A/B testing was born as a direct mail tactic, and has been adopted within the interactive space to test banner ads, emails, calls to action and landing pages. It takes the guesswork out of optimization. By testing your changes before implementing them you get quantitative answers about what approaches create positive results. By constantly testing and optimizing you can increase revenue, leads, signups, downloads, or whatever else your objective is. A/B testing differs from multivariate testing, which applies statistical modeling by which a tester can try multiple variables within the samples distributed.
A/B testing for those of us who don’t write code
A/B testing online used to require extensive coding skills. Not anymore. We, at Seamless Social use HubSpot. This enables us to do easy and affective A/B testing for event marketing and other projects. If you do not use HubSpot or some other inbound marketing or marketing automation platform, Optimizely allows you to create a test at low cost, in minutes with no coding or engineering required. A/B tests will help you convert your website visitors into customers and earn more revenue.
Optimizely: a cost-effective solution for A/B testing
With Optimizely (which starts at $17 a month), you just need to paste one line of JavaScript into every page you want to test or measure and then test all you want. You can make visual changes quickly and easily: click on some text to change copy, add new images your page, and drag page elements around to try new layouts. Optimizely begins collecting data as soon as you start your experiment and their dashboard is updated in real-time.
There’s no need to set up goals before starting an experiment. Just make sure your Optimizely snippet appears on every page you want to measure (or just paste it into your site wrapper). You can test different decisions, images, layouts, and copy without touching code–you don’t need to know HTML. Their point-and-click interface makes it easy to create and run tests in minutes. Just make sure your Optimizely snippet is on each page, and Optimizely will automatically track page views for your experiment subjects.
Optimizely can be integrated with third party analytics services such as Google Analytics and Adobe Omniture SiteCatalyst. Additionally, with the API, Optimizely can be integrated with just about any analytics service or homegrown data store.
Optimizely uses both first and third party cookies to uniquely identify visitors. When a new visitor comes to a page where you are testing, they get bucketed into one of the active variations and a tracking beacon is set. The data are then logged, statistically interpreted, and presented on the results page.
The importance of measurable outcomes and adequate samples
A/B tests should have a measurable outcome, e.g. number of sales made, click-rate conversion, number of people registering for an event, etc.
For the testing to be effective, you also need to reach an audience large enough to detect meaningful differences. Sample size affects two key variables called “confidence interval” and “confidence level.” The confidence interval (“margin of error”) is the plus-or-minus figure usually used for poll results. If you use a confidence interval of 4 and 47% percent of your sample picks an answer you can be “sure” that if you had asked the question of the entire relevant population between 43% (47-4) and 51% (47+4) would have picked that answer.
The confidence level tells you how sure you can be. It represents how often the true percentage of the population who would pick an answer lies within the confidence interval. 95% confidence means you can be 95% certain. Most researchers use the 95% confidence level. You may want to use a sample size calculator, here is one that is free.
What has been your experience with A/B testing without extensive coding skills? I would love to hear from you.
Photo courtesy of Peter Dane via Flickr’s Creative Commons
