Optimizely makes website A/B testing simple, fast, and fun

A/B testing is one of the best ways to optimize your company website. Unfortunately, it can also be a bit tedious to manually generate and track the performance of multiple small variations to a site. Optimizely (www.optimizely.com) does an impressive job of simplifying every step of A/B testing, from making the variations themselves, to routing the appropriate traffic, and keeping track of the results.

The what and the why of A/B testing
If you're not familiar with A/B testing, it basically just means creating variations of a particular page (such as the landing page of your website) and tracking their performance in terms of some metric you care about (such as your conversion rate). Often times, you'll run the different versions simultaneously such that some visitors see page A, and the others see page B. The results of an A/B test can tell you which version of the site is better, or if the change has no real impact on your metric of interest. Over time, a series of A/B tests can dramatically improve the performance of many websites.

As you may imagine (or know from experience), while A/B testing can be very useful, it can also be rather time-consuming and tedious to generate and track multiple small variations of a page. Optimizely aims to ease each step of the process of A/B testing.

How Optimizely works
At this point, you might be wondering how Optimizely manages to handle A/B testing for your website without introducing a bunch of extra hassle (or maybe you're thinking that they probably do introduce a bunch of hassle). The basic process has three steps.

1. You start by telling Optimizely the url of your website. It will then load up an annotated version of the site in which you are able to create new variations from a WSYWIG-type editor. You can rescale elements, move them around, change and reformat text, replace images, modify links, and make plenty of other changes. In many cases, this type of interface can be much easier than actually making changes to the source code for some basic testing.
2. Once you've created the variations you want to test, you save them to your account and place a small piece of javascript in the header of any pages you want to A/B test. When a visitor comes to a page with that javascript present, Optimizely will modify the page that the visitor sees according to one of the variations you've previously saved. The proportion of visitors sent to each variation can be controlled through the Optimizely site.
3. Once you've started serving the variations, Optimizely will keep track of the visitors to each version, as well as any goals that you set up (such as a conversion rate or bounce rate). It also provides some basic statistics to gauge the significance of the differences between sites, as well as integration with Google Analytics (and, I'm guessing, other analytics software). Based on this tracking, you'll be able to generate new iterations off of the successful variations to continually optimize your site.

Hopefully that gives you a decent idea of what Optimizely is all about, but I've included their intro video here as well if things are still a little unclear:

First impressions
Tyler and I currently handle all of our A/B testing manually, so I haven't had a chance to try out the full range of what Optimizely offers yet. What I have used so far is definitely easy, although, as you'd expect, you won't have as much control over the site as you would with manual A/B testing. I've primarily had a chance to mess around with page editor which seems to work well from what I can tell. As an example of the types of modifications you can easily make, here's our current LAS homepage:

And here's a variation I made of the page. I changed the size of the main logo, flipped the order of the signup form and video, and changed the wording and color of the text above the form. All of these changes were made simply by clicking the elements on the page, much like you would do in powerpoint or a website builder like Dreamweaver:

If you aren't A/B testing, get started
Like I said, I haven't had a chance to fully explore the other functionality of Optimizely, but the truth is that the editing piece is the trickiest technical part, and they seem to have done well there. One thing to look out for is the load time of your page, as the extra javascript used to create the page modifications may slow things down if you make a ton of changes. In my limited testing, I didn't run across any performance issues, but it's definitely going to be application specific, so be sure to monitor that in your own testing. I haven't been able to try out the reporting at all yet, but the screenshots and video seem to pretty well flesh that part out.

So with those caveats aside, if you haven't been doing any A/B testing of your website, there's a very good chance that you could make big gains by doing so, and Optimizely looks like it will dramatically simplify the process. Prices start at $20/month, and there's a one month free trial to try things out. Depending on the type of site you have, that first month may be all that you really need to make strong improvements, and if you have a reasonable volume of traffic, that $20/month can be well worth it for continued testing.

This article was written by...

Bracken King

Co-founder of Less Annoying CRM

We don’t have comments enabled on this article, but I'd be happy to discuss it with you. You can connect with me on: Twitter | Google+