Monday, 7 July 2014

A/B Testing using Google Experiments


Hope you all had a good weekend. I really had fun. A few days back my manager threw some light on A/B testing with the help of Google. And that is the reason why I am writing this blog.

What is A/B testing?

In short, it is a way to improve the customer experience and thereby increase revenue from your website. You can search in Google to find the best definition. 

How Google helps?

Google helps you to create 10 different variations of your content and also provides library/SDKs to integrate the variations in your application. It is called Google Content Experiments. It offers 3 different ways of implementing it.

Standard Implementation

It uses redirect method and is suitable for a multi-page application. While configuring the experiment you can define different variations of your page. For example, assume that you want to test your home with different flavors. And you have created separate pages for the same. What you have to do is to make sure that these flavors can be fetched from different URLs (keeping the domain same). 
Lets says, the URL of the original page is
And URL of first variation is
And URL of second variation is

Once you configured your experiment with these URLs, just fill in additional details such as experiment objective, minimum run time etc. Now set up the experiment code by copying the JavaScript snippet to each variation of the home page. And finally start the experiment. Based on the % traffic and traffic distribution configuration, your home page will server different flavors for each user. You also have to ensure that each page has Google analytics integrated.

Browser only Implementation

This method is suitable for single page applications (usually mobile web). If you want to switch between views or content such as image, HTML etc, make use of the Content Experiment JavaScript APIs. What you have to do is follow below given steps.

1. Identify the variations - Decide how many different variations you plan to experiment

2. Create an equivalent number of dummy variations in Google experiment portal. A screenshot is attached.

3. Get experiment code from portal as given in the screenshot given below.

4. Include following code snippet in your page.
<script src="//"></script>
Where the experiment parameter should be the one you got from 3rd step.  This client library will load synchronously and creates a global object named cxApi.

5. Now call 'chooseVariation' method of Experiment API.
var variation = cxApi.chooseVariation();

6. Based on the value of 'variation' do the appropriate action.

var variation = cxApi.chooseVariation();
var viewVariations = [
Ext.Viewport.add([{ xtype: viewVariations[variation] }])

In my sencha application, I am choosing different views for home page based on the variation received.

7. Make sure you have Google analytics integrated. If not, do it and send one hit.

  (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),

  ga('create', 'UA-XXXX-Y', 'auto');
  ga('send', 'pageview');

You will get to know who the winner is from experiment portal.

Server side Implementation

This section is a bit complicated. It is mainly intended for services which affect experiment objective even without UI interaction. It can be a db query or a business logic etc. But UI is the one which is going to reflect this variation anyway. You have to still integrate Google analytics in your page and pass on ExperimentId and variation to Google. That means a co-ordination is required between client and server.

Mobile app Implementation

Google Content Experiments also support native mobile apps with the help of Google Tag Manager. This is an interesting section. I shall come up with the details as soon as I get hold of it.

Have a nice day!

No comments:

Post a Comment