« HE:labs

Creating A/B experiments with Split

Postado por Thiago Belem em 19/08/2014

Split is a rack based ab testing framework designed to work with Rails, Sinatra or any other rack based app.

With the Split gem you can create and automate your A/B experiments and improve your website messages/buttons/colors with little to no effort! :watermelon:

Live example

I've created an example application with an A/B running on the call to action button.

On the above example, the button text can be "Sign-up!", "Join us" or "Go go go".

You can also check the Split dashboard, with the results of the experiment.

The source code for this application is available on GitHub.


We are going to use the version 0.7.2 of Split for this tutorial.

Add the gem to your Gemfile:

1 gem 'split', '~> 0.7.2'

If you have any doubt, follow the setup guide and make sure you have met all the requirements.

Create your first experiment

The simplest A/B experiment you can do is changing the color or text of a button/link on a view:

1 <div class="container" role="main">
2   <div call="hero">
3     <h1>Our products are fantastic!</h1>
5     <% ab_test('call_to_action', 'Sign-up!', 'Join us', 'Go go go') do |text| %>
6       <%= link_to(text, pages_signup_path, class: 'btn btn-primary btn-lg') %>
7     <% end %>
8   </div>
9 </div>

Every time a different user access this page, he will see one of the texts.

And then you finish the experiment when the user reaches the next page or the end of the goal:

1 <div class="container" role="main">
2   <div call="hero">
3     <h1>Thank you!</h1>
5     <% finished('call_to_action') %>
7     <%= link_to('Go back to the home', root_path, class: 'btn') %>
8   </div>
9 </div>

And that's it! The experiment is finished and the option the user saw (and clicked) gains a point.

Creating and finishing experiments via the view is not the only way of doing things with Split.. you can do the same via Controllers too. Be sure to read the README for more information.

Using another A/B algorithm

By default, Split will show a random option for every user and you can change this behavior using the algorithm option, on the split initializer:

1 # config/initializers/split.rb
3 Split.configure do |config|
4   config.algorithm = Split::Algorithms::Whiplash
5 end

The Whiplash is an implementation of a multi-armed bandit algorithm. This algorithm will automatically weight the alternatives based on their relative performance, choosing the better-performing ones more often as trials are completed.

Accessing the Split dashboard

Split comes with a very nice dashboard so you can see how every experiment/options is doing:

To enable this dashboard you need to add a require param to your Gemfile:

1 gem 'split', '~> 0.7.2', require: 'split/dashboard'

And mount the split routes:

1 # config/routes.rb
3 Rails.application.routes.draw do
4   # ... other routes ...
6   mount Split::Dashboard, at: 'split'
7 end

And when you access the /split endpoint on your browser, you should see the Split dashboard! :four_leaf_clover:

As you can see on the above picture, on the dashboard you can see the status of every experiment and options, conversion rates and etc. You can also choose an option to be used (finishing the experiment).

That's all for now folks, see you on my next post! :+1:


Sabia que nosso blog agora está no Medium? Confira Aqui!