Lightning Fast A/B Testing Twitter Style

It was interesting the other day when we discovered a fascinating A/B test that Twitter is doing on their home page, testing 5 different versions of their call-to-action text to see which one elicits better registration numbers. Registration is important for any site on the Web and for a site like Twitter that gets over 300,000 new users per day, even a 1% difference in sign-up conversion could mean 3,000 users per day! What was interesting about what Twitter was doing was that each phrase they were testing was quite different from a psychological standpoint. For example, the text “Let me in” emphasizes being on the outside of a community and wanting in. The term “Sign up” has no such implication. 

I also mentioned that at Performable we want to build the world’s fastest A/B testing platform. So I created and published the exact same test that Twitter is doing to see how long it took me.

And the results?: It took me less than 4 minutes to recreate the Twitter homepage test. That’s fast: less than one minute per test variation. (Of course, if you want to test this quickly, sign-up for our beta over at Performable.com )

Here is a screencast showing how I did it: