Back in 2013, Electronic Arts used an A/B testing strategy to boost digital downloads of its SimCity 5 game. This case study looks at how the games maker made smart use of website optimisation to sell 1.1 million copies in the first two weeks, with 50% coming from direct online downloads.
Case study summary
• Games maker wants to drive download sales of new game
• Used split testing based on several hypotheses
• Tested 100% of traffic to find optimised solution quickly
• Winning variation was a big surprise… driving 43.4% more purchases
The challenge
As EA prepared to release the new version of SimCity, they released a promotional offer to drive more game pre-orders. The offer was displayed as a banner across the top of the pre-order page. But according to the team, the promotion was not driving the increase in pre-orders they had expected. The team decided to test some other options to see what design or layout would drive more revenue.
The solution
Working with marketing tech firm Maxis, EA set up an A/B test pitting the original SimCity pre-order landing page against several variations that displayed the promotional offer in different ways.
Each variation corresponded with a hypothesis the team had set beforehand regarding placement, colour, and display of the promotional offer. One variation removed the promotional offer from the page altogether.
After building each variation in Optimizely’s visual editor, the team set their experiment goal — clicks on the “Pre-Order” button — to track purchases generated from the test.
They allocated 100% of site traffic to the experiment in order to find an optimized solution as quickly as possible during the high-traffic SimCity launch period.
The results
The Maxis digital marketing team quickly discovered shocking results – the variation with no offer messaging whatsoever drove 43.4% more purchases. This suggested people really just wanted to buy the game, no extra incentive necessary.
“We found that the variations with the promotional offer actually performed worse than just a clean version of the page. This was really surprising to us, but it had a pretty significant impact,” Burk said.
Most people believe that direct promotions drive purchases, but for EA, this turned out to be totally false. Testing gave them the information needed to maximize revenue in a way that would not have been otherwise possible.
