This spring, Google made the landmark announcement that more searches are now conducted via mobile devices than desktop. If desktop Internet connected our computers, the mobile Web connected us. Since the iPhone’s debut eight years ago, our daily lives and interactions have been absolutely transformed by mobile devices, and the trend shows no signs of slowing down.
Businesses have been challenged to innovate and meet customers across all channels. By now, you likely have a mobile-optimized version of your site, and you might have a mobile app as well. When it comes to your business, it’s undeniable that mobile matters. But are you aware of how seemingly minor changes to your customers’ mobile experience can translate to serious jumps in your bottom line?
These “small” optimizations are usually hiding in plain sight, and new tools are available to help you find them. The game has changed. Looking out across the rapidly developing mobile landscape, one thing’s for sure: there’s gold in them hills.
So where do you start looking?
Golden Opportunities: Entering the Mine
In the desktop world, A/B testing (simultaneously comparing performance between two versions of an interface or application) has remained a proven method of safely testing changes and updates prior to full deployment. But this is mainly utilized in a Web environment, where changes can be rolled out (or rolled back!) in real time. The app environment requires a bit more planning since app stores are involved, users must download new versions, and stringent rules exist on what is permissible in the realm of testing and what isn’t. When you go pan for gold in a river, you need a different approach then when you are mining for gold within a mountain. Luckily, mobile gold doesn’t lie too far beneath the surface of your digital experience.
“[T]esting to ensure that you’re not breaking anything is just as important as testing for improvements to business goals. A/B testing allows you to feature flag new functionality and then gradually roll those features out to your users. If a negative impact is observed, you can turn off the test and revert everyone back to the original experience before things get out of hand.”
Desktop and mobile experiences sometimes require different content or functions to meet different motivations. The rules that traditionally apply to desktop do not apply when you have users interacting with mobile devices in totally new ways (touch, voice, location, messaging, etc.) and in ever-changing environments. The mobile marketplace is fast changing and the rules are anything but set. What works on desktop may completely flop on a mobile platform. This is why it is important to just enter the mine and start looking.
Striking Gold: A/B Testing Is Your Metal Detector
This is where mobile A/B testing can really make for an efficient mobile experience. When it comes to mobile optimization, test, don’t guess.
Before you can start optimizing, it is vital to identify your mobile objectives and form a cohesive strategy. Whatever your current mobile target—increased average session length, total subscriptions, downloads, or conversions—the idea is the same: A/B test your features/functionality and proceed based on verified statistical significance, not hope and good intentions.
Small but intentional improvements in your mobile app can lead to big payoffs. Recently, British retailer Marks & Spencer sought to boost engagement of a newly implemented full-screen “Cooking Mode” in their mobile recipe app. Through A/B testing, the company was able to strike gold and find a 20 percent increase in engagement simply by changing its call-to-action button text from “Enter Cooking Mode” to “View Fullscreen.”
Similarly, the online book retailer MyBook was able to use A/B testing to significantly increase its mobile subscription rate. MyBook’s strategy? Differing choices for its paywall subscription (“weekly,” “monthly,” “yearly,” etc.) were tested and a winner was found. As a result, MyBook “saw a measurable increase in payments, and revenue from the winning variation represented a 25-35% increase compared to baseline.”
4 Tips for Turning Mobile A/B Testing into a Goldmine
- Start at the beginning and look closer
Put your end user’s mobile experience under the microscope. Where are the bottlenecks? Breaking down every step toward conversion into finer and finer detail for testing purposes will quickly reveal what works and what doesn’t. Any inconsistencies can be immediately identified and addressed. Do this by giving the app/mobile website to someone completely removed from your typical network and get their thoughts on it. Also dive into your analytics and check out what is happening in your funnels and around your key conversion points.
- Start digging
I promise it will not take long to hit gold if you prioritize and dig. The key is to start with the campaigns you can get out the door quickly and leverage these wins to build out other campaigns. Mobile web is generally the easiest place to start, then make your way into the app world.
- Make sure it’s not fool’s gold
You’ve A/B tested your variations and think you’ve struck gold. Great! Make sure you have a statistically significant result by following some of this great advice on sample size estimation and campaign planning.
- 4. Consistency is key
Mobile A/B testing isn’t a one-and-done deal. Those who will stand out in the mobile marketplace have a strategy in place and consistently and carefully optimize their customer’s mobile experience. Systematic, continuous A/B testing will provide a clear direction forward and allow mobile development teams to focus on specific areas for improvement, rather than a potentially disastrous “publish-and-pray” nonstrategy.
Join the Gold Rush
Mobile usage is at an all-time high and a mountain of opportunity awaits. As desktop sales continue to free fall, those who enter the mine without the right tools don’t stand a chance. Many businesses know they need to take action to compete in this mobile revolution but are unsure where to start. Mobile A/B testing is an absolutely essential tool for reaching your customers and achieving real results.