There has been a lot of discussion about App Store Optimization inside Bluecloud Select lately. So let’s take a look at one of the frequently asked questions about icon design optimization…how to split test your app icon.
This post will show you why a killer icon design is an integral part of the app conversion funnel and how Apple makes icon design testing virtually impossible. I will give you the workaround, so you can A/B test faster and find the design that converts like a madman.
Here we go…
Why You Need to Optimize Your App Icon
There are basically two parts to App Store Optimization. There is Search Optimization and what we call Appeal Optimization.
Search Optimization is done through your app’s title and the keyword field in iTunes Connect. This helps people find you in App Store search results.
Once they see your app in the search results however, that is where Appeal Optimization comes into play. Do your icon and screenshots entice them to download your app or do you send them running for the hills?
If you have to choose between working on your icon or your screenshots, you should start with your icon because it is seen the most often. It is at the heart of the branding of your app.
Don’t get me wrong, screenshots are important too, but if you have to choose where to allocate your limited resources, that is definitely the place to start.
One of my favorite illustrations of the power that image optimization can have on marketing results is this tutorial video on an experiment that was done for Facebook ad creatives. Some of the information in the beginning is dated, so you can fast forward through about the first third of the video.
But the rest of the video is timeless marketing strategy.
It’s also pretty damn funny 🙂
You may never look at ads the same way again.
[sc name=”Position – 1 – Shortcode” ]
Basic Icon Design Strategies
The elements of a great icon is not the main focus of this post, but it is an important part of the equation. So I’m going to give you a few resources that you can reference to create an app icon that will make people lock onto your app like a Death Star tractor beam.
To get some basic icon design guidelines and how to hire an app icon designer, read this post. This post also gives you some best practices on icon design, from designer Michael Flarup.
If you want some icon inspiration to get your creative juices flowing, be sure to check out this site or this site. Sometimes you just need a spark of inspiration to get you on the right track.
The Biggest Roadblock to iOS App Icon Design Testing
However, as you probably know, it can take several days for Apple to approve your app…if they approve it at all. The current wait time is 5 days, according to the App Review Times website.
That is a long time to wait to find out the impact of your icon design change. It puts a serious limit on how many icons you can test in a month.
On top of that, since you are not testing the icons at the same time, your data my be skewed by day of the week or seasonal fluctuations.
There has to be a better way.
Yes, there is.
This is what you can do to speed up the process…
How to A/B Test Your App Icon…Fast!
The key is use third party testing services to quantitatively measure your results. It goes something like this…
First, you buy paid traffic from Facebook ads, Admob, Chartboost, or any other ad platform. The problem with most in-App Store testing is that you may not currently have enough traffic to make your results statistically significant.
Heck, you may not even have an app yet. That makes it really tough to test anything.
Buying traffic solves both problems.
Second, you will create a mock app store page using one of the platforms mentioned in the next section. They allow you to change just one element on the page and measure the results.

Photo: Screenshot from Store Maven website.
We have found that the best way to bring in paid traffic is Facebook ads, but you should experiment with different platforms. To get a quickstart guide on Facebook ads, be sure to read this post. You can also get the complete Facebook Ads course inside Bluecloud University.
Tools to Use
More and more A/B testing platforms will come onto the market, but here are a couple of the top solutions that you can try right now. Be sure to test a few to find the one that fits your needs and budget.
Store Maven
This video from Store Maven is a good illustration of how the testing process works. It’s a little cheesy, but it gets the job done 🙂
They help you test all three of the major app stores, so huge bonus there.
SplitMetrics
This platform is pretty similar to Store Maven. This post on Medium gives you some background on the company and its founders.
TestNest
TestNest is another platform that gives you split testing capability. Their pages look just like Apple App Store pages, so your test results should be pretty similar to actual results.
Conclusion
So that is how you A/B test your app icon without submitting a full-on app update. This testing method will save you a lot of time and help you get to a winning design quickly.
It does cost a little bit of money, but it can be well worth it. The design that you like the most isn’t necessarily going to be the one that gets the most downloads.
In fact, in my experience with graphic design, it usually isn’t the one you like best.
There are also other platforms that allow you to split test other elements inside your app, on the fly, like the color scheme and logo design. But that is a post for another time…
Do you have any other questions about A/B testing app icons? Leave a comment below and let me know…
This is a great article, my only reservation is that now having spoken to the companies you mentioned they’re not cheap! Storemaven is very helpful but at $1500 a month it’s a difficult expenditure for indie app devs or startups. Great read though so thanks!
@Olly – StoreMaven prices start as low as $199 which includes all of the advanced reports. Happy to help you try it out.
Great piece, thanks. Would be interested to hear what other small or indie iOS devs have chosen of the three solutions presented here.
Also, would like to better understand how these three platforms interact with paid user acquisition …? We are using Facebook advertising already. Will we then be prompted to link our Facebook ad account when we sign up for StoreMaven, TestNest, or SplitMetrics?
Do you know cheaper options than StoreMaven, TestNest or SplitMetrics?