It's easy to decide to ignore all the statistics out there, or to just go with the one you like best and ignore all the others. But if you analyze the sources a bit more closely, you will get even better insight than if all the sources agree.
There are two major categories of statistics: one in which data is automatically collected across some subset of the Internet, the other in which data is collected by asking some type of person.
Ideally every request on the web, whether for a whole page or for an object of some type, has its data stored. What device, what browser, what operating system, what country, etc. Alas, that data doesn't exist. So all automatic systems are collecting only some subset.
There are two major types of automatic collection:
- Analytics installed on individual web sites using a specific platform, then aggregated into metrics. This includes normal analytics, like StatCounter. It also includes advertising platforms, most especially Millenial Media and AdMob.
- On-device reporting, such as that from Nielsen.
For any automatic system, you have to consider how the data is gathered. The advertising platforms (Millennial, Admob, etc.) are only collecting data from their network. So their numbers are affected by:
- Global coverage - each covers some countries better than others
- Ad formats - Millennial has special ad formats for iPhone, and does not promote Android to its advertisers. Admob, on the other hand, has special ad formats for Android. If you were an advertiser looking to target both platforms, which would you choose?
- Advertiser type - Millennial seems to cater to bigger brands and has less of a self-service long-tail client base than does Admob.
More fun: The Weather Channel and possibly other large mobile-savvy companies do their own ad sales, and aren't represented anywhere above. And some sites will have ads on every page, where others will limit the ads. The former will have greater representation in the data.
Nielsen, Pew, and some others use more traditional interview and survey techniques. By and large, these are done by companies who are focused on getting statistically valid data. Of course, as with any research, you have to investigate what they are asking and who they are asking it of. In particular, people don't know what type of phone they have, so asking them will result in errors.
At least one company asks mobile operator store managers which devices were sold in the past week or month. This is interesting data but doesn't tell us anything about how the devices are used.
I am especially fond of the Pew research, because it focuses on behavior, demographics, and actual use. It's helped our clients understand that many of their demographic preconceptions are wrong, and that their teenager preconceptions are merely incomplete. But that doesn't tell us what devices are being used.
Which Browser Has the Highest Mobile Web Use?
Millennial says iPhone. Admob now says Android, in the US. And StatCounter has said Opera for all but three months in 2009! Well, until you notice that iPod Touch is measured separately from iPhone.
One more bit of complexity: both Admob and Millennial serve ads to apps and web. And Admob serves ads to the top 10 free iPhone apps, so you would think there might be some bias there.
That's the wrong question for most of us. We instead want to know what smartphone/browser/phone has the highest use, in our audience.
What you should to is consider your market, and put together the data accordingly. I expect that StatCounter has disproportionate representation in India, where Opera and Nokia represent something close to 95% of the browser visits. (Admob largely agrees) Are your visitors in India?
In Indonesia, another high-volume mobile web market, there are a lot more Sony Ericsson devices (20% per Admob) but they don't make a blip on the StatCounter site. Apparently all the Sony Ericsson users are using Opera. Are your visitors in Indonesia?
StatCounter shows an uptick in Opera and a larger uptick in Symbian in the past two months. Is that a shift in adoption, or a shift in mobile web moving more into Asia? Admob data strongly suggests the latter. Or maybe Admob's sales in the region have gotten better.
Are you a premium brand? My hunch is that you should trust Millennial's data more, that their ad behavior better matches your customers' behavior.
Are you a long-tail offering? Then combine StatCounter and Admob.
Are you planning a free app? For now, I'd go with Admob. That'll change as iAd comes online, of course.
If you really want to know highest use for purchasing purposes, trust Nielsen the most. They don't measure every site out there, so do a bit of research before making ad purchasing decisions. Check out this mobile marketer article for more on choosing an ad network based on reach.
If you are deciding where to focus your development efforts and want to use just one source, again choose Nielsen. But I hope you'll look a bit deeper based on what we talked about here.