If the telecoms sector is looking at saving money, it could do a lot worse than to stop spending more than US$6 billion a year on drive testing, writes Tom Luke, a vice president at Tutela.
The limitations of traditional drive testing are well-known: it tends only to cover major cities, is conducted infrequently, and can’t tell you anything about mobile performance in homes and offices, hillsides, sports stadia, or anywhere else a car can’t go. More significantly, drive testing cannot take place everywhere, all of the time, all across the world, simultaneously, which limits the value and available use-cases for the data. What’s more, there has been known to be a degree of collaboration between independent testers and selected network operators, where the latter know exactly when and where testing will be conducted, leading to biased results.
In an era of artificial intelligence and big data analytics, the idea of employing someone to drive around the country with a clutch of mobile devices on the passenger seat seems positively antediluvian. It certainly seems a strange way of basing strategic investment decisions that will affect millions of users and billions worth of budget.
There must be a way to supplement network diagnostics with data on the actual performance from the end user’s perspective – for example, by crowdsourcing this data from everyone’s device. Speed-testing applications and mobile operator applications have been silently and anonymously measuring the user’s network quality in the background of their applications for years, transmitting this back to central databases which keep track of mobile network quality across the world.
Once again, however, this approach has its limitations, which include:
- Small amount of data – crowdsourcing in this way requires the user to download and use some specific, niche applications. These apps (mostly speed testing apps) have limited appeal to the global population and hence data is collected only from a relatively small number of users.
- Biased results – the users of these niche applications tend to be more technical users, who, by association, have more advanced phone models – so they can often achieve better performance than average. What’s worse: people generally only run a speed-test when their network quality is either very bad, or very good which heavily skews results.
- Limited to only one network – in the case of mobile operators installing monitoring software into their own self-service apps (which can overcome the two issues above), these are generally only used by their own customers, which means that a competitive view is not available. The ability to compare to peers and make competitive statements about network quality superiority is therefore lost.
The opportunity for an evolution of this model has been clear for some time. The answer is simple, but the execution is difficult: embed the monitoring software within hundreds or thousands of popular third party apps and games which are used by hundreds of millions of people, in all parts of the world and on all networks. You can then gather every bit of data you need – from download speeds to signal strength, jitter, latency and more – combined with the scale you need to make informed decisions as well as comparisons with competing networks, and all without bias.
By harnessing the power of crowdsourcing, network operators can harvest real-time app, device and connectivity data to gain an impossibly rich picture of performance anywhere and everywhere in the world – 24/7 – anywhere there are people.
In such a competitive market, where poor performance can very quickly damage a brand’s reputation for reliability and quality of service, such insight is obviously invaluable. They can make powerful marketing claims, backed-up by substantial independent data that demonstrates their superior quality, attracting new customers.
Another advantage, often forgotten in the discussion about measuring performance, is how many operators outsource network infrastructure management to third parties. Constant monitoring via crowdsourced performance data enables operators to keep a close eye on service level agreements (SLAs), ensuring that they can hold their partners to their promises.
It’s easy to take the smartphone revolution for granted: by rights we should still be astounded that we routinely carry around so much computing power in our pockets. Network operators should be harnessing the power of these incredible devices, seeing them not just as endpoints, but as windows into their subscribers’ daily experience.
