Most machine learning projects fail: Here’s how to harness analytics successfully
The majority of communications service provider (CSP) machine learning projects fail. So says David Yates, vice president of product management at Guavus. While you catch your breath at that and consider the implications for your business, here he gives five reasons for the failures, and shows how you can beat the odds.
Staying profitable in today’s telecoms climate requires some fancy footwork. The world’s incumbent CSPs are contending with 50% per year data growth on their networks, while annual infrastructure hardware costs are decreasing by only about 15%. The numbers say it all: CSPs need to act now to squeeze cost out of their organisations, in part by dramatically streamlining operations using automation. They also need to get creative in finding new sources of revenue.
The big data analytics tools that deliver the insights needed to achieve these goals are there for the taking – yet an estimated 80% of CSP analytics and automation projects that involve machine learning (ML) fail, according to STL Partners. What are the reasons and how can CSPs overcome them so they can quickly move on to greater profitability and competitiveness?
Let’s look at five common challenges that trip up CSPs and recommendations for handling each. The recommendations are based on what we’ve learned from CSP customers who’ve successfully applied artificial intelligence (AI) or ML-based analytics to meet their goals. Their key learnings could be very valuable for your business as well.
Define a clear business problem for analytics to solve
Identifying a concrete business outcome that you’d like to achieve with analytics seems simple enough. However, it can be quite difficult for CSPs, particularly incumbents with deep-rooted, isolated organisational structures in which separate groups have little or no interaction with each other. In addition, these setups have generally entailed each group keeping the data it generates and manages cordoned off from others in the organisation.
With people, operations and data isolated this way, it’s tough for any one group to see the big picture as to how analytics, run across a corporate-wide set of historical and live data, can be beneficial – to itself, another department or the service provider organisation as a whole. And that makes it difficult to set realistic business goals.
Create a cross-organisational data analytics committee to begin identifying small, targeted use cases for analytics
The group should include representatives from each of the CSP’s departments or lines of business that might benefit from successfully mining the company’s big data. In the spirit of the many successful corporate-wide security and disaster-recovery initiatives that have become common in today’s largest companies, the analytics committee would ensure that all groups’ business interests are represented. It would also ensure that cross-departmental data becomes accessible to all who could benefit from it and that bias based on history and experiences of any one group doesn’t creep into the ultimate analyses and automation. The committee would also be charged with setting data standards and creating a corporate-wide data dictionary defining what data formats to use to allow for easier data understanding.
To get started, the committee could work with a top-down, analytics consulting and knowledge organisation (see Challenge 2 recommendation, below) to identify one or more business problems, or use cases, that analytics might be able to solve. Together, the groups can work to collect business, marketing, sales and operational goals from each department represented and research what data is available and in what formats.
Demonstrate that analytics can deliver a desired outcome
Once a business outcome or use case has been clearly defined, it can be problematic to evaluate whether analytics are likely to deliver on the desired goal. If it’s not clear what data is the relevant data to include in the analysis, for example, where it’s stored, and its level of accessibility to those who need it, the people tasked with the decision to move forward with the project simply won’t have enough information to make a determination.
Create an expert analytics consulting organisation at the corporate level that business units can use free of charge
This organisation, mentioned above, would assemble large-scale analytics experts, either from an internal employee base or from an outside resource pool, and be funded by corporate. This group would consult with each business unit or the reps from each business unit in the data analytics committee to identify one or more problems or desired outcomes. The central organisation can conduct a three-week assessment about the goals of one or a few use cases to evaluate how viable it is to solve them with analytics, then provide each group with a quick yes-or-no verdict. If the response is yes, the business unit and the corporate entity can work together to begin bringing the first project to fruition, with the project cost split between the business unit and corporate.
If your company currently is simply too large and disjointed to create a single analytics oversight group, consider the approach taken by one US-based telecoms company we work with. It structured its analytics organisation into three divisions: 1) operations and customer care; 2) global supply-chain strategy; and 3) big data and AI systems. While the divisions themselves are separate, they rely on a common group to address data governance and management, data warehousing, data lakes and common analytical and AI technologies across the organisation. This helps to facilitate cross-functional, cross-organisational projects.
Identify and accumulate the right data
Today’s segregated data silos could mean that the right data needed for the analytics job at hand is inaccessible because of where it’s stored or what format it resides in – or both. Figuring out what data sets are relevant to the business goal, locating that data, and then exposing it to an analytics platform by pulling it together into a data lake or other setup have proven to be stumbling blocks for many CSPs.
Ideally, the business unit or department seeking a defined business outcome would have the necessary data or know who has it and would have the ability to get it quickly. However, that’s not always the case. In real-world situations, figuring all this out usually requires some investigation, which could take weeks that a given group can’t afford to invest, particularly if it isn’t clear who funds the effort.
Pool the efforts of the cross-departmental data analytics committee and the centralised corporate knowledge/consulting organisation
Having the centrally organised, corporate-funded expert analytics group mentioned is beneficial for assisting in this work. The organisations mentioned in recommendations one and two can go far in identifying who should do the legwork and who should pick up the cost of the data preparation – whether funding will come from a corporate budget or whether it will be a shared corporate-business unit expense.
Data drift and how to prepare data for a successful proof of concept (PoC)
The continuous maintenance of data is time-consuming and difficult, which means that personnel in charge of managing aren’t always up to date with their work at any given time. Also, different groups within a CSP organisation often record data differently, and operations personnel change frequently and adopt different procedures. For example, one group recording data that rates something on a scale of one to 10 might have different definitions of what a given number in the scale means. In mobile networks, radio access network (RAN) data about individual mobile calls might be incomplete: one group might consider recording the originating and terminating cell towers to be a complete record, when solving a network issue with data analytics also requires data about call handovers among intervening cell towers for the full picture. For all these reasons, it’s not uncommon for network-wide data to be inconsistently recorded and not in a usable form right out of the gate.
Prototype your data models before scaling them
Once the right data is known and made accessible, assign someone to prepare the data by identifying personally identifiable information (PII), anonymising the data, and ascertaining which fields have null values. Consider using simple prototype models to flush out categorical problems or reveal bad values before you scale your analytics model. A good prototype should give some indication of what basic threshold conditions should exist. You can then recreate those conditions to allow for early evaluation.
In addition, it’s important to prepare data to avoid bias: If you look at only historical data, it will be biased toward whatever was true in history. Creating data features, or attributes, that help avoid bias and excluding data columns that contribute to bias, such as gender, are ways to remove bias.
Move proof of concepts into large-scale production
Only about one in five AI projects have moved from pilot to production in CSPs to date, according to STL Partners. The reasons have to do with the isolated cultures mentioned and the general lack of centralised, top-down approaches to monetising big data in CSP organisations. In addition, analytics expertise within CSP organisations is generally scarce. Yet early adopters have attempted to take on creating or enhancing analytics platforms themselves, often in the wake of a failed attempt to squeeze carrier-class scale out of an enterprise-class solution that couldn’t accommodate the vast size and breadth of CSP networks and the data they contain.
Evaluate augmented analytics tools. Decide whether it’s best to integrate an analytics platform into your legacy system or to upgrade to a new platform
It can be intimidating to build a platform from the ground up by yourself, so be on the lookout for tools that make analytics accessible to more individuals in your company – those that can be used by personnel other than bona fide data scientists. Among these are augmented analytics with augmented data preparation capabilities that use algorithms to find relationships in data, learn users’ roles and what they’re interested in, deliver prescriptive analytics that account for situational awareness, and recommend the best approaches for cleaning, reconciling, enriching, manipulating and modeling data.
What does success look like?
Some of the achievements that have come from CSPs putting these analytics processes in place include the following:
• A European provider discovered unconnected communities in Africa and the optimal deployment locations for installing fibre into those communities to launch new services and create new revenue streams
• A Scandinavian mobile operator has slashed its new customer acquisition costs by up to 84%
• A UK-based multinational mobile operator is now using AI-based chatbots to handle two million customer interactions per month, with 70% of queries
answered the first time around
These are just a few examples of the power that running analytics across a CSP organisation can have – and the serious difference in operations and profitability they can make. As noted, though, planning and execution are critical to successful roll-outs. You need to take a big-picture view, organisationally, for opening up access to required company data while at the same time starting out laser-focused on small, with very well-defined business goals for one or a few specific use cases.
Once a pilot or PoC has successfully delivered a desired business result, it’s easier to move it into large-scale production. And from there, you’ll find it easier still to build on your experience with finding and prepping the right data and harnessing your analytics platform to further monetise the volumes of network, operational, and user data stored across your company.