To build a telco-specific LLM, look to the public cloud

Most telecom executives are well aware of the competitive edge artificial intelligence (AI) is bringing to businesses today. It’s supercharging networks, streamlining workflows, and crafting customer experiences that truly stand out. But the challenges of implementing an AI strategy within an organisation can be enough to keep telco leaders up at night.

Several of the mega tech companies have embarked on building their own Large Language Model (LLM), such as Google and Meta, which is estimated to spend $9B to build its own NVIDIA server cluster. Several telcos have also set out to build their own LLMs, like South Korea’s SK Telecom and Japan’s KDDI. While these two telcos haven’t disclosed how they are going to train their LLMs, they have basically two choices: acquire GPUs from NVIDIA or the like and build a proprietary AI datacenter, or take the easier (and cheaper) route: do it in the public cloud.

AI in the public cloud, or “Cloud AI,” can make networks smarter and faster, while helping telcos deliver more personalised customer experiences—all without busting the budget. Whereas an NVIDIA H100 server can run $25,000 – $30,000 per server (and with scarce availability, may run $40,000 on sites like eBay), the public cloud vendors have all invested in building their own custom silicon that rival NVIDIA’s chipset. By using the public cloud vendors, telcos have the ability to pay for only what they use. Plus, the public cloud’s scalability and ability to process huge amounts of data make it the perfect choice for training and deploying telcos’ data-intensive, large-scale models.

Here are three key ways Cloud AI delivers substantial value to telcos:

Lightning-Fast Speed

Telcos deal with mountains of data in real-time from voice calls, video streams, and more. That means they need to train complex models that can learn from data and make predictions—without latency or errors. The custom chips of the public cloud make this a reality. They handle AI inference and training at unparalleled speed, leaving traditional CPUs and GPUs in the dust.

Up until now, NVIDIA has been king of the hill when it comes to AI-optimised chips, but with the hyperscalers stepping up their game, today’s telcos have additional choices, such as:

• AWS’ custom-designed silicon, including:
  • Graviton4, which promises a whopping 30% performance boost, 50% more cores, and a 75% surge in memory bandwidth compared to last year’s Graviton3, which already had impressive performance;
  • Trainium2, the next-level AI model trainer, which is four times faster than its predecessor and twice as energy-efficient; and
  • Inferencia2 chips, which are designed to slash the cost of running massive, intricate machine learning (ML) models.
• Google’s AI supercomputer, TPU v4, which is reported to be 1.2x – 1.7x faster than NVIDIA while using 1.3x – 1.9x less power than the NVIDIA A100.
• Microsoft’s custom-designed AI chips, announced in November at Ignite, including:
  • Microsoft Azure Maia AI Accelerator, optimised for generative AI and AI tasks;
  • Microsoft Azure Cobalt CPU, an ARM-based processor for running compute workloads on Microsoft Cloud; and
  • Azure Boost, which makes storage and networking faster for AI processes and is now generally available.

Whichever cloud provider you choose, these specialised AI chips are the boost your AI applications need.

Cost-Effectiveness

AI workloads can be heavy on the pocket, but public cloud custom-AI chips can help reduce costs by cutting down on energy use, optimising resources, and slashing infrastructure spend.

For example, AWS Inferentia chips claim to deliver up to 45% lower cost-per-inference than GPU-based instances, and Azure’s NPU chips deliver to 50% lower cost-per-inference over CPU-based instances. Google’s TPU chips deliver up to 80% lower cost-per-training compared to GPU-based instances. This is good news for the telecoms and cloud enthusiasts interested in greater savings and smarter budgeting.

Faster Innovation

Innovation is the heartbeat of AI, and it’s the area in which the chips of the public cloud really shine. They seamlessly integrate with tools and platforms offered by cloud providers, making it easy for telcos to explore, experiment, and stay ahead of the competition.

These hyperscaler tools provide a playground for AI projects that help telcos fast-track their AI development cycle:

• Use Amazon SageMaker, a fully managed service that can build, train, and deploy ML models using AWS Inferentia, to optimise models with features like AutoML, Debugger, and Experiments. You can create models from scratch or select pre-built ones from AWS Marketplace.
• Azure Machine Learning allows you to access data from multiple sources, apply algorithms, and publish your models as APIs or web services.
• Google Cloud’s Vertex, a unified platform, offers end-to-end services for building and managing AI projects using Google TPUs. Use AI Platform to access data from Google Cloud Storage or BigQuery, train models with TensorFlow or PyTorch, and deploy them on Google Kubernetes Engine or Cloud Functions.

A Future of Smarter Telecom

The future of telecom is about optimised networks, real-time fraud detection, efficient processes, and, most importantly, owning the customer relationship. Armed with petabytes of data, the power of the public cloud, and AI-specialised chips, your telco could be the trailblazer shaping a new path forward. Telcos, the future is calling. Will you answer?