The world is quickly realising that with the rapid advancement in AI there are also caveats. In short, apart from environmental implications, it also has major significant financial ramifications.
As an example, according to Forbes, the costs of training and running AI models, especially large language models (LLMs), are substantial. These models require enormous computational resources, leading to soaring computing costs. Indeed, the cost of computing is expected to climb 89% from 2023 to 2025. A massive jump in a very short time span.
Elaborating a bit on the above, LLMs are based on transformer models, a type of neural network architecture developed by Google in recent years. But, with the mindboggling interest generated by ChatGPT in 2022, transformer models are now being used throughout the industry.
And you guessed it, unfortunately, these transformer models are costly when it is required to process these massive amounts of data.
For example, a transformer processes a large body of unlabelled data to learn the structure of the language or a phenomenon, and how nearby elements affect each other. This is extremely energy intensive exercise and cost millions of dollars to train some of the largest models.
When you then add the cost implications of data centre cooling and operational complexity into the mix, it’s easy to understand why AI is expensive; very expensive.
Power demands and over-engineering risks
As mentioned, modern AI workloads require an enormous amount of power. In fact, some estimate the power needs of AI and cloud computing are growing at such a rate that individual data centre campuses could soon use more electricity than a small city.
That said, it is also important to pinpoint exactly where more power and cooling are required; some parts of datacentre don’t deal with HPC and AI processing for example. Here, precise power design and allocation are therefore essential as over-provisioning can lead to wasted resources and under-utilised infrastructure.
Also, and this is probably relevant across many continents, AI is forcing data centre operators to augment their energy sourcing strategies. Relying solely on grid power is no longer sustainable, especially in regions where electricity prices are volatile or where utility providers impose steep tariff hikes.
Data centre operators are exploring renewable energy solutions, such as solar and hydrogen, to mitigate costs and improve sustainability.
Hybrid energy models offer another viable solution. By scheduling HPC workloads during periods of peak solar generation, data centres can reduce reliance on non-renewable sources and lower their carbon footprint.
Again, these models require careful planning and advanced scheduling systems but also promise long-term financial and environmental benefits.
Cooling
Cooling is another area where AI workloads are driving up costs. And whilst this topic has been explored at length, it’s important to recognise that traditional air-cooling methods are simply not good enough for HPC environments, especially outside naturally cold climates like the Nordics.
Advanced cooling technologies, such as liquid cooling, are now indispensable for maintaining operational efficiency. Liquid cooling offers a more effective way to dissipate heat in high-density environments, reducing energy consumption and enabling data centres to handle increased computational loads.
However, these systems come with their own set of challenges, including higher upfront costs and the need for specialised maintenance.
Managing systems to drive down costs
To manage the abovementioned complexities, data centre infrastructure management (DCIM) and building management systems (BMS) have shifted from nice-to-have to essential. These systems provide granular visibility into power and cooling dynamics, enabling operators to allocate resources more efficiently.
For instance, monitoring the heat generated by individual GPUs allows precise adjustments to cooling requirements, reducing waste and ensuring optimal performance. The ability to dynamically allocate power and cooling only where it is needed can significantly lower operational costs.
The above is extremely valuable when looking at ways to drive down the costs of AI and mitigate its environmental impact. There are solutions and these must enjoy priority in a time that what lies beneath are power hungry data centres that drive all things AI.
