The world is quickly realising that with the rapid advancement in AI, there are also caveats. Apart from environmental implications, it also has significant financial ramifications.
As an example, according to Forbes, the costs of training and running AI models, especially Large Language Models (LLMs), are substantial. These models require enormous computational resources, leading to soaring computing costs. Indeed, the cost of computing is expected to climb 89% from 2023 to 2025. A massive jump in a very short time span.
Elaborating a bit on the above, LLMs are based on transformer models, a type of neural network architecture developed by Google in recent years. But, with the mindboggling interest generated by ChatGPT in 2022, transformer models are now being used throughout the industry. Unfortunately, these transformer models are costly when required to process these massive amounts of data.
For example, a transformer processes a large amount of unlabelled data to learn the structure of a language or phenomenon and how nearby elements affect each other. This is an extremely energy-intensive exercise, and it cost millions of dollars to train some of the largest models. When you then add the cost implications of data centre cooling and operational complexity into the mix, it’s easy to understand why AI is expensive − very expensive.
Power demands and over-engineering risks
As mentioned, modern AI workloads require an enormous amount of power. In fact, some estimate the power needs of AI and cloud computing are growing at such a rate that individual data centre campuses could soon use more electricity than a small city. It is also important to pinpoint exactly where more power and cooling are required; for example, some parts of a data centre don’t deal with HPC and AI processing. Here, precise power design and allocation are essential as overprovisioning can lead to wasted resources and under-utilised infrastructure.
This is probably relevant across many continents, as AI is forcing data centre operators to augment their energy sourcing strategies. Relying solely on grid power is no longer sustainable, especially in regions where electricity prices are volatile or where utility providers impose steep tariff hikes. Data centre operators are exploring renewable energy solutions, such as solar and hydrogen, to mitigate costs and improve sustainability.
Hybrid energy models offer another viable solution. By scheduling HPC workloads during periods of peak solar generation, data centres can reduce reliance on non-renewable sources and lower their carbon footprint. Again, these models require careful planning and advanced scheduling systems, but also promise long-term financial and environmental benefits.
Cooling
Cooling is another area where AI workloads are driving up costs. While this topic has been explored at length, it’s important to recognise that traditional air-cooling methods are simply not good enough for HPC environments, especially outside naturally cold climates like the Nordics.
Advanced cooling technologies such as liquid cooling are now indispensable for maintaining operational efficiency. Liquid cooling offers a more effective way to dissipate heat in high-density environments, reducing energy consumption and enabling data centres to handle increased computational loads. However, these systems come with their own set of challenges, including higher upfront costs and the need for specialised maintenance.
Managing systems to drive down costs
To manage the abovementioned complexities, data centre infrastructure management (DCIM) and building management systems (BMS) have shifted from ‘nice-to-have’ to essential. These systems provide granular visibility into power and cooling dynamics, enabling operators to allocate resources more efficiently.
For instance, monitoring the heat generated by individual GPUs allows precise adjustments to cooling requirements, reducing waste and ensuring optimal performance. The ability to allocate power and cooling dynamically only where it is needed can significantly lower operational costs.
The above is extremely valuable when looking at ways to drive down the costs of AI and mitigate its environmental impact. This is a priority – especially in a time where what lies beneath are power hungry data centres, driving all things AI.
| Tel: | +27 11 254 6400 |
| Email: | za-ccc@se.com |
| www: | www.se.com/za/en/ |
| Articles: | More information and articles about Schneider Electric South Africa |
© Technews Publishing (Pty) Ltd | All Rights Reserved
printer friendly version