AI workloads are energy-intensive because every query and every training run triggers large numbers of mathematical operations across GPUs and other specialized chips. Even a simple prompt to a chatbot runs on hardware that draws significant power.

There are two main drivers:

• Training: Large models can require thousands of servers running continuously for weeks or months before they ever answer a single user query.
• Inference: Once deployed, models process huge volumes of requests, each consuming additional energy.

This sits on top of an already growing data center footprint. Data centers used over 1% of global electricity in 2010, rising to around 1.5% in 2024. Over the last five years, electricity consumption from data centers has grown by about 12% per year, largely due to AI.

Looking ahead, some projections suggest that in the US alone, data centers could consume 325–580 TWh per year by 2028—roughly comparable to the current annual electricity use of the UK (~320 TWh) or Germany (~460 TWh). This makes AI a major new consumer of power and puts a spotlight on how efficiently providers convert energy into useful AI output.

Nebius looks at AI infrastructure as a chain that turns energy into tokens (the pieces of text models generate). To make this transparent and measurable, we break the process into four layers of efficiency:

1. Model layer
   This is how a specific model uses the underlying hardware. Model size, architecture, and runtime optimizations all affect energy per token. For example, on the same hardware, Llama 3 405B can consume about 2.3× more energy per token than GPT‑3 175B (roughly 8.8 J/token vs. 3.6 J/token).
   
   At this layer:
   • Users choose models and configurations.
   • Vendors define hardware and software capabilities.
   • Nebius optimizes runtimes and offers tools like Token Factory to increase throughput without changing the model itself.
2. Cluster layer
   This is about how GPU clusters are built and operated. Stable, well-designed clusters reduce idle time, retries, and failures, so the same job finishes faster and uses fewer GPU-hours.
   
   For instance, one Nebius customer saw 56.6 hours of stable operation on a 3,000‑GPU production cluster, while studies show similar-sized clusters often see job interruptions roughly every 9.8 hours. Better stability directly cuts energy use for the same AI output.
3. Fabric layer
   The compute fabric is a pool of interconnected servers that act as a single resource and are shared across tenants. Utilization here is critical. When GPUs are powered on but idle, they can still draw about 12–25% of their active power, based on Nebius internal observations.
   
   Nebius uses virtualization, on-demand and preemptible instances, and smarter scheduling to capture small unused GPU “slices” that would otherwise sit idle. This increases the amount of AI work produced per MWh of available fabric capacity.
4. Data center layer
   This is how effectively a facility turns incoming electricity into power delivered to IT equipment. The key metric is PUE (Power Usage Effectiveness). In 2025, the global average annual PUE was about 1.54, meaning that for every watt used by servers and GPUs, data centers needed an additional 54% for cooling, power conversion, and other overheads.
   
   Nebius designs and operates its own data centers, which allows it to rethink cooling, power distribution, and layout to improve this ratio.

By managing all four layers—model, cluster, fabric, and data center—Nebius can reimagine how efficiently energy is turned into AI tokens and tie engineering decisions directly to sustainability outcomes.

Nebius approaches AI sustainability by combining software, hardware, and data center design under one vertically-integrated cloud. This lets us coordinate optimizations across all four efficiency layers.

Here are some of the practical measures:

1. Software: getting more tokens per unit of compute

• Token Factory for inference: Instead of running models in default configurations, Nebius Token Factory applies runtime optimizations that improve how models interact with GPUs. It increases throughput and reduces compute needs without changing the model or hardware.
• Post-training optimization tools: We add optimizations on top of open-source models so they can produce more useful output per unit of compute than their out-of-the-box setups typically allow.

2. Cluster reliability and performance

• Health checks and auto-healing: Active and passive health checks, plus automated node isolation and replacement, reduce failures and shorten recovery times after interruptions.
• Measured impact: One customer achieved 56.6 hours of stable operation on a 3,000‑GPU cluster, compared with research indicating interruptions roughly every 9.8 hours on similar-sized clusters. Fewer interruptions mean fewer wasted GPU-hours and lower energy use for the same training run.
• Topology-aware scheduling: Placing jobs on GPUs that are physically close to each other shortens total job time, improving ROI and reducing energy per completed workload.

3. Fabric utilization and virtualization

• Virtualized compute fabric: On-demand and preemptible instances let Nebius fill small unused GPU-hour “slices” that would otherwise stay idle. This is a key difference from bare-metal setups, which often leave gaps between reservations.
• Shared buffer instead of per-tenant reserves: Bare-metal environments often keep 10–20% of each tenant’s cluster as idle spare capacity. Nebius maintains a single shared buffer across many tenants and typically needs only about 4–5% of cluster capacity as buffer, cutting idle power draw.
• Unified, software-defined storage: Block, file, and object storage run on the same hardware instead of separate systems. With fewer dedicated nodes per workload, overall facility power consumption is lower.

4. Hardware and data center design

• In-house server and rack design: Nebius designs and refines its own hardware, rather than relying solely on off-the-shelf systems. This helps meet performance and reliability targets and reduces failures that would otherwise waste energy.
• Proprietary firmware: Controlling low-level software gives tighter control over reliability and behavior under load, which supports stable, efficient operation at scale.

Because Nebius operates across the model, cluster, fabric, and data center layers, it can align these measures to systematically reduce the energy required to produce each token, helping customers run AI workloads in a more power-efficient way.