Artificial intelligence has redrawn the data center design landscape, and in doing so has propelled data processing and storage to new heights of performance, complexity and cost. While GPUs, CPUs and AI accelerators often grab the headlines, storage is undergoing its own renaissance when it comes to how it’s configured and optimized, and increasingly, where within the data center rack it should be housed. NAND flash memory is at the center of this discussion, because it forms the high-density, non-volatile data pathways that help enable AI model training, pipelining and inference. This is especially true for enterprise SSDs (eSSDs). These are the NAND-based enterprise-grade solid-state storage drives that provide the data density, reliability, throughput and endurance required by today’s hyperscale computing environments.
At this juncture, data center eSSDs are evolving into two distinct branches. The first offshoot, compute-optimized eSSDs, sits close to GPUs where high-performance workloads demand high-speed, low-latency access to large datasets. Compute eSSDs also manage the exponentially growing volumes of KV cache that require both bulk storage and compute-level performance. The second evolutionary spur, capacity-optimized eSSDs, serves the dense data lakes that feed GPUs at scale by balancing high-volume data storage with the speed and reliability of flash memory. Together, these emerging NAND flash storage tiers are becoming foundational to AI-ready infrastructure.
A New Design Challenge: Stratifying Data in the AI Era
Conventional data center design yielded a relatively uniform layer in which NAND-based eSSDs supported both latency-sensitive and general-purpose storage. That’s no longer a given thanks to the widely variable workloads of today’s AI data pipelines. For data center architects, the challenge has expanded from data deployment to data stratification.
This is because AI workloads span multiple stages, from ingestion and preparation to training and inference, each with different performance and capacity requirements. Training AI models, for example, demands sustained throughput and parallel access to large datasets. Inference triggers more fragmented access patterns and typically calls for smaller and more frequent reads that emphasize data context retention. This necessitates new configurations that support both high-performance data storage and data storage at scale, often within the same deployment.
The evolution is also physical. Storage is no longer confined to a single location. Latency-sensitive data is best kept close to compute. Capacity-intensive data can be disaggregated and re-partitioned. In some deployments, such as remote networked storage, data can even be moved outside the immediate compute zone. In this scenario, data center engineers aren’t just building systems, they’re defining – and assigning – new data storage topologies. In doing so, they must now decide what data sits near compute, what’s housed elsewhere, and how each layer interacts under real-world workloads.
Near-Compute eSSDs: Feeding the GPU
Near-compute NAND flash, often referred to as compute eSSD, operates in close proximity to high-value GPUs and accelerators and can be accessed directly with technologies that bypass the CPU and system memory. These drives are optimized for low latency, high IOPS and consistent performance under heavy concurrency. They support use cases such as model loading, checkpointing and inference caching, where delays can directly impact system throughput.
AI workloads are pushing these requirements further as access patterns shift toward smaller data units and more frequent queries. Compute eSSDs are responding to the demand for increasingly granular and responsive storage by serving two completely different use cases with the same drive architecture. Initially, compute eSSds addressed data pipelining and checkpointing functions for AI model training. However, as inference and agentic AI became more prevalent – and models and parameters exploded in size – a second use case emerged in the form of KV cache offloading. Today, KV cache is one of NAND flash’s biggest growth opportunity in the AI data center.
This is why compute-optimized NAND is often described as an extension of the memory hierarchy. By acting as a high-capacity reservoir for offloaded KV cache, it occupies a space between DRAM and traditional storage and enables AI infrastructure to scale context windows beyond the physical limits of on-package memory.
High-Capacity NAND Flash: Scaling the Data Lake
At the other end of the spectrum, high-capacity NAND flash supports the rapid growth of AI data lakes. AI does not necessarily create new data types, but it changes how data is accessed. Instead of querying subsets of data, AI training often requires access to entire datasets at high speed. This has led to the rise of “fast” data lakes, where throughput and accessibility are as important as capacity. High-capacity eSSDs are optimized for density and efficiency and are defined less by latency metrics and more by how much data they can store and deliver per unit of power. This suits capacity-optimized eSSDs for large-scale “cold” storage environments where cost, space and energy budgets are tight.
In some instances, however, the need to access data more frequently is warming up cold storage. Here, data access is better served by an all-flash solution as opposed to more conventional tiered approaches that combine flash with hard drives. In these cases, advances in NAND density are altering the landscape by allowing flash-based data lakes to deliver both capacity and efficiency at scale.
QLC NAND and the Economics of Scale
NAND flash – in particular quad-level cell (QLC) NAND – is central to this data center transition. QLC allows for significantly higher storage density by accommodating four bits (and 16 voltage stages) per cell. While it does not match the performance of some higher-end NAND varieties, its value lies in delivering lower cost per bit and better energy efficiency at scale.
This makes QLC an ideal fit for capacity-optimized storage. It allows data centers to consolidate multiple hard drives into a single flash device, reducing both physical footprint and power consumption. QLC also enables a more granular memory hierarchy. Instead of relying on a single type of storage, architects can allocate different classes of NAND to different workloads. High-performance NAND can be reserved for compute-adjacent tasks, while QLC-based storage supports data lakes and bulk storage needs.
Designing for Power, Density and Scale
Increasingly, AI infrastructure is constrained as much by power as space. As GPU clusters scale, every watt becomes a critical resource. This changes how storage is evaluated. Performance remains important, but power efficiency is now a primary concern. In many cases, data centers are prioritizing storage solutions that deliver the most capacity or bandwidth per watt, rather than the highest absolute performance.
Density is another key consideration. Advances in NAND technology and packaging are helping enable significantly more capacity within a given footprint. This allows data centers to scale storage without proportionately increasing power and cooling. However, these benefits come with new planning criteria. Designers must carefully weigh how to provision storage, how to scale it over time and how to integrate it with existing infrastructure. Under-provisioning remains a common challenge, particularly when growth is underestimated, and expansion paths are not clearly defined.
Rethinking AI Data Center Memory Hierarchy
The emergence of compute- and capacity-optimized NAND flash reflects a broader transformation in data center planning. Storage is no longer a passive layer that simply holds data. It determines system performance by influencing how efficiently compute resources are used and how effectively AI workloads can scale.
This trend demands a reassessment of data center memory hierarchy – one that rewards proximity, workload alignment and data movement rather than relying on traditional storage models. It also broadens the question from how much storage to deploy, to include where it should reside. As AI evolves, architects who challenge the status quo and align their strategies to distinguish between near-compute and high-capacity storage will be better positioned to build data center infrastructure that is both scalable and efficient.
