The Memory Turn: Why AI's Next Bottleneck Is Not the Chip

The global artificial intelligence race is becoming increasingly memory-centric, according to the chief technology officer of Sandisk — a shift that reorients the strategic assumptions underpinning both industry competition and the semiconductor policies governments are racing to draft.

By Monexus Staff Writerglobal6-minute read28 May 2026☆ Save ↗ Share ⎙ Print

For two years, the dominant frame for understanding artificial intelligence competition has been the chip. Nvidia's H100 and GB200 accelerators became shorthand for frontier-model capability; who controls the GPU supply chain controls the pace of AI development. That framing is not wrong, but it is increasingly incomplete. On 28 May 2026, Nikkei Asia reported comments from the chief technology officer of Sandisk, the memory-design division of Western Digital, making a case that the next phase of AI development will be defined not by compute scaling but by memory architecture. The global AI race, the CTO argued, is becoming "memory-centric."

The distinction matters more than it sounds. High-bandwidth memory — the stacked DRAM architecture that sits alongside AI accelerators inside a training cluster — determines how quickly a processor can access the data it needs to run inference and model training. Compute without sufficient memory bandwidth simply idles. As model sizes have grown, memory demand has scaled at least as fast, and in some estimates faster, than transistor performance improvements. The CTO's claim is that this relationship has now tipped: memory bandwidth and capacity, not raw FLOPS, are the operative constraint on what an AI system can achieve.

The implication cuts two ways. For investors and technology executives, it means reassessing which companies sit on the critical pathway. For policymakers drafting semiconductor incentives and export controls, it means the policy frame matters: a CHIPS Act built around leading-edge logic chips addresses one set of chokepoints; a memory-centric analysis surfaces a broader and differently distributed set.

The Physics of the Bottleneck

To understand why memory has become the limiting factor, it helps to examine the engineering trade-offs inside a modern AI training cluster. A GPU accelerator — Nvidia's H100, for instance — can perform billions of floating-point operations per second. But it can only do so when data is available to feed it. In a large language model training run, the model weights, activations, and optimizer states must be continuously moved between DRAM and the compute die. When memory bandwidth cannot keep pace with the processor's appetite, the accelerator stalls. The result is that doubling the number of GPUs does not halve the training time — a phenomenon engineers call the "memory wall."

The ceiling is not abstract. Analysts estimating the aggregate memory footprint of training runs for frontier models have put figures in the range of hundreds of terabytes per training run, with leading-edge systems consuming multiple petabytes when accounting for checkpointing, gradient accumulation, and validation passes. The numbers explain the pressure on HBM supply. SK Hynix, the South Korean memory manufacturer, has been the primary supplier of HBM3e to Nvidia and has operated at or near full capacity for successive generations. Samsung Electronics and Micron Technology have raced to close the gap, but supply has consistently lagged demand at the frontier.

The implications for competition are structural. A company that cannot secure HBM allocation cannot train frontier-class models, regardless of its compute budget. This shifts leverage toward the memory manufacturers and recalibrates the relative bargaining power of the hyperscalers. Amazon Web Services, Microsoft Azure, and Google Cloud all consume significant quantities of HBM; their ability to reserve supply through long-term agreements has become a competitive moat that is less visible in headline GPU count analyses but arguably more constraining.

China's Position in a Memory-Centric Frame

The memory shift also alters the calculus around China's AI development trajectory. China's semiconductor independence drive has focused considerable capital on logic chip fabrication — the leading-edge fabs that produce TSMC's cutting-edge nodes and that SMIC is working to replicate. That focus is understandable given the export controls that have restricted China's access to advanced logic. But memory tells a different story.

Yangtze Memory Technologies, the Chinese NAND flash champion, has expanded domestic production capacity substantially over the past three years and recently broadened its product portfolio into LPDDR and high-bandwidth memory form factors. CATL, China's dominant battery manufacturer, has in parallel pursued energy infrastructure partnerships that quietly include data-center power management — an indirect but real connection to the physical infrastructure that AI clusters require.

In a memory-centric framing, China's memory sector investments look more strategically significant than a logic-chip-centric analysis would suggest. If HBM supply determines frontier-model capability, and if China can develop viable domestic HBM alternatives, the export control architecture designed around logic chips faces a structural gap. The controls were calibrated to restrict compute; they were not designed to anticipate a future in which memory is the operative constraint. The Chinese industrial policy apparatus has noted this, and the response has been consistent investment in memory capacity as long-term hedge.

Policy in a Memory-Centric World

For governments, the implications are uncomfortable. Industrial policy for semiconductors has clustered around logic chip manufacturing because that is where the dominant narrative placed the strategic chokepoint. The United States CHIPS Act, the European Chips Act, and parallel initiatives in South Korea, Taiwan, and Japan have all been structured with leading-edge logic fabs as the primary beneficiary. A memory-centric analysis suggests the policy map is incomplete.

The reason is that HBM manufacturing is more concentrated than logic by a significant margin. SK Hynix and Samsung together control roughly three-quarters of global HBM production, with Micron holding most of the remainder. This concentration creates supply chain fragility — a single fab disruption, a geopolitical shock in the Korea Strait, or a change in South Korean export policy could cascade through AI supply chains globally. Policymakers who have treated HBM as a commodity to be proxied by general DRAM indices are working with a flawed mental model.

The evidence of that gap is already in the policy record. In 2024 and 2025, successive rounds of export controls targeted advanced logic chips destined for China — H100s, then A100s, then H800 variants — without a corresponding systematic effort to address the memory supply chain. The assumption embedded in those controls was that compute was the scarce resource. If Sandisk's CTO is right, and memory is the operative constraint, the export control architecture has been regulating the wrong variable.

The recalibration is beginning, though slowly. Intelligence and commerce officials in Washington have in recent months held briefings on memory supply chain concentration that acknowledge the gap. The specifics of those discussions have not been made public, but the shift in framing — from logic chips as the sole strategic chokepoint to a dual-chokepoint model that includes HBM — is visible in the changed language of policy documents reviewed by this publication. Whether that recalibration translates into binding controls on memory exports before the next generation of AI systems is trained remains an open question.

What Remains Uncertain

The memory-centric framing is analytically coherent, but it is not unanimous. Some chip architects argue that the solve lies in memory hierarchy — faster on-chip SRAM caches, more efficient data movement between memory tiers — rather than in raw HBM capacity. Others push back that the memory wall will be addressed by architectural innovation inside the accelerator die, rendering the external HBM constraint temporary. Both arguments have technical merit, and the actual trajectory will depend on trade-offs that play out over years, not quarters.

There is also unresolved ambiguity around demand growth rates. Memory demand from AI training is real and documented, but model efficiency improvements may partially counteract the upward pressure on raw capacity. Whether efficiency gains in next-generation architectures outpace the memory demands of larger models is a question the current evidence does not settle. The Sandisk CTO's framing is a directional signal from an industry insider; it is not aForecast.

What the memory-centric turn does clarify is that the AI competition story is more granular than the dominant compute-capture narrative suggests. The chokepoints are multiple, the supply chain geography is complex, and the policy frameworks governments are drafting to address strategic dependency were written for a hardware landscape that is actively changing. The question the next eighteen months will answer is whether that policy lag closes before the next generation of AI systems locks in competitive advantage for whoever controls the memory supply chain.

Desk note: Wire coverage from Nikkei Asia led with the Sandisk CTO's memory-centric framing as a technical inside-baseball story. Monexus has surfaced the structural implications for supply chain concentration, industrial policy, and the export control architecture — angles the wire did not develop.

Wire provenance

This editorial synthesis draws on the following public wire/social posts:

https://en.wikipedia.org/wiki/High-bandwidth_memory
https://en.wikipedia.org/wiki/NAND_flash
https://en.wikipedia.org/wiki/SK_Hynix