HBM Demand Will Move Late Despite the AI Slowdown Debate

A debate over 'pumping the brakes' is spreading through the U.S. AI industry. Yet demand for HBM (high-bandwidth memory, a high-performance memory that helps AI read and write data quickly) is not being shaken for now. Because of long-term supply contracts and a pre-order structure, any changes may take even longer to materialize.

According to the semiconductor industry on the 20th, a new investment variable has emerged in the AI infrastructure market: inference demand for putting trained models into real-world services, on top of the computing power needed to train large language models. As AI service usage continues to grow, the direction of investment could shift as well. As investment concentrated on training large models moves toward inference for delivering actual services and agentic AI, the nature of memory demand could also change.

Against this backdrop, the spread of agentic AI is shifting the center of gravity of inference demand. As AI carries out multiple tasks in succession based on user requests, inference requests are changing from simple queries and answers to repetitive tasks. As a result, the amount of data that must be processed during inference and memory requirements are structurally increasing. Market research firm TrendForce analyzed that AI server demand, driven by the expansion of LLM training and AI inference, is boosting demand for HBM3E and high-capacity memory.

That said, increased inference demand does not translate entirely into HBM demand. Inference systems rely not only on HBM but also on server DRAM such as DDR5, SSDs, and new memory tiers. Memory makers are also expanding their product lineups for AI inference beyond HBM. SK hynix unveiled HBF, which can handle large volumes of data between HBM and SSDs, positioning it as a memory tier to address the growing data processing and capacity demands of inference.