SK Hynix announced its vision to become a 'full-stack AI memory creator,' covering everything from high-bandwidth memory (HBM) to high-capacity storage, tailored for the AI era. As models grow larger and agentic AI expands, memory demand continues to rise, necessitating layered solutions rather than single products.
Im Wi-cheol, VP of Solution Advanced Technology at SK Hynix, emphasized at the *Electronic Times* 'Tech Summit 2026' on the 29th, “While processor performance has rapidly advanced, memory has lagged, perpetuating the 'memory wall.' Now, a single memory type (HBM) alone cannot cover AI workloads.”
In large language model inference, the input understanding phase (prefill) is GPU-compute-centric, but the decoding phase—generating responses token by token—requires repeated model weight reads, making performance heavily dependent on memory bandwidth. Even high-performance GPUs face limitations here.
Bandwidth isn't the only challenge. As models grow, capacity demands also surge. SRAM-based accelerators offer high bandwidth but struggle with capacity. Early ChatGPT-level models required 350 gigabytes (GB), needing multiple wafers and hundreds of chips.
Currently, HBM is essential to balance high bandwidth and capacity. By stacking memory dies, SK Hynix increases capacity per unit area, while expanding I/O pin counts enables higher bandwidth with lower power.
Token demand is also exploding. Inference models and agentic AI generate hundreds of times more output tokens than before. Capacity scales with token count, but compute and bandwidth demands rise even faster. While algorithms reduce active memory per token or reuse cached states, longer contexts and concurrent requests increase total memory traffic.
Agentic AI structurally amplifies this demand. Traditional chatbots consume resources only when users input queries, but agents operate 24/7, interacting with tools and other agents once tasks are delegated.
SK Hynix is building a tiered lineup: active data (frequently used model weights) in HBM, high-capacity/warm data (large but not always accessed) in HBF, and less-frequent data in CXL pooled memory and SSDs. The strategy allocates frequently used data to fast memory, large-capacity data to mid-tier storage, and infrequent data to cheaper storage.
PIM (Processing-In-Memory), which handles some computations within memory, is another pillar of the full-stack lineup. While HBF expands capacity, PIM boosts bandwidth.
In conventional architectures, data moves from memory to processors (GPU/CPU) for computation, then back—creating bandwidth and power overhead. PIM performs partial computations directly in memory chips, reducing data movement and effectively increasing bandwidth.
Im concluded, “Software efficiency alone is insufficient; hardware innovation is critical. Our full-stack memory solutions will make AI more scalable and accessible.”
