Samsung Electronics has secured what's believed to be the world's first AI inference technology that embeds computing capability directly into low-power memory, a move expected to accelerate the shift toward a new era of low-power, high-efficiency AI.
According to industry sources on July 22, Samsung Electronics plans to unveil an AI inference solution in August that integrates processing-in-memory (PIM) technology into LPDDR5X, its low-power memory chip.
The announcement builds on research Samsung published in May in the form of an academic paper detailing its LP5X-PIM simulator. The August unveiling is expected to go a step further, presenting concrete performance data on speed improvements and power efficiency, along with a hardware demo and a roadmap toward commercialization.
PIM technology is designed to solve the data bottleneck that occurs when information moves back and forth between a CPU or GPU and memory. The basic idea is to embed a computing unit, known as PIM logic, directly inside the memory itself, allowing calculations to happen where the data lives and minimizing the need to shuttle data back and forth.
PIM technology itself isn't new. It's already been used consistently in high-bandwidth memory (HBM). But Samsung appears set to become the first company to combine PIM with the LPDDR family of chips, which are designed for low power consumption, and apply it specifically to AI inference. That suggests the technology has moved well beyond the research stage and is now aimed at near-term deployment in on-device AI products like smartphones.
In simulations of LPDDR5X-PIM, Samsung found that the chip could perform matrix calculations known as GEMV up to 6.2 times faster than conventional memory. In practical terms, that means AI tasks like analyzing photos or processing text could run more than six times faster.
Samsung is pursuing a two-track strategy, using HBM to target the server market and LPDDR5X-PIM to target personal devices. Unlike HBM, LPDDR5X-PIM is aimed at power-sensitive devices like laptops and smartphones, a segment where on-device AI has become increasingly important as companies look to address privacy concerns and reduce network latency.
The technology could deliver even greater efficiency gains when paired with GAIA, Samsung's in-house AI accelerator built on a 4-nanometer process. In that setup, GAIA, an NPU-centered chip, would handle the bulk of AI computation, while LPDDR5X-PIM provides efficient support at the memory level.
Rivals are pursuing similar approaches. SK Hynix has developed its own PIM product called Accelerator-in-Memory (AiM), building it into graphics DRAM under the name GDDR6-AiM, and has also unveiled AiMX, an AI accelerator that packs together multiple AiM chips.
“Meaningful technologies capable of supporting real-time AI features in AI PCs and flagship smartphones are now starting to emerge in earnest,” an industry source said. “This bodes well for the growth of the on-device AI market.”
