Technology
Samsung unveils processing-in-memory DRAM package to speed local AI inference
The LPDDR5X-PIM package delivers 614 GB/s of internal bandwidth, though software integration challenges remain.
The short version
- Samsung presented a 16 GB LPDDR5X Processing-in-Memory (PIM) chip at Hot Chips 2026 designed to accelerate local large language model inference.
- Internal bank-level compute units provide up to 614 GB/s of internal bandwidth while maintaining compatibility with standard DRAM memory controllers.
- Vendor benchmarks running Llama 3.1 8B demonstrated a 3.01x throughput increase compared to standard LPDDR5X.
- Broader adoption faces significant hurdles, including incompatible quantization formats, lack of operating system memory layout support, and unannounced pricing.
Key facts
- Samsung presented its 16 GB LPDDR5X-PIM memory package at Hot Chips 2026, featuring 614 GB/s of internal bank-level bandwidth compared to 76.8 GB/s across its external pins.[Hacker News]
- The chip uses Address Align Mode, which allows existing memory controllers to switch between standard DRAM and PIM modes without requiring redesigned memory controllers.[Hacker News]
- In a vendor benchmark using Llama 3.1 8B at a 320-token context, execution time dropped from 12.3 seconds to 5.4 seconds and throughput increased from 27 to 81.3 tokens per second.[Hacker News]
- Common inference software formats like GGUF k-quants are currently incompatible with Samsung's uniform precision hardware modes.[Hacker News]
- Mainstream operating systems such as Linux, macOS, and Windows currently lack the bank-aware contiguous physical memory allocators required by the hardware.[Hacker News]
What remains uncertain
- Samsung has not disclosed pricing for the LPDDR5X-PIM package, leaving its cost-effectiveness relative to wider memory interfaces unverified.[Hacker News]
- It is unknown how much model quality or accuracy may be lost when re-quantizing existing GGUF models into PIM-native uniform layouts.[Hacker News]