Technology
Researchers present software strategies for High Bandwidth Flash at Hot Chips 2026
Presented as an alternative to High Bandwidth Memory, High Bandwidth Flash offers higher capacity on-package but demands extensive software adaptations.
The short version
- Anurag Agarwal and Radhakrishna Giduthuri presented a tutorial at Hot Chips 2026 on how High Bandwidth Flash (HBF) could be applied to machine learning workloads.
- HBF packages flash memory cubes directly with a compute chip to offer higher capacity than High Bandwidth Memory (HBM), though with lower relative bandwidth.
- No commercial HBF products currently exist, and utilizing the technology will require significant software changes to manage block-storage requirements directly on the host.
Key facts
- Anurag Agarwal and Radhakrishna Giduthuri delivered a tutorial on High Bandwidth Flash (HBF) at Hot Chips 2026 using simulations and projections.[Hacker News]
- There are currently no commercial HBF products available on the market.[Hacker News]
- HBF implements standard SSD flash memory technology on a compute chip package, similar to how HBM is integrated, to trade off bandwidth for increased capacity.[Hacker News]
- Unlike standard system memory, HBF behaves like block storage and requires host software to manage DMA data transfers in large, aligned blocks and handle SSD controller duties such as write leveling and data retention.[Hacker News]
- Proposed applications for HBF include storing Mixture of Experts (MoE) models or KV caches, which would then be transferred into HBM or DRAM as needed.[Hacker News]
What remains uncertain
- It is unclear whether the steep software development challenges required to manage block-aligned accesses and custom memory strategies will hinder industry adoption of HBF.[Hacker News]