Technology
vLLM version 0.28.0 adds new model support and performance optimizations
The software release features optimizations for models like Kimi-K3 and DeepSeek V4 alongside configuration updates and breaking backend changes.
The short version
- vLLM version 0.28.0 was released, incorporating 584 commits from 270 contributors.
- The update introduces support for models including Muse Glimmer and Ling 3.0 Flash, alongside performance optimizations for Kimi-K3 and DeepSeek V4.
- Breaking changes include relocating bitsandbytes support to an out-of-tree plugin and removing outdated runtime parameters.
Key facts
- vLLM version 0.28.0 includes 584 commits from 270 contributors, 76 of whom are new contributors.[Hacker News]
- New model support includes Muse Glimmer, Ling 3.0 Flash, Dots3 NOTE native multimodal support, and Interns2mobius.[Hacker News]
- Optimizations were implemented across hardware backends, including NVIDIA, AMD ROCm, Intel XPU, and CPU platforms.[Hacker News]
- Breaking changes in this release migrate bitsandbytes support to an out-of-tree plugin, upgrade Transformers to version 5.15.0, and remove calculate_kv_scales and override_attention_dtype.[Hacker News]
Sources
- vLLM v0.28.0Hacker News