← Latest briefing

Technology

vLLM version 0.28.0 adds new model support and performance optimizations

The software release features optimizations for models like Kimi-K3 and DeepSeek V4 alongside configuration updates and breaking backend changes.

The short version

  • vLLM version 0.28.0 was released, incorporating 584 commits from 270 contributors.
  • The update introduces support for models including Muse Glimmer and Ling 3.0 Flash, alongside performance optimizations for Kimi-K3 and DeepSeek V4.
  • Breaking changes include relocating bitsandbytes support to an out-of-tree plugin and removing outdated runtime parameters.

Key facts

  • vLLM version 0.28.0 includes 584 commits from 270 contributors, 76 of whom are new contributors.[Hacker News]
  • New model support includes Muse Glimmer, Ling 3.0 Flash, Dots3 NOTE native multimodal support, and Interns2mobius.[Hacker News]
  • Optimizations were implemented across hardware backends, including NVIDIA, AMD ROCm, Intel XPU, and CPU platforms.[Hacker News]
  • Breaking changes in this release migrate bitsandbytes support to an out-of-tree plugin, upgrade Transformers to version 5.15.0, and remove calculate_kv_scales and override_attention_dtype.[Hacker News]

Sources