Technology
Developer releases open-source tool to run 104GB Qwen3.8 AI model on lower-memory Macs
Slotstream streams model weights from storage to enable running the 104GB Qwen3.8-Flash-Next model on Apple Silicon systems with as little as 8GB of RAM.
The short version
- A developer released Slotstream, an open-source Swift tool designed to run the 104GB Qwen3.8-Flash-Next model on memory-constrained Apple Silicon Macs.
- The software streams model weights from storage and dynamically manages memory, achieving roughly 12 tokens per second on a 48GB Mac.
- Slotstream includes a local server compatible with Ollama and OpenAI API endpoints, requiring macOS 14 or newer and at least ~110GB of free disk space.
Key facts
- Slotstream is an open-source Swift tool that enables running the 4-bit 104GB Qwen3.8-Flash-Next model on Apple Silicon Macs with a planned memory floor of 8.1GB.[Hacker News]
- Tested on an M5 Pro Mac with 48GB of RAM, the tool demonstrated a warm decode speed of approximately 12 tokens per second with peak memory auto-capped near 32GB to 33GB.[Hacker News]
- The software requires Apple Silicon, macOS 14 or higher, and around 110GB of free disk space to store the 103.8GB weight download.[Hacker News]
- Slotstream provides an integrated API server running on port 11434 that supports chat and generate endpoints compatible with OpenAI SDKs and Ollama clients.[Hacker News]
What remains uncertain
- Performance estimates for Mac configurations below 48GB of RAM are derived from hardware curves rather than actual testing on physical lower-tier hardware.[Hacker News]
- While the installer was tested, runtime stability on macOS 14 and macOS 15 remains unverified in long-term use.[Hacker News]