Technology
Perplexity announces plans to open source Lily AI engine for Apple silicon
The company reported speed gains for a specific model using a custom Rust runtime and Metal kernels, though the code has not yet been publicly released.
The short version
- Perplexity developed Lily, a specialized local inference engine designed for Apple silicon and the Qwen3.6-35B-A3B model.
- The engine relies on a Rust runtime and custom Metal kernels, bypassing PyTorch and Apple's MLX framework entirely.
- Perplexity claims significant speed improvements over MLX-LM in internal testing, but the code is not yet publicly available for independent verification.
Key facts
- Perplexity created a local AI engine named Lily optimized specifically for Apple silicon hardware and the Qwen3.6-35B-A3B model.[Slashdot]
- Lily's execution path runs via a Rust runtime and custom Metal kernels without using MLX or PyTorch.[Slashdot]
- Perplexity reported that Lily was 23 percent faster at prompt processing and 35 percent faster at token generation compared to MLX-LM in internal tests on an M5 Max MacBook Pro with 128GB of memory.[Slashdot]
- Unlike MLX-LM, which accommodates a wider variety of models and architectures, Lily is narrowly focused on specific optimizations.[Slashdot]
- Perplexity stated that it plans to release Lily as open-source software.[Slashdot]
What remains uncertain
- Perplexity has not specified a release date for the open-source code, and its performance claims currently rely entirely on internal testing without external verification.[Slashdot]