Ollama is now powered by MLX on Apple Silicon in preview

Performance Enhancements with MLX on Apple Silicon

Ollama's integration with Apple's MLX framework marks a significant step forward in optimizing large language models for Apple Silicon. MLX's unified memory architecture allows for more efficient data handling, reducing latency and improving overall performance. This is particularly beneficial for tasks that require high-speed processing, such as coding assistance and personal AI agents.

MLX and Apple Silicon: A Synergistic Relationship

The use of MLX on Apple Silicon leverages the new GPU Neural Accelerators found in the M5, M5 Pro, and M5 Max chips. These accelerators are designed to handle complex machine learning workloads, which translates into faster time to first token (TTFT) and higher generation speeds (tokens per second). This is a key improvement for users running demanding models on Apple devices.

Improved Caching and Memory Utilization

Ollama's cache has been upgraded to enhance responsiveness and efficiency in coding and agentic tasks. The new caching system reuses cache across conversations, reducing memory usage and increasing cache hits. This is especially useful when using shared system prompts with tools like Claude Code.

Support for NVFP4 Quantization

Ollama now supports NVIDIA's NVFP4 format, which allows for higher quality responses while reducing memory bandwidth and storage requirements. This format is becoming increasingly important as more inference providers adopt it for production environments. It also enables Ollama to run models optimized by NVIDIA's model optimizer, further enhancing performance and accuracy.

Future Developments and Model Support

Ollama is actively working on expanding support for future models and improving the import process for custom models. This will make it easier for users to integrate their own fine-tuned models into Ollama. Additionally, the list of supported architectures will continue to grow, ensuring broader compatibility with various machine learning frameworks.

Source: Ollama. Analysis based on publicly available information; no hands-on testing is claimed.