mlx-omni-server
MLX Omni Server is a local inference server powered by Apple's MLX framework, specifically designed for Apple Silicon (M-series) chips. It implements OpenAI-compatible API endpoints, enabling seamless integration with existing OpenAI SDK clients while leveraging the power of local ML inference.
PythonInstall
Supported APIs (2)
Other Python SDKs
Version history All releases ↗
## What's Changed * Logger now obeys `--log-level` command line arg. Added CORS support by @aperaham in https://github.com/madroidmaq/mlx-omni-server…
**Full Changelog**: https://github.com/madroidmaq/mlx-omni-server/compare/v0.5.0...v0.5.1
## What's Changed * Support Anthropic API by @madroidmaq in https://github.com/madroidmaq/mlx-omni-server/pull/66 **Full Changelog**: https://gi…
**Full Changelog**: https://github.com/madroidmaq/mlx-omni-server/compare/v0.4.8...v0.4.9
**Full Changelog**: https://github.com/madroidmaq/mlx-omni-server/compare/v0.4.6...v0.4.8
**Full Changelog**: https://github.com/madroidmaq/mlx-omni-server/compare/v0.4.5...v0.4.6
**Full Changelog**: https://github.com/madroidmaq/mlx-omni-server/compare/v0.4.4...v0.4.5
## What's Changed * Support speculative-decoding/draft-model by @madroidmaq in https://github.com/madroidmaq/mlx-omni-server/pull/52 * Refine model …