Alternatives to mlx-kquant

mlx-serve is a MIT Zig inference server for Apple Silicon that runs MLX and GGUF locally with OpenAI and Anthropic APIs, no Python.

oMLX is an Apache-2.0 Apple Silicon inference server with SSD KV cache and a macOS menu bar app for coding agents.

llama-modchip is an Apache-2.0 local inference toolkit for running selected sparse MoE GGUF models on CPU or hybrid hardware.

Tencent’s model compression toolkit combining quantization, distillation, sparse attention and speculative decoding workflows.

DeepSeek's Apache-2.0 offline inference framework for DeepSeek V4.1 Flash, targeting large-scale serving on NVIDIA SM100 GPUs or Huawei Ascend 950 NPUs.

halogen-flash-server is a Strix Halo inference server that runs Qwen3.8-Flash-Next with an OpenAI-compatible API.

MIT agent framework. Hosted traces are LangSmith Developer $0 / Plus $39 per seat. Not LangServe-as-the-product.

Document agents plus LlamaParse. 1,000 credits = $1.25. OSS framework is MIT.

TypeScript kit for MCP servers. Software $0. Not a 5-minute Claude Desktop-only tutorial.

NInfer is an Apache-2.0 C++/CUDA engine for selected Qwen checkpoints on a single RTX 5090, with OpenAI- and Anthropic-compatible local APIs.

Open-source RAG plus agents. Cloud signup exists; no public dollar table. Not a generic document scanner.

Open-source local inference engine for Qwen3.8-Flash-Next on Windows and Linux, with a chat interface and compatible agent APIs.

NVIDIA's open-source LLM vulnerability scanner with dozens of plugins and thousands of prompts for probing jailbreaks, prompt injection, and data leakage.

MCP India Stack is a MIT offline-first MCP server with 76 Indian tax, identity, and legal tools for agents.