Alternatives to DeepInfer

NInfer is an Apache-2.0 C++/CUDA engine for selected Qwen checkpoints on a single RTX 5090, with OpenAI- and Anthropic-compatible local APIs.

oMLX is an Apache-2.0 Apple Silicon inference server with SSD KV cache and a macOS menu bar app for coding agents.

halogen-flash-server is a Strix Halo inference server that runs Qwen3.8-Flash-Next with an OpenAI-compatible API.

mlx-serve is a MIT Zig inference server for Apple Silicon that runs MLX and GGUF locally with OpenAI and Anthropic APIs, no Python.

An efficient local LLM inference library built around the EXL3 quantization format, running large open models on consumer GPUs with tensor and expert parallelism, and powering the TabbyAPI server.

Open-source local inference engine for Qwen3.8-Flash-Next on Windows and Linux, with a chat interface and compatible agent APIs.

Local models plus Ollama cloud. Not a 30-minute Llama3 tutorial.

Document agents plus LlamaParse. 1,000 credits = $1.25. OSS framework is MIT.

Tencent’s model compression toolkit combining quantization, distillation, sparse attention and speculative decoding workflows.

MIT agent framework. Hosted traces are LangSmith Developer $0 / Plus $39 per seat. Not LangServe-as-the-product.

TypeScript kit for MCP servers. Software $0. Not a 5-minute Claude Desktop-only tutorial.

MCP India Stack is a MIT offline-first MCP server with 76 Indian tax, identity, and legal tools for agents.

Open-source RAG plus agents. Cloud signup exists; no public dollar table. Not a generic document scanner.

Self-hosted AI gateway for coding tools. Compare 9Router with OmniRoute and OpenRouter, and check costs, tool calls, fallback, and migration steps.