Alternatives to llama-modchip

NInfer is an Apache-2.0 C++/CUDA engine for selected Qwen checkpoints on a single RTX 5090, with OpenAI- and Anthropic-compatible local APIs.

DeepSeek's Apache-2.0 offline inference framework for DeepSeek V4.1 Flash, targeting large-scale serving on NVIDIA SM100 GPUs or Huawei Ascend 950 NPUs.

mlx-serve is a MIT Zig inference server for Apple Silicon that runs MLX and GGUF locally with OpenAI and Anthropic APIs, no Python.

oMLX is an Apache-2.0 Apple Silicon inference server with SSD KV cache and a macOS menu bar app for coding agents.

Open-source local inference engine for Qwen3.8-Flash-Next on Windows and Linux, with a chat interface and compatible agent APIs.

halogen-flash-server is a Strix Halo inference server that runs Qwen3.8-Flash-Next with an OpenAI-compatible API.

Local models plus Ollama cloud. Not a 30-minute Llama3 tutorial.

Tencent’s model compression toolkit combining quantization, distillation, sparse attention and speculative decoding workflows.

ExLlamaV3 is a local LLM inference library built on the EXL3 quantization format, fitting large open models on consumer GPUs with tensor parallelism.

MIT agent framework. Hosted traces are LangSmith Developer $0 / Plus $39 per seat. Not LangServe-as-the-product.

Document agents plus LlamaParse. 1,000 credits = $1.25. OSS framework is MIT.

TypeScript kit for MCP servers. Software $0. Not a 5-minute Claude Desktop-only tutorial.

MCP India Stack is a MIT offline-first MCP server with 76 Indian tax, identity, and legal tools for agents.

Open-source RAG plus agents. Cloud signup exists; no public dollar table. Not a generic document scanner.