TensorSharp

TensorSharp is a .NET 10 engine for running GGUF models locally. It offers a command-line interface, browser chat, Ollama-compatible and OpenAI-compatible APIs, and an engine that can be embedded in a .NET application. Its BSD 3-Clause repository is useful for developers who want to connect local inference to existing C# software.

Features and use cases

The project combines managed C# CPU kernels with native CUDA, Metal and Vulkan backends. It covers text and reasoning models, multimodal input, embeddings and selected media-generation models. Compatibility varies by model family and backend: the presence of a feature in the source does not mean every downloadable package supports it.

Serving features include continuous batching and a paged KV cache with shared prefixes. Supported models can also use speculative decoding and multi-GPU placement. TensorAgent is a related application built on the engine; desktop packages and mobile source builds have different prerequisites.

A practical first run

Start with the official releases and select a CLI or server archive for your operating system. Read the supported-model table before downloading weights. Use one documented model and a short prompt, record memory consumption, then try the same prompt through the API. This separates installation problems from model-specific failures.

Building from source requires the full .NET 10 SDK, CMake and the relevant GPU toolchain. Apple Silicon uses the Metal backend; compatible NVIDIA systems can use CUDA. Developers without a GPU can evaluate CPU backends before investing in hardware.

Cost and limitations

The repository uses BSD 3-Clause. Local execution still costs storage, memory, hardware and electricity; no hosted subscription price is asserted here. Benchmark numbers in the README apply to the recorded model, quantization, hardware and workload, rather than guaranteeing the same result on your machine.

The server defaults to listening on all interfaces and has no built-in authentication or TLS. Keep an initial test private and configure authenticated HTTPS access before sharing it. Review AgentHost file and shell tools separately from ordinary chat.

FAQ and alternatives

Does every GGUF model work? No. Check the project's model and backend coverage tables.

Is TensorAgent the engine? TensorAgent is an application using TensorSharp, with separate packaging requirements.

Compare Ollama when you want a familiar local-model service, or browse agent infrastructure for integration options. TensorSharp is particularly relevant when embedding inference in .NET matters. The next step is a reproducible small test using the official getting-started guide, followed by your own workload evaluation.