code-testing-generator

code-testing-generator is an open-source agent from Microsoft's .NET team that writes unit tests and then proves they actually work. A plain "generate unit tests" prompt leaves too much open: which code needs tests, which framework the project uses, where tests go, how the build finds them, and what they should assert. This agent settles those decisions by reading the repository before it writes anything, then it plans, writes, runs, and checks the tests it produces. It ships inside the MIT-licensed dotnet/skills repository in the dotnet-test plugin and is polyglot, covering .NET, Python, Go, TypeScript, Java, Rust, and more.

Key Features

• Repository-aware planning: instead of one-shot generation, it runs a Research-Plan-Implement pipeline. It detects language and test framework, reads existing tests for conventions, and finds the real build and test commands, which fixes the failure mode where a project builds locally but never runs in CI because nothing registered it.
• Three strategies: direct (write and validate immediately), single pass (research and plan once, then implement), and iterative (repeat for large scopes or a coverage goal).
• Verification gate: before reporting done it runs five checks, including reasoning about small changes that should make the tests fail (a lightweight form of mutation testing), looking for weak or missing assertions, mapping every requested scenario to a test, building the full workspace and running the full suite, and confirming the repo's own test command discovers the new tests.
• Safe by design: it never modifies production code and avoids tests that call external URLs, bind ports, or depend on exact timing.
• Runs where you already work: available in GitHub Copilot CLI, and in Visual Studio Code and VS Code Insiders through plugin support, with Visual Studio support on the way.

Benchmark Results

On Microsoft's internal benchmark of 152 tasks from real repositories, the agent completed 140 (92.1%) versus 120 (78.9%) for stock GitHub Copilot on the same model and prompts, which is 63% fewer failures. The gain is concentrated on vague prompts: 79 of 89 (88.8%) versus 59 (66.3%), and on diff-targeted tasks it passed 15 of 15 while stock Copilot passed none. It also generated 2.3% fewer tests (6,963 vs 7,129) at effectively identical line coverage (72.4% vs 72.2%) and completed tasks about 5.5% faster. On the harder external SWE Atlas benchmark the gap held, 16/44 versus 12/44.

Use Cases

• Backfilling tests on untested modules and raising coverage before a release gate.
• Reviewing a pull-request diff to make sure the new code is actually covered.
• Regulated or audit-heavy estates in financial services, healthcare, and the public sector that need demonstrably meaningful tests.
• Polyglot monorepos that want one standard set of conventions.

Problems It Solves

1. Tests that pass but assert nothing: the agent checks assertions and mutation, not just a green CI run.
2. Unregistered test projects: it finds the real build and test commands so CI actually runs what you wrote.
3. Vague prompts: it resolves the framework, location, and convention decisions itself.

Pricing

Open source and free.

Getting Started

1. Add the plugin to GitHub Copilot CLI: /plugin marketplace add dotnet/skills, then /plugin install dotnet-test@dotnet-agent-skills.
2. Restart the CLI and select code-testing-generator from the list of agents.
3. Try "generate unit tests" and review the plan, tests, and final checks.

Frequently Asked Questions

What does it test?

Unit tests only. Integration, end-to-end, browser, and performance tests are out of scope for now.

Is it free?

Yes, MIT-licensed open source.

Does it touch production code?

No. It reads it for context but only writes test files.

Alternatives

• Aider: a general CLI coding agent for prompts and edits.
• Claude Code: a terminal coding agent you can pair with custom skills.
• codex-cli: OpenAI's agentic CLI, which you can also extend with skills.

Tips

1. Use the agent for the "vague prompt" cases, which is where it beats a plain Copilot call by the largest margin.
2. Review the final checks, not just the green test count: assertion quality is the real win.
3. Fork the language guidance if your repo uses internal frameworks so the agent matches your conventions.

Conclusion

code-testing-generator is Microsoft's answer to "tests that pass but prove nothing." By researching the repository first and then verifying every test it writes, it turns a copy-paste prompt into a reliable trust loop, and the numbers hold up best exactly where generic assistants struggle. Try it in GitHub Copilot CLI or read the technical post.