mini-AGI

mini-AGI is a research project by Alexey Borsky (volotat on GitHub) that asks a narrow question with a lot of consequences: can a language model keep learning from a single stream of data, on one consumer GPU, without catastrophically forgetting what it already knew? The first-party repository was created on 2026-09-19 and GitHub listed 305 stars and 22 forks on 2026-09-22. It is MIT licensed, and the README is blunt that this is currently a toy-level model rather than something you would use for real work.

What It Actually Is

It is a byte-level language model that trains from scratch, batch size one, on a GPU with at least 8GB of VRAM. The reported size during the run was around 530M parameters and growing, but the parameter count is not really the interesting number. The model stores its experts as ordinary files on disk and pages them onto the card as needed, so capacity is bounded by free disk space rather than by VRAM. It grows new experts while training when it runs short, prunes the ones nothing asks for, and serves through exactly the same code path it trains through.

How the Architecture Works

Characters do not pass through a fixed stack. They pass through two dense prelude blocks, then through one recurrent block applied up to 24 times, with each application choosing its own experts from a shared pool. The latent state between applications is merged with the embedded input each round, so the loop cannot drift away from the text being read.

• Adaptive depth. A halting head scores every character at every row, and the character stops as soon as another row would not change the answer. The README measures about 8.0 rows per character while reading and 9.9 while writing.
• Routing per application, not per character. Each of up to 26 block applications picks its own top-8 experts, so one character touches far more of the pool than a single top-8 would suggest.
• No labelled experts. Nothing assigns a subject to an expert; soft top-k routing distributes capability by itself. The trade-off is that capabilities share parameters and can therefore interfere.

The Forgetting Result

The measurement the whole design rests on is a learning-rate split. Reading 524,000 characters of chess with the trunk at the same rate as the experts takes the seven unrelated subjects from 1.12 to 3.73 nats. Running the trunk at 0.1x the experts' rate brings that forgetting down to +0.0067 nats, which the project reports as 99.84% of progress retained against a chance baseline. Two further findings matter: freezing the working set is not what prevents forgetting, and the damage reads more like displacement than destruction, since three quarters of it returns after 131,000 characters of rereading.

Running It

Clone the repository, install the Python dependencies, and start with train.py and the supplied config.yaml, then use serve.py to talk to the model. A corpus builder under corpora/ assembles streams from code, arithmetic, chess games, chat, and reasoning data. The repository README documents the paging rules, the growth brakes (room, used, earning, fits, honest), and the pruning rule that an expert is dead when nothing addresses it, not when its gate is small.

Limitations

• Weights are not published yet. The README says the run is still working through its first pass over the corpus and weights go up a couple of weeks later.
• Toy-level output. The project's own samples are short, grammatically fine, and repetitive.
• You are training, not loading. There is no checkpoint to download, so the entry path is a training job on your own machine.
• One benchmark family. Its claims come from the author's own probes on his own corpus, not from a standard suite.

FAQ

Is this a model I can download?

Not yet. The code is MIT licensed and public, but the trained weights were still unreleased as of 2026-09-22.

What hardware do I need?

The README targets a PC or laptop with at least an 8GB VRAM GPU, and the design deliberately trades VRAM for disk space.

Is it comparable to a frontier model?

No, and the README says so directly. The contribution is the continual-learning mechanism, not the output quality.

Alternatives

• Bespoke Nimble: another small, self-hostable model, but trained for structured decisions rather than continual learning.
• LFM2.5 2.6B: a small open-weight model you can actually run today for practical work.
• LongCat 2.0: a much larger open model to compare on long-context behaviour.

Conclusion

mini-AGI is worth reading because it isolates one hard problem and reports the measurement that solves it, on hardware almost anyone owns. Treat it as an experiment with a reproducible recipe rather than a tool: the code is permissive, the weights are still coming, and the output quality is not the point.