Ternary Bonsai 2

Ternary Bonsai 2 is Prism ML's ternary packing of Qwen3.8-27B for llama.cpp. Weights live at prism-ml/Ternary-Bonsai-2-27B-gguf, created 2026-09-16, Apache 2.0. On 2026-09-19 Hugging Face listed 920 likes and 405,609 downloads on that GGUF repo. Those counts are heat signals, not site traffic. r/LocalLLaMA had a 1.6K-upvote launch thread plus follow-ups on 2026-09-19.

The language backbone is 27.36B parameters with hybrid attention (~75% linear). Context is 262K tokens from the base model. Compare Qwen3.8-Flash-Next if you wanted the sparse cloud MoE instead of a dense 27B on disk.

Key Features

• Ternary g128: weights in {-1, 0, +1} with FP16 scales per 128 weights. The card states 1.72 bits/weight ideal and 5.95 GB for GGUF PTQ1_0 (1.75 bits) or 7.21 GB for PQ2_0.
• First-party scores: the card reports 98.2% of FP16 intelligence retained (84.78 average on 14 thinking-mode benchmarks) and about 47 tok/s on an Apple M5 Max. Those numbers are Prism ML's, not an audit.
• Optional vision: a separate Q8_0 mmproj (~0.63 GB) loads only for images.
• Backends: Prism ML's llama.cpp fork (CUDA, Metal, CPU) and an MLX 2-bit companion.
• Sampling: thinking mode temperature=1.0, top_p=0.95; instruct temperature=0.7, top_p=0.80. Default reasoning effort is xhigh. low is not supported.

Limitation: stock llama.cpp will not load these packs. You need the Prism ML fork (or MLX fork). r/LocalLLaMA users also reported overthinking loops and high token use; that is community report, not a card claim.

Use Cases

• Laptops and one GPU that cannot hold 54 GB FP16 of Qwen3.8-27B.
• Local agents that still need thinking and tool-calling, per the card's agentic score of 74.92.
• Skip if you wanted a hosted API. Use Qwen3.8-Flash-Next.

Pricing

Getting Started

1. Read the demo README. It is the source of truth for pinned binaries.
2. Download PTQ10 (5.95 GB) or PQ20 (7.21 GB) from Hugging Face.
3. Serve with the Prism ML llama.cpp or MLX build, not stock llama.cpp.
4. Add the mmproj only when you pass an image.

First-party start: the model card and the whitepaper PDF.

Frequently Asked Questions

Is this a new architecture?

No. The card says architecture is unchanged from Qwen3.8-27B. The change is ternary packing plus custom kernels.

Can I use vanilla llama.cpp?

Not according to the card. Packed weights need the matching Hadamard runtime.

Do I need the vision tower?

No. Text serving never loads the mmproj.

Alternatives

• Qwen3.8-27B: full-precision dense base.
• Qwen3.8-Flash-Next: sparse open MoE, not a 6 GB laptop file.
• MiniMax Code: a terminal agent, not a weight pack.

Tips

1. Pin the demo's binaries. Forks move.
2. Start with PTQ10 unless a kernel table says PQ20 is faster on your GPU.
3. Treat 98.2% and 47 tok/s as card claims until you bench your box.

Conclusion

Ternary Bonsai 2 is a 6 GB ternary Qwen3.8-27B pack for people who already run llama.cpp or MLX and will install Prism ML's fork. Download the GGUF, follow the demo README, and bench it yourself. It is not a drop-in for stock llama.cpp.