OpenChat 3.5 7B

OpenChat 3.5 7B is an open chat model from the OpenChat project. The version this entry links to is openchat/openchat-3.5-0106 on Hugging Face. According to the card, it is built on Mistral-7B-v0.1, has 7B parameters, an 8,192-token context length, and the Apache-2.0 license. The OpenChat approach is described in the paper OpenChat: Advancing Open-source Language Models with Mixed-Quality Data, dated September 20, 2023.

Status

Available as open weights on Hugging Face. The card does not announce a deprecation or a successor, and this is a small 2024-era 7B model, so newer open models of similar size are likely to perform better. Check this catalog's Meta Llama 3.1 8B Instruct and Mistral 7B Instruct entries.

Key Features

• 7B on Mistral-7B-v0.1: a small base that runs on a consumer GPU. The card says it can run on a 24 GB GPU.
• 8,192-token context.
• Vendor-reported benchmarks for the 0106 version: average 64.5, MT-Bench 7.8, HumanEval 71.3, MMLU 65.8, GSM8K 77.4 and TruthfulQA 61.0. The card compares it with OpenChat-3.5-1210 (average 63.8), OpenChat-3.5 (61.6) and ChatGPT (March) (61.5). The authors describe these as their own evaluations and reported results.
• Two modes: a default prompt format (GPT4 Correct User: ... <|end_of_turn|>GPT4 Correct Assistant:) and a math mode (Math Correct User: ...). Set <|end_of_turn|> as the end-of-generation token.
• OpenAI-compatible server: the project's own server follows the OpenAI ChatCompletion API.
• License: Apache 2.0.

Use Cases

• Lightweight, self-hosted chat assistants and prototypes.
• Teaching and experiments with small open chat models.
• A baseline for comparing newer 7B models.

Limitation: it is a small model, and the benchmark scores come from the authors, not an independent audit.

Pricing

The weights are free under Apache 2.0. Cost is your own GPU time, or the pricing of a third-party host that serves the model.

Getting Started

1. Open the Hugging Face card.
2. Serve it with the project's OpenAI-compatible server: python -m ochat.serving.openai_api_server --model openchat/openchat-3.5-0106 --engine-use-ray --worker-use-ray. It listens on localhost:18888.
3. Use the model name openchat_3.5 in requests. For math mode, add "condition": "Math Correct".

Frequently Asked Questions

What is OpenChat 3.5 based on?

Mistral-7B-v0.1, according to the Hugging Face card.

What is its context length?

8,192 tokens.

Can I use it commercially?

The card lists the Apache-2.0 license.

What MT-Bench score does it report?

7.8 for OpenChat-3.5-0106, reported by the authors.

Alternatives

• Llama 3.1 8B Instruct: Meta's small instruct model with a 128K context.
• Mistral 7B Instruct: Mistral's own instruct tune of the same base family.
• WizardLM-2 7B: another open 7B chat model built on Mistral-7B-v0.1.

Conclusion

OpenChat 3.5 7B showed how far a 7B open model could go with mixed-quality data fine-tuning. It remains a convenient small model to self-host, but compare it with newer 7B and 8B models on your own tasks. The details are on the model card.