Llama v2 70B Chat

Llama 2 70B Chat is the dialogue-tuned version of the largest model in Meta's Llama 2 family. According to the Hugging Face model card, the models were trained between January and July 2023, and the accompanying paper was published on July 18, 2023. The 70B model has a 4K-token context and was pretrained on 2.0 trillion tokens.

Status

Superseded. Meta has since released Llama 3 and Llama 3.1, which are in this catalog as Llama 3 70B Instruct and Llama 3.1 70B Instruct. The Llama 2 card remains on Hugging Face and the weights are still downloadable after accepting Meta's license, and the card does not announce a deprecation date.

Key Features

• Size and context: 70B parameters (the Hub lists 69B) and a 4K-token context window.
• Chat tuning: the chat variants are tuned for assistant-like dialogue.
• English focus: the card states Llama 2 is intended for commercial and research use in English, and that other languages are out of scope.
• Knowledge cutoff: pretraining data ends in September 2022, with some tuning data up to July 2023.
• Meta-reported results: the base 70B model scores 68.9 on MMLU, according to the card. The card's safety evaluation lists a TruthfulQA score of 64.14 and a ToxiGen score of 0.01 for the chat 70B model. These are Meta's own figures.
• License: a custom Llama 2 Community License. Access requires sharing contact information with Meta.

Use Cases

• Reproducing results from 2023-era research that used Llama 2 70B Chat.
• A baseline when measuring improvements in newer Llama models.
• Existing fine-tunes and tools that still depend on the Llama 2 architecture.

Limitation: the 4K-token context is far shorter than modern models, and the card limits intended use to English.

Pricing

Meta distributes the weights free of charge under its license. Hosted API prices depend on the provider and are not quoted here.

Getting Started

1. Request access on the Hugging Face card and accept Meta's license.
2. Load the Llama-2-70b-chat-hf weights with Transformers on multi-GPU hardware.
3. For new projects, start from Llama 3.1 70B Instruct, which has a 128K context according to Meta's Llama 3.1 card.

Frequently Asked Questions

What is Llama 2 70B Chat's context length?

4K tokens, according to the Hugging Face card.

Is Llama 2 70B still available?

The weights remain on Hugging Face after you accept Meta's license. Meta has released newer Llama models since.

Which languages does it support?

The card says Llama 2 is intended for use in English.

What replaced Llama 2 70B?

Meta's later Llama 3 and Llama 3.1 models, such as Llama 3.1 70B Instruct.

Alternatives

• Llama 3.1 70B Instruct: the 70B successor in the Llama 3.1 family.
• Llama 3 70B Instruct: the direct Llama 3 generation 70B model.
• Mistral 7B Instruct: a far smaller open instruct model.

Conclusion

Llama 2 70B Chat is a historically important open chat model, but its 4K context and English focus make it a poor fit for new work. See the model card for details and move to Llama 3.1 for current projects.