NVIDIA: Llama 3.1 Nemotron 70B Instruct

Llama 3.1 Nemotron 70B Instruct is a 70B-class language model that NVIDIA customized from Meta's Llama-3.1-70B-Instruct to improve the helpfulness of its responses. According to its Hugging Face model card, NVIDIA trained it with RLHF, specifically REINFORCE implemented in NeMo Aligner, using the Llama-3.1-Nemotron-70B-Reward model and HelpSteer2-Preference prompts. The card is built with Llama, and the license is the NVIDIA Open Model License together with the Llama 3.1 Community License Agreement.

Key features

• Helpfulness tuning: the card says the model targets general-domain instruction following and helpfulness, and that it has not been tuned for specialized domains such as math.
• Long input: input up to 128K tokens, with output up to 4K tokens.
• NVIDIA-reported results: as of October 1, 2024, the card reports Arena Hard 85.0, AlpacaEval 2 LC 57.6 and MT-Bench (GPT-4-Turbo) 8.98. These are NVIDIA's own evaluations.
• Open serving options: the card lists Hugging Face Transformers, vLLM and SGLang, with hosted inference at build.nvidia.com.

Model specs

Use cases

• General chat assistants where helpful, well-structured answers matter.
• Research on RLHF and reward-model-guided alignment.
• Self-hosted deployments on NVIDIA GPUs.

Getting started

1. Review the license terms on the model card.
2. Plan for hardware: NVIDIA says it was tested with 2x A100 80GB GPUs, recommends two or more 80GB NVIDIA Ampere-or-newer GPUs, and needs at least 150GB of free disk space.
3. Load it in bfloat16 with device_map="auto" and the chat template, or serve it with vLLM or SGLang.
4. To try it without hardware, use the free hosted endpoint at build.nvidia.com.

FAQ

What is Llama 3.1 Nemotron 70B Instruct based on?

Meta's Llama-3.1-70B-Instruct, fine-tuned by NVIDIA with RLHF.

What are its benchmark scores?

NVIDIA reports Arena Hard 85.0, AlpacaEval 2 LC 57.6 and MT-Bench 8.98 as of October 1, 2024.

Is it good at math?

The card says it has not been tuned for specialized domains such as math.

What license does it use?

The NVIDIA Open Model License, with the Llama 3.1 Community License Agreement as additional information.

Alternatives

• Llama 3.1 70B Instruct: the base model it was tuned from.
• Qwen2.5 72B Instruct: another open 70B-class chat model.
• Nous Hermes 3 405B Instruct: a larger Llama 3.1 derivative.

Conclusion

Llama 3.1 Nemotron 70B Instruct shows how reward-model-guided RLHF can lift a strong open base model. Read the official model card for details, and browse the models category for newer options.