Qwen2.5-Coder-32B logo

Qwen2.5-Coder-32B

Visit

Qwen2.5-Coder-32B is Alibaba's Apache 2.0 base code model (32.5B parameters, 5.5T training tokens, up to 131,072-token context), released November 2024.

Share:
View alternatives

Qwen2.5-Coder-32B

Qwen2.5-Coder-32B is the largest base model in Alibaba's Qwen2.5-Coder family, announced in the Qwen blog post dated November 12, 2024 (the catalog's website link is the same post at qwen.ai). The Hugging Face model card describes it as a base model from the pretraining stage, built on Qwen2.5-32B, with 32.5B total parameters (31.0B non-embedding). It is not an instruction-tuned chat model, and Qwen says base models are not recommended for conversation. It is a previous-generation model, still downloadable.

Key Features

  • Training and size: 5.5 trillion training tokens including source code, text-code grounding and synthetic data; 64 layers with grouped-query attention (40 query heads, 8 KV heads).
  • Context: up to 131,072 tokens. The default config covers 32,768 tokens, and longer inputs need YaRN with a factor of 4.0. vLLM supports only static YaRN, which may hurt short-text performance.
  • License: Apache-2.0.
  • Family: the blog lists sizes 0.5B, 1.5B, 3B, 7B, 14B and 32B, each in Base and Instruct versions. Over 40 programming languages are covered by the McEval benchmark.
  • Vendor claims: for the Instruct version, Qwen described it as the current state-of-the-art open-source code model with coding ability comparable to GPT-4o, and reported an Aider score of 73.7 and a McEval score of 65.9. These are Alibaba's figures, not independent audits.

Use Cases

  • Fill-in-the-middle code completion and code-focused continued pretraining.
  • A base for custom post-training such as SFT or RLHF.
  • Code agents and developer tools, after instruction tuning or by using the Instruct sibling.

Limitation: the base model is not meant for chat. For conversational use pick Qwen2.5-Coder-32B-Instruct. Use a recent Transformers release, since versions before 4.37.0 raise a KeyError: 'qwen2'.

Pricing and availability

The weights are free under Apache-2.0. This entry quotes no hosted price because none was confirmed on a first-party page.

Getting Started

  1. Download Qwen/Qwen2.5-Coder-32B from Hugging Face.
  2. Load it with Transformers 4.37.0 or later.
  3. Enable the YaRN rope_scaling setting only when you need long context.

FAQ

Is Qwen2.5-Coder-32B open source?

Yes, it is released under the Apache-2.0 license.

What is its context length?

Up to 131,072 tokens with YaRN; the default config is 32,768 tokens.

What is the difference from the Instruct model?

This is the base model; Qwen2.5-Coder-32B-Instruct is tuned for chat and instructions.

Is there a Qwen coding CLI?

Yes, see Qwen Code.

Alternatives

Conclusion

Qwen2.5-Coder-32B is a permissively licensed base for code completion and post-training. Use the Instruct sibling for chat. See more in the models category.

Comments

No comments yet. Be the first to comment!