Key takeaways

• Gemma 4 26B A4B IT is an instruction-tuned Mixture-of-Experts (MoE) model by Google DeepMind, designed for general-purpose language tasks with multimodal input support (text, images, video) and featuring a 256K token context window.
• Despite having 25.2 billion total parameters, only 3.8 billion activate per token during inference, allowing it to achieve near-31B quality at a lower computational cost.
• The model demonstrates strong mathematical problem-solving capabilities, achieving a 94.6% pass rate on the MATH-500 benchmark by providing detailed, step-by-step solutions across various math domains. Strengths include handling complex multi-step problems and adherence to formatting.
• It struggles significantly with complex reasoning tasks, achieving a very low 5.41% accuracy on the 'Humanity's Last Exam' benchmark. Common failure modes include misinterpretation of multimodal input (images), incorrect adherence to instructions, and errors in multi-step logical deduction.
• Key limitations include a lack of specific, frontier-level domain expertise in fields like biology and medicine, and difficulties with precise calculation and interpretation of scientific data.

Technical specifications

Context length

262k tokens

Max output

—

Architecture

MoE

Modalities

text+image+video->text

Parameters

~26B

Open weights

Yes

Licence

Apache 2.0

Regions

USA

Provider

Google

Provider model key

google/gemma-4-26b-a4b-it