Key takeaways

• Gemma 3 27B is a proprietary, Transformer-based model from Google, released in March 2025, designed for general-purpose intelligent interaction, reasoning, and chat across over 140 languages, handling text and image-to-text tasks.
• The model demonstrates strong performance in elementary science reasoning, achieving 100% accuracy on the easy subset of the AI2 Reasoning Challenge with consistent format adherence and low toxicity.
• Gemma 3 27B struggles significantly with legal reasoning tasks (0% accuracy on EU Sustainability Law Benchmark) and mathematical reasoning, often due to formatting errors (e.g., incorrect LaTeX) failing to adhere to specific output formatting requirements, even when intermediate steps are logically sound.
• On the XLSUM EN news summarization benchmark, the model showed low toxicity but produced inconsistent quality, struggling with factual accuracy, coherence, message capturing, and exhibiting wildly varying readability scores.
• The model's performance on math problems was heavily influenced by the complexity of the mathematical concepts involved, with simpler problems being answered more effectively; however, issues with formatting and adherence to the required output structure persisted, particularly concerning the use of LaTeX.

Technical specifications

Context length

128k tokens

Max output

—

Architecture

Transformer

Modalities

text+image->text

Parameters

~27B

Open weights

No

Licence

Proprietary

Regions

USA

Vendor

Google

Provider model key

google/gemma-3-27b-it