Key takeaways

• Llama 4 Scout is a multimodal (text+image->text), open-weight Transformer-based model from Meta, designed for efficient tasks like summarization, code reasoning, and multilingual chat, supporting a 10 million token context window.
• The model excels in scientific reasoning, achieving 98.6% accuracy on the 'AI2 Reasoning Challenge - Easy' dataset, demonstrating strong competency in basic science concepts and clear explanations.
• A significant limitation and severe weakness of Llama 4 Scout is its complete failure (0% accuracy on the Stock BCS dataset) in financial reasoning due to misclassification of financial terms and struggles with precise numerical computations, often confusing Bear Call Spreads with Bull Call Spreads.
• While strong in science, the model can be overly verbose, and showed an instance of incorrect classification (primary vs. secondary succession), indicating potential limitations in nuanced domain knowledge despite excelling in basic science concepts.
• Llama 4 Scout consistently adhered to specified output formats and provided high-quality reasoning and explanations in scientific domains, even if sometimes more elaborate than necessary, but struggles significantly with complex mathematical problem-solving from datasets like AIME 2024, scoring 0% due to incorrect application of formulas, computational errors, and failure to adhere to strict formatting rules.

Technical specifications

Context length

172k tokens

Max output

—

Architecture

Transformer

Modalities

text+image->text

Parameters

~109B

Open weights

Yes

Licence

Llama 4

Regions

USA

Vendor

Meta

Provider model key

meta-llama/llama-4-scout