Key takeaways

• Microsoft's Phi-4 is a Transformer-based text-to-text model designed for efficient coding and specialized problem-solving, requiring minimal computational resources and released under the MIT license.
• The model excels in structured, logical, and scientific reasoning tasks, achieving 97.85% accuracy on the AI2 Reasoning Challenge - Easy benchmark for easy-level science questions with consistent adherence to response format and clear, accurate explanations.
• A significant limitation of Phi-4 is its very poor performance (2.31% accuracy) on the SimpleQA Creative Multimedia Benchmark, indicating weakness in creatively recalling and accurately retrieving obscure creative information.
• Common failure modes include an inability to access specific or obscure information, providing generalized responses, and incorrectly associating historical figures or cultural items in creative multimedia contexts. Memory recall and reasoning for creative multimedia content are particular challenges for the model, often mixing up facts.
• The model's strengths lie in high accuracy on easy-level science questions, while its weaknesses are evident in factual accuracy and memory recall for creative multimedia content, often mixing up facts and showing a lack of specific knowledge for such tasks.

Technical specifications

Context length

16k tokens

Max output

—

Architecture

Transformer

Modalities

text->text

Parameters

~14B

Open weights

Yes

Licence

MIT

Regions

USA

Vendor

Microsoft

Provider model key

microsoft/phi-4