Key takeaways

• DeepSeek-R1 is an open-source, Transformer-based Mixture of Experts (MoE) model specialized in mathematical reasoning and technical/scientific applications, released under the MIT license in January 2025.
• The model excels in evaluating complex logical expressions, achieving 89.5% accuracy on the Formal Logic Extended Dataset, but struggles with multiple negations and operator precedence, and occasionally provides verbose responses rather than just 'true'/'false'.
• DeepSeek-R1 completely fails in abstract reasoning (0% on ARC_AGI) and policy-based Q&A (0% on PolicyQA Benchmark), frequently hallucinating and failing to extract precise answers or adhere to specific output formats.
• A significant limitation is its inability to generalize beyond its specialized domain, leading to incorrect formats and verbose responses in tasks requiring abstract thought or adherence to specific output formats.
• The model exhibited inconsistent behavior and failed to provide any output in the Accounting Audit benchmark, indicating a fundamental issue with understanding prompt format or severe limitations in financial reasoning capabilities.

Technical specifications

Context length

64k tokens

Max output

—

Architecture

MoE

Modalities

text->text

Parameters

~671B

Open weights

Yes

Licence

MIT

Regions

China

Vendor

DeepSeek

Provider model key

deepseek/deepseek-r1