DeepSeek R1 0528 AI Evaluation | LayerLens - Benchmark Results & Performance

Model

DeepSeek R1 0528 by DeepSeek

text->text

Free credits included. No credit card required.

May 28th update to the original DeepSeek R1 Performance on par with OpenAI o1, but open-sourced and with fully open reasoning tokens. It's 671B parameters in size, with 37B active in an inference pass.

Evaluations

across 22 benchmarks

Latency

10m 12s

Context length

64k

tokens

Cost

$0.20 · $4.50

input · output per 1M tokens

Key takeaways

DeepSeek R1 0528 is a text-to-text Mixture-of-Experts (MoE) model, specifically designed for conditional execution and logical evaluation, particularly in tasks with clear, constrained instructions like debugging.
The model features an MoE architecture with 671 billion parameters and 37 billion active during inference. It's open-sourced under an MIT license, optimizing for efficient inference and broad task capability.
It demonstrates strong performance in straightforward logical evaluations, achieving 99.2% accuracy on the 'Formal Logic Extended' dataset. Its strengths include correctly evaluating simpler logical expressions.
The model struggles with complex reasoning, nested negations, contextual understanding, multi-step problems, and generating functional code without precise guidance. Its worst performance was on the 'Humanity's Last Exam' dataset, scoring only 3.19% for general scientific and truthfulness tasks.
Common failure modes include misinterpreting problem context, incorrect application of rules, difficulties with complex multi-step problems and precise calculations, and errors in handling deeply nested negations.

Technical specifications

Context length: 64k tokens
Max output: —
Architecture: MoE
Modalities: text->text
Parameters: ~671B
Open weights: Yes
Licence: MIT
Regions: China
Vendor: DeepSeek
Provider model key: deepseek/deepseek-r1-0528