Key takeaways

• DeepSeek V3 is an open-source, general-purpose large language model (LLM) utilizing a Mixture of Experts (MoE) architecture, pre-trained on an extensive 15 trillion tokens with a 65,536-token context window.
• The model demonstrates strong capabilities in foundational science concepts, achieving 98.57% accuracy on the 'AI2 Reasoning Challenge - Easy' dataset, indicating proficiency in straightforward knowledge retrieval and elementary science reasoning.
• A significant limitation of DeepSeek V3 is its struggle with complex reasoning, exhibiting hallucination, factual inaccuracy, lack of specificity, and misunderstanding of intricate scenarios. This is highlighted by 0% accuracy on 'ITC Infotech Additional Use Cases' and 'Accounting Audit' datasets.
• DeepSeek V3 scored 0% accuracy on the 'Accounting Audit' dataset, completely failing to respond to any of the 30 multiple-choice audit questions. This indicates a severe limitation in either understanding prompt instructions or factual knowledge within the auditing domain, or both.
• The model's failure in the 'Accounting Audit' dataset was characterized by an inability to follow explicit instructions for selecting options and provide factual accuracy in the auditing domain, often resulting in empty responses instead of valid answer choices.

Technical specifications

Context length

66k tokens

Max output

—

Architecture

MoE

Modalities

text->text

Parameters

~236B

Open weights

Yes

Licence

MIT

Regions

China

Vendor

DeepSeek

Provider model key

deepseek/deepseek-chat