Key takeaways

• Mistral Medium 3 is an enterprise-grade, Transformer-based language model designed for high-performance and cost-effective virtual banking assistance, focusing on secure query handling, RBI compliance, and customer privacy.
• The model demonstrates state-of-the-art reasoning and multimodal performance, achieving 99.07% accuracy in scientific reasoning (AI2 Reasoning Challenge - Easy) and robust adherence to privacy regulations in banking evaluations.
• Capabilities include high accuracy on easy science questions, consistent adherence to response formats, effective scientific reasoning, and maintaining secure and compliant responses in banking contexts while avoiding PII.
• Limitations include verbosity, struggles with complex calculations, susceptibility to hallucination, and a tendency to favor English over Hinglish, sometimes providing overly detailed explanations or defaulting to general English for cultural references.
• Best case evaluations showed 99.07% accuracy on 'AI2 Reasoning Challenge - Easy' demonstrating strong scientific reasoning. Worst cases included incorrect answers to specific chemistry questions, verbose explanations, and defaulting to general English for Hinglish cultural references in banking.

Technical specifications

Context length

131k tokens

Max output

—

Architecture

Transformer

Modalities

text->text

Parameters

—

Open weights

No

Licence

Commercial

Regions

France

Vendor

MistralAI

Provider model key

mistralai/mistral-medium-3