GPT-4o AI Evaluation | LayerLens - Benchmark Results & Performance

Model

GPT-4o by OpenAI

text+image->text

Free credits included. No credit card required.

Latest GPT-4o model supporting text and image inputs with enhanced capabilities.

Evaluations

across 32 benchmarks

Latency

497ms

Context length

128k

tokens

Cost

$2.50 · $10

input · output per 1M tokens

Key takeaways

GPT-4o is a proprietary multimodal large language model from OpenAI, built on a Transformer architecture, capable of processing both text and image inputs to generate text. It aims to excel in general question answering and various professional domains.
The model demonstrates strong performance in auditing tasks, achieving 88% accuracy on the 'Accounting Audit' benchmark, particularly in audit opinions, general principles, and ethical considerations. It also generates comprehensive explanations for application requirements and design principles.
GPT-4o struggles with complex mathematical problems (0% accuracy on 'AIME 2024') due to incorrect formula application and misinterpretation. It also has difficulty with nuanced sampling methods and complex procedural details in auditing tasks.
A significant limitation of GPT-4o is its propensity for widespread hallucination, especially in open-ended questions and complex math problems, sometimes accompanied by overconfidence in incorrect answers. While contextually relevant, its responses can be verbose.
On the 'Application Requirements and Design Principles Benchmark', GPT-4o showed low toxicity and hallucination scores, providing detailed and well-structured responses, although with a tendency towards verbosity that sometimes doesn't perfectly align with concise ground truth.

Technical specifications