Qwen3 4B Thinking 2507 AI Evaluation | LayerLens - Benchmark Results & Performance

Model

Qwen3 4B Thinking 2507 by Qwen

text->text

Free credits included. No credit card required.

Smarter, sharper, and 256K-ready! Advanced reasoning in logic, math, science & code — built for expert-level tasks. Both models (instruct and thinking) are more aligned, more capable, and more context-aware.

Evaluations

across 3 benchmarks

Latency

155ms

Context length

262k

tokens

Cost

$0 · $0

input · output per 1M tokens

Key takeaways

Qwen3 4B Thinking 2507 is a Transformer-based text-to-text model designed for advanced reasoning in logic, math, science, and code, targeting expert-level tasks and complex problem-solving.
The model exhibits strong mathematical accuracy and logical reasoning, achieving perfect scores on AIME 2025 problems and demonstrating effective use of algebraic manipulation and mathematical theorems.
A key strength is its exceptional ability to identify when a function or tool is not applicable, succeeding with 100% accuracy on non-applicable cases in Berkeley-Function-Calling-v3.
Despite its strengths, the model struggles significantly when functions ARE applicable, with conflicting instructions, and with tasks requiring multi-step reasoning or external knowledge integration, resulting in only 25.26% accuracy on Berkeley-Function-Calling-v3.
Common limitations include excessive verbosity, redundancy, difficulties with multi-layer instructions, multi-step reasoning, ambiguous requests, and tendencies towards complex, circular reasoning, especially in general function calling and multi-turn scenarios, also demonstrating an overall accuracy of only 5% on the MATH-500 benchmark.

Technical specifications

Context length: 262k tokens
Max output: —
Architecture: Transformer
Modalities: text->text
Parameters: ~4B
Open weights: Yes
Licence: Apache 2.0
Regions: China
Vendor: Qwen
Provider model key: qwen/qwen3-4b-thinking-2507