Nemotron 3.5 Lightning AI Evaluation | LayerLens - Benchmark Results & Performance

Processing your request

This may take a moment. We're preparing the latest results.

Evaluations Compass Benchmarks Spaces

Docs

Processing your request

This may take a moment. We're preparing the latest results.

Nemotron 3.5 Lightning AI Evaluation | LayerLens - Benchmark Results & Performance

Model

Nemotron 3.5 Lightning by Nvidia

Text->text

Free credits included. No credit card required.

Run your own evaluation

NVIDIA Nemotron 3.5 Lightning is an open mixture-of-experts model with 30B total parameters and only 3B active per token, built for always-on agentic workloads. It combines a hybrid Mamba-Transformer MoE architecture with multi-token prediction to deliver up to 4x higher throughput than similarly sized open models. The model is suited for high-throughput agentic workloads and specialized tasks that benefit from domain-specific customization, including chatbots, RAG systems, and instruction-following applications. It supports a 262K-token context window and controllable reasoning.

Evaluations

across 6 benchmarks

Latency

1m 11s

Context length

262k

tokens

Cost

$0.10 · $0.25

input · output per 1M tokens

Key takeaways

Nemotron 3.5 Lightning is an open Mixture-of-Experts (MoE) model with 30B total parameters (3B active per token), designed for high-throughput agentic workloads and specialized tasks like chatbots and RAG systems.
It utilizes a hybrid Mamba-Transformer MoE architecture with multi-token prediction, enabling up to 4x higher throughput compared to similarly sized open models and supports a 262K-token context window.
The model demonstrates strong mathematical problem-solving skills, achieving 94.5% accuracy on the MATH-500 benchmark, excelling in complex calculations and step-by-step reasoning across various math domains. Minor issues related to output formatting and specific problem interpretations were identified.
Nemotron 3.5 Lightning shows robust factual knowledge and contextual understanding (76.26% accuracy on General Purpose Question Answering), with particular strengths in Astrophysics, Chemistry, Quantum Mechanics, and Electromagnetism.
Limitations include occasional misinterpretation of problem constraints, formatting errors in mathematical outputs, and difficulty with highly nuanced biological/chemical mechanisms or subtle contextual cues in complex scientific questions.

Technical specifications

Context length: 262k tokens
Max output: —
Architecture: MoE
Modalities: text->text
Parameters: ~30B
Open weights: Yes
Licence: NVIDIA Open Model
Regions: USA
Vendor: Nvidia
Provider model key: nvidia/nemotron-3.5-lightning