VLX-VR: A Full Upgrade for Complex Video Reasoning

Highlights

• 📊 Leading among peers: On the MINERVA video-reasoning benchmark, VLX-VR reaches 78.8 accuracy, above Seed2.1 Pro (70.7), Gemini 3.5 (68.6), and Seed2.1 Turbo (65.9).
• ⏱ Stable across duration: Using the same three bins as the MINERVA paper (under 5 minutes / 5–15 minutes / over 15 minutes), accuracy is 76.70% / 78.73% / 80.92%. Cross-Duration Accuracy Variance (CDAV) is 2.97, with a three-bin standard deviation of about 1.7 percentage points. Public Gemini 2.5 Pro, GPT-4.1, and similar models drop as videos get longer; VLX-VR does not, and the long-video bin is slightly higher.
• 🧠 Checkable process: MINERVA provides a human-written reasoning trace for every question. On correctly answered items, agreement between the model’s reasoning and the reference trace is 96.2%, so correct results are not mainly lucky guesses among options.
• 🎯 Clear strengths: Reading comprehension 90.8%, situational awareness 90.3%, temporal reasoning 86.0%, numerical reasoning 85.7%—more stable on textual cues, situation, and temporal/numerical reasoning.

1. Introduction

Video understanding is moving from “what is this video about” to “complete multi-step reasoning from frames, text, and the timeline.” VLX-VR targets exactly this layer: Video Reasoning—on videos of variable length, chaining perception, temporal localization, textual cues, and logical steps into a checkable reasoning process, rather than only emitting a final choice.

Many existing VideoQA benchmarks supervise only the final answer. Models can score via language bias, elimination, or chance hits, and evaluation cannot tell “the reasoning was right” from “the answer happened to be right.” MINERVA is designed to fill that gap: every item includes a human-written reasoning trace, so one can score both accuracy and the reasoning process.

This report presents VLX-VR results on MINERVA. The focus is threefold: first, overall accuracy against peer video-reasoning models; second, a duration split to see whether long videos drag performance down; third, a skill-type profile from dataset labels, plus reasoning agreement on correctly answered items.

2. The MINERVA Dataset

MINERVA (Multimodal INterpretablE Reasoning Video Annotations) is a VideoQA benchmark for complex video reasoning. Paper: arXiv:2505.00681; data: google-deepmind/neptune. It is not a short-clip action-classification set; it is built to stress multi-step, multi-skill, multimodal reasoning.

The public setup has the following main properties.

Scale and structure. 1000+ questions covering 200+ videos; each video usually has multiple questions. Every item provides 5 options, the correct answer, and a human-written reasoning trace.

Must be multi-step and multi-skill. Each question is required to combine at least two skills, such as temporal reasoning, counting, causality, goal reasoning, situational awareness, event occurrence, state change, reading (OCR), spatial perception, numerical reasoning (math beyond counting), object recognition, and counterfactual reasoning. MINERVA therefore scores combinatorial ability, not a single perception head.

Multimodal. Some items must combine frames, the timeline, and textual cues; a single frame is not enough.

Wide duration span. Videos range from under 2 minutes to about 100 minutes, with the longest over 1.5 hours, mean about 12 minutes, covering shorts, sports, instruction, lifestyle/tutorials, travel, and other domains. This makes it suitable for testing whether a model only handles short video, or can keep reasoning quality on medium and long video.

Reasoning traces for comparison. Reference reasoning averages about 92 words; about 99.6% contain timestamps, averaging about 4 timestamps per trace, and describe key actions, objects, and logical steps. Evaluation can therefore do reference-based process analysis: whether the model localized the right interval, looked at the right objects, and completed the logic. Human accuracy in the paper is about 92.5%; an earlier frontier model (Gemini 2.5 Pro Thinking) is about 66.2%, so the set still has headroom.

For this report, MINERVA supplies three things at once: an overall Video Reasoning score, long-video ability by duration, a skill-type profile, and a gold reasoning trace to measure whether the process agrees when the answer is correct.

3. Evaluation Setup

The evaluated system is VLX-VR. The task is video reasoning on MINERVA: given a video and a question, output a choice and a reasoning process.

Peer video-reasoning numbers are Seed2.1 Pro, Seed2.1 Turbo, and Gemini 3.5, taken from the Seed site Seed2.1; public numbers from the MINERVA paper are listed as well. The primary metric is accuracy; on the process side, we compute agreement between model reasoning and the MINERVA reference trace on correctly answered items. Skill types follow the dataset annotations. Duration bins match the paper’s Fig. 3(c): under 5 minutes, 5–15 minutes, over 15 minutes, compared against the paper’s public duration curves.

Table 1 is the overall accuracy comparison; the remaining tables are slices from this evaluation.

4. Overall Results: Ahead of Peer Video-Reasoning Models

On MINERVA video reasoning, VLX-VR accuracy is 78.8, above Seed2.1 / Gemini 3.5 in this comparison, and above Gemini 2.5 Pro Thinking (66.20) as published in the paper.

Table 1. MINERVA Video Reasoning accuracy. Scores are MCQ accuracy (%). VLX-VR is this evaluation; Seed2.1 Pro, Seed2.1 Turbo, and Gemini 3.5 are from the Seed site Seed2.1; the rest are public MINERVA-paper results.

This is 8.1 points above Seed2.1 Pro, 10.2 above Gemini 3.5, 12.9 above Seed2.1 Turbo, and 12.6 above Gemini 2.5 Pro Thinking in the paper. The gap is not “crushing short-video recognition”; it is better overall on a multi-step, multi-skill set that includes medium and long video. 78.8 is still below the human ceiling of 92.54, but it leads this comparison.

5. Duration Split: Long-Video Ability Holds

One of MINERVA’s values is duration span. Paper Fig. 3(c) splits videos into three bins: under 5 minutes, 5–15 minutes, over 15 minutes. Public models (Gemini 2.5 Pro Thinking, GPT-4.1, OpenAI o1, Claude 3.5 Sonnet v2) drop clearly as duration increases. This evaluation uses the same bins.

Table 2. Accuracy by video duration (same bins as paper Fig. 3(c)).

All three bins sit around 77–81%: 76.70% under 5 minutes; 78.73% in the middle bin; 80.92% over 15 minutes, rising rather than falling. The three-bin range is 4.22 percentage points.

Table 3. Duration split versus public models in the MINERVA paper (accuracy %). Paper-model numbers are digitized from the official Fig. 3(c) SVG; VLX-VR is Table 2. Qwen2.5-VL, VideoLLaMA3, and InternVideo2.5 from the paper are omitted (open-source lag; not used in this figure comparison).

On the short-video bin VLX-VR already exceeds Gemini 2.5 Pro (76.70 vs 68.87); beyond 15 minutes, Gemini falls to 58.0 and GPT-4.1 to 47.3, while VLX-VR rises to 80.92. That is a harder point than the overall 78.8: long video does not punch through the reasoning level.

Figure 1. MINERVA accuracy by duration. VLX-VR is the red diamond; others are closed-source counterparts from paper Fig. 3(c) (Human excluded; the three open-source models excluded).

To collapse “how duration affects accuracy” into one number, define Cross-Duration Accuracy Variance (CDAV): equally weighted population variance of the three-bin accuracies

where \(n=3\) and \(\mathrm{Acc}_{i}\) is in percentage points. The three-bin mean is 78.78%, CDAV = 2.97 (percentage points\(^{2}\)), corresponding to a standard deviation of \(\sigma=1.72\) percentage points. The variance is still small, meaning accuracy does not swing hard when input duration switches among short / medium / long. The metric weights the three bins equally and is not weighted by item count.

Conclusion: Duration has a small effect on VLX-VR’s overall level (CDAV = 2.97); the three bins stay in 77–81%, and the long-video bin is slightly higher. This differs from the paper’s picture of public models sliding with duration, and indicates VLX-VR can attach reasoning ability to medium and long video.

6. Task Types: Stronger on Events, Text, and Causality

Skill types follow MINERVA annotations. Under the overall 78.8% accuracy, advantages concentrate on textual cues, situation, and temporal/numerical reasoning.

Table 4. Accuracy by skill type.

Strengths. Reading comprehension 90.8%, situational awareness 90.3%, temporal reasoning 86.0%, numerical reasoning 85.7%, all clearly above overall; event occurrence 82.8% and object recognition 82.0% are also above overall. VLX-VR is better at chaining textual cues, situation, and temporal/numerical information into a complete reasoning chain.

Weaknesses. Counting, state change, causal reasoning, spatial perception, and similar skills remain below overall; counting is still the largest gap. Further work will continue here.

7. Reasoning-Process Evaluation

What sets MINERVA apart from VideoQA sets that only give a final answer is a human reasoning trace on every item. On correctly answered items, agreement between the model’s reasoning and the reference trace is 96.2%.

That rate is high: when the answer is correct, the model’s steps align strongly with the annotated trace, so the 78.8% accuracy has process support rather than mainly option guessing. This is VLX-VR’s advantage over video-QA setups that only emit a final answer.

8. Cases

Under 5 minutes

5–15 minutes

Over 15 minutes

9. Conclusion and Next Steps

On MINERVA, VLX-VR reaches 78.8 video-reasoning accuracy, above Seed2.1 and Gemini 3.5 in this comparison. With the paper’s same three bins (under 5 minutes / 5–15 minutes / over 15 minutes) the split is 76.70% / 78.73% / 80.92%, CDAV = 2.97; public models in the paper slide with duration, VLX-VR does not, and the long-video bin is slightly higher. Reasoning agreement on correctly answered items is 96.2%, so correct results have process support. By skill type, reading comprehension, situational awareness, temporal reasoning, and numerical reasoning are clearly more stable. Counting, state change, causal reasoning, spatial perception, and similar skills still have room; further work will continue here.