# Does Chain-of-Thought Actually Hurt Your VLA?

A paper published today on arXiv from Jiarui Yang, Wen Huang, Jiale Zhang, Maowei Hu, and Hang Guo makes a pointed claim: injecting free-form textual chain-of-thought (CoT) reasoning into [Vision-Language-Action models](https://humanoidintel.ai/glossary/vision-language-action-model) doesn't improve manipulation performance — it actively degrades it. The authors present both empirical and analytical evidence for why this happens, then propose a concrete alternative called **In-Context VLA** that achieves state-of-the-art results across three simulation benchmarks (RoboCasa-GR1, SimplerEnv, and LIBERO) and eight real-world robot manipulation tasks.

The core diagnosis is precise: free-form CoT produces ungrounded reasoning, its latency breaks closed-loop timing, and — most critically — the reasoning tokens and action tokens are optimized against conflicting objectives. The policy learns to narrate rather than act. The fix, the authors argue, is not to give VLAs the ability to *generate* language, but to give them the ability to *consume* grounded language. That reframe has real implications for how the broader humanoid AI stack should be designed.

---

## The CoT Problem Is Structural, Not Incidental

The dominant training paradigm for generalist manipulation policies today is behavior cloning: a model imitates expert action chunks conditioned on a static image and a fixed instruction. When researchers have tried to improve on this by adding chain-of-thought reasoning — letting the model talk through what it's about to do before executing — the intuition seems sound. But the Yang et al. paper argues the failure is structural.

Three failure modes are identified. First, the reasoning produced by CoT is ungrounded — it doesn't anchor to specific perceptual evidence in the scene. Second, the latency introduced by generating reasoning tokens is non-trivial in closed-loop control, where timing directly determines whether a grasp succeeds. Third, and most damaging, the optimization landscape becomes contradictory: action tokens want the model to act, reasoning tokens want it to explain, and these objectives pull the gradient updates in different directions during training.

This is a technically credible critique. Closed-loop [dexterous manipulation](https://humanoidintel.ai/glossary/dexterous-manipulation) operates at timescales where inference latency is a first-order constraint, not a secondary concern. Any approach that adds token generation overhead to the hot path of a manipulation policy faces a genuine engineering headache — particularly as humanoid platforms push toward faster, more reactive whole-body behaviors.

---

## What In-Context VLA Actually Does

The paper's proposed framework has two components working in tandem.

**In-context post-training** injects perceptual evidence as structured context into the model during training, but critically, the model is supervised only on actions — not on generating any language output. This preserves the policy's ability to interpret rich linguistic and perceptual descriptions without incurring the overhead or conflicting-gradient problem of generating them.

**An agentic tool-use interface** lets the policy actively query external tools to acquire task-relevant information before acting. The tools cited in the paper are: open-vocabulary detectors, monocular depth estimation, and a vision-language model. Rather than baking scene understanding into a single monolithic forward pass, the policy can pull in structured, grounded descriptions of what it needs to know.

The data engine underpinning training is also notable. Instead of emitting a single templated caption for each scene — the approach most prior work takes — the authors' engine produces diverse, paraphrased, and evidence-conditioned spatial descriptions. The explicit goal is [zero-shot generalization](https://humanoidintel.ai/glossary/zero-shot-generalization): the policy should be able to interpret language it has never seen verbatim. This matters enormously for real deployment, where operators will inevitably issue instructions that don't match training-time phrasing.

---

## Benchmark Results and Real-World Validation

The paper reports state-of-the-art results across RoboCasa-GR1, SimplerEnv, and LIBERO — three simulation benchmarks that together cover a meaningful range of manipulation complexity. Crucially, the authors also validate on eight real-world robot manipulation tasks, which is where most VLA papers quietly fail to follow through. The claim is that In-Context VLA achieves superior performance *and* efficiency compared to CoT-based approaches under matched configurations.

Readers should note what the source material does not provide: specific percentage improvements, task success rates, or inference latency numbers are not included in the abstract or available text. The performance claims are directional rather than numerically grounded in what's available here. The full paper will need scrutiny on experimental setup — particularly whether the "matched configurations" baseline controls adequately for model size and training compute.

---

## Why This Matters for Humanoid AI Architecture

The humanoid AI stack is converging on VLAs as the glue between perception and action. [Physical Intelligence](https://humanoidintel.ai/companies/physical-intelligence), [Skild AI](https://humanoidintel.ai/companies/skild-ai), and most serious manipulation research groups are building in this paradigm. The question of how to add reasoning capability without sacrificing control quality is not academic — it's the central engineering tension for anyone trying to deploy a generalist policy on a 40-DOF bipedal platform.

The In-Context VLA paper's core insight — that a policy needs to *consume* grounded language rather than *generate* language — reframes the problem in a way that could influence how the field approaches the CoT-plus-VLA trend. If the analysis holds up, it suggests that reasoning should be offloaded to external tools and injected as structured context, keeping the hot path clean for action generation. That's an architecturally conservative but practically sensible position.

The agentic tool-use interface is the piece worth watching most carefully. Querying open-vocabulary detectors and depth models at inference time introduces its own latency and failure modes — if a tool call fails or returns noisy data mid-task, the policy needs to be robust to that. The paper's framing around this is promising, but the real-world validation on eight tasks, while meaningful, is a modest demonstration of robustness at scale.

For hardware teams building the sensor and compute stack on next-generation humanoids, this research direction also has implications: a policy architecture that relies on queryable external perception modules needs reliable, low-latency access to those modules on-device. That's a systems integration problem as much as an AI one.

---

## Key Takeaways

- **Free-form CoT provably hurts VLA performance** through ungrounded reasoning, latency in the control loop, and conflicting training objectives between reasoning and action tokens.
- **In-Context VLA flips the framing**: rather than generating language, the policy consumes structured, grounded language injected as context — supervised only on actions.
- **Agentic tool use** allows the policy to query open-vocabulary detectors, monocular depth, and a VLM to actively acquire task-relevant scene information.
- **A diverse data engine** generates paraphrased, evidence-conditioned spatial descriptions to drive zero-shot generalization to unseen language at deployment.
- **SOTA results claimed** on RoboCasa-GR1, SimplerEnv, and LIBERO benchmarks, plus eight real-world manipulation tasks — but specific performance numbers require review of the full paper.
- The architecture has direct implications for how humanoid AI stacks should handle the reasoning-vs-control tradeoff in closed-loop deployment.

---

## Frequently Asked Questions

**What is In-Context VLA and how does it differ from standard VLA training?**
In-Context VLA is a training and inference framework introduced in a 2026 arXiv paper (2608.05738) by Yang et al. Standard VLA training uses behavior cloning with static image and instruction inputs. In-Context VLA instead injects perceptual evidence as structured context and supervises the model only on actions — not on language generation — while adding an agentic interface to query external perception tools.

**Why does chain-of-thought reasoning hurt robot manipulation policies?**
According to the paper, CoT in VLAs fails for three reasons: the reasoning it produces is ungrounded in specific scene evidence; the token generation adds latency that breaks closed-loop timing; and reasoning tokens and action tokens are trained against conflicting objectives, causing the policy to learn narration rather than control.

**What benchmarks does In-Context VLA use?**
The paper evaluates on three simulation benchmarks — RoboCasa-GR1, SimplerEnv, and LIBERO — plus eight real-world robot manipulation tasks, claiming state-of-the-art performance and efficiency compared to CoT-based approaches under matched configurations.

**What external tools does the agentic interface in In-Context VLA query?**
The policy can query open-vocabulary object detectors, monocular depth estimation models, and a vision-language model to actively acquire task-relevant spatial information before generating actions.

**How does this research affect commercial humanoid robot development?**
For teams building generalist manipulation policies on humanoid platforms — including those at companies like Physical Intelligence and Skild AI — this work suggests that routing reasoning through external tools and injecting grounded context is architecturally preferable to having the policy generate chain-of-thought inline, particularly for real-time closed-loop control.