# Does Your Coding Agent Already Know How to Control a Robot?

**96.7%.** That's the success rate Stanford researchers achieved on LIBERO-Goal manipulation tasks — not with a purpose-built [Vision-Language-Action Model](https://humanoidintel.ai/glossary/vision-language-action-model) trained on thousands of robot demonstrations, but with a frontier language model agent driving a manipulator through a browser-based 3D interface. No robot-specific fine-tuning. No privileged state access. Zero robot demonstrations in training.

The paper, published today on arXiv by Hengyuan Hu, Priya Sundaresan, Jensen Gao, and Dorsa Sadigh from Stanford, introduces VIA (Visual Interface Agent for robot control). The core idea is disarmingly simple: recast robot manipulation as a computer-use task. An off-the-shelf FM-powered agent takes screenshots of a browser-based 3D interface, issues intuitive commands through a small set of general tools, observes the outcome, and adjusts. The agent perceives only visual input — the same modality it uses to operate any other piece of software.

With the strongest model tested, referred to in the paper as Fable 5, VIA achieves 96.7% success on three LIBERO-Goal tasks and 100% on a long-horizon rainbow assembly task. The system also runs [zero-shot generalization](https://humanoidintel.ai/glossary/zero-shot-generalization) with Claude Code and Codex, demonstrating the approach isn't model-locked.

---

## The VLA Bottleneck VIA Is Trying to Break

The dominant paradigm for deploying intelligence on robot hardware is fine-tuning a large pretrained model on robot-specific data to produce a VLA — a model that takes images and language instructions as input and outputs low-level actions. Companies like [Physical Intelligence (π)](https://humanoidintel.ai/companies/physical-intelligence) and [Skild AI](https://humanoidintel.ai/companies/skild-ai) have built entire businesses on variants of this approach.

The Stanford team identifies a structural problem with that path: VLAs are "often orders of magnitude smaller than frontier FMs," constrained by the limited robot demonstration data available and the compute cost of fine-tuning at scale. A fine-tuned VLA might contain a fraction of the parameters of the frontier model it was derived from, leaving most of the reasoning capability on the table.

This is not a minor quibble. The gap between a frontier model's general reasoning capability and what a robot-specific fine-tune retains has been a persistent concern in the research community. Every fine-tuning step is a tradeoff: you gain task-specific action grounding, but you risk catastrophic forgetting of the general world knowledge that makes frontier models useful in the first place.

VIA's bet is that you don't need to fine-tune at all if you design the right interface. Rather than teaching the model to speak "robot" (raw joint torques, end-effector deltas), you teach the robot to speak "agent" — presenting its state as a visual interface the model already knows how to interpret and control.

---

## How VIA Actually Works

The architecture has four components worth understanding:

**The interface layer:** A browser-based 3D visualization presents the robot's workspace. The agent interacts with this the same way a computer-use agent interacts with any GUI — by taking screenshots and issuing commands.

**The tool set:** The agent has access to a "small set of general tools." The paper does not enumerate every tool, but emphasizes that these are general-purpose rather than robot-specific. This is critical: the agent's competence transfers directly from its existing software-operation skills.

**The perception loop:** The agent receives no privileged state information — no ground-truth joint angles, no object poses from a separate tracking system. It infers everything from visual input, mirroring real-world deployment constraints.

**Closed-loop error recovery:** Because the agent observes the outcome of each action and can re-plan, VIA inherits the error correction and replanning capabilities of the underlying frontier model. This is where agentic framing pays real dividends over single-shot policy inference.

The authors frame this explicitly: "your coding or computer-use agent is, in a sense, secretly a robot-control agent." That's a provocative claim, but the LIBERO-Goal results and the 100% rainbow assembly completion rate provide empirical support rather than just rhetoric.

---

## What the Results Actually Show — and What They Don't

The benchmark results are striking, but context matters. LIBERO-Goal is a tabletop manipulation benchmark — a controlled evaluation environment with relatively constrained task diversity. The rainbow assembly task demonstrates long-horizon planning capability, which is genuinely impressive, but the physical setup remains a tabletop scenario.

VIA has not been demonstrated on:
- Mobile manipulation or whole-body control scenarios involving bipedal locomotion
- Tasks requiring sub-millimeter precision or high-speed reactive control
- Unstructured environments outside the browser interface's representation

The paper's claim that "performance improves with the scale and strength of the underlying model" is directionally important for the industry. It means VIA gets better for free as frontier models improve — a compounding advantage that fine-tuned VLAs don't automatically inherit.

The approach also raises practical questions for deployment. Browser-based 3D interfaces introduce latency. The screenshot-and-command loop, while elegant, may not match the control frequencies required for dynamic manipulation tasks. These aren't fatal objections, but they define the gap between a compelling research result and a production system.

---

## Why This Matters for Humanoid Deployability

For the humanoid robotics industry, VIA's implications cut in two directions.

**The bullish reading:** If frontier agents already possess transferable robot control skills, the data bottleneck plaguing VLA development becomes less severe. Humanoid platform vendors — from [Figure AI](https://humanoidintel.ai/companies/figure-ai) to [Agility Robotics](https://humanoidintel.ai/companies/agility-robotics) — have all cited robot demonstration data scarcity as a limiting factor. An approach that bypasses fine-tuning entirely, even partially, changes the economics of deploying new skills.

**The skeptical reading:** Tabletop [dexterous manipulation](https://humanoidintel.ai/glossary/dexterous-manipulation) via a 3D browser interface is still very far from bipedal loco-manipulation in unstructured environments. The interface abstraction that makes VIA elegant for a fixed-base manipulator becomes significantly harder to design for a humanoid executing whole-body coordination while maintaining balance. The latency and representational assumptions that work in a browser UI may not survive contact with a walking robot's real-time control requirements.

The more immediate impact may be on the software-stack side of the industry: agentic frameworks for robot planning are attracting serious investment, and VIA provides a research anchor for the argument that general-purpose agents can be interfaced to robot hardware without the full fine-tuning pipeline.

---

## Key Takeaways

- VIA achieves **96.7% success on LIBERO-Goal tasks** and **100% on a long-horizon rainbow assembly task** using frontier model agents — zero robot-specific fine-tuning required
- The framework recasts robot control as a **computer-use / agentic task**, routing a frontier FM through a browser-based 3D interface
- Results hold across multiple models including **Claude Code, Codex, and Fable 5** (strongest tested), with performance scaling with model capability
- VLAs are constrained by being "orders of magnitude smaller than frontier FMs" — VIA argues the right interface eliminates the need for fine-tuning entirely
- Demonstrated on tabletop manipulation benchmarks; extension to **bipedal humanoid platforms and whole-body control** remains an open engineering challenge
- The approach inherits frontier model **error recovery and replanning** capabilities automatically, without task-specific training

---

## Frequently Asked Questions

**What is VIA in robotics?**
VIA (Visual Interface Agent for robot control) is a framework from Stanford that controls robot manipulators using off-the-shelf frontier language model agents. Instead of fine-tuning a model on robot data, VIA presents the robot's workspace as a browser-based 3D interface that the agent controls by taking screenshots and issuing commands — the same way it would operate any software.

**Does VIA require robot training data or fine-tuning?**
No. According to the paper, the agent receives no robot-specific fine-tuning and no access to privileged state information. It perceives only visual input and acts through a small set of general tools.

**What benchmark results does VIA achieve?**
Using the strongest model tested (Fable 5), VIA achieves 96.7% success on three LIBERO-Goal tasks and 100% success on a long-horizon rainbow assembly task. The system also runs zero-shot with Claude Code and Codex.

**How does VIA compare to standard VLA models?**
Standard VLAs are fine-tuned versions of foundation models, constrained by limited robot data and compute — often orders of magnitude smaller than frontier FMs. VIA bypasses fine-tuning entirely, directly leveraging frontier model scale and capability through an interface abstraction layer.

**Can VIA control humanoid robots?**
The current research demonstrates tabletop manipulation with a fixed-base manipulator. Extension to bipedal humanoid platforms, which require whole-body coordination and real-time balance control, would require significant additional engineering. The interface abstraction that works for a stationary arm faces harder constraints on a walking robot.

**Why does model scale matter for VIA?**
The paper reports that performance improves with the scale and strength of the underlying frontier model. This means VIA benefits automatically from improvements to general-purpose LLMs — unlike fine-tuned VLAs, which must be retrained when a better base model is released.