# Does VLA Inference Latency Break RL Finetuning in Deployment?

Yes — and a joint Siemens/UC Berkeley team has a fix. Published today on arXiv (2608.23831), researchers from Siemens, UC Berkeley, Microsoft, and ETH Zurich identify a fundamental but underappreciated failure mode in deploying [Vision-Language-Action Model](https://humanoidintel.ai/glossary/vision-language-action-model)s (VLAs): their inference latency doesn't just slow robots down, it structurally breaks the Markov assumption that all standard reinforcement learning algorithms depend on. The result is that RL finetuning — currently one of the most promising paths for generalist robot policies to improve autonomously during real-world deployment — fails completely when applied to large VLAs without accounting for timing.

The team's proposed solution, **Asynchronous RL with Intermediate Information (ARLI)**, restores near-Markovian structure through two specific contributions: state augmentations that incorporate committed actions and a mid-inference observation captured while the model is still computing its next output. In simulated and real-world manipulation evaluations, ARLI enables effective RL finetuning under inference delays where standard RL algorithms produce no meaningful improvement — and in some configurations matches or exceeds standard RL performance in idealized, zero-latency settings.

For any team running a VLA on a humanoid today, this is immediately relevant engineering.

---

## The Problem: Latency Is Not Just Lag, It's a Structural RL Failure

The standard mental model of VLA deployment treats latency as an inconvenience — pauses between actions, slightly jerky movements, slower cycle times. The Siemens/Berkeley paper reframes this: latency fundamentally alters the effective environment dynamics that the RL algorithm observes.

Here's why this matters mechanically. RL algorithms assume the Markov property: that the current state contains all information necessary to predict the next state, independent of history. When a VLA takes non-trivial time to produce an action, the robot is effectively executing a *committed* action from a previous timestep while simultaneously observing new states. The observation the policy sees when it produces action $a_t$ is already stale by the time $a_t$ executes. Standard RL treats these as the same timestep — a fiction that compounds over an episode and causes training to diverge or produce degenerate policies.

This isn't a marginal issue. As [Physical Intelligence (π)](https://humanoidintel.ai/companies/physical-intelligence), [Skild AI](https://humanoidintel.ai/companies/skild-ai), and others have scaled VLA model sizes, the latency problem has grown proportionally. Larger models with stronger [zero-shot generalization](https://humanoidintel.ai/glossary/zero-shot-generalization) are precisely the ones most susceptible to this RL failure mode. The industry has been deploying these models for inference-only tasks while quietly setting aside RL-based continual improvement — this paper explains structurally why that's been necessary.

---

## ARLI: Two Targeted Engineering Contributions

The paper's framework, ARLI, sits on top of existing asynchronous inference approaches — the class of methods that interleave action generation with execution so the robot doesn't freeze while the model thinks. The key insight is that asynchronous inference alone is insufficient for RL: it hides latency from the hardware but doesn't give the RL algorithm the information it needs to maintain a valid training signal.

ARLI adds two components:

**1. Committed Action Augmentation.** The state representation is augmented to include the actions that have already been committed (i.e., dispatched to the robot) but not yet completed. This allows the RL policy to reason about where the robot *will be* during the inference window, restoring near-Markovian structure without requiring the model to have already produced its next output.

**2. Mid-Inference Observation.** Rather than waiting until the VLA finishes computing to take a new observation, ARLI captures an observation partway through the inference window. This mid-inference snapshot reduces the staleness of the state estimate that feeds the RL training loop, further tightening the Markov approximation.

The combination allows a low-latency RL policy layer to operate reactively within the inference window — acting on fresh information while the large VLA continues to compute. The authors describe this as "maximizing reactivity within the inference window," which is a useful conceptual frame for thinking about the architecture.

---

## Experimental Results: Where Standard RL Fails, ARLI Succeeds

The paper evaluates across both simulated and real-world manipulation tasks. The headline finding: standard RL algorithms fail entirely under inference delays in these settings, while ARLI achieves effective finetuning. Critically, the authors report that ARLI can match or exceed standard RL performance even in idealized no-latency conditions — meaning there is no apparent penalty for using the framework when latency is low, only upside when latency is present.

The authors do not claim ARLI is a complete solution to all VLA deployment challenges. The paper is specifically scoped to the RL finetuning problem under inference delays, evaluated on manipulation tasks. Generalization to whole-body locomotion or loco-manipulation tasks on full humanoid hardware would require additional validation.

---

## Industry Implications: Continual Improvement at Scale

The stakes here are significant for humanoid deployment timelines. Every major hardware platform — from [Agility Robotics](https://humanoidintel.ai/companies/agility-robotics)' Digit to [Figure AI](https://humanoidintel.ai/companies/figure-ai)'s Figure 02 — is nominally targeting autonomous improvement during deployment as a path to closing the capability gap with human workers. RL finetuning of a deployed generalist policy is the obvious mechanism. But if that finetuning is structurally broken for large VLAs, the path narrows considerably.

ARLI provides a framework-level fix that doesn't require shrinking models, which would sacrifice the generalization capability that makes VLAs attractive in the first place. The institutional authorship — Siemens alongside Sergey Levine's group at UC Berkeley, with contributors from Microsoft Research and ETH Zurich — signals this is intended as infrastructure-level research, not a one-off experiment.

The Siemens affiliation is also worth noting. Siemens has industrial automation interests directly relevant to humanoid-adjacent manipulation tasks. A joint team publishing RL methodology research with Berkeley's robotics leadership suggests Siemens is positioning itself in the software stack for physical AI deployment, not merely as a hardware or factory customer.

---

## Key Takeaways

- **The core finding:** VLA inference latency breaks the Markov assumption RL relies on, causing standard RL finetuning to fail completely during deployment — ARLI fixes this.
- **Mechanism:** Two state augmentations — committed action incorporation and mid-inference observation capture — restore near-Markovian structure without requiring reduced model size.
- **Performance:** ARLI enables effective RL finetuning where standard RL fails, and matches or exceeds no-latency RL baselines in tested conditions.
- **Authorship:** Joint work from Siemens, UC Berkeley, Microsoft, and ETH Zurich, with Sergey Levine as a senior author.
- **Scope:** Validated on manipulation tasks (simulated and real-world); applicability to full humanoid whole-body control is an open question.
- **Industry relevance:** Any team using a large VLA and expecting to improve it via RL during deployment needs to account for this failure mode — ARLI is the first published framework specifically addressing it.

---

## Frequently Asked Questions

**What is the ARLI framework and why does it matter for humanoid robots?**
ARLI (Asynchronous RL with Intermediate Information) is a framework developed by Siemens and UC Berkeley researchers that enables reinforcement learning finetuning of large Vision-Language-Action models during deployment. It matters for humanoid robots because VLA inference latency structurally breaks standard RL training loops — ARLI repairs this with state augmentation techniques that restore near-Markovian structure.

**Why does inference latency break RL for VLAs?**
RL algorithms assume the Markov property: the current state fully determines future dynamics. When a VLA takes significant time to produce an action, the robot executes stale actions while observing new states, violating this assumption. Standard RL algorithms exposed to this mismatch fail to produce useful policy improvements.

**What are committed actions in the context of ARLI?**
Committed actions are actions that have already been dispatched to the robot for execution but have not yet completed when the next observation is taken. ARLI augments the RL state representation with these committed actions, allowing the policy to account for the robot's in-progress trajectory during the inference window.

**Can ARLI be applied to existing VLA architectures without retraining?**
The paper does not explicitly detail integration requirements for arbitrary pre-existing VLAs. However, the framework is described as building on asynchronous inference approaches, suggesting it operates as a layer around the inference pipeline rather than requiring internal model modification. Teams evaluating adoption should consult the full paper for architecture-specific requirements.

**Does this approach generalize beyond manipulation to humanoid locomotion?**
The paper validates ARLI specifically on manipulation tasks. Generalization to full-body locomotion or [whole-body control](https://humanoidintel.ai/glossary/whole-body-control) scenarios on humanoid hardware is an open research question not addressed in this work.