# Does RTCF Fix the Long-Horizon Failure Mode in Frozen VLA Policies?

A training-free test-time framework called RTCF (Retrieve in Time, Correct in Frequency) raises aggregate success rates on the LIBERO benchmark from 86.4% to 88.4% — and more meaningfully, lifts LIBERO-Long from 61.6% to 68.6% — without any parameter updates, repeated inference passes, or additional GPU resources. The correction runs on client CPU after a single policy invocation, with median latencies summing to 10.99 ms per action chunk.

This matters because the hardest bottleneck in deploying [vision-language-action models](https://humanoidintel.ai/glossary/vision-language-action-model) on humanoid hardware is not single-step accuracy but long-horizon task completion. Accumulated execution error and visual aliasing across task stages cause VLAs to drift even when their per-step predictions look reasonable. RTCF attacks that failure mode directly, using successful prior rollouts as corrective memory rather than retraining or fine-tuning. The gains are modest in aggregate but substantial on the hardest benchmark suite — and the near-zero computational overhead means the technique is practically free to deploy alongside any frozen policy stack.

---

## The Problem: Why Frozen VLAs Drift on Long Tasks

[Vision-language-action models](https://humanoidintel.ai/glossary/vision-language-action-model) generate temporally extended action chunks — sequences of predicted motor commands issued in a single forward pass. This architecture is efficient, but it creates a compounding error problem: early execution mistakes shift the robot into visual states that were never in the training distribution, and the model's next chunk prediction is now grounded in a scene that no longer matches its priors. On short tasks, the model can recover. On long-horizon manipulation — tasks spanning many sequential sub-goals — the drift accumulates until the rollout fails.

The naive fix is retrieval: find a successful past rollout that looks similar to the current moment and blend those actions into the policy output. The problem, as the RTCF authors identify, is that naive frame retrieval returns *visually similar* moments, which may not be *progress-aligned* moments. A robot halfway through a stacking task might visually resemble the beginning of a different stacking task — and the retrieved actions would set it back rather than forward.

Direct time-domain fusion compounds this: if you simply add retrieved actions to the frozen policy's output, you overwrite the reactive high-frequency structure that makes the policy responsive to perturbations. You've traded one failure mode for another.

---

## How RTCF Separates Retrieval from Transfer

RTCF's architecture splits the problem into two independently tractable sub-problems: *which experience to retrieve* and *which part of that experience to transfer*.

**Progressive Memory Alignment (PMA)** handles the retrieval question. Rather than matching individual frames by visual similarity, PMA maintains an incrementally updated monotonic frontier that causally aligns the growing visual execution history with complete successful trajectories. This means it tracks progress through a task stage-by-stage without requiring explicit stage labels — the alignment is derived from the visual history itself, constrained to be monotonically advancing. The result is that retrieved memory is progress-aligned, not just visually-aligned.

**Frequency-domain transfer** handles the action blending question. Once RTCF identifies the aligned position in the memory trajectory, it does not replay those actions wholesale. Instead, it extracts a low-frequency residual from the motion channels — coefficient-wise-clipped to prevent large corrections that would destabilize execution — and adds it to the frozen policy's output. Higher-frequency components and gripper binary decisions are inherited entirely from the frozen policy. The reasoning: low-frequency components encode the broad directional shape of a motion (where the arm is going over the next several timesteps), while high-frequency components encode the reactive fine-structure that the policy learned to generate from current observation. Overwriting the latter with retrieved memory would eliminate the policy's ability to respond to real-time perturbations.

This is a principled decomposition. Low-frequency correction nudges the trajectory toward a successful shape without suppressing the policy's reactivity.

---

## Benchmark Results: LIBERO at Scale

The evaluation runs across all four LIBERO suites with 2,000 episodes per condition — a sample size large enough to draw meaningful conclusions. Key reported numbers from the paper:

- **Aggregate success rate:** 86.4% (baseline frozen VLA) → 88.4% (RTCF)
- **LIBERO-Long success rate:** 61.6% → 68.6%
- **Median correction latency:** 10.99 ms per action chunk
- **Additional GPU requirement:** None — correction runs on client CPU post-inference

The LIBERO-Long gain is the number that deserves attention. A 7-point absolute improvement on the suite specifically designed to stress-test long-horizon [dexterous manipulation](https://humanoidintel.ai/glossary/dexterous-manipulation) is not marginal. For context, LIBERO-Long tasks require sequentially completing multiple sub-goals across a scene, exactly the regime where accumulated error is most damaging. That RTCF recovers meaningful performance there — without touching model weights, without an extra GPU, and within 11 ms — is the result worth tracking.

The aggregate 2-point gain is smaller and worth scrutinizing. On shorter LIBERO suites where the baseline frozen VLA already performs well, there is less headroom for retrieval-based correction to help, and the method's benefit is naturally ceiling-bounded. The framework is most valuable precisely where policies are most brittle: long tasks, novel scene configurations, and deployment contexts where retraining is not an option.

---

## Industry Implications: Test-Time Correction as a Deployment Layer

The broader significance of RTCF is architectural. It demonstrates that a frozen [Physical AI](https://humanoidintel.ai/glossary/physical-ai) policy is not a static artifact — it can be improved at deployment time by a lightweight reasoning layer that has access to successful prior rollouts. This is a different design philosophy from continual fine-tuning or online reinforcement learning, both of which require compute, safety guardrails, and data pipelines that are difficult to operate on deployed hardware.

For humanoid operators — whether that is [Figure AI](https://humanoidintel.ai/companies/figure-ai) running manipulation policies in a warehouse, or [Physical Intelligence (π)](https://humanoidintel.ai/companies/physical-intelligence) shipping frozen π-series policies to hardware partners — the practical appeal is clear: ship a frozen model, accumulate successful rollouts on the target platform, and let a CPU-resident correction layer improve task completion without a retraining cycle. The 10.99 ms overhead is negligible against typical action chunk durations.

The skeptical read: RTCF requires successful rollouts to retrieve from. In a zero-shot deployment on a new task or environment with no prior successful trajectories, the memory bank is empty and the framework degrades to the frozen baseline. This is not a criticism the authors hide — it is structural to any retrieval-based approach. The method is a multiplier on prior successful data, not a substitute for it. Operators who can seed a memory bank through teleoperation or simulation would get immediate benefit; those attempting genuinely novel deployments would not.

There is also the question of how RTCF interacts with policies that already use action chunking with receding horizons (as in diffusion-based VLA variants). The paper evaluates on frozen policies specifically; whether the frequency-domain transfer logic remains well-motivated when the base policy is already recalculating chunks at high frequency is a question the research does not address.

---

## Key Takeaways

- **RTCF raises LIBERO-Long success from 61.6% to 68.6%** using only successful prior rollouts and no parameter updates
- **Aggregate LIBERO improvement is 86.4% → 88.4%** across 2,000 episodes per condition — statistically robust but modest on easy suites
- **Correction runs in 10.99 ms median latency** on client CPU after a single policy forward pass — effectively free overhead
- **Progressive Memory Alignment (PMA)** solves progress-misaligned retrieval by tracking monotonic trajectory frontiers rather than raw visual similarity
- **Frequency-domain transfer** preserves policy reactivity by only blending low-frequency motion residuals, leaving high-frequency structure and gripper decisions to the frozen policy
- **Key limitation:** performance degrades to baseline on tasks with empty memory banks — the method is a multiplier on existing successful data, not a [zero-shot generalization](https://humanoidintel.ai/glossary/zero-shot-generalization) mechanism
- The framework points toward a broader class of CPU-resident deployment-layer tools that improve frozen VLA behavior without retraining

---

## Frequently Asked Questions

**What is RTCF and what problem does it solve?**
RTCF (Retrieve in Time, Correct in Frequency) is a training-free framework that improves frozen VLA policy performance on long-horizon manipulation tasks by retrieving progress-aligned past successful rollouts and blending low-frequency corrective residuals into the policy's action chunks — without retraining, fine-tuning, or additional GPU compute.

**How much does RTCF improve VLA performance on LIBERO?**
According to the paper, RTCF raises aggregate LIBERO success from 86.4% to 88.4% and improves LIBERO-Long specifically from 61.6% to 68.6%, evaluated over 2,000 episodes per condition.

**Does RTCF require retraining the base VLA model?**
No. RTCF is explicitly training-free — it operates at test time, requires no parameter updates, and runs the correction step on client CPU after a single policy inference pass with a reported median latency of 10.99 ms per action chunk.

**What is Progressive Memory Alignment?**
PMA is RTCF's retrieval component. It causally aligns the robot's growing visual execution history against complete successful trajectories using incrementally updated monotonic frontiers, identifying the progress-aligned position in memory without requiring explicit task stage labels.

**What are the limitations of RTCF?**
The method requires an existing memory bank of successful rollouts. On truly novel tasks or environments with no prior successful trajectories, RTCF falls back to the frozen baseline policy. It also has not been evaluated on policies that use receding-horizon action chunking, which is a common variant in production VLA deployments.

**Why does RTCF transfer only low-frequency components from retrieved actions?**
High-frequency motion components and gripper decisions encode the policy's reactive structure — its ability to respond to real-time perturbations. Overwriting these with retrieved memory would suppress reactivity. Low-frequency components encode the broad directional shape of a motion and can be corrected without compromising the policy's moment-to-moment responsiveness.