# Does Splitting Proprioception and Vision Into Separate World Models Help Humanoids Navigate Obstacles?

A new paper from Rongjun Jin, Jianming Ma, and Yue Gao argues that the answer is yes — and validates it on a [Unitree Robotics](https://humanoidintel.ai/companies/unitree-robotics) G1 humanoid navigating randomized obstacle layouts. The framework, called DWMP (Dual World Model Policy), attacks a problem that has quietly constrained multimodal robot learning: existing methods treat [proprioception](https://humanoidintel.ai/glossary/proprioception) and egocentric vision as if they were the same kind of data, when they are fundamentally not.

Proprioceptive observations are low-dimensional but governed by highly nonlinear robot dynamics. Visual observations — particularly depth images from an egocentric camera — are high-dimensional, noisy, and geometrically redundant. Shoving both through a single shared encoder forces the policy network to reconcile these incompatible statistical structures, which the authors argue degrades learning efficiency and traversal performance. DWMP instead builds two separate but complementary world-model representations, fuses their latent outputs, and hands the combined embedding to a student actor policy.

Experiments show DWMP improves obstacle traversal over baselines in simulation and transfers to real-world deployment on the G1 under randomized obstacle configurations — the critical [sim-to-real transfer](https://humanoidintel.ai/glossary/sim-to-real-transfer) test that separates publishable ideas from deployable ones.

---

## The Two-Model Architecture: Koopman Meets RSSM

The conceptual core of DWMP is the deliberate architectural mismatch between its two world models, each chosen to match the statistical character of its input modality.

**Koopman-based dynamics world model (proprioceptive branch):** Koopman operator theory lifts nonlinear dynamical systems into a higher-dimensional latent space where their temporal evolution becomes approximately linear. For humanoid locomotion, where joint torques, velocities, and contact forces interact through complex, nonlinear mechanics, this linearization is substantive. A linear latent representation of robot dynamics is dramatically easier for an actor network to plan over than raw proprioceptive time series — the gradient landscape is smoother, credit assignment across time steps is cleaner, and the features carry interpretable structure related to stability and momentum.

**RSSM-based visual world model (depth vision branch):** The Recurrent State Space Model (RSSM), originally developed in the Dreamer family of model-based RL agents, compresses sequential observations into compact stochastic latent states using a combination of recurrent and variational components. Applied here to egocentric depth images, it strips away redundant texture information while preserving the geometric structure that actually matters for obstacle detection — the shape, height, and proximity of objects in the traversal path. The stochastic state representation also naturally handles sensor noise, which is a persistent problem with depth cameras on mobile platforms.

The student policy receives the fused latent representation from both branches and generates actions. The architecture follows a teacher-student distillation pattern common in locomotion research: a privileged teacher with access to ground-truth state information trains in simulation, and the student — relying only on onboard sensor observations — learns to match it.

---

## Why the Unitree G1 Is the Right Testbed Here

The G1 has become something of a standard platform for locomotion research precisely because its hardware is accessible enough that researchers outside well-funded labs can run real-world experiments. The fact that DWMP transfers from simulation to randomized real-world obstacle layouts on the G1 is meaningful validation, not just a footnote. Randomized layouts specifically stress [zero-shot generalization](https://humanoidintel.ai/glossary/zero-shot-generalization) — the robot cannot have memorized a fixed obstacle configuration during training.

The paper does not report specific quantitative metrics from the source text beyond describing improved performance over baselines, so the precise magnitude of the improvement cannot be stated here. That's worth flagging: obstacle traversal benchmarks in humanoid locomotion research are not yet standardized, which makes cross-paper comparisons difficult and means "improves over baselines" requires careful reading of which baselines and which metrics.

---

## Skeptical Analysis: What DWMP Does and Doesn't Solve

The architectural intuition is sound. Koopman linearization for robot dynamics has genuine theoretical backing, and RSSM-based compression is well-tested in visual model-based RL. The contribution here is their combination in a dual-branch framework specifically for humanoid obstacle navigation — not either component alone.

Several open questions are worth tracking as this work develops:

**Computational cost at inference.** Running two separate world models plus a fusion layer and actor network imposes a real-time compute budget. The paper does not specify inference latency figures, which matters for [whole-body control](https://humanoidintel.ai/glossary/whole-body-control) applications where control loops run at high frequency. If the dual-model approach adds significant latency, its practical advantage over simpler baselines shrinks.

**Obstacle complexity scope.** "Cluttered obstacle fields" is a specific regime. The framework is not presented as a general locomotion policy — it is tuned for structured traversal scenarios. How it handles dynamic obstacles, deformable terrain, or combined loco-manipulation tasks remains untested.

**Scalability of the Koopman approach.** Koopman linearization works well for relatively low-dimensional proprioceptive spaces. As humanoid platforms add [degrees of freedom](https://humanoidintel.ai/glossary/degrees-of-freedom) — full hand articulation, torso compliance — the dimensionality of the proprioceptive observation grows, and maintaining approximate linearity in the Koopman latent space becomes harder. This is not a fatal limitation, but it is a scaling concern.

---

## Industry Trajectory Implications

The broader significance of DWMP is its contribution to a growing consensus in humanoid locomotion research: multimodal sensor fusion should respect the statistical heterogeneity of its inputs rather than homogenizing them through a single encoder. This is a design philosophy with implications beyond obstacle traversal.

For teams building locomotion stacks — whether at hardware companies shipping G1s or at AI-first labs developing foundation policies — the Koopman branch in particular is an underexplored tool. Most current locomotion policies use transformer or MLP encoders that treat proprioceptive sequences as arbitrary time series. If Koopman linearization demonstrably improves sample efficiency and policy robustness, it could become a standard component rather than a research curiosity.

The RSSM visual branch is more familiar territory — Dreamer-derived architectures have been used in robotics for several years — but its pairing with depth-only input (rather than RGB) and explicit geometric preservation is a sensible adaptation for obstacle detection specifically.

For the competitive landscape, this research doesn't directly advantage any single hardware vendor. But it reinforces the value of platforms like the Unitree G1 as research-grade humanoid testbeds, and it adds to the accumulating evidence that task-specific world models — rather than single monolithic foundation models — remain a productive direction for locomotion in structured environments.

---

## Key Takeaways

- **DWMP** uses two separate world models — Koopman-based for proprioception, RSSM-based for depth vision — rather than a single shared encoder for humanoid obstacle traversal.
- **Koopman linearization** lifts nonlinear robot dynamics into a latent space with approximately linear temporal evolution, making proprioceptive features easier for a policy to learn from.
- **RSSM compression** reduces high-dimensional, noisy depth observations to compact stochastic states while preserving obstacle geometry.
- **Real-world validation** on a Unitree G1 under randomized obstacle layouts confirms successful sim-to-real transfer.
- **Open questions** include inference latency, performance on dynamic obstacles, and Koopman scalability as humanoid DOF counts increase.
- The paper is authored by **Rongjun Jin, Jianming Ma, and Yue Gao** and is available at arxiv.org/abs/2609.12347.

---

## Frequently Asked Questions

**What is DWMP and what problem does it solve?**
DWMP (Dual World Model Policy) is a framework for humanoid obstacle traversal that processes proprioceptive and visual sensor data through separate, architecturally distinct world models rather than a shared encoder. It addresses the mismatch between low-dimensional nonlinear proprioceptive data and high-dimensional, noisy visual data.

**What is a Koopman operator and why use it for humanoid proprioception?**
A Koopman operator lifts a nonlinear dynamical system into a higher-dimensional space where its evolution is approximately linear. For humanoid robots, where joint dynamics are highly nonlinear, this linearization makes the proprioceptive latent features significantly easier for a policy network to learn from and plan over.

**What is an RSSM and how does DWMP use it?**
RSSM (Recurrent State Space Model) is a model-based RL component that compresses sequential observations into compact stochastic latent states. DWMP applies it to egocentric depth images to extract obstacle-relevant geometric information while discarding noise and redundancy.

**Was DWMP tested on a real robot?**
Yes. The paper reports experiments on a Unitree G1 humanoid robot under randomized obstacle layouts, validating sim-to-real transfer beyond simulation-only results.

**How does DWMP compare to vision-language-action models for navigation?**
DWMP is a purpose-built locomotion policy for obstacle traversal, not a [Vision-Language-Action Model](https://humanoidintel.ai/glossary/vision-language-action-model). VLAs target semantic task understanding and generalization across task types; DWMP targets robust sensorimotor performance in a specific physical regime. The two approaches are complementary rather than competitive.