LLM-Guided Future Hypotheses for Horizon-Aware Exploration in Multi-Step Robot Manipulation
Abstract
Multi-step robot manipulation requires acting under uncertainty about how the scene will evolve. Tasks such as opening drawers, moving sliders, turning switches, and interacting with articulated objects depend on contact timing, object motion, and delayed task progress. We study whether short-horizon future videos can provide useful conditioning signals for closed-loop robot control and reinforcement-learning fine-tuning. We formulate this setting as Future-Experience Conditioning (FEC), where a policy receives recent observations together with a compact latent representation of a short future video. During training, this future signal can be taken from demonstrations. At inference time, ground-truth futures are unavailable, so we evaluate generated futures and intentionally mismatched futures to study robustness to imperfect future supervision. We conduct extensive simulation experiments on RoboCasa and CALVIN and validate generated-future-conditioned execution on a real Franka Panda across opening a kettle, closing a kettle, and closing a microwave. We compare behavior cloning, behavior cloning with RL fine-tuning, and a future-conditioned policy baseline under no-future, ground-truth-future, generated-future, and wrong-future conditions. The results show that task-consistent future conditioning improves policy execution and RL adaptation, while mismatched futures reveal the importance of reliability-aware conditioning. Together, the simulation benchmarks and real-robot demonstrations show that short-horizon visual futures provide practical structured priors for manipulation policies under imperfect future predictions.
Step-by-step future-conditioning pipeline
Approach
We study control under imperfect future supervision, where the future signal used during training is not identical to the future available at inference time. The policy is trained and evaluated with a compact representation of a short future video, while the source of that future can vary. This lets us compare no-future conditioning, ground-truth futures, generated futures, and intentionally wrong futures under the same control interface.
Task grounding and future construction
Given a task command and current scene information, the system creates a structured description of the manipulation goal, including the target object, relevant interaction part, and desired state transition. This description is used to construct a short-horizon visual future that represents the intended evolution of the task. The same future-conditioning interface is evaluated in simulation and used for closed-loop execution on real Franka Panda manipulation tasks.
Generated future signal
At inference time, ground-truth future frames from demonstrations are not available. We therefore use generated future clips as practical test-time conditioning signals in both simulation and real-robot experiments. These clips provide cues about upcoming object motion and interaction timing while naturally containing visual and temporal imperfections. Our three-panel real-robot videos show the generated future, the current camera observation, and the resulting Franka execution side by side.
Future-Experience Conditioning (FEC)
The future clip is encoded into per-frame embeddings, compressed into temporal bins, and projected into a fixed-dimensional future latent. This compact representation provides a common interface between future video signals and policy learning. We also test temporally shifted futures to evaluate how sensitive the policy is to timing mismatch in the future signal.
Reliability-Aware Future Conditioning (RAFC)
RAFC adds a lightweight reliability gate over a null future and temporally shifted future candidates. It is designed to reduce sensitivity to partially correct but misaligned generated futures, rather than to solve completely wrong future hypotheses.
Future-conditioned policy
The policy receives recent observations together with the compact future latent and outputs continuous actions in closed loop. We evaluate behavior cloning, behavior cloning with reinforcement-learning fine-tuning, and a future-conditioned comparison baseline. Simulation benchmarks measure execution quality, adaptation, temporal robustness, and sensitivity to incorrect futures, while real-robot experiments demonstrate generated-future-conditioned control on physical hardware.
Results
BibTeX
@misc{khoshnazar2026futurehypotheses,
title = {LLM-Guided Future Hypotheses for Horizon-Aware Exploration in Multi-Step Robot Manipulation},
author = {Khoshnazar, Mohammad and Melnik, Andrew},
year = {2026},
url = {https://enact2026.github.io/}
}