UltraWorld: Learning Interactive Ultrasound World Models from Untracked Videos via Acoustic Sampling Map
Anonymous submission to ICLR 2027
Ultrasound futures predicted by UltraWorld from an initial observation and a sequence of scanning actions.
Abstract
World models can enable autonomous ultrasound scanning by predicting the outcomes of probe motions from local observations. Learning this action–observation relationship typically relies on synchronized video–pose pairs, which are costly to collect at scale and largely unavailable in routine clinical recordings. Reliable action following further requires modeling ultrasound’s cross-sectional sampling geometry.
We present UltraWorld, a self-distillation recipe that transfers priors from clinical ultrasound videos into interactive world models without real action annotations. Starting from clinical videos, we adapt a video foundation model into an ultrasound generator conditioned on reference images and anatomical masks. Anatomical masks sampled along programmable trajectories through 3D anatomy provide spatial guidance for synthesizing action–video pairs. We then use these synthetic pairs to self-distill the generator into a world model that predicts future observations from local observations and actions, without requiring anatomical masks or other 3D assets at inference time. To further improve action following, we introduce the Acoustic Sampling Map (AsMap), which represents probe poses and imaging settings as pixel-wise 3D sampling positions, beam directions, and depths.
Zero-shot evaluation on real tracked ultrasound scans demonstrates the effectiveness of UltraWorld, reducing Motion Error by 39.8–86.5% and improving Motion Direction Agreement by 20.0–51.4 percentage points over the unadapted Wan2.1 baseline, without training on real video–pose pairs. We further demonstrate closed-loop robotic ultrasound planning in simulation across 24 tasks, achieving a mean final position error of 4.84 mm, a 47.1% reduction over the strongest baseline.
Overview

Acoustic Sampling Map

Self-distillation




