Publications

Spatial reasoning is essential for vision-language models (VLMs) to understand and act in the physical world. Reasoning in dynamic environments requires VLMs to perceive local state transitions caused by object motion and viewpoint changes and integrate them over long trajectories to maintain an updated spatial state. However, existing VLMs remain limited in both capabilities. Current spatial training primarily focuses on static questions about object attributes and spatial relations, providing limited direct supervision for state transitions. In contrast, interaction trajectories naturally connect a preceding observation, an action, and a subsequent observation, providing direct supervision for local state transitions, while complete trajectories reveal dependencies among consecutive transitions. We therefore introduce Spatial-Interactor, a framework that trains VLMs to model physical-world state transitions through interaction. We organize this learning process into a three-level curriculum covering L1 passive world-state transitions, L2 active self-state transitions, and L3 long-horizon interaction trajectories. Accordingly, we construct the Learning from Spatial Interaction dataset (LSI-108K) from simulated and real interaction trajectories, with tasks aligned with the objective of each level. Our two-stage training strategy applies Supervised Fine-Tuning (SFT) to L1 and L2 for local transition modeling. On-Policy Distillation (OPD) then uses privileged self-distillation: a teacher branch given segment-level transition descriptions supervises the student’s on-policy CoT, helping the student learn to integrate consecutive transitions over L3 long trajectories. Experiments across multiple VLMs and spatial benchmarks show consistent gains in local transition modeling and long-horizon integration.

Spatial intelligence is essential for agents to move from static semantic understanding toward interacting with the physical world. In the real world, many spatial tasks are grounded in continuous visual scenes, where locations, regions, and paths are more naturally expressed by pointing, marking, or drawing than by reporting precise coordinates or discrete textual symbols. However, existing spatial reasoning benchmarks usually require models to answer with coordinates, options, or textual descriptions, creating an answer-interface mismatch for image-generation models. This interface gap makes it difficult to evaluate image-generation models under the same task semantics as text-output VLMs, even though they can externalize spatial judgments directly in pixel space. To address this problem, we propose ProVisE (Protocolized Visual Evaluation), a benchmark-agnostic framework that elicits protocol-constrained visual answers from image-generation models and parses them into structured predictions compatible with the original evaluation metrics. ProVisE also includes an Agentic builder that constructs and validates task-specific protocols for new benchmarks. Building on this framework, we introduce SpatialGen-Bench, a curated diagnostic benchmark comprising 470 samples across 14 spatial subtasks, four capability levels, and diverse answer forms. We evaluate representative text-output VLMs and image-generation models in a unified setting, and validate Agentic protocol construction on six external spatial benchmarks. Experimental results show that image-generation models are competitive when spatial answers can be externalized directly in pixel space, while text-output VLMs retain a clear advantage in compositional spatial reasoning. These findings reveal the complementary strengths and limitations of pixel-space expression and text-based reasoning, while ProVisE and SpatialGen-Bench establish a metric-compatible testbed for generative spatial evaluation and future studies of spatial cognition in image-generation models.