AffordDrive3D: Affordance-Aware World-Action
Modeling with Spatial Understanding

Tianhui Cai^1,2*‡ Xinglong Sun^2* Chao Fang³ Zhenxin Li^2,4‡ Rui Song¹ Jose M. Alvarez² Yunxiang Mao² Jiaqi Ma¹ Langechuan Liu^2†

¹ University of California, Los Angeles ² NVIDIA ³ 42dot by Hyundai ⁴ University of Toronto

^* Equal contribution · ^† Corresponding author · ^‡ Work done during an internship at NVIDIA

Paper Code coming soon

91.3PDMS
NAVSIM v1

89.9EPDMS
NAVSIM v2

01 / Overview

Aligning future world modeling with the planning objective through geometric and affordance prediction

Current driving scene with a leading vehicle on an urban street. — Shared inputCurrent scene

World predictions

Predicted future RGB appearance. — Future RGBappearance

Predicted depth indicating spatial structure. — Depthgeometry

Predicted drivable area outlined in green. — Drivable areaaffordance

Predicted collision-critical regions outlined in red. — Collision-criticalaffordance

Planning outcomes

DriveVLA-W0 trajectory collides with a vehicle in this example. — RGB-only model

AffordDrive3D trajectory safely stops behind the leading vehicle. — AffordDrive3D

HumanRGB WAMAffordDrive3D

RGB-onlyPredicting future scene appearance

RGB-based world-action models predict the scene’s future appearance. For planning, however, it is more important for a model to understand future geometry and identify the regions that matter for the ego vehicle’s future actions.

02 / Method

The AffordDrive3D framework

AffordDrive3D framework with a shared VLM backbone and affordance, RGB world, geometry, and action branches.

A causal VLM processes multi-view images, navigation commands, ego state, and four groups of learnable tokens.

What matters for action

Temporal affordance forecasting

Affordance tokens forecast drivable areas and collision-critical regions across the current and future scene, explicitly modeling how action-relevant space evolves over time.

03 / Results

Planning performance on NAVSIM

AffordDrive3D achieves 91.3 PDMS on NAVSIM v1 and 89.9 EPDMS on NAVSIM v2.

NAVSIM v1

PDMS ↑ higher is better

91.3

+0.5vs. best baseline

Baseline methods
Method	World modelling supervision	PDMS
ReCogDrive	—	90.8
OWMDrive	3D Occupancy	90.8
ExploreVLA	RGB + Depth	90.4
DriveVLA-W0	RGB	90.2

NAVSIM v2

EPDMS ↑ higher is better

89.9

+1.1vs. best baseline

Baseline methods
Method	World modelling supervision	EPDMS
ExploreVLA	RGB + Depth	88.8
DriveDreamer-Policy	RGB + Current Depth	88.7
DriveSuprim	—	87.1
DriveVLA-W0	RGB	86.1

04 / Visualizations

Challenging driving scenarios

AffordDrive3D demonstrates strong planning performance in challenging scenarios, such as narrow curvy roads and interactions with collision-critical objects.

Narrow curvy road

Following the road through a tight curve

AffordDrive3D’s drivable-area forecasts capture the narrow road structure over time, so its trajectory remains near the lane center. The RGB WAM and action-only baselines both go off-road.

Trajectory prediction

AffordDrive3D trajectory on the narrow curvy road. — AffordDrive3D

RGB world-action model trajectory on the narrow curvy road. — RGB WAM

Action-only trajectory on the narrow curvy road. — Action-only

Zoomed trajectory comparison with an off-road label placed beside the two circled baseline violations. — Zoomed region

HumanOursRGB WAMAction-onlyEgo footprint at the violationZoom-in region

AffordDrive3D world modelling

Leading vehicle braking

Responding to a braking vehicle

AffordDrive3D’s collision-critical forecasts track the leading vehicle across future horizons, while RGB-consistent depth captures its distance and surrounding geometry. AffordDrive3D maintains a safe distance, while both baselines collide.

Trajectory prediction

AffordDrive3D trajectory for the braking scenario. — AffordDrive3D

RGB world-action model trajectory for the braking scenario. — RGB WAM

Action-only trajectory for the braking scenario. — Action-only

Zoomed trajectory comparison using the PowerPoint layout, with a large collision label to the left of the baseline violation footprints. — Zoomed region

HumanOursRGB WAMAction-onlyEgo footprint at the violationZoom-in region

AffordDrive3D world modelling

05 / Citation

Citation

@article{cai2026afforddrive3d,
  title={AffordDrive3D: Affordance-Aware World-Action Modeling with Spatial Understanding},
  author={Cai, Tianhui and Sun, Xinglong and Fang, Chao and Li, Zhenxin and Song, Rui and Alvarez, Jose M and Mao, Yunxiang and Ma, Jiaqi and Liu, Langechuan},
  journal={arXiv preprint arXiv:2610.11060},
  year={2026}
}