TL;DR: UniLM-Nav is a zero-shot MLLM framework that bridges object navigation and manipulation by selecting task-relevant views, grounding affordances, and reasoning about manipulation-ready base poses for open-vocabulary mobile manipulation.
Abstract
Mobile manipulation requires a robot to navigate to a target object or receptacle and then perform intended manipulation. However, reaching the vicinity of the target does not guarantee a manipulation-ready base pose, a problem known as last-mile navigation. Prior methods for last-mile navigation either rely on manual pose annotation or task-specific training, limiting their scalability to open-vocabulary settings with fine-grained spatial constraints.
We propose UniLM-Nav, a unified framework for zero-shot open-vocabulary last-mile navigation. UniLM-Nav decomposes last-mile navigation into view selection, task-conditioned affordance grounding, and geometry-aware base-pose reasoning, all resolved with a shared multimodal large language model (MLLM) backend. Specifically, UniLM-Nav first selects a reference view that best captures the target object or receptacle from recently collected observations. It then grounds task-relevant affordance point in the selected view and lifts the result into the robot-centric coordinate frame. Finally, conditioned on the grounded affordance, task context, and robot geometry, it infers a manipulation-ready base pose for the robot.
We evaluate UniLM-Nav on the OVMM benchmark where it outperforms the previous state-of-the-art method, MoTo, by 3.13 percentage points. Analyses show that the components of our method are crucial to final performance, and that the choice of MLLM also has a substantial effect. We further deploy UniLM-Nav on a Unitree B2 quadruped robot with a 6-DoF Unitree Z1 manipulator, validating its applicability to real-world mobile manipulation tasks.
Method
UniLM-Nav decomposes last-mile navigation into view selection, affordance grounding, and base-pose reasoning. Given a task instruction, the MLLM first selects a suitable view from recent observations, then grounds the task-relevant affordance and lifts it to 3D with depth. Conditioned on the 3D affordance, robot configuration, and task instruction, the MLLM predicts a manipulation-ready base pose for execution.
Results on OVMM-Bench
(1) UniLM-Nav achieves the best performance on the OVMM validation set. (2) With a lightweight RoboBrain-2.5-4B as its MLLM backend, UniLM-Nav still outperforms most alternative methods.
| Method | Partial Success Rates | Overall SR |
Average SR |
||
|---|---|---|---|---|---|
| FindObj (↑) | Pick (↑) | FindRec (↑) | |||
| HomeRobot (RL) | 66.60% | 61.10% | 50.90% | 14.80% | 48.30% |
| HomeRobot (Heuristic) | 65.40% | 54.80% | 43.70% | 7.30% | 42.80% |
| MoManipVLA | 66.10% | 62.60% | 53.10% | 15.80% | 49.40% |
| UniTeam | 66.13% | 62.65% | 54.69% | 17.96% | 50.36% |
| MoTo | 66.67% | 60.95% | 49.87% | 20.64% | 49.53% |
| UniLM-Nav (w/ Gemini-3-Flash-Preview) | 69.47% | 66.22% | 54.55% | 23.77% | 53.50% |
| UniLM-Nav (w/ RoboBrain-2.5-4B) | 68.97% | 63.05% | 52.38% | 19.19% | 50.90% |
Error Analysis
Left: Error Breakdown. Right: Representative last-mile navigation failures involving view selection, affordance grounding, and base-pose reasoning. The green annotations indicate preferred views, affordance regions, or base poses, while the red annotations show the corresponding erroneous predictions.
BibTeX
@misc{zhang2026unilmnav,
title={UniLM-Nav: A Unified Framework for Zero-Shot Last-Mile Navigation},
author={Zhang, Zhuofan and Wang, Tianxu and Zhang, Guoxi and Lin, Yixiong and Wang, Xilin and Xu, Hongming and Li, Qing and Zhu, Song-Chun and Fan, Lifeng},
year={2026},
eprint={2607.06537},
archivePrefix={arXiv},
primaryClass={cs.RO},
url={https://arxiv.org/abs/2607.06537},
}