MUST-Loc: Multi-view Uncertainty-aware Semantic Token Association for Object-level Global Localization

Object-level global localization is highly sensitive to semantic uncertainty from viewpoint variations in open-set scenarios. To address this problem, we present MUST-Loc, a multi-view, uncertainty-aware semantic token matching framework. The key idea is to aggregate object-level tokens through online updates in the mapping process to form mean–variance descriptors, capturing viewpoint-induced variability while maintaining semantic consistency. At the localization query, we compute uncertainty-aware semantic similarity, which down-weights high-variance token dimensions to establish reliable correspondences under semantic ambiguity. Finally, the camera pose is estimated by selecting the solution that maximizes the Wasserstein-based alignment score between observed detections and projected landmark hypotheses. For rigorous validation, we evaluate on challenging TUM RGB-D sequences with occlusions, label noise, and diverse categories, showing consistent improvements over baselines in association and pose accuracy.