How Far Are VLMs from Privacy Awareness in the Physical World?
Abstract
ImmersedPrivacy is an interactive audiovisual evaluation framework for privacy awareness in physical environments. It separates perception, privacy awareness, and agentic alignment across cluttered scenes, shifting social contexts, and conflicts between explicit commands and inferred privacy constraints. Across 12 VLMs, no model exceeds 65% accuracy under social-context shifts.
Type
Publication
NeurIPS 2026
ImmersedPrivacy uses a Unity-based environment to test whether VLMs can perceive sensitive information, adapt to social context, and preserve privacy when task completion creates pressure to reveal it.
Xinjie Shen 沈鑫杰
PhD Student @ Georgia Tech
I study how to train capable agents and make them safe and reliable in open-ended environments.