Spatially-Grounded Communication for Mental Model Alignment in Human-Robot Teams

Abstract: There is great potential for humans and autonomous robots, each possessing their own capabilities and strengths, to perform tasks collaboratively across a number of domains, achieving greater performance than either could on their own. Effective teamwork, however, requires a great deal of coordination. In other words, human and robots must maintain well-aligned mental models regarding the shared task and each agent's role within it, communicating to rectify those models during times of mismatched expectation. However, since humans and robots "think" in vastly different planning spaces, this communication is non-trivial. In this talk, I will discuss my research developing novel systems, algorithms, and interfaces for explicitly synchronizing mental models via agent-to-agent communication during live human-robot collaboration, in tasks ranging from tabletop manipulation to environment navigation and search. In particular, I will focus on spatially-grounded communication methods, including augmented reality interfaces capable of displaying visual information at key locations within a shared environment, and natural language explanations tied to such spatially-grounded features. Such methods establish shared context between human and robot teammates, allowing for compact bi-directional communication of environment and task information, thus facilitating the alignment of mental models between agents and improving subjective and objective measures of team performance.

Bio: Matthew B. Luebbers is a PhD candidate in the Department of Computer Science at the University of Colorado Boulder, advised by Bradley Hayes. His research interests lie at the intersection of robotics, multi-agent reinforcement learning, and human factors, focusing especially on the development of novel algorithms and communication interfaces to enable effective human-robot teaming, including work nominated for best paper awards at the AAMAS 2022 and HRI 2024 conferences. Prior to Colorado, Matthew received a BA in Computer Science from Cornell University in 2018. He has also spent many summers interning at NASA's Jet Propulsion Laboratory, working in various roles for three Martian surface missions - the Curiosity rover, the InSight lander, and the Perseverance rover. While working on these missions, he has participated in multiple rover drive sequencing shifts, accruing a career Martian odometry of 228 meters (86 m on Curiosity, 142 m on Perseverance). This fall, Matthew will be joining the Georgia Institute of Technology as a postdoctoral fellow in the School of Interactive Computing, advised by Matthew Gombolay.