Entry Date:
January 25, 2017

Modeling and Verification of Language-Based Interaction

Principal Investigator Nicholas Roy

Project Start Date August 2014

Project End Date
 July 2019


Many autonomous systems today, such as personal or service robots, are designed primarily to perform tasks independently and in isolation. Integrating these robots with human partners can often result in poor performance, as the robot does not know how to interpret human interaction, and cannot merge information from this interaction with a model that guarantees robot performance. This research brings together key elements that are just now reaching a sufficient level of maturity for integration: firstly, natural language processing and probabilistic modeling to capture human input, and secondly probabilistic synthesis and verification of the combined human-robot systems to ensure correct performance. The outcome will be theory and software to enable correct, effective and natural interactions between robots and humans to be realized. This research will impact most future autonomous systems which require interactions with humans, including service, personal and planetary robots.

The goal of this research is to develop models and algorithms for synthesizing and verifying an integrated human-plus-robot system based on natural language interaction. Algorithms are being developed for probabilistic modeling and inference of natural language, including the grounding of the constituents of the language into the physical world and the human's expectations. These models will enable the development of a distribution over specifications for control synthesis, which will in turn enable the development and verification of correct-by-construction controllers to a particular level of probability. The out years will consider interactive human-robot dialogue to resolve conflicts, and "open world" scenarios to enable on-line learning of new models over time. It is expected that this research will enable high reliability and performance in many autonomous systems because of the inherent interaction with humans. Outcomes include open source data and software; community workshops; and undergraduate and graduate student education in the unique area of language, modeling and verification for robotics.