Prof. Deb K Roy

When used by people, words are symbolic reflections of rich mental networks that encode knowledge of the physical and social world. Beneath words and sentences lie complex cognitive structures shaped by evolutionary and lifetime learning, linked in various ways to the goals and environment of language users. The meaning of words in everyday language depends on two very different kinds of relations. On one hand, language refers to (is about) the world -- spatial, temporal, and social factors situate communication. On the other hand, people use language to pursue their interests -- speech acts are purposeful. A grand challenge for the cognitive sciences is to develop a computational framework that simultaneously models referential and functional meaning. In practical terms, progress towards meeting this challenge will have significant impact on a broad spectrum of applications in human-machine communication and machine-mediated human-human communication.

Professor Roy's research group is driven by this challenge. We are developing natural language processing systems that ground language in physical and social context. We have implemented conversational robots, natural language understanding in video games, sensor-grounded computational models of language acquisition, and context-aware speech interfaces for home and work environments. A common denominator in all of the domains we tackle is that the meaning of words cannot be fully determined without paying attention to non-linguistic context.