Principal Investigator Boris Katz
Co-investigator Shimon Ullman
Project Website http://cbmm.mit.edu/research/projects-thrust/visual-intelligence/grounded-langu…
Children learn to describe what they see through visual observation while overhearing incomplete linguistic descriptions of events and properties of objects. We have in the past made progress on the problem of learning the meaning of some words from visual observation and are now extending this work in several ways. This prior work required that the descriptions be unambiguous, meaning that they have a single syntactic parse.
Children learn to describe what they see through visual observation while overhearing incomplete linguistic descriptions of events and properties of objects. We have in the past made progress on the problem of learning the meaning of some words from visual observation and are now extending this work in several ways. This prior work required that the descriptions be unambiguous, meaning that they have a single syntactic parse. We are aiming at learning from ambiguous descriptions: simultaneously learning the meanings of the words while discovering the correct parse of the sentence. Ultimately, we aim for this work to simultaneously acquire a lexicon and a parser through visual observation.