Prof. James DiCarlo

Peter de Florez Professor of Neuroscience
Director, MIT Quest for Intelligence
Investigator, McGovern Institute

Primary DLC

Department of Brain and Cognitive Sciences

MIT Room: 46-6161A

Areas of Interest and Expertise

Neuronal Mechanisms Underlying Object Recognition
Object Recognition in the Visual System
Systems and Computational Neuroscience

Research Summary

The research goal of DiCarlo's laboratory is to understand the mechanisms underlying visual object recognition. Specifically we seek to understand how sensory input is transformed by the brain from an initial representation (essentially a photograph on the retina), to a new, remarkably powerful form of representation -- one that can support our seemingly effortless ability to solve the computationally difficult problem of object recognition. We are particularly focused on patterns of neuronal activity in the highest levels of the ventral visual stream (primate inferior temporal cortex, IT) that likely directly underlie recognition. At these high levels, individual neurons can have the remarkable response property of being highly selective for object identity, even though each object’s image on the retinal surface is highly variable – for example, due to changes in object position, distance, pose, lighting and background clutter. Understanding the creation of such neuronal responses by transformations carried out along the ventral visual processing stream is the key to understanding visual recognition.

To approach these very difficult problems, the work of our laboratory is directed along three main lines: 1) characterize the computational usefulness of patterns of IT neuronal activity for supporting immediate visual object recognition, 2) test and develop computational theories of how visual input is transformed along the ventral processing stream from a pixel-wise representation, to a powerful representation in IT, 3) understand the spatial organization of this representation. Our primary research approaches are: neurophysiology in awake, behaving non-human primates, functional brain imaging (fMRI), human psychophysics, and computational modeling. Across all of these endeavors we aim to develop innovative methods and tools to facilitate this work in our laboratory and others. Our approaches are often synergistic with those of other MIT laboratories, and this has greatly enhanced our progress.

Research is focused on the crux problem of visual object recognition, which is called the “invariance” problem. This problem results from the fact that each object category can present an essentially infinite number of images to us – due to changes in object position, distance, pose, lighting, background, deformation, and exemplar variation. Yet somehow the brain is able to determine that all of these images still contain the same object category.

Recent Work

  • Video

    5.25.21-Quest-for-Intelligence-DiCarolo-Yildiz

    May 25, 2021Conference Video Duration: 113:41

    James DiCarlo
    Director, MIT Quest
    Peter de Florez Professor of Neuroscience
    Head, Brain and Cognitive Sciences

    Bilge Yildiz
    Professor, Nuclear Science and Engineering
    Professor, Materials Science and Engineering

    2.23.21-AI-DiCarlo

    February 23, 2021Conference Video Duration: 18:5
    James DiCarlo
    Peter de Florez Professor of Neuroscience
    Co-Director, MIT Quest for Intelligence 
    Head, Brain and Cognitive Sciences
    Investigator, McGovern Institute

    Making Sense of Sight

    October 17, 2016MIT Faculty Feature Duration: 21:3

    James DiCarlo
    Head, Department of Brain and Cognitive Sciences, Peter de Florez Professor of Neuroscience