Biologically Plausible Implementations of i-Theory for Feedforward Face Recognition and Object Recognition in the Ventral Stream

The current main topics of our research are:

The first Turing++ Question: who is there?. Our goal is to develop a model of face identification in the ventral stream that perform well, correlates with human performance and accounts for physiology data in the macaque. The standard approach to unconstrained face recognition in natural photographs is via a detection, alignment, and recognition pipeline. While that approach has achieved impressive results, it does not pass our test, mainly because of its lack of biological plausibility. A recent theory of invariant recognition (i-theory) by feedforward hierarchical networks, implies an alternative approach to unconstrained face recognition similar to, but more general than, HMAX, some convolutional networks, or possibly the ventral stream. This approach accomplishes detection and alignment implicitly by storing transformations of training images (called templates) rather than explicitly detecting and aligning faces at test time. We have shown that models designed using this approach compare favorably to the best systems that operate on aligned, closely cropped images. We plan to show that it accounts for several properties of neurons in the face patches and for human performance in face recognition.

Unsupervised learning from videos: Populations of neurons in inferotemporal cortex (IT) maintain an explicit code for object identity that also tolerates transformations of object appearance e.g., position, scale, viewing angle. Though the learning rules are not known, recent results suggest the operation of an unsupervised temporal-association-based method e.g., Foldiak's trace rule. Such methods exploit the temporal continuity of the visual world by assuming that visual experience over short timescales will tend to have invariant identity content. As shown formally by i-theory, by associating representations of frames from nearby times, a representation that tolerates whatever transformations occurred in the video may be achieved. We have been investigating systems that learn in an unsupervised way from natural videos gathered from the internet, and are able to perform an unconstrained face recognition task on natural images.