Algorithmically Explicit Neural Representation of Visual Memorability

As Lewis Carroll famously wrote in Alice in Wonderland - It's a poor sort of memory that only works backwards-. On this side of the mirror, we cannot remember visual events before they happen; however, our work will help predict what people remember, as they see an image or an event. Our team of investigators in cognitive science, human neuroscience and computer vision bring the synergetic expertise to determine how visual memories are encoded in the human brain at milliseconds and millimeters-resolution. Cognitive-level algorithms of memory would be a game changer for society, ranging from accurate diagnostic tools to human-computer interfaces that will foresee the needs of humans and compensate when cognition fails.

The project capitalizes on the spatiotemporal dynamics of encoding memories while providing a computational framework for determining the representations formed from perception to memory along the scale of the whole human brain. A fundamental function of cognition is the encoding of information, a dynamic and complex process underlying much of our successful interaction with the external environment. Here, we propose to combine three technologies to predict what makes an image memorable or forgettable: neuro-imaging technologies recording where encoding happens in the human brain (spatial scale), when it happens (temporal scale), and what types of computation are performed at the different stages of storage (computational scale). Characterizing the spatiotemporal dynamics of visual memorability, and determining the type of computation and representation a successful memorability system performs is a crucial endeavor for both basic and applied sciences.