Understanding Translucency: Physics, Perception, and Computation

People care greatly about the appearance of translucent materials such as food, skin, soap, and marble, and they are able to distinguish subtle differences in these materials based on their appearance. The translucent appearance of these materials is caused by internal volumetric scattering, which is challenging to simulate, especially because humans are so sensitive to their subtleties. Since in the natural world scattering materials are the norm, not the exception, it makes sense that the human visual system is so well engineered to analyze them. However, very little is known about how this analysis is achieved because the perception of volumetric translucency is almost unstudied.

This collaborative project, involving faculty from three universities with complementary expertise in computer graphics, human vision, machine learning, and computer vision, addresses the fundamental unsolved problem of understanding translucency for graphics. The PIs will develop a perceptually-motivated pipeline for translucency, contributing new scattering representations, perceptual dimensions, and computational algorithms to computer graphics. The scattering representations, based on a polydispersion model, will provide analytic expressions for wavelength-dependent bulk scattering properties of translucent media; this will significantly expand the range of materials that can be simulated with high visual fidelity. Finding perceptual knobs that relate physical scattering parameters with visual appearance will be achieved by coupling large-scale computation (using cloud computing) with controlled perceptual studies. Novel acquisition approaches that employ hyperspectral imaging will be created, as will editing and rendering applications that use the new perceptual representations of translucency. Low-dimensional models to represent scattering media will be developed and used to enable efficient and accurate acquisition and rendering. A suite of test materials and scenes will be developed to evaluate the fidelity of rendered images based on the developed theory and computational applications.

Broader Impacts: Currently, the simulation of translucency presents challenges in terms of both computation and visual fidelity. This restricts the ability of practical algorithms to predictively simulate translucent materials, thus fundamentally limiting the use of graphics in real applications. By building the computational tools to characterize, study, and use knowledge of translucency perception, this research will fundamentally change the graphics pipeline for translucent materials. and will potentially revolutionizing industrial design, interior design, skin care and cosmetics, and entertainment.

The project includes an education program that is tightly coupled to the research program. The PIs have already been meeting twice a week for more than six months, and their graduate students already share data, code, and equipment. During the activity, the students will make week-long and month-long visits to each other's laboratories to collaborate, and in this way the project will produce a generation of researchers who are "T-shaped" in the sense of being both deep in their respective fields and able to work effectively across these synergistic disciplines. The PIs also plan to organize a workshop that will brins together researchers in vision science, computer graphics, and computer vision, so that the important ties between these fields are strengthened even further.