ILP Institute InsiderOctober 16, 2012
A Little Bit of Digital
Bringing the digital world into everyday objects and bringing everyday objects into the "Internet of Things."
He calls himself a "gadget head," but is "easily frustrated by technology." As the go-to IT guy among his family and friends, he knows he is far from alone.
Sound like anyone you know?
Since Holtzman joined the Media Lab in 1985, he has blazed new trails in digital interfaces, radio frequency ID (RFID) applications, and the "Internet of Things." Today he is the lab's Chief Knowledge Officer, as well as co-director of MIT's Digital Life Consortium and director of the Information Ecology research group. In the latter role he orchestrates an eclectic assortment of projects ranging from social-savvy mobile apps to novel digital devices.
Here we examine three: a "Proverbial Wallet" that expresses one's financial state through tactile response; "Droplet" and "StackAR" devices that can swap data with touchscreens using optical communications; and the "Glass Infrastructure," which combines RFID and public touchscreen displays to provide a personalized kiosk network. In the accompanying video, Holtzman reveals several other projects, including a "NeXtream" social network TV application and a "MindRider" bicycle helmet that translates EEG signals into colored LED displays that broadcast the rider's mental state.
The guiding principle behind these projects is to create more intuitive technology that is focused on a few tasks presented in the appropriate context. Holtzman's projects also tend to bridge the human/computer divide by combining wireless networking with multimodal sensory I/O.
"We're trying to explore the full range of human senses to give us a richer experience and interaction,” says Holtzman. "We want to help the technology learn more about the person."
The Proverbial Wallet
Holtzman's Information Ecology group has received considerable attention for its "Proverbial Wallet," a device that uses tactile response to further the goal of simplicity. The normal-looking wallet incorporates a Bluetooth radio to connect to a smartphone app, which in turn links up to a user's financial websites. The app synthesizes one's current financial situation into signals that drive servos and vibrators within the wallet. For example, if you are living beyond your means, the hinge of the wallet is restricted.
"As your wallet gets harder to open, you maybe know you need to change your lifestyle a bit," explains Holtzman. "It's not something that's always there in your face, but not something you'll forget about, either."
The Proverbial Wallet does only a few things very simply. This "goes against the grain" of the trend of cramming multiple capabilities into "the ultimate device," such as a smartphone or tablet, says Holtzman.
While that trend will likely continue for several years, with continuing reductions in the cost of processors, sensors, and wireless radios, Holtzman believes this dynamic will give way to a networked ecosystem of devices. These more focused gadgets will interact with the user, other devices, and cloud-based services.
Instead of being forced to configure, learn, and search for apps and functions on a smartphone, people will increasingly tap into an array of devices that provide more focused and intuitive functionality, says Holtzman. Some will be carried or worn, but most will be shared technology embedded in our homes, businesses, and public spaces.
"Carpenters don't use a Swiss Army Knife—they use a whole toolbox full of tools," says Holtzman. "We are trying to build devices that are more tailored for a particular task or context. Instead of concentrating all our digital access through a four-inch screen, we take a little bit of digital and put it all around us."
A Droplet of enlightened I/O
The Information Ecology group's Droplet and StackAR projects apply a similarly direct approach. The Droplet is a ping-pong-ball-sized half dome that can be applied directly to a touchscreen to interact with a particular screen element via capacitance sensors and optical communications. Light flashes exchanged with a Droplet-enabled app can swap selected data to and from the device more quickly and intuitively than is possible by finding and inserting a USB flash drive and loading it with data, says Holtzman.
In addition to communicating with the screen, the Droplet uses light to communicate with the user, glowing or flashing different colors based on its stored data. For example, as a calendar appointment stored on the device grows near, it can change from blue to red to flashing red.
"Droplets can exchange data such as a website bookmark, so you could apply it to your screen and then toss it to a friend across the room," says Holtzman. "They'll be so inexpensive you can pass them between people like you're passing paper notes."
The Droplet can also act as a "user interface tool on its own," says Holtzman. By holding it up to a screen, the information can be programmed to visually grow out of it as an onscreen graphic. Twisting the Droplet one way or the other like a dial lets the user directly interact with the visualization.
"The Droplet can act as an anchor for a piece of information," says Holtzman. "For example, we've developed a radial clock like representation for appointments, showing information like the time and date."
A related StackAR project uses the concept to "make a very nice portable development environment for embedded devices," says Holtzman. StackAR integrates Droplet technology into an add-on shield for an Arduino Lilypad controller board. When you place the shield-enabled Lilypad on a touchscreen, it generates a graphical depiction of the device, including all pinouts.
"The StackAR display shows where all the signals are, so you can virtually wire up a circuit and try out programs," says Holtzman. "You can then flash the program back into the Arduino and plug it into the real version of the circuit. The point is that you don't always have to have a full-fledged UI with a screen and a bunch of buttons that will only be used occasionally. The device can have the bare minimum, and if you need to reconfigure it, you can bring it together with another device, and have an in-depth interaction grow very naturally."
Glass Infrastructure adds depth to kiosks
Holtzman's Information Ecology group has been leading the way on one of the Media Lab's more ambitious projects in recent years: the Glass Infrastructure. A collaboration among multiple departments, the project combines touchscreen technology with a familiar workhorse of Holtzman's career: RFID.
Operational today in the Media Lab, the Glass Infrastructure appears on its surface like any other modern, multimedia-ready kiosk one might find in a museum. Yet thanks to the dropping price of large touchscreens, there are several dozen networked displays distributed throughout the building, and they're big, ranging in size from 46 to 70 inches.
The key innovation of the Glass Infrastructure is that it can personalize experiences by reading visitors' RFID badges, which are preloaded with personal profiles and preferences. The system knows who you are as you move between screens, and it suggests projects based on previous visits, pointing users to multimedia content posted for each project.
"The Glass Infrastructure allows people to see what's happening in the building in a more cerebral way than they can with their eyes," says Holtzman.
In addition, the system takes advantage of the size of the screens to encourage discussion between visitors standing nearby. "It's designed for creating a rich dynamic social space," says Holtzman. "It recognizes multiple people at once, and shows you what you have in common, letting you trade bookmarks and share interests."
A number of organizations have approached the Media Lab for advice on building similar networks. "We've been talking to companies that want to create an environment to give their workers a better idea of what everyone is doing and how it relates," says Holtzman.
Beyond general kiosk applications, financial service providers are looking to apply the concept to their customer centers. "Instead of talking to your financial advisor about your investment strategy across a desk, you can both stand together in front of a touchscreen," says Holtzman. Meanwhile, a major conference service provider is interested in using the technology to provide attendees with a more interactive and personalized view of presentations and exhibitors.
Digital signage and advertising may prove to be the biggest application for the Glass Infrastructure. Companies are interested in extending the capabilities of digital billboards to offer targeted advertising and promotions.
"If a billboard is not interesting to a particular person, it's a wasted opportunity," says Holtzman. "But if the billboard knows something about the people in front of it, it can target the message, and presumably they'll pay more attention."