Remot-IO: A System for Reaching into the Environment of a Remote Collaborator

Remot-IO is a system for mobile collaboration and remote assistance around Internet-connected devices. It uses two head-mounted displays, cameras, and depth sensors to enable a remote expert to be immersed in a local user's point of view, and to control devices in that user's environment. The remote expert can provide guidance through hand gestures that appear in real time in the local user's field of view as superimposed 3D hands. In addition, the remote expert can operate devices in the novice's environment and bring about physical changes by using the same hand gestures the novice would use. We describe a smart radio where the knobs of the radio can be controlled by local and remote users. Moreover, the user can visualize, interact, and modify properties of sound waves in real time by using intuitive hand gestures.