Augmented Reality with X-Ray Vision

X-AR is a new AR headset that enables users to see things that are hidden. This new headset helps users find items that are inside closed boxes, under piles, or behind occlusions.  The headset uses holograms to guide users toward desired items and verify when they have picked them up.

How does it work? X-AR uses wireless signals and computer vision to enable users to perceive things that are invisible to the human eye (i.e., to deliver non-line-of-sight perception). It combines new antenna designs, wireless signal processing algorithms, and AI-based fusion of different sensors.

This design introduces three main innovations:
(1)  AR-conformal wide-band antenna that tightly matches the shape of the AR headset visor and provides the headset with Radio Frequency (RF) sensing capabilities. The antenna is flexible, lightweight, and fits on existing headsets without obstructing any of their cameras or the user's field of view.

(2) AR-Based SAR localization algorithm that fuses RF sensing from the antenna and visual sensing from the headset's cameras to localize RF-tagged objects, even when they  are hidden from the view, as the user naturally walks in the environment.

(3) RF-Visual Verification Primitive that fuses RF and vision to deliver actionable tasks to end users such as picking verification.

How well does X-AR work? X-AR is very accurate:
(*) It can locate hidden items with an accuracy of less than a foot (9.8 centimeters).
(*) It achieves over 95% accuracy in picking verification.

These results show that X-AR is successful in extending AR systems to non-line-of-sight perception, with important implications to warehousing, retail, e-commerce fulfillment, and manufacturing applications.