Vision for the Birds

We will perform an initial technical demonstration of vision-based control of aggressive UAV maneuvers.

Birds can fly at high speeds through cluttered environments with apparent ease. We believe that experiments with birds operating at their performance limits in a controlled environment can provide a unique window into perceptual and motor control systems in biology, and that attempting to reproduce this performance in our machines could motivate the next technical revolutions in machine vision and nonlinear control.

We seek to perform an initial technical demonstration of vision-based control of aggressive UAV maneuvers. Tedrake’s lab has already developed controls technology to allow a fixed-wing, and more recently a flapping-wing, UAV to execute a dynamically challenging post-stall maneuver in order to land on a perch like a bird - but these results depend on the vehicle operating in a motion capture studio with offboard sensing of the vehicle location relative to the perch. We propose to leverage and transition Freeman’s vision technology, including image deblurring to compensate for the motion blurr induced by the motion of the aircraft, into an onboard solution which would enable the robotic plane or bird to locate the perch using onboard vision.

This can be done in stages – first using offline processing of image sequences taken during a perching maneuver in the motion capture studio, then to online offboard processing of the realtime image data, and finally to a complete online onboard resource-limited vision solution. In the initial phases of the work, the vision system will have a uniquely accurate source of calibration data coming from the existing motion capture system. Success will require considerably more than just juxtaposing the vision system and the control system – rather the severe dynamic constraints of the system require deep connections between the two – with the vision and estimation system making available the complete statistics of belief for the control system to reason about, and the control system taking actions to maximize the effectiveness of the vision system.