Collective Prediction: Combining Human and Machine Intelligence in Prediction Economies

This project will attempt to combine human and machine intelligence in flexible new ways to make accurate predictions about future events such as product sales, political events, and outcomes of medical treatments.

Think of a domain in which you would like accurate predictions of future events: sales volumes for a company's products, outcomes of sporting events or military conflicts, crime rates by neighborhood, terrorist actions, new products introduced by a company's competitors, or the clinical outcomes of different medical treatments someone you know might receive for cancer.

Now imagine a network of humans and computers that makes predictions in this domain--not perfectly, but better than was possible before. And imagine that these predictions get better and better over time as the network learns from its own experience. We propose to do some of the essential research needed to help create such networks.

A number of researchers have recently developed prediction markets in which participants buy and sell predictions about uncertain future events and are paid only if their predictions are correct. Such prediction markets have been found to be surprisingly accurate in a wide range of situations (including forecasting product sales and US Presidential elections).

We propose to build on this previous work to develop prediction economies -- networks of people and computers paid (either in currency or points) for accurate predictions about future events. A prediction economy can include (a) one or more prediction markets (b) markets for various other kinds of information relevant to the events being predicted, and (c) markets for services by people (such as image analysis) or by machines (such as multiple regression, machine learning, and data mining).

Importantly, both people and their automated agents will be allowed to participate in any part of the economy. For instance, automated agents can do "program trading" in two related prediction markets whenever they see inconsistent prices in the two markets. In this way, prediction economies provide a flexible new approach to integrating human and machine expertise: People have an incentive to create new automated agents whenever they can codify useful expertise algorithmically, and they have an incentive to participate in markets directly when they can do a better job than the existing automated agents. But when people can't improve on what the automated agents are already doing, then they have no incentive to intervene.

Drawing on theories in organization science, computer science, cognitive science, and economics, this work will develop new forecasting and collaboration tools that blend human and machine capabilities to more accurately forecast risks and opportunities, thus helping to build more agile systems in many domains.