The innovation consulting industry has created hundreds of potential prescriptions ranging from open innovation tools to 20% time to broad culture training. Most methods are promoted as silver bullets that will improve an organization’s innovativeness with one simple shot. Yet evidence about the effectiveness of each method, or even how to do each method well, is often much less powerful than the sales pitch. We examine one particular innovation method, the innovation contest. We compare two different methods of evaluating innovation ideas using detailed data on more than ten thousand submissions over four years of a large firm’s successful process. This contest, which requires very little direct investment, has generated more than fifty winning ideas, many of which are now moving into production. It has also been credited with instilling a culture of innovation and collaboration among many employees who would not have seen innovation as part of their roles in the past. The selection mechanisms exhibit different cost profiles and different drivers of success. A combined approach can improve selection while optimizing use of expensive resources. The findings can also energize employees, provide advice to potential innovators, and help the firm identify hidden innovators across the organization.