Negin (Nicki) Golrezaei

Nicki Golrezaei joined the faculty at MIT in 2018 after spending a year as a postdoctoral researcher with Google Research in New York, designing and testing new algorithms and mechanisms for online marketplaces. Her research areas of interest include machine learning, statistical learning theory, mechanism design, and algorithm optimization with applications in revenue management, pricing, and online markets. For the past few years, her research has focused on designing machine learning and data-driven algorithms for online marketplaces and efficient resource allocation in online marketplaces.

In an online marketplace, third-party companies (sellers) connect with customers (buyers) through a digital middleman (the marketplace) like Amazon. “Our ability to access the internet anywhere, anytime has fueled the emergence of fast-growing online marketplaces,” says Golrezai. The pandemic served to accelerate the trend, driving businesses and consumers online at unprecedented rates. Some experts forecast that by 2024 global business-to-business (B2B) online marketplace sales could reach $US 3.6 trillion annually, while business-to-consumer online marketplace sales for the same period are estimated to hit US$ 3.5 trillion. Meanwhile, B2B e-commerce site sales alone are predicted to reach nearly $US 1.77 trillion in 2022—an increase of 12 percent from the year prior.

Bad data leads to bad decisions. Garbage in – garbage out.

Given the sheer size of these marketplaces and their contributions to the global economy, maintaining a healthy ecosystem is essential. But, as Golrezaei points out, it’s tricky. One of the great challenges for companies looking to optimize business decisions in online marketplaces with machine learning algorithms is that their algorithms are designed to trust the data available. But what if the data isn’t trustworthy? Put simply, “Bad data leads to bad decisions. Garbage in – garbage out,” writes Golrezaei in her article, “How to Produce Cleaner Data for Robust Pricing.”

Frequently, in a two-sided marketplace (buyers on one side, sellers on the other) bad data is generated by what Golrezaei calls “strategic players” with vested interests. In other words, companies populating the marketplace looking to game the system. Sellers train their algorithms on data that is influenced by the actions of buyers. Unscrupulous buyers, aware that their behavior informs seller decision-making processes, can be tempted to manipulate sellers into charging less for their product or service.

Take online advertising markets, otherwise known as ad exchanges. These are digital marketplaces that enable advertisers and publishers to buy and sell advertising space through real-time auctions. Both sides of the market leverage machine learning algorithms to optimize their auction parameters, mostly around pricing. The seller typically designs a learning policy to set prices based on past sales data. According to Golrezaei, it is not uncommon for advertisers to intentionally submit low bids intended to demonstrate a low willingness to pay for ad space, thereby manipulating the ad exchange’s pricing algorithm.

In an effort to curb the buyer’s strategic behavior and limit price manipulation, Golrezaei and her colleagues designed a pricing algorithm that uses the outcomes of the auctions rather than the submitted bids. “The outcome of the auction is really just a binary signal that tells you whether or not a particular advertiser wins or not,” Golrezaei explains. 

With Golrezaei’s solution in place, if an advertiser lowers their bid, changing the binary signal, they lose the auction altogether. “My algorithm helps to provide a healthy ecosystem for the online advertising market, allowing both sides to use their algorithms, but making it costly for advertisers to manipulate the system,” she says.

My algorithm helps to provide a healthy ecosystem for the online advertising market, allowing both sides to use their algorithms, but making it costly for advertisers to manipulate the system.

Online retail stores and search engines also suffer from data manipulation. When we shop online, we’re inundated with a barrage of options, and the order in which those options appear is important because human beings have a limited attention span, and we tend to home in on what is readily apparent. “Evidence suggests that how products are ranked and displayed on platforms heavily influences customer purchasing decisions and content engagement,” explains Golrezaei. In her paper, “Product Ranking on Online Platforms,” Golrezaei cites Millward Brown’s findings that 70 percent of Amazon users never go beyond the first page of search results.

Recognizing our tendencies, sellers and content providers engage in a race for visibility that can lead to fraudulent behavior. The internet is awash with blogs and sites offering tricks and hacks to help sellers boost sales. In 2018, The Wall Street Journal published an article detailing, among other scams, the rise of click-farms, groups of low-wage workers hired to repeatedly click on the links of a particular enterprise to artificially inflate engagement statistics and gain access to higher-visibility positions. Customers presented with content or products that don’t match their search queries will eventually leave a platform, and distorted data due to click fraud leads to billions of dollars lost for e-commerce sites yearly.

Golrezaei and her colleagues designed machine learning algorithms that demonstrate how platforms can efficiently learn the optimal product ranking despite fake users. Their solution is to keep parallel copies of a learning algorithm with different levels of conservatism. “Being conservative helps algorithms avoid being easily manipulated by fraudulent data,” she explains. “Our parallel algorithms communicate with each other to learn the right level of conservatism and to make decisions about how to rank products.” 

In recognition of her research, Golrezaei received the 2021 Young Investigator Program (YIP) Award from the Office of Naval Research (ONR). Known as one of the oldest and most selective U.S. science and technology basic-research programs, the ONR YIP funds early-career academic researchers “whose scientific pursuits show outstanding promise for supporting the Department of Defense, while also promoting their professional development.”

Her three-year project, "Finding a Needle in a Haystack: Utilizing Structures and Predictive Information in Online Optimization," involves designing powerful, fast-learning algorithms that take advantage of the structures and predictive information regarding the underlying time-varying combinatorial environments where decision-makers have a great deal of uncertainty about those environments to facilitate decision-making processes. Examples of combinatorial environments include portfolio optimization, job scheduling, influence maximization in viral marketing, and product ranking on online platforms. 

Machine learning algorithms are a great low-cost method to mitigate problems in online marketplaces and beyond. Working with industry is about finding that sweet spot, combining academic research with practicality.

“Decision-makers are often faced with too many options and not enough time to experiment,” says Golrezaei.” Her research aims to limit the necessity of experimenting with all of the options while still being able to identify the best course of action as quickly as possible. "I want to better equip decision-makers who face an exponentially large number of actions from which to choose,” she says.

Golrezaei’s work helping online marketplaces leverage new developments in machine learning is applicable to a variety of industries, and she's worked extensively with social media companies and organizations from the insurance sector looking to benefit from her cutting-edge research. “Machine learning algorithms are a great low-cost method to mitigate problems in online marketplaces and beyond,” says Golrezaei. “Working with industry is about finding that sweet spot, combining academic research with practicality.”