Modeling Urban Nature: The Role of AI and Digital Twins in Tree Health Monitoring
Sara Beery
Assistant Professor of AI and Decision-Making, MIT Department of Electrical Engineering and Computer Science
Natural world images collected by communities of enthusiast volunteers provide a vast and largely uncurated source of data. As an example, iNaturalist has enabled the collection of over 250 million images that are taxonomically identified by their community and then contributed to GBIF as species occurrence records. But these images contain a wealth of "secondary data" that gets lost when we label images with species alone, including crucial insights into interactions, animal social behavior, morphology, habitat, and co-occurrence. The analysis needed to surface valuable scientific insight beyond species is currently costly, time-consuming, and expert-dependent. We propose interactive, open-ended image retrieval as a mechanism to support scientific discovery in these collections, and introduce INQUIRE, a novel text-to-image retrieval benchmark built to provide a rigorous evaluation that challenges models to demonstrate advanced knowledge and visual reasoning on expert, scientifically impactful retrieval tasks. We demonstrate several case studies exploring the use of our tool to rapidly test ecological hypotheses, and discuss the need for innovation in statistical techniques to understand uncertainty in retrieval-derived trends.