John Williams

“We don't fully understand the properties of generative AI, which makes it difficult to predict how it will behave,” says Williams, Professor of Information Engineering in the Department of Civil and Environmental Engineering at MIT. “We need to learn how to test, align, and constrain these complex systems.”

As a physicist and systems design analyst with expertise in building simulations, Williams brings a novel perspective to the AI problem. Over his diverse and storied career, he has applied his expertise to building simulations for non-continuum mechanics, computer networking, IoT, cybersecurity, civil and environmental engineering, and software systems design.

Williams views generative AI as one of many complex adaptive systems he has studied over his career. The term, which describes a system that is never in the same state twice, can also be applied to the global climate, which has grown even more unpredictable due to greenhouse gas accumulation.

It is difficult to simulate systems such as an advanced AI model or the climate, which at some stages may become chaotic

“It is difficult to simulate systems such as an advanced AI model or the climate, which at some stages may become chaotic,” says Williams. “To simulate the climate, you must account for fluid flow, ice mechanics, solid mechanics, and atmospheric physics, as well as systems such as transportation, agriculture, and carbon capture. All these systems are interacting with each other at various spatial and time scales. No wonder the predictions are so different.”

From Ice Flows to the Internet of Things
Williams was born in Wales and studied physics at Oxford University where he played on the rugby team that beat the world champion Springboks. As he moved on to UCLA and then back to Swansea University in Wales for his PhD, he became fascinated with physics simulations.

Following his PhD, Williams joined an international consulting company and applied his simulation skills to studying the impact of ice flows on offshore oil rigs in the Arctic. “When I was visiting the oil rigs, slabs of ice up to 13 feet thick would ride up the sides, threatening to take off the deck,” he recounts. “The first night was terrifying. The rig was shaking like it was being ripped off the seabed.”

This gripping experience led Williams to the study of non-continuum mechanics. “The mechanics of ice is not continuous, and the assumptions of continuum theory don't hold,” says Williams. “You've got a complex internal structure with flaws and fractures that during impact undergo high stresses and explosive-like failures, shattering the ice into discrete blocks.” 

Williams co-founded a startup called Applied Mechanics where he led the team that invented the Discrete Element Method (DEM), a way to simulate the movement of ensembles of deformable particles.  DEM led to the development of CICE, a commercial simulation application sold to oil companies.

Non-continuum mechanics can also be used to explain the behavior of granular materials such as soil, sand, and the powders in drug capsules. Civil engineers, for example, use non-continuum mechanics to study the transformation of soil during an earthquake. “Soil is normally solid, but in an earthquake it can liquify, causing buildings to collapse,” says Williams.

After a brief stint back at Swansea where he co-wrote a book on rock mechanics, Williams moved on to take a faculty position at MIT. Here he developed software for simulating particulate systems, which led to a surprising turn toward computer networking.

“I realized that the same concepts used to program forces acting between particles in non-continuum mechanics could be viewed as messages being sent between networking nodes,” says Williams. “I helped Ford simulate its global computer network, which enabled them to reduce the number of data centers they needed.”

Williams’ unique combination of expertise led him to become the second director of the MIT Auto ID Center after Sanjay Sarma. At the Auto-ID Center, he helped develop the first RFID tags. He also served on the EPCGlobal Architectural Review Committee (ARC), which designed the early architecture of the Internet of Things.

Williams went on to co-lead MIT’s System Design and Management (SDM) program where he studied the behavior of complex systems. At SDM, he helped develop Product Development 21, a movement led by Xerox, Ford, and MIT. The project was a precursor to the Lean Dev-Ops and Agile movements that transformed software development.

Can Generative AI be Tamed?
Today Williams spends much of his time studying Large Language Models (LLMs) and generative AI, which offers immense opportunities along with some weighty challenges. “It is difficult to predict the behavior of generative AI or know when it will hallucinate,” says Williams. “When you chat with ChatGTP, you are not talking directly to an LLM, but to a front-end agent that mediates your conversation by leveraging several other AI systems. The underlying transformer neural network uses an ‘attention’ mechanism to predict the next word, but it does not allow us to predict when it will hallucinate. Testing is difficult because even a slight change in the prompt terms or the word order can lead to vastly different responses.”

Lack of explainability is a related challenge. “Current AI platforms lack the self-reflection to explain how they came up with an answer,” says Williams. He adds, however, that humans are not much better. “You don't know what your neurons are doing. If I throw you an object, you can mentally calculate its trajectory. Yet you are not calculating F = MA. You are using another subconscious method we do not fully understand.”

Williams is exploring several methods that might make generative AI more dependable and explainable. “One approach is to have one LLM monitor and constrain another,” says Williams. “You can also limit the AI to answer from a constrained set of documents fed to the LLM via the context window. These Retrieval Augmented Generation (RAG) systems should improve explainability and reduce hallucinations but are limited by the accuracy of the documents retrieved.”

Despite all the challenges with generative AI, Williams is excited about its potential, especially in education. He has developed a generative AI agent modeled on himself, which has been trained on over a thousand videos from previous courses.

My AI Tutor can interact with students at precisely their level of understanding, addressing the challenge of teaching a diverse range of students in MIT classes

“My AI Tutor can interact with students at precisely their level of understanding, addressing the challenge of teaching a diverse range of students in MIT classes,” says Williams. “Even better, it is always available, consistently polite, and never gets tired.”

Digital Transformation
Generative AI plays a leading role in a course Williams teaches on digital transformation. “Technology and society are changing so rapidly that industry and governments are struggling to keep pace,” he notes. “Organizations are already struggling with the fast pace of innovation, the complexity of the cloud enabled distributed data pipelines, and new development processes using Agile and DevOps frameworks. Now generative AI, while promising enormous benefits, is adding even more complexity and uncertainty.”

Rapidly advancing software is making it increasingly difficult for companies to accommodate change, says Williams. “As Patty McCord explains in her book, ‘Powerful’, the processes companies develop to manage complexity often become a burden on employees and a barrier to change. The biggest challenge of digital transformation is not technology, but people. Part of the problem lies in the C suite and company structure, which has always depended on top-down command and control. This works well when change is slow and largely predictable. However, as David Snowden points out with his Cynefin Framework, chaotic and complex adaptive systems must be managed very differently compared to predictable systems.”

Some companies have begun to push decision-making downward to where the expertise lies, so the ideas “percolate up,” says Williams. “Yet, this is a difficult transition.”

In the face of so much uncertainty around technology and societal changes, long-term planning is difficult, says Williams. The world is increasingly becoming a complex adaptive system that defies our predictions.

“Traditional five-year plans don’t work anymore,” says Williams. “Yet companies can still plan for a year or two. You can no longer plan every step, but you can still get to your destination.”