Stanford’s Michael Rau is on a mission to reimagine theater for the digital age. As an assistant professor of directing and devising in the Theater and Performance Studies (TAPS) program in the School of Humanities and Sciences, Rau has long explored how theater can mirror the complexities of modern life. Now, with a seed grant from Stanford’s Institute for Human-Centered Artificial Intelligence (HAI), he’s taking that mission to a new frontier: integrating artificial intelligence into a live performance.

Rau’s work encompasses three approaches:

• Image generation: An AI-powered image generator receives a video feed from a live performance and interprets a prompt to alter the image of the actors. The AI-adjusted feed is then projected onto the stage simultaneous to the human production to create an image feed that might be seen as a dream sequence, an alternative story, or a different dimension.

• Large language model (LLM) live-scripts: Suggestions from the audience are plugged into an LLM, which generates a script based on the suggestion. Text-to-speech algorithms take the lines of the script and turn them into a voice. Each role is then separated and transmitted to an actor wearing an earbud, who speaks the dialogue written by the AI. This allows the actors to perform unencumbered by holding a script or reading lines off a monitor. Instead, they can respond in the moment, using the lines delivered via their headphones.

• Pose analysis: This analytical tool for scholars uses an AI algorithm to create a 3D image of actors’ body positions for every frame of a recorded performance. Developed in collaboration with Peter Broadwell, Simon Wiles, and Vijoy Abraham at the Center for Interdisciplinary Digital Research at Stanford Libraries, its potential applications include helping film scholars examine the choreography of an actor in a film, or enabling political scientists to analyze the gestures used by a politician during a speech.

Here, Rau describes how his innovative projects are sparking conversations about how technology can expand the boundaries of storytelling while preserving theater’s essential humanity.

How did you become interested in the intersection of theater and AI?

I firmly believe that good theater reflects the lives we’re living now. Our experiences today are mediated heavily by technology, but you rarely see that reality showing up on stage, because watching someone using technology isn’t interesting. I wanted to make plays in a different way – to figure out how people are using technology and how that could be portrayed effectively on stage. I began in 2014 making several performances that integrated technology into a theatrical performance. In 2019, looking at emerging technologies, I took an interest in machine learning algorithms.

How can AI enhance theater?

I want to find the intersection where the human is still making the art and the AI is either enhancing it or adding an interesting new aspect to the performance. The technologies I’m working with allow us to add a story layer on top of the choreography the audience sees on stage, to create spontaneous scripts that result in unrehearsed and very exciting performances, and to assist scholars in evaluating movement and poses using completely different tools. In all instances the human remains the main focus.

On stage, actors depict a sword-fight scene while an AI mimics their movements. | Birgit Hupfeld

Many express concern that the use of AI in the arts will be only superficial and derivative, without meaning or intent. Are these technologies art or merely gimmicks?

I struggle with this question all the time. The definition of art has always been subjective. This art form has existed for centuries, with nearly everything being tried in one way or another, so it’s hard to say what is or isn’t a gimmick. I’ve been very intentional in the specific and narrow ways I apply AI so as not to take away the interesting and expressive parts of human theatrical performance. What makes me feel more comfortable about AI art is that I always include a “human in the loop” element.

What are the benefits of AI in theater?

Honestly, we don’t know what the benefits will be yet. But given Stanford’s research mission, I think it’s better that we explore these technologies, and perhaps something will show up that is unexpected and exciting. The alternative, to ignore this technology, may end up calcifying this art form in a way that I don’t think will be helpful. It’s my hope that AI will give us more interesting creative options, allow us to work in different ways, and provide a layer of specificity and quantitative support to ongoing scholarly work.

What are its potential drawbacks?

First, there are major intellectual property and copyright concerns that are baked into the models; that must be solved fairly and equitably. Second, I’m concerned about the homogenizing effect of AI in the creative fields, such as its use in content curation to narrowly tailor our entertainment experiences. Third, I worry about a loss of “human-centered” creativity. Making art is one of the most exhilarating things that you can do – and if we hand off too much of that responsibility to machine learning systems, the artists will suffer. In the work that I do, I try to ensure that the human art making is still at the center of the work.

Will AI change the nature of theater or the relationship between writers, actors, and directors?

Technology has always changed the theater. For example, we’ve moved from candlelight to gas lamps to electric spotlights. That shift alone resulted in changes to makeup, costumes, scenery, and staging, and many felt it would destroy the theater. We’re at a moment now where we’re again discovering new technologies and figuring out which aspects are useful and how theater can change to accommodate them.

What I love about this art form is that both can exist; you can go to the Globe Theatre in London and see how Shakespeare was originally performed, and you can also go to the West End and see all kinds of new technologies deployed on our stages. It’s possible to preserve a particular style or method of performance, and it is also possible to try new ways of making a performance.

What is one project in AI you’re excited about?

I’m in development on a project with playwright Michael Yates Crowley for a play tentatively called Hamlet.AI. The plot focuses on a playwright and a director whose clash over the use of AI on stage turns into a discussion and battle about the future of art. The piece will incorporate both the image and text generation systems I’m working with now, as well as some other machine learning technologies that we’re still developing. It’s going to be a great melding of this technology and my complicated feelings about generative AI in the arts.