Diagnosing autism can take half a day or more of clinical observation, and that’s the quick part – often, families wait years just to get to that point. Now, in hopes of speeding things up, Stanford researchers are developing a smartphone app that could drastically reduce the time it takes to get a diagnosis.

A smartphone app that could help diagnose autism uses a game to encourage kids to act out concepts such as being an artist or the feeling of being surprised. The app takes video of the kids at play to analyze and detect indicators of autism. (Image credit: Courtesy Wall Lab)

The heart of the app, called GuessWhat, is a game that encourages kids to act out concepts such as playing baseball or the feeling of being happy. But just as important, says creator Dennis Wall, an associate professor of pediatrics and of biomedical data science, is the fact that the app takes video of kids at play – video that preliminary work suggests can be analyzed to figure out if kids have autism.

With help from a Neuroscience:Translate seed grant from the Wu Tsai Neurosciences Institute, Wall, James Landay, a professor of computer science, and colleagues are expanding GuessWhat’s capabilities as not just a diagnostic tool but perhaps a therapeutic one as well.

“Children are missing an opportunity” to get help with autism, Wall said, and if the project is successful, it will “address a critical need in the diagnosis of autism.”

Charades as diagnostic tool

The original idea for GuessWhat, Wall said, came to him while playing a smartphone-based version of charades. In that game, players hold a phone on their foreheads, screen facing out, so that others can see a cue – a picture of an astronaut, for example – and try to guess that cue from what their friends act out.

Wall realized that by getting kids to act out a variety of different concepts – astronauts and the like, but also emotions or social situations – he might be able to capture video of children and use machine learning algorithms on that video to determine the probability any one child had autism. That, Wall said, could be useful both for diagnosis and for tracking developmental progress. And for parents, it could be done relatively quickly and without having to wait years for a visit to a specialized clinic.

“I thought if we could do something like this for autism, if could be really powerful,” Wall said.

Here’s how it currently works. After parents or other adults open the app and sign in, they hold the phone up to their foreheads, screen facing out so a kid can see it. The screen then displays an image – pictures of emojis or people in various jobs or social situations – for the child to act out. The adult then tries to guess what the image represents.

The difference from usual charades is the video. While a child plays, the smartphone’s camera captures video, which serves two purposes. In the initial stages, Wall, Landay and colleagues already know which kids have autism and which don’t, and the point is to analyze the video using machine learning methods to figure out which facial expressions, movements or other behaviors distinguish those with autism. From that, the app would learn to detect indicators of autism, which a child’s doctor could then use to screen kids without having to see them in a specialized clinic. Preliminary experiments, Wall said, suggest the strategy could work – and that the time is right to expand the team’s efforts.

Charades as therapy

Now, Wall said, “our goal is to build it up, and that’s where the seed grant comes in.” With that funding, the team is gearing up for field tests with a wider group of families, who will participate in co-designing the next version of the app. The team will also continue to gather data that could help the app better distinguish between kids with and without autism.

The seed grant will also go toward developing GuessWhat into a therapeutic as well as diagnostic app, creating what Wall calls an action-to-data feedback loop. “That could enable us to track progress using GuessWhat game play as a metric while treating the children” to be more able to function well in social situations, Wall said. “Once they’re more social, many will switch tracks from a delayed development track consistent with autism to a more typical development track.”

In the coming months, Wall and colleagues will work with clinicians to incorporate elements of two standard autism therapies, known as discrete trial learning and pivotal response training, into GuessWhat. Some features of those therapies, such as flashcards that teach kids to discriminate between emotions and games that emphasize imitation and social interaction, could be relatively easy to implement in a smartphone app, Wall said. Ultimately, the hope is to get ready for clinical trials to test GuessWhat’s therapeutic value sometime in the next few years.

But the most important goal may be simply to keep track of a lot of data – for example, which diagnostic decisions are made and why. “Medicine in general has failed to do a good job of record management,” Wall said. “So, when something happens in a doctor’s office – identifying a breathing abnormality with a stethoscope, a visual screen of a developmental delay, a screen of the eyes – much of what drives that doctor to arrive at a decision is lost.” By actually storing lots of data on kids playing games, researchers have a better place to start when trying to understand what autism is and how to address it.

“No one has ever captured this data before,” Wall said. “That creates an opportunity to do much, much more for developmental pediatric health going forward.”

Wall is a member of Stanford Bio-X, the Maternal & Child Health Research Institute and the Wu Tsai Neurosciences Institute. Landay is a member of the Wu Tsai Neurosciences Institute. Additional collaborators are Haik Kalantarian, a postdoctoral fellow in pediatrics and biomedical data science; Peter Washington, a graduate student in bioengineering; researchers Aaron Kline and Qandeel Tariq; and clinical research coordinators Kaitlyn Dunlap and Jessey Schwartz.

The research was supported in part by the Stanford Precision Health and Integrated Diagnostics Center, the Stanford Institute for Human-Centered Artificial Intelligence, SPARK Stanford and the Wu Tsai Neurosciences Institute’s Neuroscience:Translate Program.

To read all stories about Stanford science, subscribe to the biweekly Stanford Science Digest.