Identifying struggling young readers can be a time-consuming and costly task for schools, requiring a teacher or reading specialist to sit with students one-on-one to gauge their proficiency as the child reads aloud.

Go to the web site to view the video.

A new online tool developed at a Stanford lab lifts that burden without compromising any of the reliability of one-to-one assessments while advancing research into why some kids have trouble with reading in the first place.

The Rapid Online Assessment of Reading (ROAR), developed at the Brain Development & Education Lab at Stanford, introduces a way for school districts to assess their entire student population for struggling readers in the time it currently takes to run a standard assessment on a single student.

In addition to giving teachers useful insight into the challenges a particular student faces, the collective data generated by the assessment is helping to further the lab’s research into factors linked to learning differences in young readers.

“With the ROAR, schools and clinics can assess and monitor kids’ progress at a scale that just wasn’t possible before,” said Jason Yeatman, an assistant professor at Stanford Graduate School of Education (GSE) and the School of Medicine, who directs the Brain Development & Education Lab. “And because the tool is tied to research that’s ongoing, it gives us data that can answer a lot of questions about the mechanisms of reading development – data that can help us understand why some kids struggle and others don’t.”

A pandemic workaround takes off

The ROAR actually originated as a COVID-19 research workaround for the lab, where Yeatman and his team study the mechanisms in the brain that underlie learning.

One focus of the lab is to develop interventions for young children with dyslexia, research that relies on brain imaging studies to observe the effects an intervention has on the brain. That work includes developing targeted measures to evaluate a child’s growth in specific reading skills.

“All of this requires working in close, one-on-one or small group settings with children,” said Yeatman. “When the pandemic hit, with all of its uncertainty, we realized that we needed to come up with some new innovations in order to continue pushing our research forward.”

The researchers thought an online assessment might function reasonably well in place of the standardized in-person version they had been using, and they developed a prototype of the ROAR. Subsequent validation studies showed a remarkably high-reliability rate: Scores generated by the ROAR correlated strongly with those from standardized in-person screenings, including the Woodcock-Johnson assessment of basic reading skills, widely considered the gold standard.

“The correlation was almost perfect, meaning these measures are tapping into a very similar construct,” said Yeatman. “Honestly, we didn’t expect it to work as well as it did.”

Meanwhile, Yeatman and his team had been working with local school officials around new California guidelines issued to help teachers identify and support students with dyslexia. That relationship – developed through the Stanford-Sequoia K-12 Research Collaborative, a partnership launched in 2018 to connect GSE researchers with San Mateo County school districts to explore challenges facing the districts – laid the groundwork for helping to pilot and refine the ROAR.

“We’d had a lot of conversations [with Yeatman] about how we could get more data, where we could find a good universal screener, how to go about implementing the guidelines – we learned a lot from him and his team,” said Marta Batlle, student services director at the Woodside School District, one of the districts in the collaborative. “When they asked if we wanted to partner to administer the ROAR, we were really interested in having more data, and we wanted to be part of the research process.”

Making a game of it

The ROAR, which runs on any web browser, launched with an assessment that measures students’ ability to quickly recognize words, a foundational skill for reading fluency and comprehension. It’s designed to feel like a computer game, with cartoon-like characters walking students through the activity and encouraging them along the way. A mix of actual and made-up words flash on the screen; the student’s task is to determine whether each word is real or not.

“The kids love it,” Batlle said. “They’re engaged, and they want to do it.”

Students as young as first-graders can do the assessment on their own, in the classroom or at home.

The lab collaborated closely with school leaders to develop interactive score reports for the educators, providing statistics and visualizations from the district level down to individual classrooms and students.

“The reports provide immediate, actionable data back to reading specialists and teachers, right after students have taken the ROAR,” said Amy Burkhardt, director of research and partnerships at the Brain Development & Education Lab. (The scores are stripped of any identifying student data for Stanford’s use.)

“With the ROAR, we got a measure for every student in the classroom in about the same amount of time it takes us to administer our usual screener with one child,” said Ching-Pei Hu, assistant superintendent at the Belmont-Redwood Shores School District, another district in the Stanford-Sequoia K-12 Research Collaborative that worked with Stanford to pilot the ROAR. “It’s fast, it’s efficient, and it was reliable in giving us a baseline of who we need to dig into some more.”

Looking ahead

As they work with schools and clinics to refine the ROAR’s tool for assessing single-word recognition, Yeatman and his team are developing others for more complex reading skills, including phonological awareness (the ability to recognize and manipulate sounds) and the ability to silently read and understand full sentences quickly and accurately.

Through pilot funding from Stanford’s Office of Community Engagement, the team recently expanded their research collaborations to include KIPP Schools of Northern California. The lab is also collaborating with the Developmental-Behavioral Pediatrics clinic in the Stanford School of Medicine to explore the use of these assessments as a screener for children with learning differences who enter Stanford Hospitals and Clinics.

Beyond providing a practical tool for clinicians and educators and a better understanding of learning differences, the researchers hope the data generated by the ROAR will ultimately help in developing tailored approaches to help struggling readers.

“You can see kids from the same background – the same families, even – entering kindergarten, where they learn how letters can be combined to make words and represent human language,” said Yeatman. “Some kids dive right into that and they’re suddenly reading, but others struggle deeply for many years to learn this mapping, to make it automatic, to be able to use written language as a fluid form of communication. We’re trying to understand the difference – and the hope is that as we begin to understand the barriers for some kids, we can develop targeted interventions.”