Trouble viewing? Open in web browser.

Stanford News Service
January 19, 2017

Brief interventions help online learners persist with coursework, Stanford research finds

New research shows people in underdeveloped parts of the world are not as likely to complete massive open online courses, or MOOCs. But small psychological activities could help motivate them, closing the global achievement gap. View video here.

By Alex Shashkevich

Millions of people have taken free massive open online courses, or MOOCs, which have been touted as democratizing access to educational opportunities around the world. But whether learners are likely to succeed in a MOOC largely depends on where they live, according to new Stanford-led research.

René Kizilcec

René Kizilcec, doctoral candidate in Stanford’s Department of Communication, is lead author of a study showing how affirmation activities encourage MOOC completion in underdeveloped nations. (Image credit: L.A. Cicero)

A study published in the Jan. 20 issue of Science found that people in less-developed countries are completing MOOCs at a lower rate than those in the more developed parts of the world.

But, the researchers found, brief psychological interventions that affirm class takers’ sense that they belong can help close the global achievement gap.

“MOOCs have expanded access to education but this doesn’t guarantee equal opportunities for people around the world,” said René Kizilcec, the lead author of the study and a doctoral candidate in the Department of Communication. “Providing access to the Internet and courseware is not enough. People need to feel welcome in online-learning environments to reach their potential.”

Discovering the global gap

Online education gained momentum in 2011, when institutions and entrepreneurs began developing different MOOC platforms such as Coursera and edX. The higher-education landscape shifted and online course delivery became a viable option to extend learning opportunities to a global audience. But learning data made available by the courses themselves revealed that while many people enrolled in free, online courses, far fewer completed them.

Kizilcec began researching MOOCs at the onset. Together with an interdisciplinary group of graduate students, he co-founded Stanford’s Lytics Lab in the fall of 2012. Over the past five years, with support from the Office of the Vice Provost for Teaching and Learning and the Graduate School of Education, the lab has focused on studying the experiences of people who take MOOCs to advance the science of learning and instruction.

Kizilcec said his previous research showed that achievement rates in online courses varied based on gender and education level of the learners. But the biggest gap was geographical.

The geographic gap was quantified and visualized in the new study with data on completion rates of about 1.8 million people who have enrolled in Stanford’s MOOCs between 2012 and 2015 and with the United Nations’ Human Development Index, which measures the countries’ level of human development based on factors such as life expectancy, education and standard of living. The lower the development index was for the country associated with a learner’s IP address, the less likely the learner was to complete the course.

“Identifying this global achievement gap is one of the most profound things we were able to find out through this study,” said Andrew Saltarelli, a co-author of the research and a senior director of teaching design and practice in the Office of the Vice Provost for Teaching and Learning. “Maybe people had inklings that the gap was there, but there has never been an analysis of data like this to identify it.”

Psychological interventions

While limited Internet access and low English proficiency may contribute to the global achievement gap, there may be more to it, according to researchers.

“It’s not the average person in less-developed countries who is taking online courses,” Kizilcec said. “These learners tend to be well educated and highly motivated. So, instead of focusing on structural barriers that are complex and expensive to overcome, we tested a psychological explanation.”

Kizilcec and others theorize that a psychological barrier contributes to the global gap in MOOCs, namely social identity threat, which is a fear of being seen as less competent because of a social identity. Research has demonstrated that social identity threat can impair a person’s working memory and academic performance.

But Graduate School of Education and psychology Professor Geoffrey Cohen, who is a co-author on the study, showed in his previous work that simple interventions that encourage people to feel like they belong in the group helped reduce the effects of the social identity threat.

“Psychology matters even in a diverse worldwide population over the Internet, and a little gesture of inclusion can have a big effect on learning outcomes,” Cohen said.

Successful interventions Cohen used in face-to-face experiments were adapted for the online environment and implemented into the design of two MOOCs researched as part of the new study.

In both classes, learners were asked to complete an online activity before starting the MOOC. Some learners were randomly assigned a social belonging activity, which had them read and summarize testimonials from previous students about how they worried about belonging in the course at first but felt more comfortable with time. Others were assigned to an affirmation activity, which had them write about how taking the course reflects and serves their most important values.

The first experiment, which was conducted in a computer science MOOC offered by Stanford, collected data on 2,286 learners, of whom 16 percent were in less developed countries. The experiment was replicated in a public policy MOOC offered by Harvard with 1,165 learners.

The results showed that both types of interventions had a large effect on the performance of learners in less-developed countries, doubling their persistence and essentially eliminating the global achievement gap. The affirmation intervention even reversed the gap by raising completion rates for learners in less-developed countries from 17 percent to 41 percent.

“It is an impressive result which suggests that social identity threat can be a barrier to performance in international learning contexts, even in online environments with little social interaction,” Kizilcec said. “It goes to show that a small change to the online experience can have profound and lasting effects if it influences people’s perceptions of an environment.”

In one of the classes, the affirmation intervention, which helped those in less-developed countries the most, slightly reduced the completion rates of those in developed parts of the world. More research needs to be done to understand why, but some studies have shown that affirmation can cause people who aren’t under a psychological threat to disengage.

Improving the online environment

Using this new information about interventions, the next step for educators is to design exercises that target the right groups of people.

Over time, researchers hope that online courses at Stanford and other institutions will evolve as more studies from the Lytics Lab shine light on how best to approach teaching and learning in virtual environments.

“Many of us at Stanford believe that education is a public good,” Saltarelli said. “If there are groups of people who can’t be successful in learning experiences because of certain barriers, we have a responsibility to identify and remove the barriers.”

An additional co-author on the study is Justin Reich of the Massachusetts Institute of Technology.



Judith Romero, Office of the Vice Provost for Teaching and Learning: (650) 725-7289,

Alex Shashkevich, Stanford News Service: (650) 497-4419,


Update your subscription

Phone: (650) 723-2558

More Stanford coverage

Facebook Twitter iTunes YouTube Futurity RSS

©2017 Stanford University. Stanford, California 94305. (650) 723-2300.