Print

Teaching online yields lessons about learning, Stanford scholars say

MOOCs on computer science and math yield rich rewards, scholars tell a forum audience. Among them is the feedback they get about how students learn.

Mandy McLean Keith Devlin, Scott Klemmer, Dan Boneh

Stanford scholars, from left, Keith Devlin, Scott Klemmer and Dan Boneh reported at a faculty forum on "Research Opportunities in Online and Blended Learning" how teaching MOOCs yields lessons about learning.

What's Darwinian, thrilling and so rewarding it's like a drug you just want to take again and again?

Online teaching, at least according to three Stanford scholars who have been there and back. They participated May 22 in the third of a series of faculty forums sponsored by the Office of the Vice Provost for Online Learning (VPOL) and the Graduate School of Education (GSE). The trio were among Stanford's pioneers in the world of massive open online courses, or MOOCs.

Panelists at the forum, titled "Research Opportunities in Online and Blended Learning," were Keith Devlin, a mathematician and executive director of Stanford's Human Sciences and Technologies Advanced Research Institute (H-STAR); Scott Klemmer, associate professor of computer science and co-director of Stanford's Human-Computer Interaction Group; and Dan Boneh, professor of computer science and of electrical engineering. The moderator was Mitchell Stevens, associate professor of education.

Devlin's course, Introduction to Mathematical Thinking, was first offered in fall 2012 and will reappear this coming fall.

"Mathematical thinking is not the same as math," he said. "It's the technique of taking a real problem in the real world that hasn't been solved before and conceptualizing it in mathematical terms."

Many people drop out of math classes in school, Devlin said, and then are unable to see that mathematical thinking can unlock many human problems. "It's like learning to ride a bike; there's only one way to learn: Get on, fall off, get on, fall off, until it clicks. In math, that click usually happens only when you're a grad student grading papers." In the world of MOOCs, it means tinkering, looking at feedback, reading the forums, adjusting and doing it again. "It's a Darwinian form of education," he said.

Devlin is known to most people as "The Math Guy" on NPR Radio's Weekend Edition. As someone used to speaking on the airwaves, he said, a MOOC was impossible to resist. And just as the trick to good radio work is to imagine you are speaking to only one person, not millions, so too with MOOCs. That direct link between student and professor is essential, but so is collaboration among students. Learning has to be a collective enterprise, he said, and that's where peer assessment comes in.

Students grading students

Klemmer and his associates developed a system a year ago by which MOOC students must grade five of their peers' work on class projects. Unbeknownst to the students, one of the five projects they assess already has been assessed by faculty, so it has what Klemmer calls a "ground truth grade," which enables instructors to see how far wide of the mark the peer grades are. The system has been used since last spring in some 50 MOOCs, including Devlin's.

Feedback is delivered in a single aggregate number, but to make it slightly more qualitative, peer assessors can attach what Klemmer calls a fortune cookie, a little piece of standardized advice that might also go out to other students but which the assessor thinks is especially suited to the project he or she is looking at. Klemmer's group created a video about peer assessment in the hope that more courses will incorporate it.

Klemmer's enthusiasm for online teaching stems in part from the fact that he was trained as a designer. His goal, he said, is to mimic the experience of the studio, and he thinks they got pretty close with the Human-Computer Interaction course. The studio model of education, he said, "is very similar to the best research group meeting you've ever been at." The web can put that sort of communal, iterative experience within the reach of MOOC students.

Boneh, Klemmer's colleague in the Computer Science Department, works on cryptography and computer security. He currently is teaching his online Cryptography I course for the second time. Cryptography has a lot of math, which most people don't know, and it bears no resemblance to its portrayal in Hollywood movies in which a genius somehow figures out how to break the system.

"The challenge," Boneh said, "is to take something very appealing, very sexy, and explain that it's not what you see in the movies; it's math. The first time I recorded the class, I realized I got it wrong." People were asking him very basic questions on the online forums. They didn't understand symbols. A 12-year-old asked him what probability was. "I wasn't aiming at that audience," Boneh said. "I was thinking Stanford. And that was not the right way of thinking about it."

As a result, the second iteration explains everything and assumes nothing. Not only that, but Boneh has taken the lessons he learned online and applied them to his Stanford course. He also has learned how to teach 12-year-olds.

Interacting on the forums

For Boneh, the forums were tremendously rewarding. "Interaction is a key part" of online teaching, he said. He would spend maybe an hour a day talking to his students, answering questions, pointing them in new directions.

"There was an unbelievable response," he said. "Students were so happy that they're actually interacting with a Stanford professor – people around the world who have never even seen a professor. The feedback was so rewarding it was like a drug – I wanted to do more and more of it."

Devlin too hung out on the forums, though he said he tended to sit back and watch the discussion rather than intervene, later posting the URLs of those threads he thought were most productive. The forums are "remarkably self-regulating," he said. "People start flaming, and they quickly disappear. There's a strong sense of 'we're learning, let's get the most out of this experience.'"

Boneh raised a problem he doesn't have an answer for: the extraordinary pace of change in the materials he teaches. Just a few months after he records a video on computer security, the material might be obsolete. It helps that video segments are short, so that only outdated sections need to be re-recorded. Still, faculty will somehow need to be incentivized to continually update the materials, he said.

Boneh, as the security expert, was asked about cheating, to which he replied that he didn't really care. "It's very interesting learning how to teach by not caring about cheating. You cheat, it's your problem ... I'm there to teach, not to evaluate," he said, to which the moderator, Stevens, a sociologist, added, "you've been freed of the responsibility of assessment."

Klemmer, who will be taking a position at the University of California–San Diego in the fall, responded to a question from Bruce Clemens, a Stanford engineering professor who taught a MOOC last fall called Solar Cells, Fuel Cells and Batteries. How can these online mechanisms teach us about teaching? Clemens asked.

"We have a really interesting opportunity here," Klemmer said. "Many faculty at Stanford are excellent teachers, but many of them know little about the literature about learning. What's fascinating to me is that over the past year," since MOOCs began being taught, "I've had more conversations with colleagues here about teaching than during the previous nine years."

R. F. MacKay is a writer for the Office of the Vice Provost for Online Learning.