Every day, 10 hours a day, for three weeks, three times every year, Stanford researchers and their local partners would wade into rivers and estuaries under the baking Senegalese sun and gather samples. For each of the more than 30 sites, they would cordon off a small quadrant of roughly a square meter and tear up every bit of aquatic vegetation within the area, capture all the snails, weigh everything, meticulously record results, and repeat, 15 times per site. They would place the snails in plastic vials and then into coolers for the drive back to the lab, where the snails were classified and dissected.

The team – led by Giulio De Leo, professor of oceans and of Earth systems science in the Stanford Doerr School of Sustainability and co-director of the Stanford program for Disease Ecology in a Changing World, and Andy Chamberlin, a Stanford research associate – wanted to unveil the environmental drivers of schistosomiasis, a debilitating parasitic disease of poverty, one of the most important of the so-called Neglected Tropical Diseases, affecting 250 million people worldwide, the vast majority school age children in sub-Saharan Africa.

The research team gather snails to study for schistosomiasis, a parasitic disease spread by freshwater snails. | Giulio De Leo

“Andy with our research partners in Senegal from Espoir Pour la Santé, and some of our other students dissected zillions of snails,” says De Leo, a senior fellow of the Stanford Woods Institute for the Environment. “The work was absolutely necessary to discover these risks, but we can only do so much locally. We needed to replicate this on a much bigger scale.”

A 2018 seed grant from the Stanford Institute for Human-Centered AI (HAI) laid the foundation for this scaling. With the $50,000 grant, De Leo recruited Zac Liu and John Bauer, two Stanford AI researchers, to develop convolutional neural networks that could identify images of medically relevant snails as well as vegetation associated with schistosomiasis. De Leo was able to parlay this proof-of-concept into a second seed grant by the Woods Institute, and then into $2.5 million of funding from the National Science Foundation under the Ecology and Evolution of Infectious Diseases program. Using this money, he and his team have developed a machine learning platform that integrates data from fieldwork, drones, and satellite imagery to provide schistosomiasis risk analyses across huge swaths of land.

“I cannot overstate how catalytic this early support has been,” De Leo says. “Without the bridge funds provided by HAI, which gave us the freedom to explore this novel application, none of this would have been possible.”

Inferring snails by satellite

In the project’s early days, the research team had tried to sketch out schistosomiasis transmission patterns using relatively low-quality satellite images. But the maps were too fuzzy to correlate individual pixels with the kinds of details they were finding through their surveys on the ground.

Chamberlin, who has become an expert in applying machine learning to disease ecology, flew mapping missions by drone to bridge this gap, capturing high-quality images of water access points in which individual vegetation types could be discerned; the researchers knew that particular vegetation types, in turn, predict higher schistosomiasis infection rates year after year.

The view of a sample-collecting site by a drone | Andy Chamberlin

“We were able to extrapolate what we know from really fine-scale field work to these larger drone images with a high degree of accuracy,” Chamberlin says. “And then we could use that to evaluate satellite imagery over the same time period and a much broader area, which enabled us to do more regional-scale analysis and monitoring.”

The linchpin in the research, supported by the initial HAI grant, came from Liu and Bauer: a set of machine learning tools that stitched together these three streams of information, ultimately providing a picture of potential infection hotspots.

The methodology today can be used both to monitor populations for rates of schistosomiasis and to prioritize public health outreach in populations that are at risk of exposure.

Investing in local capacity

Through this work, De Leo and his team have developed a close partnership with Station d’Innovation Aquacole and the Université Gaston Berger (UGB) in Saint Louis, Senegal. This past fall, Chamberlin ran a week-long workshop, co-organized with research partners Maïssa Mbaye and Cheikh Bamba Dione, professors of computer science at UGB. The workshop was designed to train master’s and graduate students in computer science on image processing, how a computer represents an image in data, and deep learning techniques for imagery analysis.

De Leo and his team teach workshops on computer vision and deep learning techniques. | Maïssa Mbaye

The skills and tools imparted from this collaboration along with other partnerships between Stanford and Senegalese researchers has implications far beyond schistosomiasis and disease ecology. De Leo noted that one near-term application of this work is to estimate rice production in the lower basin of the Senegal River using drone imagery in a follow-up project supported by the Stanford Sustainability Accelerator. The techniques can also be applied to environmental monitoring more broadly, Chamberlin explained.

Mapping deforestation, finding mosquitos

De Leo and his team are already finding novel ways to take what they’ve learned studying snails and apply it in other areas. They have, for instance, integrated highly detailed lidar maps of Indonesian Borneo’s rainforest with coarser satellite images to estimate canopy height across the forest. While impossible from satellite alone, machine learning and image analysis tools helped identify patterns between the lidar data and the satellite images, which can provide a picture of deforestation and lead to important conservation and general land management policies. (This work was also supported by an HAI grant.)

Another project, also in Indonesia, led by Joelle Rosser and Desiree LaBeaud at Stanford School of Medicine, uses machine learning to identify discarded tires in urban environments; tires are particularly attractive environments for mosquito breeding, which makes them a potential source of malaria and dengue.

“It’s one thing to capture images, but it’s another to really analyze them, and for a long time the challenge of analysis has blocked a lot of research around the world,” Chamberlin says. “Combining these drone platforms with machine learning has been a journey. We’ve learned a lot. I’ve learned a lot, personally. And there’s much more to learn – and it started with snails in muddy water.”

Related story

Researchers work to outrun the risk of ‘river fever’

At Stanford and in Senegal, experts are tackling schistosomiasis in rice paddies, using fish to cut disease transmission while boosting food security and local economies.