Earlier this year, a robot took the internet by storm by sauteing shrimp. In addition to this culinary feat, Mobile ALOHA was shown performing other acts associated with the long-held dream of general-purpose household robots, such as vacuuming, doing laundry, and watering plants.

While some of the tasks Mobile ALOHA has performed on video weren’t autonomous, its ability to someday perform nearly any daily chore is not far-fetched. Behind the dazzling demonstrations is a unique teleoperation structure that allows Mobile ALOHA to learn many complex activities, impressively fast.

“We’re providing a general framework that allows people to show robots how to do a task and then allows robots to learn from what they were shown,” said Chelsea Finn, assistant professor of computer science and of electrical engineering at Stanford. Finn advises the Mobile ALOHA team, which is led by computer science graduate students Zipeng Fu and Tony Z. Zhao.

Some general-use robots are programmed step-by-step and others, including Mobile ALOHA, learn through demonstration. But the Mobile ALOHA researchers took imitation learning a step further: The researchers strap themselves into a teleoperation system directly behind the robot’s arms and puppeteer the robot through the desired actions.

Once Mobile ALOHA is operated through a task in a set environment about 50 times, powerful imitation learning algorithms help it make the leap to doing that task independently. (These algorithms are similar to the large language models behind popular chatbots, but for physical action instead of words.)

So far, the researchers have taught Mobile ALOHA to autonomously put away a cooking pot in a cabinet, call an elevator, push in chairs, sauté shrimp, clean up a wine spill, and give high-fives. Future versions of the robot may be smaller with greater freedom of movement, and be easier for non-experts to operate.

“The robot isn’t just a cool machine, we also wanted it to be fun and appeal to what people think a future robot should look like,” said Zhao.

“Helping robots is a very promising future of the field where we – as AI researchers and roboticists – can make a positive impact in society,” said Fu.