Robots in the wild: UTM’s Florian Shkurti teaches machines to learn and work with humans
The technology behind self-driving cars has been progressing swiftly, and so long as the vehicles are on familiar streets with familiar sights they do very well. But when they encounter something unfamiliar, things become much more challenging, explains Florian Shkurti, an assistant professor in the Department of Computer Science at U of T Mississauga. He gives an example: Suppose there’s a truck in front of the self-driving car, “and there’s a wind gust – and now the snow is coming at you, so you can’t see anything. And suppose your LIDAR (light detection and ranging system) misperceives the snow as an array of objects, so it thinks there are a million small objects coming at the car.” Naturally, this will lead to trouble, says Shkurti, who runs UTM’s Robot Vision and Learning (RVL) lab.
Shkurti’s research extends far beyond self-driving cars to autonomous systems in general: How they learn, how we can make them learn better, and, especially, how they can successfully navigate complex environments at the service of humans. That means making sure that robots can handle so-called “edge cases,” like the snowy truck example – cases where the robot “comes across a rare scenario, for which there’s little or no training data,” says Shkurti. “Then you really have to either collect more data, or you have to accept that there will be these rare events that your perception system won’t recognize.”
An important tool along the way is simulation: self-driving cars, for example, can be trained on simulated roads and highways before they’re let loose on actual city streets. Another major challenge is scalability: If an autonomous system has to be specially trained for every possible scenario it might encounter, progress will be haltingly slow; there will be no way to take what’s been learned from one scenario and scale it up so that the system can handle more general cases. Roughly speaking, it would be good if the robot could learn similar to the way a human would.
Shkurti gives the example of robots that help scientists collect data underwater – an effort he’s been involved with for several years. Suppose the scientist wants to dive underwater to check on the health of coral reefs. A human diver “has to collect data manually, one data point at a time, one location at a time,” says Shkurti. “It’s painstaking work; it’s not scalable.” But now suppose that an autonomous robot, capable of maneuvering underwater and equipped with a camera and other sensors, could take over the data collection process. “If the robot could understand what it’s doing – if it has a model of what the scientist thinks is important to pay attention to, in a particular environment, then the robot could collect data on behalf of the scientist.” Scaling it up now becomes straightforward: It’s much cheaper to deploy additional robots than to train more scientists. It also frees up the scientist to look after higher-level tasks, now that they don’t have to devote so many hours to data collection. “The scientist can give the robot some hints as to where to collect the data – but then the robot can take care of the rest,” Shkurti says.
Shkurti has recently received a Connaught New Researcher Award for a project titled “Robotics and Machine Learning in the Wild: New Directions in Automated Environmental Monitoring.”
Shkurti did his undergraduate studies at U of T before earning his PhD at McGill in computer science. He was hired by U of T in 2018. While everything about computer science fascinates him, the field of robotics has always held a special appeal. “Robotics lets you play in different ‘playgrounds,’ like control, perception, and machine learning,” he says. “It allows you to examine these different fields, and I really valued that – and I still value it.”
As for the lofty philosophical questions that sometimes crop up when people talk about advanced computer systems – like the question of whether machines could be said to “think” – Shkurti remains firmly agnostic. Machines can reason, he says; they can try to act optimally as they strive to achieve their goals. “If that’s thinking, then they’re doing it,” he says. “But I don’t spend very much time worrying about ‘consciousness.’ I have enough other things to worry about.” And despite Hollywood’s obsession with robots that turn on their human masters or accelerate doomsday in some fashion, Shkurti sees robots as helpers; he’s looking forward to a new and richly-rewarding era of human-robot collaboration. “No roboticist I know has a pessimistic view of the field,” he says.