VR Trains Home Robots On Arbitrary Tasks

The good news is Japanese tech giant Toyota’s Research Institute (TRI) may have a solution: using virtual reality (VR) to let humans quickly transfer their knowledge to robots – enabling them to perform human-level tasks in real homes.

“When teaching a task, a person can try different approaches, making use of their creativity to use the robot’s hands and tools to perform the task,” TRI researchers wrote in a press release. “This makes leveraging and using different tools easy, allowing humans to quickly transfer their knowledge to the robot for specific situations.“

To help motivate this aspect of learning, instead of feeding the bot a set of predetermined parameters, researchers decided to use VR training systems to allow human teachers to see what the robot was seeing live, in 3D, from its sensors. Based on that information, the teachers then instructed the robot and annotated the 3D scene, such as equating parts of the scene to a behavior, specifying how to mop the floor, grasp a handle, or pick up a cup. This approach allowed human trainers to quickly transfer their every day skills to the robot, making it more flexible in certain situations.

TRI’s researchers said robots are required to continuously compute a plan of motions based on how they perceive their surroundings. In this case, however, the system does not need prior object models or maps. It simply needs to be taught to understand the objects that are relevant to a behavior being performed.

That said, TRI’s home helper robot isn’t perfect yet. Researcher note that a key limitation to this proof-of-concept VR-based system bot is that it can’t generalize what it has learned and apply similar skills to other robots or different situations. Still, the VR concept is very important when it comes to the future of robotics. If we enable robots to learn either from a person or in simulation, and then share data in the cloud about their experiences with other robots, then bot capabilities are going to grow exponentially with profound impacts on robotics.