Researchers from computer hardware giant NVIDIA have created a deep learning system that can teach robots to carry out a task just by watching a person’s actions.
The hope is that the new system could help collaborative robots, or ‘cobots’, work alongside human beings more easily and intuitively.
NVIDIA’s researchers have developed neural networks that enable a test robot to learn and mimic a task from a single demonstration in the real world. The test robot’s camera allows a pair of neural networks to judge the positions of, and relationships between, objects in real time.
It then generates a human-readable description of the steps necessary to recreate the task. This allows users to identify and correct any mistakes in the robot’s interpretation before final execution of the task by the machine.
In the demo, researchers trained the robot using coloured blocks and a toy car. The system was taught the physical relationships between the objects, whether they were stacked on top of one another or placed next to each other by a human trainer, and was then able to recreate the arrangements.
Learning from synthetic data
The researchers relied on synthetic data to teach the system first. They explained that current approaches to training neural networks require large amounts of labelled training data, which is a major bottleneck in programming such systems. Using synthetic data generation, an almost infinite amount of labelled training data can be produced with very little effort.
According to NVIDIA, this is the first time that an image-centric domain-randomisation approach has been used to train a robot. Domain randomisation is a technique to produce synthetic data with large amounts of diversity, which then fools the system into seeing real-world objects as variations of the training data.
“The perception network, as described, applies to any rigid real-world object that can be reasonably approximated by its 3D bounding cuboid,” the researchers said. “Despite never observing a real image during training, the perception network reliably detects the bounding cuboids of objects in real images, even under severe occlusions.”
The researchers will present their work at the International Conference on Robotics and Automation (ICRA), in Brisbane, Australia, this week. They said they will continue to explore the use of synthetic training data to extend robots’ capabilities into new scenarios.
Internet of Business says
Robot programming can be a laborious and time-consuming task, as can compiling the data for training computer vision systems, so this research – while primitive at present – represents a significant breakthrough in robotics and AI.
NVIDIA is most closely associated with manufacturing high-end graphics processing units (GPUs) for use in enterprise data centres, cryptocurrency mining rigs, and games platforms, among other applications.
However, in recent months it has intensified its presence in AI and driverless vehicle technologies – a canny move, given that AI at scale relies on exactly the capabilities that its GPUs specialise in: enterprise-grade number crunching.
Earlier this year, the company partnered with Pure Storage to produce an AI supercomputer in a box, AIRI, which can be slotted into data centres as a dedicated unit for AI processing.
But NVIDIA has also suffered the slings and arrows of technology fortune in 2018. While its share price has soared over the past four years – tracking the growth of the cryptocurrency market almost exactly, while denying that its fortunes are closely linked with that sector – it suffered badly in the wake of two fatal accidents in March involving an autonomous Uber car, and a Tesla vehicle running on Autopilot.