Google has released an open source framework for reinforcement learning (RL) based on TensorFlow, its machine learning library.
The Google Brain team has developed the framework with flexibility, stability, and reproducibility in mind.
Reinforcement learning – a branch of machine learning in which computer programmes learn to take actions to maximise a certain outcome or reward – has come a long way in recent times.
The best-known working examples of this type of AI come from DeepMind, the University College London startup that Google acquired for a reported £400 million at the beginning of 2014.
DeepMind’s RL projects have included DQN, a programme able to master a range of Atari 2600 games and, more recently, the successful AlphaGo, AlphaGo Zero, and Open AI Five systems – all of which were able to play games beyond the ability of expert-level humans.
The challenge of building reinforcement learning systems
Developing RL systems often requires researchers to iterate designs quickly, sometimes disrupting established frameworks with no obvious path to follow to find a solution.
Writing in a Google AI blog published this week, Google researchers Pablo Samuel Castro and Marc Bellemare pointed out how flaws in this methodology can slow down the development process and restrict exploration.
“Most existing RL frameworks do not provide the combination of flexibility and stability that enables researchers to iterate on RL methods effectively, and thus explore new research directions that may not have immediately obvious benefits,” they wrote.
“Further, reproducing the results from existing frameworks is often too time-consuming, which can lead to scientific reproducibility issues down the line.”
The package from Google seeks to offer ease of use, tools that reflect the importance of reproducibility, and a raft of training data that will allow new researchers to benchmark their ideas swiftly against established methods.
More details can be found in the GitHub repo, named Dopamine.
Internet of Business says
As well as releasing its RL framework and training data, Google is launching a website that will enable developers to visualise training runs for multiple agents – a move that should save an incalculable amount of time for those exploring new avenues in RL.
“Inspired by one of the main components in reward-motivated behaviour in the brain, and reflecting the strong historical connection between neuroscience and reinforcement learning research, this platform aims to enable the kind of speculative research that can drive radical discoveries,” said Castro and Bellemare.
“Our hope is that our framework’s flexibility and ease of use will empower researchers to try out new ideas, both incremental and radical. We are already actively using it for our research and finding it is giving us the flexibility to iterate quickly over many ideas. We’re excited to see what the larger community can make of it.”