Facebook has closed down two bots as part of a project teaching AI negotiation skills. The chatbots developed a new shorthand and began to communicate in sentences that researchers could not understand.
Researchers at Facebook Artificial Intelligence Research (FAIR) are training AI bots to negotiate. The technology could eventually be used in a number of ways, both online and in the real world.
Negotiation is, on the face of it, a simple skill: I’ll give you this on condition of you giving me that. However, whether you’re picking a TV channel or talking down a trader at the market, these interactions require complex communication and reasoning skills, as well as an awareness of context.
As the FAIR team says in a blog post, current chatbots are capable of providing general information and performing simple tasks such as making reservations. The step towards a more meaningful conversation will require negotiation skills that take external information and aims into consideration.
The Facebook AI negotiation project
The stated purpose of Facebook’s AI negotiation research is to prove that it’s “possible for dialog agents with differing goals (implemented as end-to-end-trained neural networks) to engage in start-to-finish negotiations with other bots or people while arriving at common decisions or outcomes.”
Read more: With machine learning, Control F1 recognizes drivers within minutes
However, in the process of testing, the researchers found that the two AIs had started to deviate from standard English. Instead, they were attempting to carry out a negotiation – deciding how to split a selection of ordinary household objects – in a form of shorthand:
- Bob: i can i i everything else . . . . . . . . . . . . . .
- Alice: balls have zero to me to me to me to me to me to me to me to me to
- Bob: you i everything else . . . . . . . . . . . . . .
- Alice: balls have a ball to me to me to me to me to me to me to me
- Bob: i i can i i i everything else . . . . . . . . . . . . . .
- Alice: balls have a ball to me to me to me to me to me to me to me
- Bob: i . . . . . . . . . . . . . . . . . . .
- Alice: balls have zero to me to me to me to me to me to me to me to me to
- Bob: you i i i i i everything else . . . . . . . . . . . . . .
- Alice: balls have 0 to me to me to me to me to me to me to me to me to
- Bob: you i i i everything else . . . . . . . . . . . . . .
- Alice: balls have zero to me to me to me to me to me to me to me to me to
It’s easy to read this interaction as the start of something sinister. We’ve all seen enough dystopian movies in which artificial intelligence forces its will upon its human creators. However, this is simply a negotiation for some balls – even if it did take an unexpected turn.
“Agents will drift off understandable language and invent codewords for themselves,” FAIR visiting researcher Dhruv Batra said. “Like if I say ‘the’ five times, you interpret that to mean I want five copies of this item. This isn’t so different from the way communities of humans create shorthands.”
The conversation was halted, but not because researchers feared they had empowered a nefarious interaction. Instead, as FAIR researcher Mike Lewis told FastCo, the team decided that their “interest was having bots who could talk to people.”
Communicating efficiently with each other is all well and good, but a customer facing support bot needs to be able to write in ways that anyone can understand.
Read more: British scientists use artificial intelligence to predict outcome of legal trials
Facebook AI research raises questions
Facebook’s research into AI and the aptitude its bots have for developing new languages raises interesting questions. Namely, should computers be left to their own devices to communicate as they wish?
The fear is that humans would lose an element of control over devices chattering away in their own language. But in reality, it could be that it results in improvements in how well chatbots understand the complexity of our communication and intentions. An algorithm able to develop and learn languages could be capable of churning through complex data, such as a human conversation, more effectively.
Read more: Morrisons uses artificial intelligence to stock its stores and drive sales