Researchers at MIT’s Media Lab have developed a machine learning system that ‘reads’ facial expressions to determine human emotions.
The rush to develop machines capable of interpreting the intricacies of human communication and expression comes at a time when some believe that affective computing could provide the missing link between conventional robotics, software, and solutions that are genuinely beneficial for certain people.
Potential applications of emotion-reading systems range from unobtrusively tracking mental wellbeing to gauging student interest in classrooms or customer interest in stores.
The challenge of interpreting emotion
Being able to ‘read’ people is a skill that many human beings take for granted. For machines, it’s not as easy as it is for an intuitive or emotionally intelligent person. Particularly when there are so many variables at play and context is so important – not to mention cultural differences.
One person may express emotions differently to another, for example. And that’s before factors such as age, culture, and gender are added to the mix. Delving deeper, the time of day, the presence of other people in the interaction, and a person’s mood in the moment all serve to make reading emotions a complex process.
Which is where machine learning comes in. With enough training data, MIT Media Lab researchers have developed a system that outperforms existing models in recognising small facial expressions and their corresponding emotions.
The team claims that, with “a little extra training data”, the model can be adapted to analyse new contexts, such as a new group of people, without sacrificing efficacy.
“This is an unobtrusive way to monitor our moods,” said Oggi Rudovic, a Media Lab researcher and co-author of a paper describing the model, which was presented last week at the Conference on Machine Learning and Data Mining. “If you want robots with social intelligence, you have to make them intelligently and naturally respond to our moods and emotions, more like humans.”
The MIT team’s machine learning model differs from traditional affective computing through the use of a technique called ‘mixture of experts’. Instead of training the system on one set of images and mapping the various facial expressions and their corresponding emotions to new ones, the method is combined with individual neural network models – the ‘experts’.
These ‘experts’ are each trained to specialise in a separate processing task and to produce a single output from it. The new system also relies on a ‘gating network’, which calculates the probabilities of which expert will best detect a mood.
“Basically the network can discern between individuals and say, ‘This is the right expert for the given image,’” said lead author of the study, Michael Feffer.
- Read more: IBM takes steps to tackle AI bias
Internet of Business says
The endgame for research like this is to bring humans and machines closer together. With a degree of social awareness, machines can begin to take on more sensitive tasks and better serve individuals’ needs.
Eventually, the technology could be used to monitor people’s health and wellbeing, noticing when their emotions appear to be deviating from the average or when they might be at risk. It could also be used in educational scenarios, helping robot assistants alter their persona to better suit different audiences.
A programme could feasibly run in the background on a computer or smartphone, track its user’s video-based conversations and pick up anomalies in facial expressions. “You can have things like smartphone apps or websites that are able to tell how people are feeling and recommend ways to cope with stress or pain, and other things that are impacting their lives negatively,” suggested Feffer.
Such a system may sound like something out of a George Orwell novel, but applied in the right way it could have a positive impact in some situations and help provide timely interventions.
- Read more: MIT introduces brain-controlled robots
Roddy Cowie, professor emeritus of psychology at the Queen’s University, Belfast, believes that the progress made by MIT “illustrates where we really are” in the field of affective computing.
“We are edging toward systems that can roughly place, from pictures of people’s faces, where they lie on scales from very positive to very negative, and very active to very passive,” he said.
“It seems intuitive that the emotional signs one person gives are not the same as the signs another gives, and so it makes a lot of sense that emotion recognition works better when it is personalised.
“The method of personalising reflects another intriguing point, that it is more effective to train multiple ‘experts,’ and aggregate their judgments, than to train a single super-expert. The two together make a satisfying package.”