EmoNet: new AI can read 11 emotions from your selfie

Credit: DesignNews

EmoNet is an AI developed by researchers from the University of Colorado and Duke University and designed to identify 11 different emotions just from a picture.

Imagine telling your younger self – let’s say 15-20 years ago – that in the future, robots will be able to identify you by your face to unlock a personal, hand-held computer. Sounds like elite-level tech, right? Not only is this widely-accepted technology in 2019, it’s been surpassed by EmoNet.

EmoNet is an AI that can predict not who you are, but how you feel. University of Colorado and Duke University researchers classified 137,482 video frames from 2,187 videos, into 27 distinct emotion categories, including anxiety and sadness. After the training, the researchers validated their results with 25,000 images.

The team asked 18 people to view over 100 images, whilst measuring their brain activity. They showed the same images to EmoNet and compared the rests to train the AI. Whilst the AI struggled to classify confusion and surprise accurately, it was particularly spot on with horror and sexual desire.

Similar facial expressions, such as amusement and joy, were said to be difficult to classify.

How could EmoNet be used?

One of the most obvious and consumer-focused uses for this research could be to apply this AI in a household robot or assistant. IoT devices that could pick up on your emotions would be more likely to provide assistance related to what you require, or even just leave you alone, should your facial expression suggest that’s what you want.

It’s thought, however, that this research could be more valuable in mental health studies. Depression and anxiety sufferers, for example, who are currently dependent on keeping mood diaries could potentially just have to log their facial expression in future and let the AI work out how they’re feeling.


When it comes to measuring emotions, we’re typically still limited only to asking people how they feel. Our work can help move us towards direct measures of emotion-related brain processes.

Tor Wager, Professor of Psychology and Neuroscience at the University of Colorado Boulder

“Moving away from subjective labels such as ‘anxiety’ and ‘depression’ towards brain processes could lead to new targets for therapeutics, treatments, and interventions,” said Philip Kragel, one of the researchers on the study.

There is big potential too for technology that analyses emotion to be used in customer service. If a robot could sense how you’re feeling, it could deliver a more personalised, tactful customer service. By implementing sensors in shops or restaurants, for example, an organisation may be able to gauge the sentiments of its customers and use this data when helping them.

With emotion-reading AI like EmoNet though comes new challenges. The data that can be gleaned from identifying someone’s emotion is sensitive.


The robots are coming: prepare your business in an hour with our AI trends white paper


Wearing your heart on your sleeve is seen as a good attribute in many cultures, however, a robot reading your emotion could be seen as intrusive. It’s essentially one step away from mind-reading. We volunteer most of our data willingly – providing, of course, that we know we’re doing it – but facial expressions are often an unconscious conveyance of feeling.

Intrusion is a difficult enough topic when it comes to data. Some experts believe there just isn’t enough justification to implement tools like EmoNet into the wider world.

What now for emotional recognition?

The Association for Psychological Science conducted a review recently entitled Emotional Expressions Reconsidered: Challenges to Inferring Emotion From Human Facial Movements. Five scientists were asked to assess emotion science and examine the data as to “whether a person’s emotional state be reasonably inferred from that person’s facial movements.”

In two years of data examination, the findings were that emotional recognition is inconclusive. As is often the case for artificial intelligence, scientists on the review felt it didn’t quite factor in the multifaceted nature of human emotion. Facial features are often just one expression of an emotion and an expression – such as frowning – could be used for several emotions.

AI can easily confuse emotions based on expressions. / Credit: Cnet

“When I say to people, ‘Sometimes you shout in anger, sometimes you cry in anger, sometimes you laugh, and sometimes you sit silently and plan the demise of your enemies,’ that convinces them,” Lisa Feldman Barrett, a Professor of Psychology at Northeastern University and one of the review’s five authors, told The Verge. “I say, ‘Listen, what’s the last time someone won an Academy Award for scowling when they’re angry?’ No one considers that great acting.”

This is almost certainly not the endpoint for EmoNet. The technology is still raw and these findings will no doubt improve over time, as AI develops. The challenges of the technology are clear: will they prove to be too great in the development of emotional recognition?

Luke Conrad

Technology & Marketing Enthusiast

How E-commerce Marketers Can Win Black Friday

Sue Azari • 11th November 2024

As new global eCommerce players expand their influence across both European and US markets, traditional brands are navigating a rapidly shifting landscape. These fast-growing Asian platforms have gained traction by offering ultra-low prices, rapid product turnarounds, heavy investment in paid user acquisition, and leveraging viral social media trends to create demand almost in real-time. This...

Why microgrids are big news

Craig Tropea • 31st October 2024

As the world continues its march towards a greener future, businesses, communities, and individuals alike are all increasingly turning towards renewable energy sources to power their operations. What is most interesting, though, is how many of them are taking the pro-active position of researching, selecting, and implementing their preferred solutions without the assistance of traditional...

Is automation the silver bullet for customer retention?

Carter Busse • 22nd October 2024

CX innovation has accelerated rapidly since 2020, as business and consumer expectations evolved dramatically during the Covid-19 pandemic. Now, finding the best way to engage and respond to customers has become a top business priority and a key business challenge. Not only do customers expect the highest standard, but companies are prioritising superb CX to...

Automated Testing Tools and Their Impact on Software Quality

Natalia Yanchii • 09th October 2024

Test automation refers to using specialized software tools and frameworks to automate the execution of test cases, thereby reducing the time and effort required for manual testing. This approach ensures that automation tests run quickly and consistently, allowing development teams to identify and resolve defects more effectively. Test automation provides greater accuracy by eliminating human...

Custom Software Development

Natalia Yanchii • 04th October 2024

There is a wide performance gap between industry-leading companies and other market players. What helps these top businesses outperform their competitors? McKinsey & Company researchers are confident that these are digital technologies and custom software solutions. Nearly 70% of the top performers develop their proprietary products to differentiate themselves from competitors and drive growth. As...

The Impact of Test Automation on Software Quality

Natalia Yanchii • 04th October 2024

Software systems have become highly complex now, with multiple interconnected components, diverse user interfaces, and business logic. To ensure quality, QA engineers thoroughly test these systems through either automated or manual testing. At Testlum, we met many software development teams who were pressured to deliver new features and updates at a faster pace. The manual...