Skip to main contentSkip to navigationSkip to navigation
Ellie, a machine that can spot depression
Ellie is a virtual interviewer created by USC Institute for Creative Technologies’ SimSensei project. A sensor and webcam scan people’s facial and body movements and instruct Ellie on what questions to ask. Photograph: Teresa Dey/USC Institute for Creative Technologies
Ellie is a virtual interviewer created by USC Institute for Creative Technologies’ SimSensei project. A sensor and webcam scan people’s facial and body movements and instruct Ellie on what questions to ask. Photograph: Teresa Dey/USC Institute for Creative Technologies

Meet Ellie, the machine that can detect depression

This article is more than 8 years old

Technology is an important channel to treat mental illness, but should we be worried about an age of virtual therapy when only our problems are real?

Ellie is on the screen, she’s welcoming and gentle with a soft encouraging voice. She asks probing questions in a non-judgmental way. Her body language is encouraging and her face is interested and responsive. She’s obviously highly trained and skilful at picking up cues.

But Ellie isn’t real. She’s a virtual interviewer created by a team of scientists at the Institute for Creative Technologies (ICT) at the University of Southern California. She may be an avatar, but she’s great to talk to.

Ellie introduces herself and asks a series of questions that can pick up signs of depression and post-traumatic stress disorder, or PTSD. A sensor and webcam scan your facial and body movements and tone of voice, and instruct Ellie how best to interact with you. She’s not listening just to the words, but also judging the tone and facial expression that accompany the words.

Ellie is part of the university’s SimSensei project. She had a three-part gestation: first, the scientists analysed human beings interacting to unpick the linguistic and behavioural nuances that make up our conversations. Then they created a virtual human, with a person in the next room pressing the buttons telling the virtual human how to respond. They called this the “Wizard of Oz”. Finally, they added facial-movement sensors and dialogue managers to create a system that can read and react to human emotion. The technology is called Multisense.

Ellie’s co-creator, Professor Louis-Philippe Morency, says Ellie is not a substitute for a real therapist. She’s a decision support tool designed to gather information – diagnosis and treatment decisions are still made by human clinicians.

“The best analogy I give people is the blood test,” Morency says. “When a human doctor has questions about the symptoms of a patient, he/she will order a test of a blood sample. These results will help with the diagnosis of the illness. Ellie is there to help gather and analyse an ‘interaction sample’.”

The results of the interaction with Ellie and the analysis of Multisense are given to the human clinician to help with the diagnosis of the mental illness.

Although Ellie will not replace real therapists, she can send information back to be analysed so vulnerable individuals can be offered help. Speaking to Ellie may be easier for people who feel there is a stigma attached to “seeing a shrink”.

“We’ve seen huge interest from medical centres in the US in using technology to bring objective measures of non-verbal behaviours and correlate them with clinical scales of mental illness,” says Morency. “We still need to prove that it works.”

The computer technology relies on detecting minute facial and body movements. Anxious people fidget with their hands, distressed people may smile as often as happy people but their smiles are shorter and less intense and they tend to avoid eye contact, according to the ICT team.

When you smile at Ellie, she compares your smile with a database of controls made up of civilians and military veterans. She measures your pauses, notes whether you lean forward and which direction you’re looking in. People with depression tend not to pronounce their vowel sounds clearly because they don’t move their muscles of speech as much as those who aren’t depressed. This can be detected in computerised recordings of their conversation.

Ellie and her fellow avatars could help transform cash-strapped mental health services. And the scale of the problem is huge. One in four people in the UK experiences a mental health problem. And one in 20 people over 16 report having attempted suicide at some stage in their lives. But many people never seek help or get a diagnosis that would allow them to access treatment. In 2007, only 24% of people in the UK with a common mental disorder were getting treatment.

While avatars can collect indicators of mental health problems, can they also deliver solutions? Eve Critchley, digital community manager at Mind, agrees that some people find it easier to talk to an avatar in an online space where they’re not speaking face to face and says technology can be a positive channel for maintaining mental wellbeing.

The NHS, struggling with long waiting lists for therapists, offers a range of therapies online: computerised CBT therapy has a stamp of approval from the National Institute for Health and Clinical Excellence, or Nice, and Beating the Blues is an interactive computer programme available on the NHS in some areas to treat depression and anxiety. FearFighter is another programme used to treat anxiety and phobias.

There is a plethora of health apps available and the NHS Healthy Apps Library offers a guide to the most effective and relevant. Mindfulness by Digipill is an app that plays guided meditation for stress management and WorkGuru helps manage workplace stress.

We have become used to interacting with screens nowadays. There is some concern that addiction to screens is damaging our brains, especially in young people who are growing up knowing nothing else, according to neuroscientist Baroness Susan Greenfield of Oxford University.

Greenfield’s views are controversial and some experts question whether there is sufficient evidence to back her claims. But it does seem that being sedentary can predispose young men to depression and it is well known that exercise is particularly beneficial if you’re depressed. Even if screen time in itself doesn’t harm the brain, sitting still in front of one for hours on end is clearly not good for mental wellbeing.

Critchley agrees. “Of course, it can be hard to remind yourself to take a break from the screen,” she says. “It’s about balance. You need to make sure you’re getting outdoors if you can and looking after what you eat and your sleep. You may come across content online that makes you feel worse so we recommend support that’s in moderated spaces, like the Mind online community, Elefriends.”

Technology may be part of the answer to our growing mental health issues, but it may also be part of the problem. Is it time to turn off screens to protect our mental health? And do we really want to see a day when our therapists are all virtual and only our problems are real?

Comments (…)

Sign in or create your Guardian account to join the discussion

Most viewed

Most viewed