What Emotion-Reading Computers Are Learning About Us

Rana el Kaliouby, a Cambridge- and MIT-trained scientist and leader in facial-recognition technology, is deeply concerned about what happens when children grow up around technology that does not express emotion and cannot read our emotion. Does that cause us, in turn, to stop expressing emotion? El Kaliouby does not believe the solution lies in ridding the world of technology. Instead, she believes we should be working to make computers more emotionally intelligent.

In 2009, she cofounded a company called Affectiva, just outside Boston, where scientists create tools that allow computers to read faces, precisely connecting each brow furrow or smile line to a specific emotion. The company has created an "emotion engine" called Affdex that studies faces from webcam footage, identifying subtle movements and relating them to emotional or cognitive states. The technology is sophisticated enough to distinguish smirks from smiles, or unhappy frowns from the empathetic pursing of lips. It then uses these data to measure the subject's level of joy, surprise, or confusion. "The technology is able to deduce emotions that we might not even be able to articulate, because we are not fully aware of them," El Kaliouby tells me. "When a viewer sees a funny video, for instance, the Affdex might register a split second of confusion or disgust before the viewer smiles or laughs, indicating that there was actually something disturbing to them in the video." As Affectiva's chief scientist, El Kaliouby believes the applications of emotionally intelligent technology are potentially endless. Affectiva's technology is currently being used in political polling to identify how people react to political debate, and in games that help to train children with Asperger's to interpret emotion. But imagine the possibilities: At some point in the future, El Kaliouby suggests fridges might be equipped to sense when we are depressed in order to prevent us from binging on chocolate ice cream. Or perhaps computers could recognize when we are having a bad day, and offer a word of empathy -- or a heartwarming panda video.