Humans are measured, their data is annotated to create ground truth, machine learning continuously improves accuracy and performance metrics are modelled against the human data.
Human Measurement
Real anonymous, opt-in viewers, attention and emotion measured via webcam. No images stored.
Ground Truth Annotation
Professional annoators label frames; majority agreement only. Machine learning is continuously trained on annotated data.
Modelled Human Data
0–100 score, calibrated to business outcomes: Sales Imapact, Brand Awareness and Interactions.
We measure Visual Attention (eyes on screen) and Emotions (facial expressions) through webcams whilst people are viewing ads.
Attention
Does the ad creative break through to capture and retain attention?
Emotion
Does the creative resonate by achieving facial reactions? Confusion, Happiness, Surprise or Negativity Peaks.
Impact
Does the ad creative drive brand equity? Brand Recognition, Ad Likability.
The training data for our models comes from our proprietary dataset, collected from over 12+ years of developing facial coding models. Camera recordings are collected across different countries, age groups and sexes, to achieve fair and balanced representation globally.
Those recordings are annotated for attention and facial expressions by a large pool of qualified psychologists and annotators. Employing a “wisdom of the crowds” approach, video frames are only labelled when 3 to 7 annotators reach a majority agreement on each specific frame. As an additional step, annotated labels then go through a standardized quality assurance review.
...recordings are annotated for attention and facial expressions by a large pool of qualified psychologists and annotators.
Billions of frames of viewing mapped to attention and emotion from 18m+ webcam observations of people watching 90k ads across 90 countries.
Billions of frames of viewing mapped to attention and emotion from 18m+ webcam observations of people watching 90k ads across 90 countries.
Our AI recognizes human attention and emotional expression in the same way that people do, by separating background from the foreground, the ability to focus and detect the presence of a face and its countenance.
Facial coding picks up the same subtle behavioral cues that the human brain instinctively processes to gauge someone’s attention to stimuli, and any changes in their expression.
We use ‘in the wild’ datasets, employing machine learning that teach our algorithms to cope with complexities such as poor lighting, heavy shadows, thick facial hair, spectacles, and other occlusions that would typically make attention and expression tracking more challenging.
For each video frame, the AI instantly interprets the information like a human: detecting the existence of a face, separating it from the background, with the ability to focus on facial features, tracking the 3D head position, eye position, eyelids (palpebral aperture) and the shape of its expression.
Additionally, we create a person-dependent baseline, or mean face shape. By measuring expressions as they deviate from a ‘neutral’ face, this accounts for people who naturally look more positive or negative to tolerate any bias.
Realeyes uses a proprietary Convolutional Neural Network (CNN), a type of deep neural network specifically designed for processing and analyzing images and videos. Data is processed in the cloud in three steps.
AI detects face presence by the existence of facial features – not their identity
AI isolates and extracts the facial features from each camera frame
AI uses facial landmarks to track the position and movement of attention and expression
We define attention as the awareness to a stimulus while ignoring other stimuli. Here we quantify the focus or interest by tracking head pose, face direction, eye lid openness, and gaze direction in response to stimulus (ad content).
Eyes focused or fixated on a stimulus is interpreted as full attention, whereas closed eyes, looking away, or obscuring the eyes (e.g. with hands) indicates the absence of attention, often described as distraction.
When a person is attentive, their head position tends to be aligned with the stimulus (the target of their attention). Turning left, right, up or down, moving the head position away so that eyes no longer see the stimuli indicates negative attention.
In addition to eye movement and head pose, when individuals are attentive, their facial expressions, may exhibit specific patterns attributed to emotional engagement: Happiness, Surprise, Sadness, Confusion or Concentration for example.
These are the movement and configurations of facial muscles, including the movements of the eyebrows, eyes, nose, mouth and cheeks.
Every day, people rely on facial expressions to communicate how they think and feel to others. This kind of non-verbal communication is a natural part of our behavior.
Our patented technology recognizes and interprets human expression through their facial cues. These facial expressions are classified into metrics based on the specific muscle movements and configurations of facial features.
A smile is formed from the cheeks rising and the corners of the mouth pulling up respectively.
A surprised expression is formed from a combination of raised eyebrows, eyes wide open (raised eyelids) and the jaw dropping to reveal an open mouth.
A frown is formed with the lowering of the brow, a raising and narrowing of the eyelids, and a tightening of the lips.
Disgust is expressed by the nose wrinkling, a downturn of the lower lip and the corners of the mouth moving downwards.
This expression is characterized by raised eyebrows, widened eyes, nose wrinkling, a tensed mouth and elevated cheeks resembling a startled appearance.
This expression is formed by a slight curling of the lip corners, often accompanied by a tightening of the jaw muscles.
This expression is characterized by a downturned mouth, furrowed, or lowered eyebrows, narrowed eyes and tensed cheeks to appear slightly raised.
We trained our 18m webcam session dataset to create a 0–100 performance score that is 80% accurate to human. It’s calibrated to Sales Impact, Brand Awareness and Interactions.
2025 analysis of +$1bn ad spend on +1,000 ad formats, across a range of social platforms and geographies.
Sales Impact Score
40% Sales Variance.
25% CPM Variance.
Sales lift potential reported by Mars Inc. in its first validation of AI-powered attention scoring and decision-making.
Level & Scene Tagging, Isolated Element Tags and Deeper Abstractions, analysing brand mentions and presence holistically from audio and video.
Scene & Level Tags
Isolated element tags (e.g., dog) and more complex narrative events (e.g., unboxing).
Deeper Abstractions
Emotion tonality, storytelling, presentation style, expressivity of characters, social interactions, intended audience.
Subscribe to our low frequency news letter to get our latests updates delivered to your inbox
