New Enabling of Advertising Research Using Facial Coding and Real Time Response Measurement Online Dr. Olaf Wenzel (SKOPOS) Christian Lauterbach (Deutsche Telekom)
Deutsche Telekom AG – Marketing Communications Strategy & Media in Cooperation with SKOPOS GmbH & Co. KG March 2014
WHAT‘S RIGHT WITH OUR TV-SPOT? IS THERE SOMETHING WRONG? CLASSIC RESEARCH IS NEEDS TO ASK QUESTIONS OVER QUESTIONS ... An overall rating is important but it does not tell us the whole story.
So we are asking open questions to get a deeper understanding of the performance of a TV-Spot. And what do you think about this sequence? Tell me about your feelings watching this.
And what do you think about this sequence? Tell me about your feelings watching this.
Finally we have to conduct content analysis to grasp which sequence works well, and why that is so.
And what do you think about this sequence? Tell me about your feelings watching this.
Deutsche Telekom AG Marketing Communications Strategy & Media in cooperation with SKOPOS GmbH & Co. KG
March 2014
2
CAN FACIAL CODING OR REAL TIME RESPONSE MEASURM. HELP US? WHAT ARE THE BENEFITS? WHAT ARE THE PITFALLS? REAL TIME RESPONSE MEASUREMENT Test person is rating the stimulus while watching it using the mouse wheel: • Wheel up = like • Wheel down = don‘t like
RESULT: One explicit measure over time.
TECHNICAL REQUIREMENTS: • Webcam • Software plugin hosted on an external server. Source: http://www.nviso.ch/technology.html
TECHNICAL REQUIREMENTS: • Wheelmouse • Software plugin hosted on an external server
AUTOMATED FACIAL CODING Test person is watching the stimulus. The testperson‘s face is captured on video using its webcam.
RESULTS: Seven implicit measures decoded from facial expressions • Emotion lift (combined overall measure) • Anger, Disgust, Happiness Fear, Sadness, Surprise Deutsche Telekom AG Marketing Communications Strategy & Media in cooperation with SKOPOS GmbH & Co. KG
March 2014
3
LET‘S GIVE IT A TRY ... TESTING TWO TV-SPOTS USING THREE DIFFERENT APPROACHES.
RANDOM ORDER
1.
2.
3.
A. CLASSIC APPROACH Presenting the 1st video Asking questions, rating the video Presenting the 2nd video Asking questions and rating the video B. FACIAL CODING (FC) 1. Presenting the 1st video and measuring emotions via facial coding 2. Presenting the 2nd video and measuring emotions via facial coding 3. Presenting the 1st video again 4. Asking questions, rating the video 5. Presenting the 2nd video again 6. Asking questions and rating the video C. REAL TIME RESPONSE MEASUREMENT (RTRM) 1. Presenting the 1st video and rating via RTRM 2. Presenting the 2nd video and rating via RTRM 3. Presenting the 1st video again 4. Asking questions, rating the video 5. Presenting the 2nd video again 6. Asking questions and rating the video
Deutsche Telekom AG Marketing Communications Strategy & Media in cooperation with SKOPOS GmbH & Co. KG
March 2014
4
AT A FIRST GLANCE THE OUTCOMES SEEM TO BE QUITE SIMILAR. FC AND RTRM RATING CURVES SHOW QUITE SIMILAR GRADIENTS.
RTRM
RTRM
Facial Coding
Facial Coding
* FACIAL CODING CURVES ARE REPRESENTING EMOTION LIFT. Emotion lift is a combined measure of the six basic emotions measured via facial coding: anger, disgust, fear, surprise, happiness, sadness.
Deutsche Telekom AG Marketing Communications Strategy & Media in cooperation with SKOPOS GmbH & Co. KG
March 2014
5
AT A FIRST GLANCE THE OUTCOMES SEEM TO BE QUITE SIMILAR. FC AND RTRM RATING CURVES SHOW QUITE SIMILAR GRADIENTS.
RTRM
RTRM
Facial Coding
Facial Coding
* FACIAL CODING CURVES ARE REPRESENTING EMOTION LIFT. Emotion lift is a combined measure of the six basic emotions measured via facial coding: anger, disgust, fear, surprise, happiness, sadness.
Deutsche Telekom AG Marketing Communications Strategy & Media in cooperation with SKOPOS GmbH & Co. KG
March 2014
6
FC REACTS FASTER ON CHANGES OF EMOTIONAL STATE. RTRM SEEMS TO SUFFER FROM SOME KIND OF LATENCY. FACIAL CODING (EMOTION LIFT) VS. REAL TIME RESPONSE MEASUREMENT (SPOT 2)
Whilst FC shows instant reactions to the stimulus …
Probands need to think first about what they see before they react via RTRM.
… RTRM occasionaly seems to lack a little behind. RTRM Facial Coding
Deutsche Telekom AG Marketing Communications Strategy & Media in cooperation with SKOPOS GmbH & Co. KG
March 2014
7
WHAT MAKES YOU HAPPY? WHAT IS MAKING YOU SURPRISE? SOME FC MEASURES ARE CORRESPONDING WELL WITH THE CONTENT. HAPPINESS CURVE (SPOT 1 VS. SPOT 2) / FACIAL CODING Happiness goes only slightly down when the last scene is on display.
Happiness goes quickly down when construction worker is on display.
Deutsche Telekom AG Marketing Communications Strategy & Media in cooperation with SKOPOS GmbH & Co. KG
March 2014
8
WHAT MAKES YOU HAPPY? WHAT IS MAKING YOU SURPRISE? SOME FC MEASURES ARE CORRESPONDING WELL WITH THE CONTENT. SURPRISE CURVE (SPOT 2) / FACIAL CODING When (all of a sudden) the city is emerging the surprise measure goes straight up.
Once the lines have been identified as magenta colored veins the surprise measure goes down.
Deutsche Telekom AG Marketing Communications Strategy & Media in cooperation with SKOPOS GmbH & Co. KG
March 2014
9
RTRM MEASURES EXPLICIT. FC MEASURES IMPLICIT. HIGH CORRELATION OF RTRM RESULTS WITH RATING SCALES. CORRELATIONS OF AGGREGATED MEASURES WITH OVERALL RATING OF THE SPOT (FC vs. RTRM) Spot 1
Spot 2 FC measures show no substantial correlations with explicit overall ratings ot the spot.
0,14
0,11
0,09 -0,03 -0,07
0,10 0,08
0,63 0,48
0,05
0,04
-0,03
-0,06 -0,16
FACIAL CODING MEASURES Emotion lift Surprise Sadness
Happiness
RTRM measure correlates highly significant with explicit overall ratings of the spots.
Fear
Disgust
-0,01 -0,13
Anger
Deutsche Telekom AG Marketing Communications Strategy & Media in cooperation with SKOPOS GmbH & Co. KG
RTRM March 2014
10
MEASURES ARE OF USE ONLY IF PREASSUMPTIONS ARE PROFOUND. INTERPRETATION SHOULD CONSIDER COMMUNICATION TARGETS. DISGUST SADNESS
SURPRISE
ANGER EMOTION LIFT
HAPPINESS FEAR
?
?
?
INTERPRET WITH CAUTION! Most of the measures can not be claimed to be „good“ or „bad“ per se.
CONSIDER COMMUNICATION TARGETS! Examples Target: Perceive company (sender) in a happy mood! Focus on happiness level before logo is displayed Target: Induce activation at the beginning of the spot! Focus on surprise, disgust, fear at the beginning
Deutsche Telekom AG Marketing Communications Strategy & Media in cooperation with SKOPOS GmbH & Co. KG
March 2014
11
REMEMBER THAT THE TEST SITUATION IS OUT OF YOUR CONTROL. IN 24% OF ALL CASES FC WAS NOT WORKING FOR TECHNICAL REASONS. SOME SCREENSHOTS OF TEST PERSONS (AVAILABLE FOR QUALITY CONTROL)
FACIAL CODING WORKS WELL
FACIAL CODING MIGHT WORK
Deutsche Telekom AG Marketing Communications Strategy & Media in cooperation with SKOPOS GmbH & Co. KG
FACIAL CODING NOT POSSIBLE
March 2014
12
REMEMBER THAT THE TEST SITUATION IS OUT OF YOUR CONTROL. IN 27% OF THE CASES OF RTRM THE MOUSE WHEEL WAS NOT USED. DIFFERENT GROUPS OF RTRM RESPONDENTS 73% of the respondents are really using their mouse wheel to rate the stimulus.
27% of the respondents are doing nothing but watching („Flatliners“)
Deutsche Telekom AG Marketing Communications Strategy & Media in cooperation with SKOPOS GmbH & Co. KG
March 2014
13
FC PROVIDES A CLEAR BENEFIT. RTRM NOT THAT MUCH. ONLINE TESTING WORKS. BUT OVERSAMPLING IS REQUIRED. RTRM suffers from latency. It is not quite clear (and variing) how fast a decision is made about liking or not liking a scene. The measure is not precise regarding the timeline. We end up with an unprecise explicit measure not carring much more information than the classic approach via asking questions. Facial coding has some advantages. • Precise in terms of measuring spontaneous reactions implicitely. No latency enforced by cognitive appraisal. • Adding value in terms of adding new qualitative information going beyond common explicit measures. • Speeding up the research process by making long and repeated questions refering to different scenes obsolete. But we don‘t know yet which level of emotion lift or of the other measures has to be achieved in terms of being really substantial according to our communication tagets. Benchmarking is required! From a methodological point of view: • RTRM and FC both are valid options for online research. Face validity and criterion-related validity are high. • But substantial oversampling is required for both methods due to technical reasons and due to the lack of control of the interview situation. Deutsche Telekom AG Marketing Communications Strategy & Media in cooperation with SKOPOS GmbH & Co. KG
March 2014
14
THANK YOU!
Christian Lauterbach Senior Expert Communications Deutsche Telekom AG
Dr. Olaf Wenzel Research Director SKOPOS GmbH & Co. KG
Deutsche Telekom AG Marketing Communications Strategy & Media in cooperation with SKOPOS GmbH & Co. KG
March 2014
15
