The Say-Do Gap is the Insight (not the problem)

I've seen some interesting discussions lately about AI-driven research unlocking more "honest" responses: the idea that people open up more to AI than human moderators because they worry less about judgment. Some even claim that that makes AI-driven research superior to 'traditional' research

It's a compelling observation, and we've seen similar patterns with online qual/communities over the years. It was actually one of the main 'selling points'. Reduced social presence does tend to yield more candid responses on sensitive topics.

But here's what concerns me: if we optimize purely for "honesty" 𝘄𝗲 𝗺𝗶𝗴𝗵𝘁 𝗺𝗶𝘀𝘀 𝘁𝗵𝗲 𝗽𝗼𝗶𝗻𝘁 𝗲𝗻𝘁𝗶𝗿𝗲𝗹𝘆.

David Ogilvy famously said: "𝘛𝘩𝘦 𝘵𝘳𝘰𝘶𝘣𝘭𝘦 𝘸𝘪𝘵𝘩 𝘮𝘢𝘳𝘬𝘦𝘵 𝘳𝘦𝘴𝘦𝘢𝘳𝘤𝘩 𝘪𝘴 𝘵𝘩𝘢𝘵 𝘱𝘦𝘰𝘱𝘭𝘦 𝘥𝘰𝘯'𝘵 𝘵𝘩𝘪𝘯𝘬 𝘸𝘩𝘢𝘵 𝘵𝘩𝘦𝘺 𝘧𝘦𝘦𝘭, 𝘵𝘩𝘦𝘺 𝘥𝘰𝘯'𝘵 𝘴𝘢𝘺 𝘸𝘩𝘢𝘵 𝘵𝘩𝘦𝘺 𝘵𝘩𝘪𝘯𝘬, 𝘢𝘯𝘥 𝘵𝘩𝘦𝘺 𝘥𝘰𝘯'𝘵 𝘥𝘰 𝘸𝘩𝘢𝘵 𝘵𝘩𝘦𝘺 𝘴𝘢𝘺."

This quote gets weaponized to discredit research. But actually it's one of the strongest arguments FOR proper research. 𝗕𝗲𝗰𝗮𝘂𝘀𝗲 𝘁𝗵𝗲 𝘀𝗮𝘆-𝗱𝗼 𝗴𝗮𝗽 𝗶𝘀𝗻'𝘁 𝗮 𝗿𝗲𝘀𝗲𝗮𝗿𝗰𝗵 𝗯𝘂𝗴 𝗯𝘂𝘁 𝗶𝘁'𝘀 𝘁𝗵𝗲 𝘃𝗲𝗿𝘆 𝗽𝗵𝗲𝗻𝗼𝗺𝗲𝗻𝗼𝗻 𝘄𝗲 𝗻𝗲𝗲𝗱 𝘁𝗼 𝘂𝗻𝗱𝗲𝗿𝘀𝘁𝗮𝗻𝗱.

Between what people say (or genuinely intend) and what they ultimately do sits a complex web:

• social norms
• context
• convenience
• habit
• emotion
• identity
• competing priorities
• etc

The public performative self and the private honest self are both real, and the tension between them is often where the most powerful insights live.

Bad research sees the say-do gap as failure "people lied to us."

Good researchers see it as the insight: "𝘸𝘩𝘢𝘵'𝘴 𝘴𝘵𝘰𝘱𝘱𝘪𝘯𝘨 𝘵𝘩𝘦𝘮? 𝘞𝘩𝘢𝘵 𝘸𝘰𝘶𝘭𝘥 𝘯𝘦𝘦𝘥 𝘵𝘰 𝘤𝘩𝘢𝘯𝘨𝘦? 𝘞𝘩𝘢𝘵 𝘥𝘰𝘦𝘴 𝘵𝘩𝘪𝘴 𝘨𝘢𝘱 𝘵𝘦𝘭𝘭 𝘶𝘴 𝘢𝘣𝘰𝘶𝘵 𝘩𝘰𝘸 𝘥𝘦𝘤𝘪𝘴𝘪𝘰𝘯𝘴 𝘢𝘤𝘵𝘶𝘢𝘭𝘭𝘺 𝘨𝘦𝘵 𝘮𝘢𝘥𝘦?"

Skilled qual researchers know how to navigate this by creating safety for honest disclosure while also observing and probing the social performance. Or by using the different tools in their toolbox to facilitate this. If AI can systematically capture both layers and help us understand what sits between them, that's valuable. But if it only optimizes for one dimension, we've lost something critical.

The real value of research isn't just collecting what people say, honest or otherwise. It's understanding why they say it, what they actually do, what sits in between, and how to design for reality rather than aspiration. That's where you find your behavior change levers, your positioning gaps, your cultural truths.

So AI-driven interviews: yes, they'll have a compelling place in our toolbox, but only if we resist the temptation to optimize for a single dimension of truth. Real business impact comes from understanding the complexity of layers and designing strategies that account for the gap between them.