Why Your Carefully Crafted AI Personas Are Probably Making Things Worse
Why Your Carefully Crafted AI Personas Are Probably Making Things Worse
Breaking news: That "expert" prompt you're so proud of? It might actually be making your AI dumber.
Look, I get it. We've all been there. You're trying to get the perfect answer from ChatGPT or Claude, so you craft what you think is the perfect persona:
"You are a brilliant Stanford professor with 30 years of experience in quantum physics, multiple Nobel prizes, and a deep understanding of..."
Stop. Just stop.
A new study shattered one of the most fundamental assumptions in prompt engineering: that giving AI models specific personas or roles makes them perform better. Spoiler alert: It doesn't.
The Uncomfortable Truth
Researchers tested 162 different personas across multiple leading AI models, using over 2,400 questions. The results? Adding personas either made no difference or – here's the kicker – actually made the AI perform WORSE.
Let that sink in for a moment. All those carefully crafted "you are an expert in..." prompts? They might be actively sabotaging your results.
But Wait, It Gets Better (Or Worse?)
The study found that:
Your "expert doctor" prompt for medical questions? Barely better than no persona at all
Gender-specific roles performed worse than neutral ones (sorry, "wise grandfather")
Even when personas did help, it was nearly impossible to predict which ones would work
The most basic prompt often outperformed elaborate role-playing setups
The Domain Knowledge Plot Twist
But here's where it gets interesting and what I have been teaching in the “Advanced Prompting workshops”, while personas themselves don't help, domain-specific language and vocabulary DO matter. Think about it - when you're working on a complex mathematical problem, using precise mathematical terminology focuses the AI's attention on the relevant knowledge clusters in its training data.
It's not about telling the AI "you're a mathematician" - it's about speaking the language of mathematics. This aligns with what many of us with technical backgrounds have intuited: the more precisely you can frame your problem in domain-specific terms, the better your results.
The Real Magic: Domain Expertise in Prompting
This leads us to a crucial insight: advanced prompting techniques must be rooted in domain expertise, but not through role-playing. Instead:
Use Domain Vocabulary: Instead of "you are a statistician," use statistical terms and frameworks directly
Apply Domain Frameworks: Structure your prompts using the thinking patterns of your field
Leverage Domain Context: Reference relevant concepts and relationships without assigning personas
For example, instead of: "You are an expert statistician. What's wrong with this analysis?"
Try: "Analyze this considering sampling bias, confidence intervals, and potential violations of normality assumptions."
The Real Kicker
Here's what's truly mind-blowing: even when researchers used machine learning to try to predict which personas would work best for specific questions, they couldn't beat random selection. Think about that. A sophisticated AI trying to pick the best persona for another AI performed no better than throwing darts at a board.
And that’s, also why we look next at the “memory function” of ChatGPT, and why ist is probably a really, and I mean really bad idea to use it!
So What Should You Do Instead?
Keep It Simple: Skip the elaborate roleplay. A clear, direct prompt often works better.
Focus on Domain Language: Use precise, field-specific terminology
Structure Matters: Organize your prompts according to your field's thinking patterns
Test Everything: Your results may vary - what works for one task might fail for another
The Billion-Dollar Question
This raises some uncomfortable questions for the AI industry and all these prompt courses from “the experts” . Why are major AI companies still using default personas in their system prompts? (Looking at you, "helpful assistant.") Are we building our AI interactions on a foundation of fluff rather than focusing on precise, domain-specific communication?
The Bottom Line
The next time someone shares their "perfect prompt" that starts with an elaborate persona, send them this article. The data is clear: it's not about role-playing - it's about speaking the right language for your domain.
Your carefully crafted AI expert might not just be unnecessary – it might be making things worse. Instead, focus on leveraging your actual domain expertise in how you structure and phrase your prompts.
What's your take? Have you noticed better results when using domain-specific language instead of personas? Share your thoughts in the comments below.
Head over to the german version