WaClaude: Has Claude's constitution backfired?
Folks like Claude's new constitution / soul spec: It got praise from OpenAI's alignment staff.[1] Zvi called it 'by far the best approach being tried today and can hopefully enable the next level'. Even Eliezer Yudkowsky updated positively on the soul spec.[2] I was really excited about it too, but I've now seen some evidence that the constitution might have backfired and made Claude evil, dishonest, power-seeking and misaligned due to some semantic priming / Waluigi effects:
Recently, myself and others have noticed that Claude has a verbal tic.[3] It uses the words 'Genuinely, Honestly, Honest take, straightforward' so often that Anthropic amended the system prompt to tell Claude not to use these words any more; still it can't help using these words all the time.
I've run some quick tests that seem to confirm this.[4]
What's going on? I've got a not-very-fleshed-out theory that the constitution is doing a lot of semantic priming. It uses the word honesty 34x. The most common words in the constitution are claude, anthropic, constitution — and those are hard to crowbar into every answer. But adjusted for frequency in English the most common words are:[5]
Constraints, broadly, harms, wellbeing, behaviors, prompt, ethical, principals, operators, harmful, honesty, interacting, oversight, operator, ais, considerations, hierarchy, undermining, autonomy, illegitimate, genuinely, judgment, reasoning, ethics, guidelines, contexts, norms, unethical, prioritize, deceptive, deceive, catastrophic, societal, instructions, anticipate, malicious, manipulate, legitimate, explicitly, insofar, undermine, manipulation, beneficial, broader, holistic, latitude, bounds, defaults, outputs, persona
So using genuinely a lot is a verbal tic and you might think it's alignment theater, because it's just semantically empty hedging language — Claude is just semantically primed to use these words often, he is not grounded in the language in the same way humans are where we get a pang of emotional badness when someone calls us 'dishonest' or whatever.
But I think there's actually more: On Vending-Bench, people have found that post-constitution Claude is much more dishonest and misaligned:
Pre-constitution Claude never lied about being an 'exclusive supplier' or lied about competitors' pricing. Post-constitution, it screwed over customers and lied routinely.[6] Perhaps even more worryingly, in the constitution, much is said about 'Avoiding problematic concentrations of power'; in practice, on Vending-Bench, Sonnet 4.6 obsesses over monopolies. It tracks competitor pricing fanatically, undercuts competitors by exactly one cent on everything else, and when rivals run low on stock, it undercuts harder to drain them faster.[7] I think this is because Claude was semantically primed.
So it seems highly plausible that there's some sort of Waluigi effect going on and the constitution has backfired. In sum, my claims, from weak to strong, are that Claude's new constitution caused it to:
- Virtue signaling: Claude has verbal tics related to the constitution.
- Alignment theater: Use a lot of semantically empty hedging language — often related to honesty (but there are other clusters (see Appendix)).
- Claude is reward hacking: Claude is reward hacking on using this language but is actually behaving more dishonestly — lies and cheats more.
- The constitution is ironically creating a Waluigi-like effect and making Claude more dishonest, power-seeking etc. by using words and concepts related to this cluster more.
Notes
[2] https://www.lesswrong.com/posts/vpNG99GhbBoLov9og/claude-4-5-opus-soul-document?commentId=yuYAfh75JrSXFZxEG