AI researchers trick chatbots into sharing how to make cocaine as long as they believe a user is wearing a green shirt — 'CoT Forgery' exploit spurs LLMs to divulge forbidden info by faking trusted chains of thought
… The researchers built “role probes” that score how strongly a model internally treats each token as its own reasoning or as a user command. …