🪞
Don't trust its account of its own thinking
Ask an AI why it gave a certain answer, and it will tell you a story. The story sounds coherent. It's often wrong.
Why it happens
The model doesn't have access to its actual reasoning process, only to what a plausible explanation would look like after the fact.
What to do
If you need to know what really happened, look at the output itself, not the explanation attached to it.
"Show me the actual data or steps you used, not just your summary of them."