🎲
That confidence percentage means nothing
When an AI tells you it's "85% confident," it hasn't measured anything. It's producing a number that sounds precise because precise numbers sound trustworthy.
Why it happens
There's no calculation behind it. The model produces a plausible-sounding number the same way it produces a plausible-sounding sentence.
What to do
If you want to know how solid an answer is, ask a better question instead of a number.
"Skip the confidence percentage. Tell me what would make this answer wrong."