When I say I’m 80% sure I will have that project done by Friday, what do I mean by that? What I should mean is that in 8 out of 10 similar situations I would be done by Friday and in 2 out of 10 I would not.

In reality people are frequently miscalibrated. When they say 80% it may in fact turn out that only in 5 out of 10 similar projects they would be done by the predicted date.

When I ask you whether Brasil or China has a bigger landmass you might say “China”. Then I ask “How certain are you?”. You might reply 80%.

What do you mean by that? What you should mean is that given similar questions you would get 8 out 10 right and 2 wrong.

Why is being miscalibrated bad? You might take a bet that you shouldn’t. You might defend some position because you are not asking your own gut how sure you really are about that argument you are using.

Knowing yourself and assessing whether you can trust your intuitions or whether you shouldn’t is a valuable skill.

The good news is that personal calibration can be trained really well. The bad news is that it only transfers moderately well to other domains. So even if you become very well calibrated for trivia questions you might not automatically see an improvement in task estimation at work.

Below is a sheet (licensed by CC-SA-BY-4.0 - meaning you can use as you like if you credit us e.g. by NOT removing the footer of the document) that we use in our meetups.

Download

• personal-calibration.odt (LibreOffice / Microsoft Word)
• personal-calibration.pdf (PDF)

Learn more

• https://calibration.lazdini.lv/
• https://www.howtomeasureanything.com/3rd-edition/
• http://credencecalibration.com
• https://en.wikipedia.org/wiki/Calibrated_probability_assessment
• https://www.lesswrong.com/posts/dciuB5nTG2o9PzJWe/test-your-calibration
• https://acritch.com/credence-game/
• Julia Galef’s “The Scout Mindset” Chapter 6