Science Puzzle
The Armour Dilemma
Engineers study fighter planes returning from missions and map the bullet holes. The damage is dense on the wings and the main body, but almost absent around the engines.
Should they add extra armour where the holes are, or where the holes are not?
The Answer
Where the holes are not. Specifically around the engines. This feels backwards until you realise that the planes you can examine are not a random sample of all the planes that flew.
A plane hit in the wing can still limp home. A plane hit in the engine cannot. So engine hits are almost absent on the returning planes precisely because planes that took engine damage never made it back to be counted. The wings show where a plane can afford to be hit.
This is survivorship bias. When you can only study the survivors, the damage they show reveals where things can go wrong safely. The missing data, the planes that never returned, contains the critical signal about the fatal vulnerabilities.
The principle: Survivorship bias. When only survivors can be studied, the damage they show is not where it matters most. The fatal weaknesses are found in the data that is missing.