Science Puzzle
One Result Is Not a Result
A laboratory announces a striking result. Five independent laboratories attempt the same experiment. Four find nothing at all; one finds a much weaker effect. No fraud has occurred anywhere.
Why is replication treated as the step that decides, rather than the original finding? And why is a failure to replicate not automatically an accusation?
The Answer
Because a single result is a claim about the world made by one group of people, once, with one set of equipment, one sample, and one analysis. Any of those can go wrong in ways nobody noticed, and there is a much duller possibility that is always on the table: chance.
Consider what has to be true for a real effect to vanish in four labs out of five. Very little. If the original result was a statistical fluke, the sort that turns up roughly one time in twenty, then this pattern is exactly what you expect. If a piece of equipment was subtly miscalibrated, or the sample was unrepresentative, or the analysis contained a small error, the same pattern follows. None of these requires anyone to have behaved badly, and all of them are common.
This is why replication is the load-bearing step. It is the only procedure that systematically catches the errors nobody knew they had made, because it varies everything the original team held fixed: their instruments, their reagents, their technicians, their assumptions, their luck. An effect that survives all that variation is unlikely to be an artefact of any one of them.
It also explains why a failed replication is not an accusation. The most likely explanation is not fraud but ordinary noise, and the second most likely is a hidden difference in conditions that nobody realised mattered, which is itself a discovery worth having. Treating replication failure as an insult is what makes people defensive about it, and that defensiveness is a large part of why the problem persists.
The uncomfortable finding of the last decade is how often replication fails. Large-scale attempts in psychology and cancer biology have found that a substantial fraction of published results do not reproduce. That is not a sign that science is broken. It is a sign that the self-correcting mechanism is working, and that it was not being run often enough. A finding is provisional until it has been obtained by someone who wanted it to fail.
The principle: Replication. A single result cannot distinguish a real effect from chance, error, or unnoticed local conditions. Only independent repetition, which varies everything the original held fixed, can do that.