Think of the last time you were handed a problem with no instruction manual. There are lots of examples. A parent who suddenly needed care; a new job in which nobody explains how things really work; your own body when it starts doing something new and unwelcome.
But here is my nominee for an acute problem like that: A baby who will not stop crying.
Now ask yourself a question. How did you figure out what to do?
I will bet it was not by demanding proof. You tried something. You watched. You tried something else. You noticed that the rocking helped a little and the singing helped a lot. You held onto what seemed to work, loosened your grip on what did not, and kept your eyes open. You never were certain sure.
If you had insisted on knowing for sure before you acted, the baby would still be crying.
A messy, humble, incremental way of learning like that is not a failure. It is how most learning works. But a large part of the scientific establishment, my own field included, has spent a century pretending otherwise.
The verdict habit
Here is how most research is done. You start by assuming that nothing is happening. The new treatment does nothing. The new idea is worthless. Anything you already know about how the world works is not allowed to enter into the conversation because that would supposedly buy us the results. You focus on the null hypothesis. Essentially you assume you know nothing.
Then you gather data and ask a single yes or no question: is this result enough to force me to abandon that assumption if I assume nothing but chances operating? If yes, the finding is declared real. If no, it goes in a drawer.
This is a courtroom mentality. The idea is the defendant, presumed innocent of being true. The study is the trial. The verdict is binary and final.
It gets weirder. Every trial starts with the jury’s memory wiped. What was learned last time is not supposed to count. The forty studies that came before are not evidence; they are a separate case. And a verdict of “not proven” quietly gets treated as “proven not,” so a treatment that helped some people in some circumstances is filed away as a treatment that does not work.
No parent would learn this way. No gardener, no cook, no coach, no person learning to play the guitar.
Finding the submarine
In May 1968 the American nuclear submarine Scorpion vanished in the Atlantic with 99 people aboard. No distress call. No debris. Thousands of miles of ocean.
The Navy handed the problem to a scientist named John Craven. He had only had one search ship. He did a version of what you would do with the crying baby, only more carefully.
He divided the ocean into squares and gave each square a number. He gathered veteran submarine commanders and asked them to guess which number contained the sub. Just guess. Where might it have gone down, and why? He let them bet on their scenarios, sometimes with even with bottles of scotch as an incentive.
Every time the ship swept a square and found nothing, that empty square was not a failure. It lowered the chance for that square and raised it for the others. The map moved – the votes changed. He searched again where the map now said to look.
In late October the ship found the Scorpion nearly two miles down, about 460 miles southwest of the Azores.
The same idea found the wreck of Air France 447, lost between Rio and Paris in 2009. Two years of searching had turned up nothing. A team of mathematicians took those failures, treated every empty pass as a clue rather than a dead end, redrew the map, and the next search found the plane in a week.
Notice what these searchers did not do. They did not start by assuming the submarine was nowhere. They did not throw away a search because it came up empty. They began with their best honest guess, they let reality push the guess around, and they got a little less wrong every day until they were right.
Some science. Just a little bit.
There is a long and sometimes bitter argument among statisticians about whether science should work like the courtroom or like the search. I am not going to drag you into it, and I am not going to use any of the geeky vocabulary, because that is exactly what has kept ordinary people (and researchers!) from realizing they already know the answer.
The search says: knowledge is a map of how confident we are in our guesses. New evidence moves the map. Old evidence stays on the map. A study that finds nothing still tells you something. Disagreement is allowed, but it is a disagreement about degrees, and two people who disagree can still share what they have seen.
The verdict habit produces winners and losers. Somebody’s idea gets convicted or acquitted, camps form, careers ride on it, and a field spends decades relitigating instead of learning. I’m in a field like that and its painful to watch.
Now add in the simple fact that we are not clones. What helps your neighbor may not help you, and what helped you last year may not help you now. A verdict about the average person cannot tell you that. A map that you keep updating for this person, in this life, can. The way you figured out your baby is the way a good clinician figures out a client, and the way, if we are wise, we will make progress on the rest.
Where this lands
We are living through a period in which the challenges are large, moving, and unfamiliar. Climate. Fractured communities. Technologies that change faster than our institutions can. Nobody has the manual. Waiting for proof before we act is a recipe for standing still while the water rises. Pretending to a certainty we do not have is a recipe for driving off a cliff with confidence.
The alternative is older, humbler, and more human. Make your best honest guess. Say out loud how sure you are. Act on it, watch what happens, and let what happens change the guess. Treat the things that did not work as clues. Pool your map with other people’s maps.
That is how a crying baby gets soothed. It is how a lost submarine gets found. It is how, one square of ocean at a time, we might find our way through what we are facing now.
You already know how to do this. You have been doing it your whole life. The only truly new step is to stop apologizing for it.
