posted Sep 20, 2014
in Bloviation

The debate about the reproducibility of science bubbles onward, with everyone agreeing that it's a problem but of course nobody with power to fix it doing anything about it.

Recently I've been thinking that science as we know it sits in a very unpleasant middle ground.

On the one hand, despite the propaganda institutional science is biased against replication. This holes it below the waterline, and any serious scientist (alas) must consider fixing this in their field their top priority or they are consenting to just spin their wheels forever. We do not work formally enough to produce good results, because merely reaching "Peer Approved Once" and getting published is provably not a solid foundation to build on.

If one is inclined to take offense to that, consider the fact that scientists are supposed to be building on the work of others. It's very simple math to see that even if a uniformly-distributed 95% of the papers published are perfectly correct, that 5% has a disproportional impact on the accuracy of a tower of knowledge; as the tower grows, the chances of any particular new result containing a false result in the set of results it is building on approaches 1 quickly.

Many scientific disciplines would be lucky to have a 95% accuracy rate.

On the other hand, scientists are also not allowed to just "fool around", by virtue of not being able to get funding for it. Even simple experiments must be submitted, approved, funded, etc, all involving processes a great deal more complicated than the simple little English words imply. As a second-order effect it becomes a waste of time to go through the process for a small experiment, making the small experiments even less likely to be conducted than you would initially think. And yet, historically, a lot of great stuff happened from very skilled, knowledgeable scientists just fooling around. In only a few fields can a scientist afford to fool around on their own time and money, mostly the mathematical ones.

The system both crushes away the rigor we're promised in the brochure, and also crushes away any chance of serendipity or discovery on the cheap. The miracle is when we get any science at all.


Site Links


All Posts