Thursday, June 7, 2012

The Problem With Statistics in Science

They're not proof:
"Cornell’s Daryl Bem case is instructive. He’s an academic who has published several notable peer-reviewed articles which claim that (several different versions of) ESP is real. Trouble is, despite the prominence of the journals, and the peer review, almost none of his peers believe his results. "They publish his papers anyway because the papers meet the statistical criterion of success, which is to say the papers contain wee p-values, which are p-values less than the magic number. Bem always finds, at least in the papers he submits for publication, publishable p-values. In his latest work he touts, “all but one of the experiments yielded statistically significant results.” This is code for “p-values less than the magic number.” "This sets up a conflict in the mind of the researcher. Small p-values are thought to be the proof definitive..."
A really great post. I'll note that there's something wrong with a scientific journal if their only criteria for inclusion is the statistical result...