Big names in statistics want to shake up much-maligned P value

August 8, 2017

Big names in #statistics want to shake up…#Pvalue Stronger significance cutoffs (.005?) but danger of FNs

“Lowering P-value thresholds may also exacerbate the “file-drawer problem”, in which studies with negative results are left unpublished, says Tom Johnstone, a cognitive neuroscientist at the University of Reading, UK. But Benjamin says all research should be published, regardless of P value.

Other scientific fields have already cracked down on P values — and in 2015, one psychology journal banned them. Particle physicists, who collect reams of data from atom-smashing experiments, have long demanded a P value below 0.0000003 (or 3 × 10−7) because of concerns that a lower threshold could lead to mistaken claims, notes Valen Johnson, a statistician at Texas A&M University in College Station and a co-lead author of the paper. More than a decade ago, geneticists took similar steps to establish a threshold of 5 × 10−8 for
genome-wide association studies, which look for differences between people with a disease and those without across hundreds of thousands of DNA-letter variants.”