Are research results accurate? New work reveals why previous scientific principles are crumbling

Posted: 25 February 2015

Dr Halsey reveals that the measure typically used by scientists to evaluate their data – the ‘P value’ – is not reliable.

Dr Halsey says: “The P value is central to most researchers’ statistical interpretations, yet very few scientists realise how fickle P is. In my research paper we demonstrate with simple models that the P value can vary dramatically from one replicate of a study to another, therefore the P value is fickle and a weak way to assess the strength of a finding from a study.”

There is a strong belief among scientists that the P value can be the central metric used to interpret their results; if the P value is sufficiently low then the results are considered to be strong evidence that the findings are convincing, and consequently are not often repeated to be sure the results are correct. Thus, many studies reporting a low P value are never challenged or replicated. However Dr Halsey’s research contradicts this common belief. Dr Halsey said: “A better understanding of why P is so unhelpful should encourage scientists to reduce their reliance on this misleading concept, and in turn reduce the number of study findings that crumble under the scrutiny of replication.”

Read more about Dr Halsey’s exploration of the P value and the replicability of scientific results on his recent paper ‘The fickle P value generates unrepeatable results’, published in the Nature Methods this Thursday 26th February 2015.

Lewis Halsey is a Senior Lecturer in Comparative and Environmental Physiology, who enlightens his students with his active research in environmental physiology, energetics, respiratory physiology, and the relationships between the behaviour, ecology and energetics of king penguins. He is also on committee at the Society for Experimental Biology.