The Fickle P Value Generates Irreproducible Results

The fickle P value generates irreproducible results

The article, ‘The fickle P value generates irreproducible results’ by Halsey et al. explores the key problems associated with p-values. The article seeks to provide suggestions for alternatives that can help in ensuring that published research results can be easily replicated. The main problem that the article sought to address is the difficulty in replicating research results even though the scientific method is founded on the principle of achieving findings that can be reproduced. Halsey et al. (179) attribute the problem to the large sample-to-sample variability that is demonstrated in the P values. The authors note that most studies focus on the P value in statistical analysis, often without recognizing that the value may fail to aid in the data interpretation, especially if a study’s statistical power is too low.

Problems associated with P values

Halsey et al. argue that one of the problems associated with the P values is that most scientists that are not statisticians fail to realize that a test’s statistical power is highly relevant whenever one is considering results that are statistically significant. In this context, the authors maintain that the statistical power can dramatically affect the capacity to interpret the P value and the test results. According to Halsey et al., the P value demonstrates large sample-to-sample variability unless in cases where the statistical power is exceptionally high. The second problem associated with the P values is the treatment of the P value as a dichotomous rather than as a continuous variable. The approach involves focusing on threshold values, such as 0.05, which are often unfounded. Another problem that the article highlights in relation to P values is the exaggeration of effect sizes. Samples that are less representative may lead to a low P-value, which further contributes to an exaggerated impression of the variations between the initial populations.

The need for change

Conclusively, Halsey et al. postulate that the P value problems emanate from the poor statistical understanding that researchers demonstrate, which threatens reliability in research. Given that the P value is still held as the major statistic applied in the interpretation and reporting of research findings, there is a need to change this aspect and adopt alternative statistical methods. Although the authors that there is no one perfect approach, they acknowledge that one of the most tractable and intuitive approaches is reporting effect size estimates as well as their precision for statistical interpretation, which should be accompanied by graphical representation. The authors maintain that the approach helps to overcome the P value problems by emphasizing on the precision and importance of the projected effect size. The approach achieves the objective by determining the strength or size of the association or relationship between variables.

Works Cited

Halsey, Lewis G., et al. “The fickle P value generates irreproducible results.” Nature methods 12.3 (2015): 179.