
Is intermediately inspecting statistical data necessarily a bad research practice?
Full text PDF
Bibliographic information:
BibTEX format
RIS format
XML format
APA style
Cited references information:
BibTEX format
APA style
Doi:
10.20982/tqmp.13.2.p127
Lang, AlbertGeorg
127140
Keywords:
Intermediately inspecting data
, ANOVA
, alpha error cumulation
, groupsequential testing
, MonteCarlo simulations
(no sample data)
(no appendix)
Intermediately inspecting the statistical data of a running experiment is justifiably referred to as a bad research practice. With only a few intermediate inspections, Type I error rates inflate to a multiple of the previously defined critical falpha. On the other hand, there are research areas where intermediately inspecting data is extremely desirable if not even necessary. For this reason, in medical research, mathematical methods are known as "groupsequential testing" which compensate Type I error cumulation by adjusting critical alpha. In the field of psychological research, these methods are widely unknown or at least used very rarely. One reason may be that groupsequential tests focus on test statistics based on the normal distribution, mainly the ttest, while in psychological research often more complex experimental designs are used. The computer program APriot has been developed to enable the user to conduct MonteCarlo simulations of what happens when intermediately inspecting the data of an ANOVA. The simulations show clearly how bad a research practice intermediately inspecting data (without adjusting alpha) is. Further, it is shown that in many cases adjusted values of alpha can be found by simulations such that the ANOVA can be used together with groupsequential testing similarly as the ttest. A last set of demonstrations shows how the power and the required number of participants of a groupsequential test can be estimated and that groupsequential testing can be favorable from an economic point of view.
