Now Is the Time to Assess the Effects of Open Science Practices with Randomized Control Trials

We issue a call for the design and conduct of experimental trials to test
the effects of researchers' adoption of Open Science (OS) research
practices. OS emerged to address lapses in the transparency, quality, integrity,
and reproducibility of research by proposing that investigators institute
practices, such as preregistering study hypotheses, procedures and statistical
analyses, prior to launching their research. These practices have been greeted
with enthusiasm by some parts of the scientific community, but empirical
evidence of their effects relies mainly on observational studies; furthermore,
questions remain about the time and effort required by these practices and their
ultimate benefit to science. To assess the outcomes of OS research practices, we
propose they be viewed as behavioral interventions for scientists and tested in
randomized controlled trials (RCTs), to identify potential benefits and
(unintended) harms. As this is a call to action rather than an action plan
per se, we sketch out four potential trial designs to
encourage further deliberation and planning. Experimental tests to document the
outcomes of OS practices can provide evidence to optimize how scientists,
funders, policymakers, and institutions utilize these strategies to advance
scientific practice.