What Policies would I want to Implement if I was a Journal Editor?

If I was asked to consider being an editor of a psychology journal, here is a wish list of policies I would like to implement*:

For all submitted articles

  • Encourage pre-registration of (a) hypotheses, and (b) data analytic plans, and provide guidance to authors on how to pre-register different types of studies (e.g., prior to data collection, following data collection, longitudinal designs)
  • The manuscript needs to include a discussion of sample size rationale, inclusion/exclusion criteria, data collection stopping rules (already implemented at many journals of course)
  • A list of all study materials/scales available to the researcher(s) when conducting analyses for studies presented in the manuscript, and the order of presentation, needs to be available to the reviewers and editor
  • Additionally, all study materials (e.g., copies of questionnaires, coding schemes, stimuli) and procedures presented in the manuscript needs to be available to the reviewers and editor
  • The data analytic code needed to reproduce the results in the manuscript needs to be available to reviewers and the editor
  • The data, and meta-data, needed to reproduce the results presented in the manuscript needs to be available to reviewers and the editor
  • Prior to the manuscript being sent out for review, the data analytic code provided will be run using the data provided to reproduce the results presented in the manuscript. If major discrepancies emerge that make the reported results difficult or impossible to interpret, the manuscript will be returned to the authors. Otherwise, a report of the outcome of this process will be provided to the reviewers, and ultimately to the author(s).
  • All reviews and editorial correspondence will be made publicly available, but reviewers will not be required to sign their reviews

For all accepted articles

  • The list of all study materials/scales available to the researcher(s) when conducting analyses for studies presented in the manuscript, and the order of presentation, needs to be made publicly available prior to acceptance
  • All study materials and procedures presented in the manuscript must be made publicly available prior to acceptance (copyrighted material excluded)
  • All data analytic code required to reproduce the results in the manuscript needs to be made publicly available prior to acceptance
  • The data and meta-data needed to reproduce the results in the manuscript need to be placed on a third party server. Author(s) can petition the editor for the data to not be made publicly available, but available upon request. A policy would need to be established to allow the editor to approve requests for data when the corresponding author is not available (e.g., retired, not responding to email, or has passed away).
  • Prior to acceptance, a pre-print of the manuscript needs to be made publicly available (if one has not yet been made available)

Other offerings

  • Have a registered reports option
  • Strongly encourage direct replications of presented results within each submitted manuscript when feasible (e.g., via cross validation, or other independent sample)
  • Adopt the “pottery barn rule”—replications of studies that were previously published in the journal will be considered for publication

I have likely missed some important policies to promote and encourage being more open and transparent during the research process, but this seems like a good start.

* I realize this will likely mean I will not be asked to consider being a journal editor anytime in the near future