I Think This Much is True*

Here are 12 things that I** think are true regarding the future of how we do science based on meta-scientific discussions over the past few years and observing the varied developments in many different places (many that have sprung up as part of these discussions):

Ten years*** from now,

  1. How we share the results of our scientific efforts will not look the same. The “article” format will very likely continue to be the primary vehicle for sharing research results, but the publishing and distribution of articles will take on many different forms (as is already beginning to occur).
  2. Science will be more open. It seems highly unlikely that researchers will return to only sharing limited output of the research process, at the end of the research process, in print publications. Instead, it seems that mechanisms for sharing even more of the research process will continue to grow. And it will grow in ways not anticipated today.
  3. Trust in the integrity and reproducibility of data will become more important than trust in the research, because data and code will be openly available for the vast majority of articles (perhaps reaching between 75-90%).
  4. Researchers will be officially credited with what are today not yet considered contributions (e.g., shiny-apps, blogs, published data sets, etc.). Publications will always be important, but other types of contributions will be recognized by the scientific community and by University administrators.
  5. Findings that are replicable (whether context specific results, or results obtained across many contexts) will be valued more than those that are “surprising”.
  6. Statistics training in graduate programs will still be insufficient. But, statistics training will be viewed less as a necessary evil and simply necessary.
  7. Computational science will become more popular and prevalent. Some computational approaches include the use of (a) machine learning, (b) simulation of complex models, (c) advanced statistical approaches, and (d) understanding large and complex data sets (e.g. big data).
  8. When researchers call upon theory it will be more frequently to generate hypotheses rather than help explain an obtained pattern of results post-hoc. Hopefully this means that more theories start to be subjected to risky tests, and thus developed in a cumulative manner (but this second part is not a prediction, but rather a wish).
  9. More articles will contain efforts to directly replicate initial results in a line of research. Direct replications will complement, not replace, conceptual replications.
  10. More effort will be made to distinguish between confirmatory and exploratory analyses. Pre-registration of hypotheses and data analytic plans will be a big part of this effort.
  11. There will be a growing use of within-person, compared to between-person, research designs. One reason for the growing popularity of these designs will be the realization of the greater statistical power and sensitivity of these designs.
  12. Because of all of the above, a much larger proportion of published findings will be (demonstrably) replicable across independent labs. The “Many Labs 100”**** project will support this prediction.

It is my sincere wish to be around ten years from now to see the degree to which these things I think are true come to be. It will also be interesting to observe developments that were not on the horizon (or least on my horizon) today!

* sorry Wally Lamb

** not just me actually. Etienne LeBel (@eplebel; curatescience.org) contributed some novel ideas to this post. I also want to see him around ten years from now.

*** seemed like a good timeline

**** hard to predict, really, how many “Many Labs” projects will have been ushered into being in ten years

2 thoughts on “I Think This Much is True*”

  1. My prediction is that a lot of the science that we do will be the same, but honestly reported. As such, the majority of papers will be exploratory, and that will be OK. Confirmatory science is a more mature science built upon exploratory investigations, and the ratios should reflect this (and note which is which).

    1. “As such, the majority of papers will be exploratory, and that will be OK”

      I never understood why this is OK. Since you can make anything significant with a few tricks in your exploratory analyses, why publish these “findings”? No need to publish what “could be possible”, because we already know that by using a few tricks anything is possible, even the impossible (http://pss.sagepub.com/content/22/11/1359.full.pdf).

      I think publishing exploratory work will (continue) to lead to 1) a big waste of resources, and 2) prevent the literature from actually being interpretable, cumulative, and useful.

Leave a Reply

Your email address will not be published. Required fields are marked *