Presentation Title: Three Proposals for Pushing Forward the Theory of Results Reproducibility

Time & Location: Friday, December 1st, 2:00pm; WIRB 3000

Abstract:

In this talk, we describe three fundamental theoretical problems to move forward in understanding results reproducibility in sciences. We believe that these problems recur or are on the horizon, and they constitute major challenges in understanding reproducibility in mathematical, philosophical, and applied contexts. We propose some ideas toward overcoming these problems to motivate discussion, and obtaining interdisciplinary input. The problems are as follows: 1) Experiments condition on background knowledge about the system being modeled. Background knowledge affects the modeling assumptions and the results of a study. There is no clear theory of how to capture background knowledge and it is challenging to pinpoint mathematically and philosophically. Classical probability experiments make their straightforward assumptions and move on to technical solutions. Applied scientists may not be aware of these assumptions, and often not equipped with tools to incorporate these assumptions in their studies. We propose the use of simulation studies to partially alleviate this problem. 2) What is a theoretically sound way to capture the elusive concept of results reproducibility? It does not seem to apply to different fields in the same way (e.g., physics vs. psychology). For a sound overarching theory valid to sciences across fields, we seem to be lacking an understanding about the nature of replication studies in different fields. We categorize replication studies into three types mathematically and philosophically, and discuss their aims, frequencies, and relevance in assessing results reproducibility. 3) Issues related to results reproducibility have been recognized by a few fields due to direct consequences of non-reproduced results on their fundamental theories. However, other fields in which the models, methods, and results do not pass rigorous scrutiny seem to be oblivious to these issues. Assuming that eventually a collective awareness of these issues will arise, a major problem is: How can we describe and measure the concept of results reproducibility across scientific fields. We define a novel probability distribution up to the task, and describe its properties.

SPEAKER PROFILE:

Erkan Buzbas is a professor of Statistics at the University of Idaho, and a statistician and mathematical modeler.
His background is in stochastic modeling of population-level phenomena, and development of computational statistical methods to perform inference under uncertainty. He has a broad interest in stochastic modeling
of complex systems and statistical theory.

His current work is focused on:

• Mathematical and statistical aspects of metascience. He uses  statistical theory, mathematical modeling,
  and simulations to investigate:
  • • How scientific processes of accumulating knowledge work
    • How sciences progress
    • What is the role of reproducibility of scientific findings in advancing knowledge.