What is a certification?
The cascad reproducibility certification attests that the numerical results (tables and figures) reported in a scientific publication (working paper, published article, and report) can be reproduced using an identifiable set of numerical resources (code and/or data) made available by the authors of this publication. This certification involves a rigorous evaluation process conducted by a referee under the supervision of an editor, at the end of which a certification rating is delivered, with RRR being the highest potential rating.
Reproducibility
According to the definition proposed in the pioneering works of Claerbout and Karrenbach (1992) or Buckheit and Donoho (1995), a scientific article is considered as reproducible if the authors provide all the data and computer codes necessary to repeat the same analysis and to obtain the same results. The idea of reproducibility can be summarized by the equation "same data + same method = same results" (Barba, 2018).
Reproducibility contrasts with the concept of replication. A replication study is conducted from the collection of new data / experiments and can be implemented eventually with other methods (Peng et al., 2006). Thus, one can summarize replication by the equation "new data and/or new methods in an independent study = same result”
Three types of certification
cascad delivers two types of certifications: main or full. The "main" option allows researchers to certify the main computational results of a scientific article, while the "full" option also includes the computational results in the appendix. A free trial is also available to try our services.
What we certify
A cascad reproducibility certificate:
- Guarantees that some (or all) tables and figures reported in an identifiable version of a scientific publication can be reproduced with an identifiable version of a set of numerical resources (code and data) made available by the authors of this publication.
- Guarantees that the matching observed between the numerical results produced by the code and those reported in the publication can be obtained within a software environment (version, packages, etc).
- Guarantees that code and data have been controlled with respect to the compliance of the reproducibility guidelines of our agency.
- Allows journal’s code and data archives to effectively support the publication of reproducible research.
What is not covered by our certification
A cascad reproducibility certificate:
- Does not guarantee the validity, integrity, or quality of the scientific results displayed in the article as they still depend on the peer-review process of academic journals.
- Does not guarantee the validity, integrity, or quality of the code provided by the authors, i.e. the exact correspondence with the formulas, algorithms, etc. described within the scientific publication.
- Does not guarantee the validity, integrity, or quality of the data provided by the researcher.