Reproducibility and RAPs
π― Objectives
This page introduces the foundations of reproducibility and reproducible analytical pipelines (RAPs) for simulation modelling..
- π Reproducibility: What it means, how it differs from replicability and reusability, and why it matters.
- βοΈ Reproducible analytical pipelines (RAP): What RAPs are and why they matter.
- π§ Reproducible work v.s. RAPs: Key distinctions.
- π Focus of this book: Supporting both reproducibility and full RAPs.
π Reproducibility
Reproducibility is the ability to regenerate the published simulation results - such as tables and figures - by running the provided model code and data.
Itβs helpful to distinguish reproducibility from related concepts:
- Reusability means the model code can be adapted for new simulation projects or different settings (for example, using a hospital model developed for Hospital A to study patient flow in Hospital B).
- Replicability is when independent researchers, using only your published description (not your code), can build their own version of the model and get similar results for the same objectives and case study.
Why does reproducibility matter?
Advantages for others
β Trust. If work is reproducible, it builds trust, as others can independently confirm your results and see that your methods are transparent.
β Reuse. If others can reproduce your work, this verifies that the code is working as expected when they run it. Knowing that the underlying code works as expected, they are then able to confidently reuse and adapt the code for new contexts and projects.
βοΈ Reproducible analytical pipelines (RAP)
Reproducible analytical pipeline (RAP) = systematic approach to data analysis and modelling in which every step of the process (end-to-end) is transparent, automated, and repeatable.
Rather than relying on manual processes or undocumented decisions, a RAP ensures that the entire workflow - from data ingestion and cleaning, to modelling, and through to analysis (and sometimes also reporting) - is scripted, documented, and reproducible.
There should be no (or minimal) manual intervention required to generate the results. The requirements of a RAP are outlined in the NHS RAP Maturity Framework.
π§ Reproducible work v.s. RAPs
You can do reproducible work without building a full RAP - but a RAP takes things further. Reproducible analysis may still involve manual steps, selective re-running of scripts, or undocumented decisions. This leaves room for human error and inefficiency, and makes it harder to check and re-run the work.
A RAP is distinct because it automates the entire workflow, minimising manual intervention and using software engineering best practices like version control, peer review, and modular code. This makes your analysis more efficient and robust, and easier to re-run and review.
π Focus of this book
This book does both! As detailed on the guidelines page:
- It aims to make your work reproducible, drawing on recommendations from Heather et al. (2025) for reproducible discrete-event simulation (DES)
- It also guides you through building a full RAP, following the NHS βLevels of RAPβ maturity framework.