As part of the STARS project, a series of computational reproducibility assessments were conducted. From these, several recommendations were shared to support reproducibility of healthcare discrete-event simulation (DES) models, as described in:
Heather, A. Monks, T. Harper, A. Mustafee, N. Mayne, A. On the reproducibility of discrete-event simulation studies in health research: an empirical study using open models (2025). Journal of Simulation. https://doi.org/10.1080/17477778.2025.2552177.
Those marked with a star (⭐) were identified as having the greatest impact in Heather et al. 2025.
This book adopts the STARS reproducibility recommendations because they were developed from a study of real healthcare DES models and focus directly on computational reproducibility. They give a clear, healthcare-specific checklist for making DES studies easier to re-run, troubleshoot, and reuse, which this book follows throughout.
Recommendations to support reproduction
| Recommendation | Covered on page… |
|---|---|
| Set-up | |
| Share code with an open licence ⭐ | Licensing |
| Link publication to a specific version of the code | Changelog |
| List dependencies and versions | Environments |
| Running the model | |
| Provide code for all scenarios and sensitivity analyses ⭐ | Scenario and sensitivity analysis |
| Ensure model parameters are correct ⭐ | Full run |
| Control randomness | Randomness |
| Outputs | |
| Include code to calculate all required model outputs ⭐ | Performance measures |
| Include code to generate the tables, figures, and other reported results ⭐ | Producing tables and figures |
Recommendations to support troubleshooting and reuse
| Recommendation | Covered on page… |
|---|---|
| Design | |
| Separate model code from applications | Not covered as we don’t create apps in this book, but simply: keep model code separate from app code. As an example, in previous work we made a simulation as a package (treat-sim) hosted on PyPI, then imported that to the streamlit web app |
| Avoid hard-coded parameters | Parameters from script Parameters from file |
| Minimise code duplication | Code organisation |
| Clarity | |
| Comment sufficiently | Docstrings |
| Ensure clarity and consistency in the model results tables | Performance measures |
| Include run instructions | Documentation |
| State run times and machine specifications | Documentation |
| Functionality | |
| Optimise model run time | Parallel processing |
| Save outputs to a file | Scenario and sensitivity analysis Producing tables and figures |
| Avoid excessive output files | Scenario and sensitivity analysis |
| Address large file sizes | Scenario and sensitivity analysis |