STARS framework
This page evaluates the extent to which the original study meets the recommendations from the STARS framework for the sharing of code and associated materials from discrete-event simulation models (Monks, Harper, and Mustafee (2024)).
Of the 8 essential STARS components:
- 7 were met fully (β )
- 1 was not met (β)
Of the 5 optional STARS components:
- 1 was met fully (β )
- 1 was partially met (π‘)
- 3 were not met (β)
| Component | Description | Met by study? | Evidence/location |
|---|---|---|---|
| Essential components | |||
| Open license | Free and open-source software (FOSS) license (e.g. MIT, GNU Public License (GPL)) | β Fully | BSD 3-Clause License on GitLab, and MIT license on Figshare. |
| Dependency management | Specify software libraries, version numbers and sources (e.g. dependency management tools like virtualenv, conda, poetry) | β Fully | Only dependencies are Python and Simpy. Python major + minor version is stated (3.9). Simpy is listed as a dependency in the repository, and in the other linked repository, a specific version is also given. Since both repositories are linked from the happy, I am satisfied it meets this requirement. The simpy dependency was as a virtualenv (requirements.txt) |
| FOSS model | Coded in FOSS language (e.g. R, Julia, Python) | β Fully | Python |
| Minimum documentation | Minimal instructions (e.g. in README) that overview (a) what model does, (b) how to install and run model to obtain results, and (c) how to vary parameters to run new experiments | β Fully | README.md provides brief description of model, explains how to install and run model, with the inputs being example CSV files. CHARM_INFO.md provides further information on all of the input parameters (like a data dictionary) to explain what these are, with the example .csv provided to show how these parameters are structured in a file. |
| ORCID | ORCID for each study author | β Not met | CONTRIBUTING.md lists the full name of the author who developed the model as well as the supporting team, and acknowledgements for concept, but no ORCID IDs provided in repository or in FigShare |
| Citation information | Instructions on how to cite the research artefact (e.g. CITATION.cff file) | β Fully | In the repository itself, CONTRIBUTING.md makes it much clearer who you would want to cite than a typical repository, but no explicit instructions on how to cite the artefact are provided. However, the repository does provide a DOI linking to long-term storage of the software on Figshare - and there, citation instructions are provided: βAnagnostou, Anastasia (2022). CHARM: dynamiC Hospital wARd Management. Brunel University London. Software. https://doi.org/10.17633/rd.brunel.18517892.v1β |
| Remote code repository | Code available in a remote code repository (e.g. GitHub, GitLab, BitBucket) | β Fully | https://gitlab.com/anabrunel/charm |
| Open science archive | Code stored in an open science archive with FORCE11 compliant citation and guaranteed persistance of digital artefacts (e.g. Figshare, Zenodo, the Open Science Framework (OSF), and the Computational Modeling in the Social and Ecological Sciences Network (CoMSES Net)) | β Fully | Code is archived on Figshare: https://doi.org/10.17633/rd.brunel.18517892.v1 |
| Optional components | |||
| Enhanced documentation | Open and high quality documentation on how the model is implemented and works (e.g. via notebooks and markdown files, brought together using software like Quarto and Jupyter Book). Suggested content includes: β’ Plain english summary of project and model β’ Clarifying license β’ Citation instructions β’ Contribution instructions β’ Model installation instructions β’ Structured code walk through of model β’ Documentation of modelling cycle using TRACE β’ Annotated simulation reporting guidelines β’ Clear description of model validation including its intended purpose |
β Not met | - |
| Documentation hosting | Host documentation (e.g. with GitHub pages, GitLab pages, BitBucket Cloud, Quarto Pub) | β Not met | - |
| Online coding environment | Provide an online environment where users can run and change code (e.g. BinderHub, Google Colaboratory, Deepnote) | β Not met | - |
| Model interface | Provide web application interface to the model so it is accessible to less technical simulation users | β Fully | Not run as not in scope to run the app, but code is available at: https://gitlab.com/anabrunel/charm-app |
| Web app hosting | Host web app online (e.g. Streamlit Community Cloud, ShinyApps hosting) | π‘ Partially | The web app was hosted with heroku (https://charm-des-app.herokuapp.com/), although heroku is no longer available, so this is now inactive. |
References
Monks, Thomas, Alison Harper, and Navonil Mustafee. 2024. βTowards Sharing Tools and Artefacts for Reusable Simulations in Healthcare.β Journal of Simulation 0 (0): 1β20. https://doi.org/10.1080/17477778.2024.2347882.