Introduction#
The publication of discrete-event simulation (DES) studies in healthcare has grown significantly, with over 100 studies published annually in academic journals. These studies employ DES to analyse critical aspects of healthcare delivery that impact patients internationally: spanning areas such as stroke, cancer, and dementia care. However, literature reviews have shown that 90% of these studies fail to provide access to the underlying computer simulation models used to generate their reported findings. This lack of transparency is particularly evident in older journal articles, where it is very rare to find computer model. In summary, the number of DES computer models that are accessible is a small fraction of the total number of DES models that have been developed and used in healthcare research studies. The majority of studies use commercial software, but Free and Open Source Software (FOSS) and simulation packages, such as Python’s SimPy, have emerged as valuable tools in promoting open publication of computer models.
Traditionally, publishing a healthcare DES study has involved an academic journal article outlining the model’s logic, performance metrics, aggregate input parameter data, assumptions, and simplifications, often accompanied by a logic or flow diagram depicting a patient’s journey through the care pathway. A small subset of these publications adheres to reporting guidelines like the Strengthening the Reporting of Empirical Simulation Studies for DES (STRESS-DES). Replicating and constructing these models is a demanding task for specialized researchers and the NHS, often leading to duplication of research efforts and infrequent model reconstruction.
Large language models (LLMs) such as GPT3.5 or 4.0 are now widely accessible to the public via OpenAI’s ChatGPT, Google’s Bard and Perplexity AI. LLMs, are trained on open data, such as Github, GitLab and the large number of code tutorials and code Q&A sites online. LLMs offer general code generative capabilities to health researchers, but with some challenges. Users interact with LLMs using plain English, receiving sophisticated and detailed code outputs in response. The training data, likely containing code based models built in Python and SimPy (and other packages and languages), as well as dashboarding/interface and web application tools like Streamlit, may also include broken code, outdated packages, and hidden logic bugs.
Large language models (LLMs) have opened up new avenues for the healthcare research and practitioner community:
When the original computer model is unavailable, LLMs can now be employed to create a FOSS version based on the model description provided in a journal article. Following thorough testing, this FOSS model could be utilized for comparable research or NHS initiatives, to replicate the study’s findings, or to educate others in constructing FOSS models.
Certain published models rely on commercial software with restrictive licensing, limited backward compatibility, and usage fees. By leveraging the model description, an alternative FOSS implementation can be developed, archived, and preserved for future use.
LLMs can generate comprehensive documentation for the code itself. For instance, after a model has been created, an LLM can be tasked with explaining the code to both Simpy beginners and experienced users who may want to reuse the model.