Reflections
This page contains reflections on the facilitators and barriers to this reproduction, as well as a full list of the troubleshooting steps taken to reproduce this work.
What would have helped facilitate this reproduction?
Provide environment (packages and versions)
- Would’ve been handy to have a list of packages and versions used
Ensure provided scripts match exactly to the article
- These were really, really similar with one tiny exception - it was missing the smoothing on figure 7
State full run time
- The long run time wasn’t obvious from reading paper, and I got lucky that set to run over weekend on remote machine, and that it therefore had time for its 48 hour run time.
Provide comments
- There were very few comments in the code. I didn’t end up needing to dig into the code much as it ran pretty easily, but if I had needed to, this would have been more of an issue.
Minimise output file sizes if possible
- The large file sizes for
outp_raw.csvwere difficult to work with, as it required that I compress it (.csv.gz) set up GitHub Large File Storage to track. Gladly only one of the twenty raw results files was needed for the analysis. If there were any way this results file could be shrinked (for example, if there’s any information not actually needed, or so on), this would be handy.
What did help facilitate this reproduction?
Followed convention of naming main script master/main, making it easy to spot that that was the run, despite no documentation.
Include R scripts to generate figures and tables as they appear in the article.
The R scripts generating those figures and tables are informatively named (e.g. fig7.R) and the output files likewise (e.g. fig7.png).
Script is set-up in a way that you would just need to set the working directory at the top of the script, and other file paths are generated based on that.