Amy Heather
July 15, 2024
Consensus on evaluation + reflections + research compendium. Total evaluation time: 1h 45m.
Pulled together to share with Tom and Alison, to get a second opinion on these, and emailed over a link. Later, input responses below. Agreed with all decisions, so no changes required.
Badges:
STARS framework:
Reporting guidelines:
| Item | My comments | Thoughts from Tom |
|---|---|---|
| STRESS-DES 1.2 Model outputs. Define all quantitative performance measures that are reported, using equations where necessary. Specify how and when they are calculated during the model run along with how any measures of error such as confidence intervals are calculated. | It does describe the measures, and how these are calculated, and so I have said it met these criteria, although I did find it hard to understand/calculate the relative probability of waiting, and would’ve benefited from further detail/equations. Currently marked as fully met. | Agree with decision. |
| STRESS-DES 1.3 Experimentation aims. If the model has been used for experimentation, state the objectives that it was used to investigate. (A) Scenario based analysis – Provide a name and description for each scenario, providing a rationale for the choice of scenarios and ensure that item 2.3 (below) is completed. | I feel the paper does describe the scenarios clearly - my only hesitation is that I have been unable to successfully implement the exclusive use scenario - but that feels like a coding issue rather than a description issue? As, on the face of it, the article describes everything I need to know. Currently marked as fully met. | Agree with decision. Argue that description in article is a reasonable explanation of the logic in play - “First, in the “exclusive-use” scenario, angioINR is not available for elective IR patients. Its use is restricted to stroke, elective INR and emergency IR patients”” Huang et al. (2019) |
| STRESS-DES 3.2 Pre-processing. Provide details of any data manipulation that has taken place before its use in the simulation, e.g. interpolation to account for missing data or the removal of outliers. | None provided, so presumed not applicable - but hard to say, as maybe there was pre-processing that simply wasn’t mentioned. But as not possible to know either way, assumed not-applicable | Agree with decision. Give benefit of the doubt by its absence - although ideally they would state no data pre-processing was used. |
| ISPOR SDM 12 Is cross validation performed and reported | Wasn’t certain whether to mark this is unmet (❌) or not applicable (N/A)? Currently set as unmet. Evidence - stating there is a gap in the Introduction: “In contrast to other healthcare fields, a resource-use optimization model has not been implemented for comprehensive stroke services.” Huang et al. (2019) |
Agree with decision. |
| ISPOR SDM 15 Is the model generalizability issue discussed? | Not sure if it is partially (🟡) or fully met (✅)? Currently marked as fully. Evidence - Discussion: “The quality of the ECR service appears to be robust to important parameters, such as the number of radiologists. The simulation findings apply to ECR services that can be represented by the model in this study. As such, utilization of this model to its maximum capacity requires tailoring the model to local needs, as institutional bottlenecks differ between providers. We specifically developed this model using an open source programming language so that the source code can serve as a basis for future model refinement and modification.”Huang et al. (2019) |
Agree with decision. |
import sys
sys.path.append('../')
from timings import calculate_times
# Minutes used prior to today
used_to_date = 91
# Times from today
times = [
('08.22', '08.30'),
('08.37', '08.41'),
('10.53', '10.55')]
calculate_times(used_to_date, times, limit=False)Time spent today: 14m, or 0h 14m
Total used to date: 105m, or 1h 45mDid some further research into options for dependency management in R.
Completed reflections.qmd.
Tried forking and running actions from existing repositories that render and publish an R-based Quarto book on GitHub pages.
Then updated my action to be similar to the rap4all actions and tried it. This failed - “configuration failed because libcurl was not found”. I add installation of libcurl and ran it again, but this all failed just like before, with the error there is no package called 'rmarkdown'.
Some further work on the research compendium stage.
testthat to environmentrenv::snapshot() once it completed. However, I cancelled it as realised could run without devtools (and devtools would be alot of extra dependencies!)testthat::test_dir("tests")Links: