Day 12

reproduction
Author

Amy Heather

Published

October 11, 2024

Note

Ran two scripts on remote machine with 50 million agents. Total time used: 17h 42m (44.2%)

09.50-09.59, 13.15-13.20: Attempting to run with 50 million agents for one entire .Rmd file

Returning to this as long delays and minimal response in getting access to the HPC, and figured its worth seeing if we can manage to run this locally on our high-powered machine that I have used previously, over the weekend.

Set up Case_Detection_Results.Rmd (3 yrs and 5yrs) to run with 50 million agents (so from 5e5 to 50e6).

Rscript -e "rmarkdown::render('scripts/Case_Detection_Results.Rmd')"

This used apx 4400 memory (so still lots spare). Adding another in parallel bumped it up to over 5000, but at that point, the machine got a bit noisy.

Timings

import sys
sys.path.append('../')
from timings import calculate_times

# Minutes used prior to today
used_to_date = 1048

# Times from today
times = [
    ('09.50', '09.59'),
    ('13.15', '13.20')]

calculate_times(used_to_date, times)
Time spent today: 14m, or 0h 14m
Total used to date: 1062m, or 17h 42m
Time remaining: 1338m, or 22h 18m
Used 44.2% of 40 hours max