Day 12

compendium
Author

Amy Heather

Published

July 19, 2024

Note

Finishing up research compendium: final bit of troubleshooting docker, then GHCR.

Untimed: Research compendium

Docker

Resuming where I left off yesterday.

Current challenge is to try and install stringi and openssl into the renv, so the later renv::restore() doesn’t fail.

Important commands when troubleshooting

Reminder of key commands:

  • Build image: sudo docker build --tag huang2019 . -f ./docker/Dockerfile
  • Delete image: sudo docker image rm huang2019
  • Create container and open RStudio: (sleep 2 && xdg-open http://localhost:8888) & sudo docker run -it -p 8888:8787 -e DISABLE_AUTH=true --name huang2019_docker huang2019
  • Delete container: sudo docker rm huang2019_docker

Unsuccessful attempts

I tried variants of:

  • Permissions (USER root, USER rstudio)
  • Installation instructions (remotes, renv, install.packages).

I seemed to be getting success with:

RUN R -e "renv::activate()"
USER root
RUN R -e "renv::install('stringi@${STRINGI_VERSION}')"
RUN R -e "renv::install('openssl@${OPENSSL_VERSION}')"
USER rstudio
#RUN R -e "remotes::install_version('openssl', version='${OPENSSL_VERSION}')"
RUN R -e "renv::restore(lockfile=\"${PROJDIRECTORY}/renv.lock\", repos = c(CRAN = \"https://packagemanager.rstudio.com/all/__linux__/focal/latest\"))"

This installed stringi and openssl into the renv successfully, although I seemed to need to have USER root permissions. For renv::restore(), it installed fine without needing to be USER root, but it then failed when installing stringi, so it appears to not have resolved the issue.

Error in dyn.load(file, DLLpath = DLLpath, ...) : 
    unable to load shared object '/tmp/Rtmp8ogTfT/renv-staging-7b6bc3cd/stringi/libs/stringi.so':
  libicui18n.so.66: cannot open shared object file: No such file or directory

I tried running all the renv installation whilst being the USER root. The image built successfully, but it did not open as an active project, and the renv/ folder only contained renv.

I tried just running renv::install() (and not specific installs or restore), but this had errors of:

  • renv appears to be unable to access the requested library path: /home/rstudio/renv/library/linux-ubuntu-jammy/R-4.4/x86_64-pc-linux-gnu. Check that the 'rstudio' user has read / write access to this directory.
  • Error: error downloading 'https://cloud.r-project.org/src/contrib/mime_0.12.tar.gz' [cannot open URL 'https://cloud.r-project.org/src/contrib/mime_0.12.tar.gz']

Minimal example (small .lock and specific files) succeeds

I tried temporarily replacing my lockfile with a simple one with only my packages markdown and mime, and renv::restore() under user rstudio (similar to the GitHub issue from yesterday).

  • When running renv::restore(), it seemed to install its own copy of renv (and didn’t use the one I’d installed above) - and indeed, I found I could remove that from the Dockerfile with no impact on the outcome
  • The library was successfully installed and accessible when I opened the container!
{
  "R": {
    "Version": "4.4.1",
    "Repositories": [
      {
        "Name": "CRAN",
        "URL": "https://cloud.r-project.org"
      }
    ]
  },
  "Packages": {
    "markdown": {
      "Package": "markdown",
      "Version": "1.13",
      "Source": "Repository",
      "Repository": "CRAN",
      "Requirements": [
        "R",
        "commonmark",
        "utils",
        "xfun"
      ],
      "Hash": "074efab766a9d6360865ad39512f2a7e"
    },
    "markdown": {
      "Package": "markdown",
      "Version": "1.13",
      "Source": "Repository",
      "Repository": "CRAN",
      "Requirements": [
        "R",
        "commonmark",
        "utils",
        "xfun"
      ],
      "Hash": "074efab766a9d6360865ad39512f2a7e"
    }
  }
}

When I did that successful run, I’d only copied over a few essential / example files

COPY renv.lock renv.lock
COPY reproduction.Rproj reproduction.Rproj
COPY scripts scripts

Instead of

COPY . .

However, when I switched back to just copying everything, I get issues that it cannot copy and move files and that permission is denied.

I tried manually specifying everything from the folder (exc. .Rhistory and .Rproj.user)…

COPY DESCRIPTION DESCRIPTION
COPY docker docker
COPY outputs outputs
COPY README.md README.md
COPY renv.lock renv.lock
COPY reproduction.Rproj reproduction.Rproj
COPY scripts scripts
COPY tests tests

…And that built fine.

So I tried again with COPY . ., but adding .Rhistory and .Rproj.user to the .dockerignore - but this failed like before. Hence, I decided to stick with specifying the files being copied (and regardless, I guess that has the benefit of being specific about what we include). It appears I must be copying over something undesirable when running COPY . . which is causing us issues, although I can’t spot what this might be.

Troubleshooting stringi

I ran renv::snapshot() to restore the lock file back to being complete, and then tried to build the image again. This failed with the libicui18n.so.66 error from before. I reverted the lockfile back to the simple version (so the various test runs are quicker), but adding stringi, getting the same error.

Tried adding install before restore - RUN R -e "renv::install('stringi')" (which automatically installs renv for us before running)…. and it was successfull….! The library contained stringi and the other packages from the lockfile.

So, I then, again, restored the full lockfile, add an install for openssl, and tried once more.

This was successful!

Giving read, write and save access to folders

All the files are present and can be run, and the renv is active with all the required files. However, if we want to save anything (e.g. modifying qmd, or saving output from model with ggsave), we don’t currently have permissions.

Based on the docker documentation, I tried using --chown with the COPY statements to specifically give permissions to the user rstudio. I also modified stringi and openssl so they have the specific versions.

This worked!

GitHub Container Registry

Activated and ran but hit issue on COPY renv/activate.R renv/activate.R. It cannot find the file - although the previous commands with relative file paths all ran without issue.

ERROR: failed to solve: failed to compute cache key: failed to calculate checksum of ref 70b54bab-f7c4-4333-a94b-def3ee959db6::iqqrpwic0590fua1x3sv0utsd: "/renv/activate.R": not found

Checking on GitHub, the activate.R file is included. However, in my prior example of this (Shoaib et al. 2022), I ran the dockerfile from the main folder rather than the reproduction folder. Hence, I tried updating the Dockerfile so that the file paths are relative to reproduction’s parent, which fixed the issue.

I then tested downloading the image from GHCR and ran one of the scripts within that, and it all worked fine. Add instructions for this to the README.

Summary report

Filled out the summary report template.