R packages, secret keys, and testing on Appveyor

In the wake of defending my dissertation and landing a new job, I’ve been relaxing by polishing up an R package, PastecR that wraps the API for Pastec, an open-source fuzzy image matching engine.

The creator of Pastec recently launched a hosted version of the service of the service. I’ve just updated PastecR to talk to both self-hosted instances of Pastec, as well as the SaaS instance at api.pastec.io.

Because I use both Travis-CI as well as Appveyor, I needed to jump through hoops to make sure that I could provide the CI services with encrypted versions of my api.pastec.io keys, while also making sure that I didn’t send extraneous keys (encrypted or not) to CRAN.

Jennifer Bryan has a great rundown of how to work with encrypted keys on Travis. But what about Appveyor?

Save key objects as .rda

In order to run tests against the hosted version of Pastec, I need to provide R with keys to my account on the server. api.pastec.io requires both an index_id as well as an auth_key. Encrypting multiple files is a bit of a pain on both Travis and Appveyor, so I sidestepped this by save()-ing both strings into one .rda object that I can easily load again.

# Save your keys BUT DO NOT SAVE THIS CODE
index_id <- "XXXXXXXXXXXXXXXXXXXX"
auth_key <- "XXXXXXXXXXXXXXXXXXXX"

save(index_id, auth_key, file = "tests/testthat/hosted_keys.rda")
# NOW DELETE THIS CODE!

You’ll want to load() these keys before running tests, BUT you also want to make sure that CRAN does not try to load this code while building your package:

if (identical(Sys.getenv("NOT_CRAN"), "true")) {
  load("hosted_keys.rda")
  x <- some_server_function(index_id, auth_key)
}

# ...a bunch of tests...

Encrypt your keys and setup appveyor.yml

Appveyor uses a little tool called secure-file to encrypt files for its service. This installs via nuget, which you can install with homebrew if you are developing on OS X.

# Odd tip: nuget downloads files to your current working directory, so make sure
# to cd elsewhere lest you add an errant executable to your R package
cd
brew install nuget
nuget install secure-file -ExcludeVersion

Now cd back to your package directory, and encrypt. secure-file requires a secret string. Let’s use AV2016SECRET in this example:

# On OS X, you'll run secure-file.exe via mono, which homebrew will install
# alongside nuget
mono ~/secure-file/tools/secure-file.exe -encrypt tests/testthat/hosted_keys.rda -out tests/testthat/av_hosted_keys.rda.enc -secret AV2016SECRET

Note that I’ve added the prefix av_ to the .rda.enc file. If you are also running Travis-CI, you’ll need to save a distinct encrypted version of the same hosted_keys.rda file.

You’ll now have av_hosted_keys.rda.enc sitting in tests/testthat. In order to decrypt this file when it goes to Appveyor, we need to modify the default appveyor.yml generated by devtools::use_appveyor() describing how Appveyor should decrypt av_hosted_keys.rda.enc into hosted_keys.rda.

Change:

install:
  ps: Boostratp

to:

install:
  - nuget install secure-file -ExcludeVersion
  - secure-file\tools\secure-file -decrypt tests/testthat/av_hosted_keys.rda.enc -secret %my_secret% -out tests/testthat/hosted_keys.rda
  - ps: Bootstrap

See %my_secret% in there? That’s a secret environment variable that we need to provide to Appveyor. To do so, go to Appveyor’s config data encryption tool and enter the secret we used above, AV2016SECRET:

Now add the following lines to appveyor.yml, using the encrypted version of the secret phrase:

environment:
  my_secret:
    secure: NhtVX5iFHBg8Ftkc9sdnQQ==

.ignore the right things

We’ll follow generally the same guidelines that Bryan sets out in setting up encrypted keys for Travis-CI:

1. .gitignore the unencrypted hosted_keys.rda. We do not want to add the raw keys the repository.
2. do not .gitignore the encrypted av_hosted_keys.rda.enc. This needs to be added to the repo so that Appveyor can access it.
3. .Rbuildignore the encrypted av_hosted_keys.rda.enc. This should not be added into the built package source.
4. do not .Rbuildignore the unencrypted hosted_keys.rda. They need to be added to the built source so Appveyor can access it. (note: you will .Rbuildignore av_hosted_keys.rda before submitting to CRAN, preferrably off of a separate CRAN submission branch like Bryan describes.)
5. In addition to the previous steps, which all need to be followed for Travis-CI as well, we also need to .Rbuildignore the secure-file directory by adding the line ^secure-file/*$. Why? Because when Appveyor runs appveyor.yml, it will download secure-file directly into the package directory before building (I know), so that it can decrypt av_hosted_keys.rda.enc. Ignore the directory during build, or R CMD check will yell at you about having a binary executable hiding in your package.

Breathe.

And git push! With any luck, your package will now build and test successfully on Appveyor.