Repository: FHIR/sushi
Pull request #470 (2020-06-01 19:34:27)

Only load current packages when newer version is available
Addresses https://standardhealthrecord.atlassian.net/browse/CIMPL-364.

We used to always load a new package to the cache if the version was specified as `#current`. Now, if we are loading a `#current` package, and that package is already cached, we try to be intelligent about if we need to download that package or not. Current packages are downloaded from build.fhir.org, and each current package has a `package.manifest.json` file which lists the `date` for that package. When loading a current package that is already locally cached, we will now compare the date on the local copy with the date on the `package.manifest.json` hosted on build.fhir.org. If the dates match, we do nothing. If the dates do not match, we trigger a download.

All other loading behavior should remain the same, but adding this change required a bit of refactoring, so the `loadDependency` code has changed quite a bit. Additionally, I realized that our tests for `loadDependency` could be a little more comprehensive, so I tried to flesh out those tests more in this PR.

I would suggest that reviewers also do a test run our two to ensure that loading is working as expected on their machine on this branch. The tests rely on mocking out certain functions, so it is nice to have further confirmation that things are working when the non-mocked functions are called.

ngfreiter: >First, I noticed that the directory structure of "current" packages is different than the directory of versioned packages. The "current" packages put everything in a package subfolder. The versioned packages (e.g. hl7.fhir.us.core#3.1.0) only put JSON in package and other folders are siblings to package. I assume this is a known thing? And that if the IG Publisher downloads current, it does it the same way?

I don't think this is something that is consistent across all packages. From what I have found, spot checking some packages, most packages just have everything in the package subfolder. Which is the documented structure of packages: https://confluence.hl7.org/pages/viewpage.action?pageId=35718629#NPMPackageSpecification-Format.

So it kinda just seems like a weird issue with the 3.1.0 version of US Core hosted on packages.fhir.org. If you visit packages.fhir.org/hl7.fhir.us.core/3.1.0 you will download the package from there, and you can see it does indeed have directories *not* within the package subfolder. What is weird though is that when the IG Publisher downloads US Core 3.1.0, it gets a version that *does* follow the correct structure, and has everything nested in a package subfolder. I had been made to beleive when first implementing this that the IG Publisher also got its packages from packages.fhir.org, so I don't know if that has changed or I was under the wrong impression. But it seems to be specific to this US Core package. 
cmoesel: Hmmm... I guess this behavior is probably not new to this PR -- but I wonder if this might cause issues...  If the IG publisher somehow installs the (correct?) expected folder structure, but SUSHI installs the oddball folder structure, might the publisher have problems processing the cached US Core package that SUSHI installed?  So far, it seems like it hasn't been a problem, but I actually find that... a little surprising!
ngfreiter: So I don't *think* it will be a problem because
1. It hasn't been a problem yet.
2. The definitions themselves are still in the same place, and isn't that all that should matter.

That being said it is a bit concerning, so I've done some further investigation, and the IG Publisher is doing something that it did not used to do. There is now a `packages-client` folder in my `.fhir` cache: 
![image](https://user-images.githubusercontent.com/52747882/83526237-1f096780-a4b4-11ea-9e8c-16dffc590e09.png)

And that folder contains the `.tgz` file from packages.fhir.org for hl7.fhir.us.core:
![image](https://user-images.githubusercontent.com/52747882/83526300-3d6f6300-a4b4-11ea-9600-afebaf23b9e4.png)

But then what is weird is that this zip file does have the incorrect directory structure, aka folders are at the same level as `package`, not nested within it. So is the IG Publisher converting this package into the proper form? If so, why is it doing that and not just requiring that packages are in the documented format? My investigations have not yet gone that far. But I don't think we want to get in the business of converting packages into the proper format if they are in some improper format.

You will also notice that I chose to keep actual dates out of the informational messages, and limit the number of messages. I feel like we are getting to a place where there is a bit of information overload coming from package loading, and I don't know if we want to add more info about why certain things are happening, when I imagine most users don't really care why stuff is happening, but what is actually happening. The other reason not to add dates is that the IG Publisher uses a weird non-documented format for its dates, so we are then tied to that format changing on a whim.

cmoesel: Perhaps we should just raise it w/ Grahame to ensure we understand the issue and that our assumption (that it's harmless) is correct?  I notice that `examples` and `spec.internals` aren't in the `package` folder -- but as long as the IG Publisher never tries to load those for dependencies, I guess we're probably OK.
ngfreiter: > Perhaps we should just raise it w/ Grahame to ensure we understand the issue and that our assumption (that it's harmless) is correct? I notice that `examples` and `spec.internals` aren't in the `package` folder -- but as long as the IG Publisher never tries to load those for dependencies, I guess we're probably OK.

Alright, we can keep an eye on this thread https://chat.fhir.org/#narrow/stream/215610-shorthand/topic/dev.20dependencies for any further explanation of what is going on here.
ngfreiter: @cmoesel I felt like I was too quick to think messages with a date wouldn't be helpful, so I added them as debug messages in https://github.com/FHIR/sushi/pull/470/commits/4d94a5aa8e59dafc1e88e3facf1772872421c3e5. That way people won't have to see them, but if they are confused about what is going on and use the debug flag, they can get more detailed info. But this way it's also easy for me to make them info if you feel strongly they should be, so let me know what you think.
ngfreiter: @cmoesel Moment is no more in https://github.com/FHIR/sushi/pull/470/commits/97a340fecb2d468286d546112459594487a37109.
ngfreiter: Alright @cmoesel I added a formateDate function. Good suggestion, not nit-picky! 

There has also been a response on our relevant zulip thread. I'll let you read it for yourself. I'm not sure what is the best way forward. The way I see it we can either do nothing and wait to see if this ever is a problem, or we can align with the IG Publisher approach and fix packages that have the old format. 

Patch:
diff --git a/src/app.ts b/src/app.ts
index e455aa5..9dbc8d2 100644
--- a/src/app.ts
+++ b/src/app.ts
@@ -121,7 +121,6 @@ async function app() {
     dependencyDefs.push(
       loadDependency(dep, config.dependencies[dep], defs)
         .then(def => {
-          logger.info(`Loaded package ${dep}#${config.dependencies[dep]}`);
           return def;
         })
         .catch(e => {
diff --git a/src/fhirdefs/load.ts b/src/fhirdefs/load.ts
index 5192c9f..fc88ee1 100644
--- a/src/fhirdefs/load.ts
+++ b/src/fhirdefs/load.ts
@@ -26,55 +26,88 @@ export async function loadDependency(
   let fullPackageName = `${packageName}#${version}`;
   let loadPath = path.join(cachePath, fullPackageName, 'package');
   let loadedPackage: string;
-  if (version != 'current') {
-    logger.info(`Checking local cache for ${fullPackageName}...`);
+
+  // First, try to load the package from the local cache
+  logger.info(`Checking local cache for ${fullPackageName}...`);
+  loadedPackage = loadFromPath(loadPath, fullPackageName, FHIRDefs);
+  if (loadedPackage) {
+    logger.info(`Found ${fullPackageName} in local cache.`);
+  } else {
+    logger.info(`Did not find ${fullPackageName} in local cache.`);
+  }
+
+  // When a dev package is not present locally, fall back to using the current version
+  // as described here https://confluence.hl7.org/pages/viewpage.action?pageId=35718627#IGPublisherDocumentation-DependencyList
+  if (version === 'dev' && !loadedPackage) {
+    logger.info(
+      `Falling back to ${packageName}#current since ${fullPackageName} is not locally cached. To avoid this, add ${fullPackageName} to your local FHIR cache by building it locally with the HL7 FHIR IG Publisher.`
+    );
+    version = 'current';
+    fullPackageName = `${packageName}#${version}`;
+    loadPath = path.join(cachePath, fullPackageName, 'package');
     loadedPackage = loadFromPath(loadPath, fullPackageName, FHIRDefs);
-    if (loadedPackage) {
-      logger.info(`Found ${fullPackageName} in local cache.`);
-    } else {
-      logger.info(`Did not find ${fullPackageName} in local cache.`);
-    }
   }
-  if (!loadedPackage) {
-    let packageUrl: string;
-    if (version === 'dev') {
-      // When a dev package is not present locally, fall back to using the current version
-      // as described here https://confluence.hl7.org/pages/viewpage.action?pageId=35718627#IGPublisherDocumentation-DependencyList
-      logger.info(
-        `Falling back to ${packageName}#current since ${fullPackageName} is not locally cached. To avoid this, add ${fullPackageName} to your local FHIR cache.`
-      );
-      version = 'current';
-      fullPackageName = `${packageName}#${version}`;
-      loadPath = path.join(cachePath, fullPackageName, 'package');
+
+  // Even if a local current package is loaded, we must still check that the local package date matches
+  // the date on the most recent version on build.fhir.org. If the date does not match, we re-download to the cache
+  let packageUrl;
+  if (version === 'current') {
+    const baseUrl = 'http://build.fhir.org/ig';
+    const res = await axios.get(`${baseUrl}/qas.json`);
+    const qaData: { 'package-id': string; date: string; repo: string }[] = res?.data;
+    // Find matching packages and sort by date to get the most recent
+    let newestPackage;
+    if (qaData?.length > 0) {
+      const matchingPackages = qaData.filter(p => p['package-id'] === packageName);
+      newestPackage = matchingPackages.sort((p1, p2) => {
+        return Date.parse(p2['date']) - Date.parse(p1['date']);
+      })[0];
     }
-    if (version === 'current') {
-      // Current packages need to be loaded using build.fhir.org
-      const baseUrl = 'http://build.fhir.org/ig';
-      const res = await axios.get(`${baseUrl}/qas.json`);
-      const qaData: { 'package-id': string; date: string; repo: string }[] = res?.data;
-      // Find matching packages and sort by date to get the most recent
-      let newestPackage;
-      if (qaData && qaData.length > 0) {
-        const matchingPackages = qaData.filter(p => p['package-id'] === packageName);
-        newestPackage = matchingPackages.sort((p1, p2) => {
-          return Date.parse(p2['date']) - Date.parse(p1['date']);
-        })[0];
+    if (newestPackage?.repo) {
+      const [org, repo] = newestPackage.repo.split('/');
+      const igUrl = `${baseUrl}/${org}/${repo}`;
+      // get the package.manifest.json for the newest version of the package on build.fhir.org
+      const manifest = await axios.get(`${igUrl}/package.manifest.json`);
+      let cachedPackageJSON;
+      if (fs.existsSync(path.join(loadPath, 'package.json'))) {
+        cachedPackageJSON = fs.readJSONSync(path.join(loadPath, 'package.json'));
       }
-      if (newestPackage && newestPackage.repo) {
-        // Find the package based on the first two parts of the package's 'repo' property.  E.g.,
-        //   "repo": "HL7/US-Core-R4/branches/test-branch-tweak/qa.json"
-        // means to find the package at:
-        //    http://build.fhir.org/ig/HL7/US-Core-R4/package.tgz
-        // See: https://chat.fhir.org/#narrow/stream/179165-committers/topic/Build.20Problem/near/187610137
-        const [org, repo] = newestPackage.repo.split('/');
-        packageUrl = `${baseUrl}/${org}/${repo}/package.tgz`;
+      // if the date on the package.manifest.json does not match the date on the cached package
+      // set the packageUrl to trigger a re-download of the package
+      if (manifest?.data?.date !== cachedPackageJSON?.date) {
+        packageUrl = `${igUrl}/package.tgz`;
+        if (cachedPackageJSON) {
+          logger.debug(
+            `Cached package date for ${fullPackageName} (${formatDate(
+              cachedPackageJSON.date
+            )}) does not match last build date on build.fhir.org (${formatDate(
+              manifest?.data?.date
+            )})`
+          );
+          logger.info(
+            `Cached package ${fullPackageName} is out of date and will be replaced by the more recent version found on build.fhir.org.`
+          );
+        }
       } else {
-        throw new CurrentPackageLoadError(fullPackageName);
+        logger.debug(
+          `Cached package date for ${fullPackageName} (${formatDate(
+            cachedPackageJSON.date
+          )}) matches last build date on build.fhir.org (${formatDate(
+            manifest?.data?.date
+          )}), so the cached package will be used`
+        );
       }
     } else {
-      // All non-current packages are stored at packages.fhir.org
-      packageUrl = `http://packages.fhir.org/${packageName}/${version}`;
+      throw new CurrentPackageLoadError(fullPackageName);
     }
+  } else if (!loadedPackage) {
+    // If the package is not locally cached, and it is not a current or dev version, we attempt to get it
+    // from packages.fhir.org
+    packageUrl = `http://packages.fhir.org/${packageName}/${version}`;
+  }
+
+  // If the packageUrl is set, we must download the package from that url, and extract it to our local cache
+  if (packageUrl) {
     // Create a temporary file and write the package to there
     temp.track();
     const tempFile = temp.openSync();
@@ -83,25 +116,29 @@ export async function loadDependency(
     const res = await axios.get(packageUrl, {
       responseType: 'arraybuffer'
     });
-    logger.info(`Downloaded ${fullPackageName}`);
-
-    fs.ensureDirSync(targetDirectory);
-    fs.writeFileSync(tempFile.path, res?.data);
-    // Extract the package from that temporary file location
-    tar.x({
-      cwd: targetDirectory,
-      file: tempFile.path,
-      sync: true,
-      strict: true
-    });
-
-    // Now try to load again from the path
-    loadedPackage = loadFromPath(loadPath, fullPackageName, FHIRDefs);
-    if (!loadedPackage) {
-      // If we fail again, then we couldn't get the package locally or from online
-      throw new PackageLoadError(fullPackageName);
+    if (res?.data) {
+      logger.info(`Downloaded ${fullPackageName}`);
+      fs.ensureDirSync(targetDirectory);
+      fs.writeFileSync(tempFile.path, res.data);
+      // Extract the package from that temporary file location
+      tar.x({
+        cwd: targetDirectory,
+        file: tempFile.path,
+        sync: true,
+        strict: true
+      });
+      // Now try to load again from the path
+      loadedPackage = loadFromPath(loadPath, fullPackageName, FHIRDefs);
+    } else {
+      logger.info(`Unable to download most current version of ${fullPackageName}`);
     }
   }
+
+  if (!loadedPackage) {
+    // If we fail again, then we couldn't get the package locally or from online
+    throw new PackageLoadError(fullPackageName);
+  }
+  logger.info(`Loaded package ${fullPackageName}`);
   return FHIRDefs;
 }
 
@@ -187,3 +224,14 @@ export function loadFromPath(
     return targetPackage;
   }
 }
+
+/**
+ * Takes a date in format YYYYMMDDHHmmss and converts to YYYY-MM-DDTHH:mm:ss
+ * @param {string} date - The date to format
+ * @returns {string} the formatted date
+ */
+function formatDate(date: string): string {
+  return date
+    ? date.replace(/(\d{4})(\d{2})(\d{2})(\d{2})(\d{2})(\d{2})/, '$1-$2-$3T$4:$5:$6')
+    : '';
+}


Test patch:
diff --git a/test/fhirdefs/fixtures/sushi-test#0.1.0/package/StructureDefinition-MyPatient.json b/test/fhirdefs/fixtures/sushi-test#0.1.0/package/StructureDefinition-MyPatient.json
new file mode 100644
index 0000000..4f65cc4
--- /dev/null
+++ b/test/fhirdefs/fixtures/sushi-test#0.1.0/package/StructureDefinition-MyPatient.json
@@ -0,0 +1,25 @@
+{
+  "resourceType": "StructureDefinition",
+  "id": "MyPatient",
+  "url": "http://hl7.org/fhir/sushi-test/StructureDefinition/MyPatient",
+  "name": "MyPatient",
+  "status": "active",
+  "fhirVersion": "4.0.1",
+  "kind": "resource",
+  "type": "Patient",
+  "baseDefinition": "http://hl7.org/fhir/StructureDefinition/Patient",
+  "derivation": "constraint",
+  "differential": {
+    "element": [
+      {
+        "id": "Patient.deceased[x]",
+        "path": "Patient.deceased[x]",
+        "type": [
+          {
+            "code": "boolean"
+          }
+        ]
+      }
+    ]
+  }
+}
diff --git a/test/fhirdefs/fixtures/sushi-test#0.1.0/package/package.json b/test/fhirdefs/fixtures/sushi-test#0.1.0/package/package.json
new file mode 100644
index 0000000..99f065f
--- /dev/null
+++ b/test/fhirdefs/fixtures/sushi-test#0.1.0/package/package.json
@@ -0,0 +1,24 @@
+{
+  "name": "sushi-test",
+  "version" : "0.1.0",
+  "canonical" : "http://hl7.org/fhir/sushi-test",
+  "url" : "http://hl7.org/fhir/sushi-test",
+  "title" : "FSH Test IG",
+  "description": "Provides a simple example of how FSH can be used to create an IG",
+  "dependencies": {
+     "hl7.fhir.r4.core": "4.0.1",
+     "hl7.fhir.us.core": "3.1.0",
+     "hl7.fhir.uv.vhdir":"current"
+  },
+  "date": "20200413230227",
+  "language": "en",
+  "author": "James Tuna",
+  "maintainers": [
+    {
+      "name": "Bill Cod",
+      "email": "cod@reef.gov",
+      "url": "https://capecodfishermen.org/"
+    }
+  ],
+  "license": "CC0-1.0"
+ }
\ No newline at end of file
diff --git a/test/fhirdefs/fixtures/sushi-test#current/package/StructureDefinition-MyPatient.json b/test/fhirdefs/fixtures/sushi-test#current/package/StructureDefinition-MyPatient.json
new file mode 100644
index 0000000..4f65cc4
--- /dev/null
+++ b/test/fhirdefs/fixtures/sushi-test#current/package/StructureDefinition-MyPatient.json
@@ -0,0 +1,25 @@
+{
+  "resourceType": "StructureDefinition",
+  "id": "MyPatient",
+  "url": "http://hl7.org/fhir/sushi-test/StructureDefinition/MyPatient",
+  "name": "MyPatient",
+  "status": "active",
+  "fhirVersion": "4.0.1",
+  "kind": "resource",
+  "type": "Patient",
+  "baseDefinition": "http://hl7.org/fhir/StructureDefinition/Patient",
+  "derivation": "constraint",
+  "differential": {
+    "element": [
+      {
+        "id": "Patient.deceased[x]",
+        "path": "Patient.deceased[x]",
+        "type": [
+          {
+            "code": "boolean"
+          }
+        ]
+      }
+    ]
+  }
+}
diff --git a/test/fhirdefs/fixtures/sushi-test#current/package/package.json b/test/fhirdefs/fixtures/sushi-test#current/package/package.json
new file mode 100644
index 0000000..99f065f
--- /dev/null
+++ b/test/fhirdefs/fixtures/sushi-test#current/package/package.json
@@ -0,0 +1,24 @@
+{
+  "name": "sushi-test",
+  "version" : "0.1.0",
+  "canonical" : "http://hl7.org/fhir/sushi-test",
+  "url" : "http://hl7.org/fhir/sushi-test",
+  "title" : "FSH Test IG",
+  "description": "Provides a simple example of how FSH can be used to create an IG",
+  "dependencies": {
+     "hl7.fhir.r4.core": "4.0.1",
+     "hl7.fhir.us.core": "3.1.0",
+     "hl7.fhir.uv.vhdir":"current"
+  },
+  "date": "20200413230227",
+  "language": "en",
+  "author": "James Tuna",
+  "maintainers": [
+    {
+      "name": "Bill Cod",
+      "email": "cod@reef.gov",
+      "url": "https://capecodfishermen.org/"
+    }
+  ],
+  "license": "CC0-1.0"
+ }
\ No newline at end of file
diff --git a/test/fhirdefs/fixtures/sushi-test#dev/package/StructureDefinition-MyPatient.json b/test/fhirdefs/fixtures/sushi-test#dev/package/StructureDefinition-MyPatient.json
new file mode 100644
index 0000000..4f65cc4
--- /dev/null
+++ b/test/fhirdefs/fixtures/sushi-test#dev/package/StructureDefinition-MyPatient.json
@@ -0,0 +1,25 @@
+{
+  "resourceType": "StructureDefinition",
+  "id": "MyPatient",
+  "url": "http://hl7.org/fhir/sushi-test/StructureDefinition/MyPatient",
+  "name": "MyPatient",
+  "status": "active",
+  "fhirVersion": "4.0.1",
+  "kind": "resource",
+  "type": "Patient",
+  "baseDefinition": "http://hl7.org/fhir/StructureDefinition/Patient",
+  "derivation": "constraint",
+  "differential": {
+    "element": [
+      {
+        "id": "Patient.deceased[x]",
+        "path": "Patient.deceased[x]",
+        "type": [
+          {
+            "code": "boolean"
+          }
+        ]
+      }
+    ]
+  }
+}
diff --git a/test/fhirdefs/fixtures/sushi-test#dev/package/package.json b/test/fhirdefs/fixtures/sushi-test#dev/package/package.json
new file mode 100644
index 0000000..99f065f
--- /dev/null
+++ b/test/fhirdefs/fixtures/sushi-test#dev/package/package.json
@@ -0,0 +1,24 @@
+{
+  "name": "sushi-test",
+  "version" : "0.1.0",
+  "canonical" : "http://hl7.org/fhir/sushi-test",
+  "url" : "http://hl7.org/fhir/sushi-test",
+  "title" : "FSH Test IG",
+  "description": "Provides a simple example of how FSH can be used to create an IG",
+  "dependencies": {
+     "hl7.fhir.r4.core": "4.0.1",
+     "hl7.fhir.us.core": "3.1.0",
+     "hl7.fhir.uv.vhdir":"current"
+  },
+  "date": "20200413230227",
+  "language": "en",
+  "author": "James Tuna",
+  "maintainers": [
+    {
+      "name": "Bill Cod",
+      "email": "cod@reef.gov",
+      "url": "https://capecodfishermen.org/"
+    }
+  ],
+  "license": "CC0-1.0"
+ }
\ No newline at end of file
diff --git a/test/fhirdefs/fixtures/sushi-test-no-download#current/package/StructureDefinition-MyPatient.json b/test/fhirdefs/fixtures/sushi-test-no-download#current/package/StructureDefinition-MyPatient.json
new file mode 100644
index 0000000..4f65cc4
--- /dev/null
+++ b/test/fhirdefs/fixtures/sushi-test-no-download#current/package/StructureDefinition-MyPatient.json
@@ -0,0 +1,25 @@
+{
+  "resourceType": "StructureDefinition",
+  "id": "MyPatient",
+  "url": "http://hl7.org/fhir/sushi-test/StructureDefinition/MyPatient",
+  "name": "MyPatient",
+  "status": "active",
+  "fhirVersion": "4.0.1",
+  "kind": "resource",
+  "type": "Patient",
+  "baseDefinition": "http://hl7.org/fhir/StructureDefinition/Patient",
+  "derivation": "constraint",
+  "differential": {
+    "element": [
+      {
+        "id": "Patient.deceased[x]",
+        "path": "Patient.deceased[x]",
+        "type": [
+          {
+            "code": "boolean"
+          }
+        ]
+      }
+    ]
+  }
+}
diff --git a/test/fhirdefs/fixtures/sushi-test-no-download#current/package/package.json b/test/fhirdefs/fixtures/sushi-test-no-download#current/package/package.json
new file mode 100644
index 0000000..f2911df
--- /dev/null
+++ b/test/fhirdefs/fixtures/sushi-test-no-download#current/package/package.json
@@ -0,0 +1,24 @@
+{
+  "name": "sushi-test",
+  "version" : "0.1.0",
+  "canonical" : "http://hl7.org/fhir/sushi-test",
+  "url" : "http://hl7.org/fhir/sushi-test",
+  "title" : "FSH Test IG",
+  "description": "Provides a simple example of how FSH can be used to create an IG",
+  "dependencies": {
+     "hl7.fhir.r4.core": "4.0.1",
+     "hl7.fhir.us.core": "3.1.0",
+     "hl7.fhir.uv.vhdir":"current"
+  },
+  "date": "20100413230227",
+  "language": "en",
+  "author": "James Tuna",
+  "maintainers": [
+    {
+      "name": "Bill Cod",
+      "email": "cod@reef.gov",
+      "url": "https://capecodfishermen.org/"
+    }
+  ],
+  "license": "CC0-1.0"
+ }
\ No newline at end of file
diff --git a/test/fhirdefs/fixtures/sushi-test-old#current/package/StructureDefinition-MyPatient.json b/test/fhirdefs/fixtures/sushi-test-old#current/package/StructureDefinition-MyPatient.json
new file mode 100644
index 0000000..4f65cc4
--- /dev/null
+++ b/test/fhirdefs/fixtures/sushi-test-old#current/package/StructureDefinition-MyPatient.json
@@ -0,0 +1,25 @@
+{
+  "resourceType": "StructureDefinition",
+  "id": "MyPatient",
+  "url": "http://hl7.org/fhir/sushi-test/StructureDefinition/MyPatient",
+  "name": "MyPatient",
+  "status": "active",
+  "fhirVersion": "4.0.1",
+  "kind": "resource",
+  "type": "Patient",
+  "baseDefinition": "http://hl7.org/fhir/StructureDefinition/Patient",
+  "derivation": "constraint",
+  "differential": {
+    "element": [
+      {
+        "id": "Patient.deceased[x]",
+        "path": "Patient.deceased[x]",
+        "type": [
+          {
+            "code": "boolean"
+          }
+        ]
+      }
+    ]
+  }
+}
diff --git a/test/fhirdefs/fixtures/sushi-test-old#current/package/package.json b/test/fhirdefs/fixtures/sushi-test-old#current/package/package.json
new file mode 100644
index 0000000..6d55a7b
--- /dev/null
+++ b/test/fhirdefs/fixtures/sushi-test-old#current/package/package.json
@@ -0,0 +1,24 @@
+{
+  "name": "sushi-test-old-available",
+  "version" : "0.1.0",
+  "canonical" : "http://hl7.org/fhir/sushi-test-old-available",
+  "url" : "http://hl7.org/fhir/sushi-test-old-available",
+  "title" : "FSH Test IG",
+  "description": "Provides a simple example of how FSH can be used to create an IG",
+  "dependencies": {
+     "hl7.fhir.r4.core": "4.0.1",
+     "hl7.fhir.us.core": "3.1.0",
+     "hl7.fhir.uv.vhdir":"current"
+  },
+  "date": "20100413230227",
+  "language": "en",
+  "author": "James Tuna",
+  "maintainers": [
+    {
+      "name": "Bill Cod",
+      "email": "cod@reef.gov",
+      "url": "https://capecodfishermen.org/"
+    }
+  ],
+  "license": "CC0-1.0"
+ }
\ No newline at end of file
diff --git a/test/fhirdefs/load.test.ts b/test/fhirdefs/load.test.ts
index 97e627f..50ade5e 100644
--- a/test/fhirdefs/load.test.ts
+++ b/test/fhirdefs/load.test.ts
@@ -62,8 +62,8 @@ describe('#loadDependency()', () => {
   let tarSpy: jest.SpyInstance;
   let ensureDirSpy: jest.SpyInstance;
   let writeSpy: jest.SpyInstance;
+  let cachePath: string;
   beforeAll(() => {
-    defs = new FHIRDefinitions();
     axiosSpy = jest.spyOn(axios, 'get').mockImplementation((uri: string): any => {
       if (uri === 'http://build.fhir.org/ig/qas.json') {
         return {
@@ -93,9 +93,50 @@ describe('#loadDependency()', () => {
               version: '4.0.0',
               tool: '4.1.0 (3)',
               repo: 'HL7/US-Core-R4/branches/newbranch/qa.json'
+            },
+            {
+              url:
+                'http://hl7.org/fhir/sushi-test-no-download/ImplementationGuide/sushi-test-no-download-0.1.0',
+              name: 'sushi-test-no-download',
+              'package-id': 'sushi-test-no-download',
+              'ig-ver': '0.1.0',
+              repo: 'sushi/sushi-test-no-download/branches/master/qa.json'
+            },
+            {
+              url: 'http://hl7.org/fhir/sushi-test-old/ImplementationGuide/sushi-test-old-0.1.0',
+              name: 'sushi-test-old',
+              'package-id': 'sushi-test-old',
+              'ig-ver': '0.1.0',
+              repo: 'sushi/sushi-test-old/branches/master/qa.json'
+            },
+            {
+              url: 'http://hl7.org/fhir/sushi-test/ImplementationGuide/sushi-test-0.1.0',
+              name: 'sushi-test',
+              'package-id': 'sushi-test',
+              'ig-ver': '0.1.0',
+              repo: 'sushi/sushi-test/branches/master/qa.json'
             }
           ]
         };
+      } else if (
+        uri === 'http://build.fhir.org/ig/HL7/US-Core-R4/package.manifest.json' ||
+        (uri.startsWith('http://build.fhir.org/ig/sushi/sushi-test') && uri.endsWith('json'))
+      ) {
+        return {
+          data: {
+            date: '20200413230227'
+          }
+        };
+      } else if (
+        uri === 'http://packages.fhir.org/sushi-test/0.2.0' ||
+        uri === 'http://build.fhir.org/ig/sushi/sushi-test-old/package.tgz' ||
+        uri === 'http://build.fhir.org/ig/HL7/US-Core-R4/package.tgz'
+      ) {
+        return {
+          data: {
+            some: 'zipfile'
+          }
+        };
       } else {
         return {};
       }
@@ -103,69 +144,150 @@ describe('#loadDependency()', () => {
     tarSpy = jest.spyOn(tar, 'x').mockImplementation(() => {});
     writeSpy = jest.spyOn(fs, 'writeFileSync').mockImplementation(() => {});
     ensureDirSpy = jest.spyOn(fs, 'ensureDirSync').mockImplementation(() => {});
+    cachePath = path.join(__dirname, 'fixtures');
   });
 
   beforeEach(() => {
+    loggerSpy.reset();
+    defs = new FHIRDefinitions();
     axiosSpy.mockClear();
     tarSpy.mockClear();
     writeSpy.mockClear();
     ensureDirSpy.mockClear();
   });
 
-  it('should not try to download a package that is already in the cache', async () => {
-    defs = new FHIRDefinitions();
+  // Packages with numerical versions
+  it('should not try to download a non-current package that is already in the cache', async () => {
+    const expectedDefs = new FHIRDefinitions();
     loadFromPath(
-      path.join(__dirname, '..', 'testhelpers', 'testdefs', 'package'),
-      'test#1.1.1',
-      defs
+      path.join(cachePath, 'sushi-test#0.1.0', 'package'),
+      'sushi-test#0.1.0',
+      expectedDefs
+    );
+    await expect(loadDependency('sushi-test', '0.1.0', defs, cachePath)).resolves.toEqual(
+      expectedDefs
     );
-    await expect(
-      loadDependency(
-        'test',
-        '1.1.1',
-        defs,
-        path.join(__dirname, '..', 'testhelpers', 'testdefs', 'package')
-      )
-    ).resolves.toEqual(defs);
     expect(axiosSpy.mock.calls.length).toBe(0);
   });
 
-  it('should try to load a package from packages.fhir.org when a non-special package version is loaded', async () => {
-    await expect(loadDependency('hl7.fhir.hspc', '1.1.1', defs, 'foo')).rejects.toThrow(
-      'The package hl7.fhir.hspc#1.1.1 could not be loaded locally or from the FHIR package registry'
+  it('should try to load a package from packages.fhir.org when a non-current package is not cached', async () => {
+    await expect(loadDependency('sushi-test', '0.2.0', defs, 'foo')).rejects.toThrow(
+      'The package sushi-test#0.2.0 could not be loaded locally or from the FHIR package registry'
+    ); // the package is never actually added to the cache, since tar is mocked
+    expect(axiosSpy.mock.calls).toEqual([
+      ['http://packages.fhir.org/sushi-test/0.2.0', { responseType: 'arraybuffer' }]
+    ]);
+    expect(ensureDirSpy.mock.calls[0]).toEqual([path.join('foo', 'sushi-test#0.2.0')]);
+    expect(tarSpy.mock.calls[0][0].cwd).toBe(path.join('foo', 'sushi-test#0.2.0'));
+  });
+
+  it('should throw PackageLoadError when a package with a non-current version is not cached or available on packages.fhir.org', async () => {
+    await expect(loadDependency('sushi-test', '0.3.0', defs, 'foo')).rejects.toThrow(
+      'The package sushi-test#0.3.0 could not be loaded locally or from the FHIR package registry'
+    );
+    expect(loggerSpy.getLastMessage('info')).toMatch(
+      /Unable to download most current version of sushi-test#0.3.0/
+    );
+    expect(axiosSpy.mock.calls).toEqual([
+      ['http://packages.fhir.org/sushi-test/0.3.0', { responseType: 'arraybuffer' }]
+    ]);
+    expect(ensureDirSpy.mock.calls).toHaveLength(0);
+    expect(tarSpy.mock.calls).toHaveLength(0);
+  });
+
+  // Packages with current versions
+  it('should not try to download a current package that is already in the cache and up to date', async () => {
+    const expectedDefs = new FHIRDefinitions();
+    loadFromPath(
+      path.join(cachePath, 'sushi-test#current', 'package'),
+      'sushi-test#current',
+      expectedDefs
     );
-    expect(axiosSpy.mock.calls[0][0]).toBe('http://packages.fhir.org/hl7.fhir.hspc/1.1.1');
-    expect(ensureDirSpy.mock.calls[0]).toEqual([path.join('foo', 'hl7.fhir.hspc#1.1.1')]);
-    expect(tarSpy.mock.calls[0][0].cwd).toBe(path.join('foo', 'hl7.fhir.hspc#1.1.1'));
+    await expect(loadDependency('sushi-test', 'current', defs, cachePath)).resolves.toEqual(
+      expectedDefs
+    );
+    expect(axiosSpy.mock.calls).toEqual([
+      ['http://build.fhir.org/ig/qas.json'],
+      ['http://build.fhir.org/ig/sushi/sushi-test/package.manifest.json']
+    ]);
   });
 
-  it('should try to load the latest package from build.fhir.org when a current package version is loaded', async () => {
+  it('should try to load the latest package from build.fhir.org when a current package version is not locally cached', async () => {
     await expect(loadDependency('hl7.fhir.us.core.r4', 'current', defs, 'foo')).rejects.toThrow(
       'The package hl7.fhir.us.core.r4#current could not be loaded locally or from the FHIR package registry'
-    );
-    expect(axiosSpy.mock.calls[0][0]).toBe('http://build.fhir.org/ig/qas.json');
-    expect(axiosSpy.mock.calls[1][0]).toBe('http://build.fhir.org/ig/HL7/US-Core-R4/package.tgz');
+    ); // the package is never actually added to the cache, since tar is mocked
+    expect(axiosSpy.mock.calls).toEqual([
+      ['http://build.fhir.org/ig/qas.json'],
+      ['http://build.fhir.org/ig/HL7/US-Core-R4/package.manifest.json'],
+      ['http://build.fhir.org/ig/HL7/US-Core-R4/package.tgz', { responseType: 'arraybuffer' }]
+    ]);
     expect(ensureDirSpy.mock.calls[0]).toEqual([path.join('foo', 'hl7.fhir.us.core.r4#current')]);
     expect(tarSpy.mock.calls[0][0].cwd).toBe(path.join('foo', 'hl7.fhir.us.core.r4#current'));
   });
 
-  it('should load the current package from build.fhir.org when a dev package is loaded and not locally cached', async () => {
-    await expect(loadDependency('hl7.fhir.us.core.r4', 'dev', defs, 'foo')).rejects.toThrow(
-      'The package hl7.fhir.us.core.r4#current could not be loaded locally or from the FHIR package registry'
+  it('should try to load the latest package from build.fhir.org when a current package version has an older version that is locally cached', async () => {
+    await expect(
+      loadDependency('sushi-test-old', 'current', defs, cachePath)
+    ).resolves.toBeTruthy(); // Since tar is mocked, the actual cache is not updated
+    expect(axiosSpy.mock.calls).toEqual([
+      ['http://build.fhir.org/ig/qas.json'],
+      ['http://build.fhir.org/ig/sushi/sushi-test-old/package.manifest.json'],
+      ['http://build.fhir.org/ig/sushi/sushi-test-old/package.tgz', { responseType: 'arraybuffer' }]
+    ]);
+    expect(ensureDirSpy.mock.calls[0][0]).toEqual(path.join(cachePath, 'sushi-test-old#current'));
+    expect(tarSpy.mock.calls[0][0].cwd).toBe(path.join(cachePath, 'sushi-test-old#current'));
+  });
+
+  it('should revert to an old locally cached current version when a newer current version is not available for download', async () => {
+    const expectedDefs = new FHIRDefinitions();
+    loadFromPath(
+      path.join(cachePath, 'sushi-test-no-download#current', 'package'),
+      'sushi-test-no-download#current',
+      expectedDefs
+    );
+    await expect(
+      loadDependency('sushi-test-no-download', 'current', defs, cachePath)
+    ).resolves.toEqual(expectedDefs);
+    expect(axiosSpy.mock.calls).toEqual([
+      ['http://build.fhir.org/ig/qas.json'],
+      ['http://build.fhir.org/ig/sushi/sushi-test-no-download/package.manifest.json'],
+      [
+        'http://build.fhir.org/ig/sushi/sushi-test-no-download/package.tgz',
+        { responseType: 'arraybuffer' }
+      ]
+    ]);
+    expect(ensureDirSpy.mock.calls).toHaveLength(0);
+    expect(tarSpy.mock.calls).toHaveLength(0);
+  });
+
+  // Packages with dev versions
+  it('should not try to download a dev package that is already in the cache', async () => {
+    const expectedDefs = new FHIRDefinitions();
+    loadFromPath(path.join(cachePath, 'sushi-test#dev', 'package'), 'sushi-test#dev', expectedDefs);
+    await expect(loadDependency('sushi-test', 'dev', defs, cachePath)).resolves.toEqual(
+      expectedDefs
     );
+    expect(axiosSpy.mock.calls).toHaveLength(0);
+  });
+
+  it('should load the current package from build.fhir.org when a dev package is loaded and not locally cached', async () => {
+    await expect(loadDependency('sushi-test-old', 'dev', defs, cachePath)).resolves.toBeTruthy();
     expect(
       loggerSpy
         .getAllMessages('info')
         .some(message =>
           message.match(
-            /Falling back to hl7.fhir.us.core.r4#current since hl7.fhir.us.core.r4#dev is not locally cached./
+            /Falling back to sushi-test-old#current since sushi-test-old#dev is not locally cached./
           )
         )
     ).toBeTruthy();
-    expect(axiosSpy.mock.calls[0][0]).toBe('http://build.fhir.org/ig/qas.json');
-    expect(axiosSpy.mock.calls[1][0]).toBe('http://build.fhir.org/ig/HL7/US-Core-R4/package.tgz');
-    expect(ensureDirSpy.mock.calls[0]).toEqual([path.join('foo', 'hl7.fhir.us.core.r4#current')]);
-    expect(tarSpy.mock.calls[0][0].cwd).toBe(path.join('foo', 'hl7.fhir.us.core.r4#current'));
+    expect(axiosSpy.mock.calls).toEqual([
+      ['http://build.fhir.org/ig/qas.json'],
+      ['http://build.fhir.org/ig/sushi/sushi-test-old/package.manifest.json'],
+      ['http://build.fhir.org/ig/sushi/sushi-test-old/package.tgz', { responseType: 'arraybuffer' }]
+    ]);
+    expect(ensureDirSpy.mock.calls[0]).toEqual([path.join(cachePath, 'sushi-test-old#current')]);
+    expect(tarSpy.mock.calls[0][0].cwd).toBe(path.join(cachePath, 'sushi-test-old#current'));
   });
 
   it('should throw CurrentPackageLoadError when a current package is not listed', async () => {