Repository: medplum/medplum (TypeScript)

Issue:
Cannot add StructureDefinition. Problem with FHIRPath replaceMatches()
Related: https://github.com/medplum/medplum/pull/6760

## Problem
When I try to add [this StructureDefinition](http://fhir.de/StructureDefinition/informationrecipient) to medplum, I get an error. The same error happens with many StructureDefinitions inside the [de.basisprofil.r4](https://simplifier.net/packages/de.basisprofil.r4/1.5.4) package that use differentials.

`Constraint sdf-8a not met: In any differential, all the elements must start with the StructureDefinition's specified type for non-logical models, or with the same type name for logical models ({"fhirpath":"(%resource.kind = 'logical' or element.first().path.startsWith(%resource.type)) and (element.tail().empty() or element.tail().all(path.startsWith(%resource.differential.element.first().path.replaceMatches('\\\\..*','')&'.')))"}) (StructureDefinition.differential)`

<img width="997" height="714" alt="Image" src="https://github.com/user-attachments/assets/a9106d8d-6227-4722-8d24-1a7944dcd925" />


## Deep Dive
After doing some research, I found the following: The FHIRPath in [sdf-8a](https://hl7.org/fhir/R4/structuredefinition-definitions.html) uses `replaceMatches('\\..*','')` to turn e.g. `"Extension.url"` into `"Extension"`. This transformation however fails in the current @medplum/core, because it internally creates the regex `/\\..*/` instead of `/\..*/`.


## Try it out
```ts
test('Test replaceMatches()', () => {
  expect(evalFhirPath(`foo.replaceMatches('\\\\..*','')`, { foo: 'Extension.url' })).toStrictEqual(['Extension']); // ❌ Fails currently
  //                                         ^ Why 4 backslashes?
  // Because the FHIRPath expression for sdf-8a is literally \\..*
  // That's why we need to escape them in the javascript string
  // Instead of '\\\\..*' we could also write String.raw`\\..*`
});
```


## Solution(?)
My first thought would be to manually remove the "over-escaped" backslashes. But I'm not sure if that's the right place to make the change. It could also be a bug upstream, and the [source file](https://simplifier.net/packages/hl7.fhir.r4.core/4.0.1/files/2831303/~xml) should actually say `\..*` instead of `\\..*`. But someone else should be the judge of that.

https://github.com/medplum/medplum/blob/5db6a9472dd897cd8c5fe3d9e355577042dba314/packages/core/src/fhirpath/functions.ts#L1301

```diff
-      (str, pattern, substition) => str.replaceAll(new RegExp(pattern, 'g'), substition)
+      (str, pattern, substition) => str.replaceAll(new RegExp(pattern.replaceAll('\\\\', '\\'), 'g'), substition)
```



Discussion:
Thanks for the detailed writeup @fardolieri — this appears to be an issue in the base FHIR definitions themselves.  The [website](https://hl7.org/fhir/R4/structuredefinition-definitions.html#StructureDefinition.differential) shows that the correct stringified regular expression is `"\\..*"`: an escaped backslash, followed by two periods and an asterisk.  However, the [raw JSON](https://hl7.org/fhir/structuredefinition.profile.json) `StructureDefinition` that is used to generate that page contains the following:

```json
{
  "key" : "sdf-8a",
  "expression" : "(%resource.kind = 'logical' or element.first().path.startsWith(%resource.type)) and (element.tail().empty() or element.tail().all(path.startsWith(%resource.differential.element.first().path.replaceMatches('\\\\..*','')&'.')))",
}
```

Note that the backslash in this string is, in fact, __double-escaped__: since Medplum uses these JSON definition files as the source of truth, the erroneous string is carried all the way through to the FHIRPath function and used directly as the regex, resulting in no matches.

In this case, since the base StructureDefinition is pretty clearly incorrect, we can probably fix this issue just by editing the JSON directly.  I'll follow up with a PR to do just that
Awesome response time. Appreciate it!

I thought about it some more and I'm not yet sure whether our assumption is entirely correct. I think the JSON base StructureDefinition having 4 backslashes is actually correct because we are looking at a JSON string where backslashes need to be escaped.

- `replaceMatches('\\..*','')` [FHIRPath Spec (Human Readable)](https://hl7.org/fhir/R4/structuredefinition-definitions.html#StructureDefinition.differential) `2x \` ✅ Source of Truth (SoT)
- `replaceMatches('\\..*','')` [XML Spec](https://hl7.org/fhir/R4/structuredefinition.profile.xml.html) `2x \` ✅ Placing SoT into XML text node. XML does not need escapes.
- `replaceMatches('\\\\..*','')` [JSON Spec](https://hl7.org/fhir/R4/structuredefinition.profile.json) `4x \` ✅ Placing SoT into JSON string. JSON needs escapes.

Now the exciting question is, why does the original FHIRPath have two backslashes instead of just one? It's its own pseudo-language and could do whatever it wants including saying `'\..*'`, right? I think the answer is that [they simply chose to have their own kind of escape characters in FHIRPath strings](https://build.fhir.org/ig/HL7/FHIRPath/#string). That way they are more flexible and can support stuff like representing single quotes inside a FHIRPath string `'\''`.

This brings me to the conclusion that we should actually not change the content of the original JSON base StructureDefinition but instead deal with the short list of FHIRPath escape characters by hand.

> No other escape sequences besides those listed above are recognized.

```ts
  replaceMatches: (context: AtomContext, input: TypedValue[], regexAtom: Atom, substitionAtom: Atom): TypedValue[] => {
    return applyStringFunc(
      (str, pattern, substition) => {
        /**
         * See https://build.fhir.org/ig/HL7/FHIRPath/#string
         */
        const unescapedPattern = (pattern as string)
          .replaceAll("\\'", "'") // \' -> Single-quote
          .replaceAll('\\"', '"') // \" -> Double-quote
          .replaceAll('\\`', '`') // \` -> Backtick
          .replaceAll('\\r', '\r') // \r -> Carriage Return
          .replaceAll('\\n', '\n') // \n -> Line Feed
          .replaceAll('\\t', '\t') // \t -> Tab
          .replaceAll('\\f', '\f') // \f -> Form Feed
          .replaceAll('\\\\', '\\') // \\ -> Backslash
          .replaceAll(/\\u[0-9A-Fa-f]{4}/g, (match) => String.fromCharCode(parseInt(match.slice(2), 16))); // \uXXXX -> Unicode character, where XXXX is the hexadecimal representation of the character
        return str.replaceAll(new RegExp(unescapedPattern, 'g'), substition as string);
      },
      context,
      input,
      regexAtom,
      substitionAtom
    );
  },
```

I haven't tested this yet, but it should probably be thoroughly tested.

What do you think?

**EDIT:**
I should note that the [docs](https://build.fhir.org/ig/HL7/FHIRPath/#string) also say

> If a \ is used at the beginning of a non-escape sequence, it will be ignored and will not appear in the sequence.
>
> define TestEscape1: '\p' // 'p'
> define TestEscape2: '\\\\p' // '\p'
> define TestEscape3: '\3' // '3'
> define TestEscape4: '\u005' // 'u005'
> define TestEscape5: '\\' // ''

My solution above does currently not deal with those cases. But I'm not sure if this is important to deal with because someone would have to write a faulty FHIRPath in the first place for that to take place.
Do you mind if I open a PR that also includes some tests for the edge cases?
@fardolieri  Sure, go ahead 👍 



Patch:
diff --git a/packages/core/src/fhirpath/functions.ts b/packages/core/src/fhirpath/functions.ts
index 20182f1844..1f2f72f8db 100644
--- a/packages/core/src/fhirpath/functions.ts
+++ b/packages/core/src/fhirpath/functions.ts
@@ -1887,7 +1887,9 @@ function applyStringFunc<T>(
   if (typeof value !== 'string') {
     throw new Error('String function cannot be called with non-string');
   }
-  const result = func(value, ...argsAtoms.map((atom) => atom?.eval(context, input)[0]?.value));
+
+  const args = argsAtoms.map((atom) => atom?.eval(context, input)[0]?.value);
+  const result = func(value, ...args);
   if (result === undefined) {
     return [];
   }
diff --git a/packages/definitions/dist/fhir/r4/profiles-resources.json b/packages/definitions/dist/fhir/r4/profiles-resources.json
index 13f91bede5..62914ae737 100644
--- a/packages/definitions/dist/fhir/r4/profiles-resources.json
+++ b/packages/definitions/dist/fhir/r4/profiles-resources.json
@@ -337822,7 +337822,7 @@
             "key" : "sdf-8a",
             "severity" : "error",
             "human" : "In any differential, all the elements must start with the StructureDefinition's specified type for non-logical models, or with the same type name for logical models",
-            "expression" : "(%resource.kind = 'logical' or element.first().path.startsWith(%resource.type)) and (element.tail().empty() or element.tail().all(path.startsWith(%resource.differential.element.first().path.replaceMatches('\\\\..*','')&'.')))",
+            "expression" : "(%resource.kind = 'logical' or element.first().path.startsWith(%resource.type)) and (element.tail().empty() or element.tail().all(path.startsWith(%resource.differential.element.first().path.replaceMatches('\\..*','')&'.')))",
             "xpath" : "count(f:element)=count(f:element[f:path/@value=ancestor::f:StructureDefinition/f:type/@value or starts-with(f:path/@value, concat(ancestor::f:StructureDefinition/f:type/@value, '.'))])"
           }],
           "isModifier" : false,


Test patch:
diff --git a/packages/core/src/typeschema/validation.test.ts b/packages/core/src/typeschema/validation.test.ts
index 020b2010ea..71c90ab5fc 100644
--- a/packages/core/src/typeschema/validation.test.ts
+++ b/packages/core/src/typeschema/validation.test.ts
@@ -316,6 +316,81 @@ describe('FHIR resource validation', () => {
     expect(() => validateResource(medplumBundle)).not.toThrow();
   });
 
+  test('StructureDefinition with differential', () => {
+    const sd: StructureDefinition = {
+      resourceType: 'StructureDefinition',
+      url: 'http://fhir.de/StructureDefinition/informationrecipient',
+      name: 'ExtensionInformationRecipient',
+      status: 'active',
+      kind: 'complex-type',
+      abstract: false,
+      context: [
+        {
+          type: 'element',
+          expression: 'Composition',
+        },
+      ],
+      type: 'Extension',
+      baseDefinition: 'http://hl7.org/fhir/StructureDefinition/Extension',
+      derivation: 'constraint',
+      differential: {
+        element: [
+          {
+            id: 'Extension.extension',
+            path: 'Extension.extension',
+            max: '0',
+          },
+          {
+            id: 'Extension.url',
+            path: 'Extension.url',
+            fixedUri: 'http://fhir.de/StructureDefinition/informationrecipient',
+          },
+          {
+            id: 'Extension.value[x]',
+            path: 'Extension.value[x]',
+            slicing: {
+              discriminator: [
+                {
+                  type: 'type',
+                  path: '$this',
+                },
+              ],
+              ordered: false,
+              rules: 'open',
+            },
+            min: 1,
+            type: [
+              {
+                code: 'Reference',
+                targetProfile: [
+                  'http://hl7.org/fhir/StructureDefinition/Practitioner',
+                  'http://hl7.org/fhir/StructureDefinition/Device',
+                  'http://hl7.org/fhir/StructureDefinition/Patient',
+                  'http://hl7.org/fhir/StructureDefinition/RelatedPerson',
+                  'http://hl7.org/fhir/StructureDefinition/PractitionerRole',
+                  'http://hl7.org/fhir/StructureDefinition/Organization',
+                ],
+              },
+            ],
+          },
+          {
+            id: 'Extension.value[x]:valueReference',
+            path: 'Extension.value[x]',
+            sliceName: 'valueReference',
+            min: 1,
+            max: '1',
+            type: [
+              {
+                code: 'Reference',
+              },
+            ],
+          },
+        ],
+      },
+    };
+    expect(() => validateResource(sd)).not.toThrow();
+  });
+
   test('Profile with restriction on base type field', () => {
     const patient: Patient = {
       resourceType: 'Patient',