Repository: obsidian-tasks-group/obsidian-tasks (ts)

Issue:
Make the `explain` instruction show how regular expressions are interpreted
### ⚠️ Please check that this feature request hasn't been suggested before.

- [X] I searched previous [Ideas in Discussions](https://github.com/obsidian-tasks-group/obsidian-tasks/discussions/categories/ideas-any-new-feature-requests-go-in-issues-please?discussions_q=category%3A%22Ideas%3A+Any+New+Feature+Requests+go+in+Issues+please%22+-label%3A%22status%3A+released%22+-label%3Aduplicate+sort%3Atop) didn't find any similar feature requests.
- [X] I searched previous [Issues](https://github.com/obsidian-tasks-group/obsidian-tasks/issues?q=is%3Aopen+is%3Aissue+label%3A%22type%3A+enhancement%22+sort%3Acomments-desc) didn't find any similar feature requests.

### 🔖 Feature description

This instruction:

```
folder regex matches /^Home/Family/Us/
```

Is actually interpreted as this because of #1037:

```
folder regex matches /^Home/
```

That issue still needs fixing, but in the meantime I think it would be helpful for `explain` on regex instructions to show how they have been interpreted...

### ✔️ Solution

Change this output of `explain`

```
folder regex matches /^Home/Family/Us/
```

To:

```
folder regex matches /^Home/Family/Us/ =>
  folder regex matches /^Home/
```


### ❓ Alternatives

Fix #1037 so that such misleading lines are detected and not run.

But still it would be useful to see the interpretation, because it will be useful to see how the flags are interpreted, for example.


### 📝 Additional Context

_No response_

Pull request:
feat: explain now shows regex interpretation
<!--- Provide a general summary of your changes in the Title above -->

# Description

This adds information to the explanation of `regex matches` and `regex does not match` queries, to show how the regular expression is interpreted.

Example output:

```
description regex matches /^$/ =>
  using regex:            '^$' with no flags

description regex matches /#context/pc_photos|#context/pc_clare|#context/pc_macbook/i =>
  using regex:            '#context' with no flags

description regex matches /#context\/pc_photos|#context\/pc_clare|#context\/pc_macbook/i =>
  using regex:            '#context\/pc_photos|#context\/pc_clare|#context\/pc_macbook' with flag 'i'

description regex matches /[⏫🔼🔽📅⏳🛫🔁]/u =>
  using regex:            '[⏫🔼🔽📅⏳🛫🔁]' with no flags

folder regex matches /root/sub-folder/sub-sub-folder/ =>
  using regex:       'root' with no flags

folder regex matches /root\/sub-folder\/sub-sub-folder/ =>
  using regex:       'root\/sub-folder\/sub-sub-folder' with no flags
```

## Motivation and Context

<!--- Why is this change required? What problem does it solve? -->
<!--- If it fixes an open issue, please link to the issue here. -->

Fixes #2108.

## How has this been tested?

<!--- Please describe in detail how you tested your changes. -->
<!--- Include details of your testing environment, tests ran to see how -->
<!--- your change affects other areas of the code, etc. -->

## Screenshots (if appropriate)

## Types of changes

<!--- What types of changes does your code introduce? Put an `x` in all the boxes that apply: -->

Changes visible to users:

- [x] **New feature** (prefix: `feat` - non-breaking change which adds functionality)
- [ ] **Documentation** (prefix: `docs` - improvements to any documentation content **for users**)

Internal changes:

- [x] **Tests** (prefix: `test` - additions and improvements to unit tests and the smoke tests)

## Checklist

<!--- Go over all the following points, and put an `x` in all the boxes that apply. -->
<!--- If you're unsure about any of these, don't hesitate to ask. We're here to help! -->

- [x] My code follows the code style of this project and passes `yarn run lint`.
- [x] My change requires a change to the documentation.
- [ ] I have [updated the documentation](https://publish.obsidian.md/tasks-contributing/Documentation/About+Documentation) accordingly.
- [x] My change has adequate [Unit Test coverage](https://publish.obsidian.md/tasks-contributing/Testing/About+Testing).

## Terms

<!--
By submitting this pull request, you must agree to follow our
[contributing guide](https://publish.obsidian.md/tasks-contributing) and
[Code of Conduct](https://github.com/obsidian-tasks-group/obsidian-tasks/blob/main/CODE_OF_CONDUCT.md).
Put an x in the boxes to confirm you agree.
-->

- [x] My contribution follow this project's [contributing guide](https://publish.obsidian.md/tasks-contributing)
- [x] I agree to follow this project's [Code of Conduct](https://github.com/obsidian-tasks-group/obsidian-tasks/blob/main/CODE_OF_CONDUCT.md)


Patch:
diff --git a/src/Query/Filter/TextField.ts b/src/Query/Filter/TextField.ts
index 80b17897..07203a3c 100644
--- a/src/Query/Filter/TextField.ts
+++ b/src/Query/Filter/TextField.ts
@@ -2,7 +2,6 @@ import type { Task } from '../../Task';
 import { SubstringMatcher } from '../Matchers/SubstringMatcher';
 import { RegexMatcher } from '../Matchers/RegexMatcher';
 import type { IStringMatcher } from '../Matchers/IStringMatcher';
-import { Explanation } from '../Explain/Explanation';
 import type { Comparator } from '../Sorter';
 import type { GrouperFunction } from '../Grouper';
 import { Field } from './Field';
@@ -50,7 +49,7 @@ export abstract class TextField extends Field {
         // and tests if it matches the string filtering rule
         // represented by this object.
         const negate = filterOperator.match(/not/) !== null;
-        const filter = new Filter(line, this.getFilter(matcher, negate), new Explanation(line));
+        const filter = new Filter(line, this.getFilter(matcher, negate), matcher.explanation(line));
         return FilterOrErrorMessage.fromFilter(filter);
     }
 
diff --git a/src/Query/Matchers/IStringMatcher.ts b/src/Query/Matchers/IStringMatcher.ts
index b837af27..26d465de 100644
--- a/src/Query/Matchers/IStringMatcher.ts
+++ b/src/Query/Matchers/IStringMatcher.ts
@@ -1,3 +1,5 @@
+import type { Explanation } from '../Explain/Explanation';
+
 /**
  * An interface for determining whether a string value matches a particular condition.
  *
@@ -18,4 +20,13 @@ export abstract class IStringMatcher {
     public matchesAnyOf(stringsToSearch: string[]) {
         return stringsToSearch.some((s) => this.matches(s));
     }
+
+    /**
+     * Return an {@link Explanation} object, with any extra detail
+     * about the behaviour of this matcher.
+     *
+     * If there is no extra detail, just pass {@link instruction} in to
+     * the {@link Explanation} constructor.
+     */
+    public abstract explanation(instruction: string): Explanation;
 }
diff --git a/src/Query/Matchers/RegexMatcher.ts b/src/Query/Matchers/RegexMatcher.ts
index 7a2e97e3..8ef99499 100644
--- a/src/Query/Matchers/RegexMatcher.ts
+++ b/src/Query/Matchers/RegexMatcher.ts
@@ -1,3 +1,4 @@
+import { Explanation } from '../Explain/Explanation';
 import { IStringMatcher } from './IStringMatcher';
 
 /**
@@ -41,4 +42,57 @@ export class RegexMatcher extends IStringMatcher {
     public matches(stringToSearch: string): boolean {
         return stringToSearch.match(this.regex) !== null;
     }
+
+    explanation(instruction: string): Explanation {
+        const intro = 'using regex: ';
+        const explanationText = alignRegexWithOriginalInstruction(instruction, intro, this.regexAsString());
+        return new Explanation(explanationText);
+    }
+
+    private regexAsString() {
+        let result = `'${this.regex.source}' with `;
+
+        switch (this.regex.flags.length) {
+            case 0:
+                result += 'no flags';
+                break;
+            case 1:
+                result += `flag '${this.regex.flags}'`;
+                break;
+            default:
+                result += `flags '${this.regex.flags}'`;
+                break;
+        }
+
+        return result;
+    }
+}
+
+/**
+ * Align the regex we are actually using with the regex in the original query.
+ * For example, this creates the second line in this instruction/explanation pair:
+ *
+ * ```text
+ * description regex matches /waiting|waits|wartet/ =>
+ *   using regex:            'waiting|waits|wartet' with no flags
+ * ```
+ *
+ * @param instruction
+ * @param intro
+ * @param regexAsString
+ */
+function alignRegexWithOriginalInstruction(instruction: string, intro: string, regexAsString: string) {
+    const match = instruction.match(/\//);
+    if (!match) {
+        return 'Error explaining instruction. Could not find a slash character';
+    }
+
+    // The explanation will be indented 2 characters from the parent instruction,
+    // so pad the explanation so that the start of the explained regex
+    // aligns with the start of the regex in the original instruction.
+    // This makes any differences in the regex much easier to spot.
+    const indentation = 2;
+    const startOfRegex = (match.index ?? indentation) - indentation;
+    const prefixPadded = intro.padEnd(startOfRegex);
+    return `${prefixPadded}${regexAsString}`;
 }
diff --git a/src/Query/Matchers/SubstringMatcher.ts b/src/Query/Matchers/SubstringMatcher.ts
index 6071b867..d75f83f4 100644
--- a/src/Query/Matchers/SubstringMatcher.ts
+++ b/src/Query/Matchers/SubstringMatcher.ts
@@ -1,3 +1,4 @@
+import { Explanation } from '../Explain/Explanation';
 import { IStringMatcher } from './IStringMatcher';
 
 /**
@@ -26,4 +27,10 @@ export class SubstringMatcher extends IStringMatcher {
     public static stringIncludesCaseInsensitive(haystack: string, needle: string): boolean {
         return haystack.toLocaleLowerCase().includes(needle.toLocaleLowerCase());
     }
+
+    explanation(instruction: string): Explanation {
+        // We don't currently have any specific explanation of substring-searching,
+        // so just return the original instruction line.
+        return new Explanation(instruction);
+    }
 }


Test patch:
diff --git a/tests/Query/Filter/TextField.test.explains_regular_expression_searches_bulk_test.approved.txt b/tests/Query/Filter/TextField.test.explains_regular_expression_searches_bulk_test.approved.txt
new file mode 100644
index 00000000..6fa8ccb5
--- /dev/null
+++ b/tests/Query/Filter/TextField.test.explains_regular_expression_searches_bulk_test.approved.txt
@@ -0,0 +1,109 @@
+description regex does not match /(buy|order|voucher|lakeland|purchase|\spresent)/i =>
+  using regex:                   '(buy|order|voucher|lakeland|purchase|\spresent)' with flag 'i'
+
+description regex does not match /^$/ =>
+  using regex:                   '^$' with no flags
+
+description regex matches /  / =>
+  using regex:            '  ' with no flags
+
+description regex matches /#context/pc_photos|#context/pc_clare|#context/pc_macbook/i =>
+  using regex:            '#context' with no flags
+
+description regex matches /#context\/pc_photos|#context\/pc_clare|#context\/pc_macbook/i =>
+  using regex:            '#context\/pc_photos|#context\/pc_clare|#context\/pc_macbook' with flag 'i'
+
+description regex matches /#tag\/subtag[0-9]\/subsubtag[0-9]/i =>
+  using regex:            '#tag\/subtag[0-9]\/subsubtag[0-9]' with flag 'i'
+
+description regex matches /(buy|order|voucher|lakeland|purchase|\spresent)/i =>
+  using regex:            '(buy|order|voucher|lakeland|purchase|\spresent)' with flag 'i'
+
+description regex matches /[⏫🔼🔽📅⏳🛫🔁]/ =>
+  using regex:            '[⏫🔼🔽📅⏳🛫🔁]' with no flags
+
+description regex matches /[⏫🔼🔽📅⏳🛫🔁]/u =>
+  using regex:            '[⏫🔼🔽📅⏳🛫🔁]' with no flags
+
+description regex matches /^$/ =>
+  using regex:            '^$' with no flags
+
+description regex matches /^Log/i =>
+  using regex:            '^Log' with flag 'i'
+
+description regex matches /waiting|waits|wartet/ =>
+  using regex:            'waiting|waits|wartet' with no flags
+
+description regex matches /waiting|waits|wartet/i =>
+  using regex:            'waiting|waits|wartet' with flag 'i'
+
+filename regex does not match /^Tasks User Support Kanban\.md$/ =>
+  using regex:                '^Tasks User Support Kanban\.md$' with no flags
+
+folder regex matches /root/sub-folder/sub-sub-folder/ =>
+  using regex:       'root' with no flags
+
+folder regex matches /root\/sub-folder\/sub-sub-folder/ =>
+  using regex:       'root\/sub-folder\/sub-sub-folder' with no flags
+
+heading regex matches /^Exactly Matched Heading$/ =>
+  using regex:        '^Exactly Matched Heading$' with no flags
+
+path regex does not match /^_meta/ =>
+  using regex:            '^_meta' with no flags
+
+path regex does not match /^_templates/ =>
+  using regex:            '^_templates' with no flags
+
+path regex does not match /sadfasfdafa/i =>
+  using regex:            'sadfasfdafa' with flag 'i'
+
+path regex does not match /sadfasfdafa/igm =>
+  using regex:            'sadfasfdafa' with flags 'gim'
+
+path regex does not match /w.bble/ =>
+  using regex:            'w.bble' with no flags
+
+path regex matches /(george phone|exercise|100th|knee|TRVs|HWRC|2023-03-21 Python Pairing on the Hub with so-and-so|Epson|Fred's flat|1519 - feat - Theme-ability|McDermid)/i =>
+  using regex:     '(george phone|exercise|100th|knee|TRVs|HWRC|2023-03-21 Python Pairing on the Hub with so-and-so|Epson|Fred's flat|1519 - feat - Theme-ability|McDermid)' with flag 'i'
+
+path regex matches /(george)/i =>
+  using regex:     '(george)' with flag 'i'
+
+path regex matches /Log/ =>
+  using regex:     'Log' with no flags
+
+path regex matches /clare/i =>
+  using regex:     'clare' with flag 'i'
+
+path regex matches /root/sub-folder/sub-sub-folder/index\.md/ =>
+  using regex:     'root' with no flags
+
+path regex matches /root\/sub-folder\/sub-sub-folder\/index\.md/ =>
+  using regex:     'root\/sub-folder\/sub-sub-folder\/index\.md' with no flags
+
+path regex matches /sadfasfdafa/i =>
+  using regex:     'sadfasfdafa' with flag 'i'
+
+recurrence regex does not match /\d/ =>
+  using regex:                  '\d' with no flags
+
+recurrence regex matches /\d/ =>
+  using regex:           '\d' with no flags
+
+tag regex does not match /./ =>
+  using regex:           '.' with no flags
+
+tag regex matches /#book$/ =>
+  using regex:    '#book$' with no flags
+
+tag regex matches /#t$/ =>
+  using regex:    '#t$' with no flags
+
+tags regex does not match /(home|town)/ =>
+  using regex:            '(home|town)' with no flags
+
+tags regex matches /(home|pc_mac|town)/ =>
+  using regex:     '(home|pc_mac|town)' with no flags
+
+Error: None
diff --git a/tests/Query/Filter/TextField.test.ts b/tests/Query/Filter/TextField.test.ts
new file mode 100644
index 00000000..f264f02c
--- /dev/null
+++ b/tests/Query/Filter/TextField.test.ts
@@ -0,0 +1,76 @@
+import { verify } from 'approvals/lib/Providers/Jest/JestApprovals';
+import { DescriptionField } from '../../../src/Query/Filter/DescriptionField';
+import { Query } from '../../../src/Query/Query';
+
+describe('explains regular sub-string searches', () => {
+    it('should explain simple string search', () => {
+        const instruction = 'description includes hello';
+        const field = new DescriptionField().createFilterOrErrorMessage(instruction);
+        expect(field).toHaveExplanation('description includes hello');
+    });
+});
+
+describe('explains regular expression searches', () => {
+    it('should explain regex matches search', () => {
+        const instruction = 'description regex matches /hello/';
+        const field = new DescriptionField().createFilterOrErrorMessage(instruction);
+        expect(field).toHaveExplanation("using regex:            'hello' with no flags");
+    });
+
+    it('should explain regex does not match search', () => {
+        const instruction = 'description regex does not match /hello/';
+        const field = new DescriptionField().createFilterOrErrorMessage(instruction);
+        expect(field).toHaveExplanation("using regex:                   'hello' with no flags");
+    });
+
+    it('bulk test', () => {
+        /* This test uses Approval Tests' verify() to quickly get good coverage on a wide variety
+           of regular expression examples.
+         */
+
+        // Arrange
+        // Alphabetical order please
+        const source = `
+            description regex does not match /(buy|order|voucher|lakeland|purchase|\\spresent)/i
+            description regex does not match /^$/
+            description regex matches /  /
+            description regex matches /#context/pc_photos|#context/pc_clare|#context/pc_macbook/i
+            description regex matches /#context\\/pc_photos|#context\\/pc_clare|#context\\/pc_macbook/i
+            description regex matches /#tag\\/subtag[0-9]\\/subsubtag[0-9]/i
+            description regex matches /(buy|order|voucher|lakeland|purchase|\\spresent)/i
+            description regex matches /[⏫🔼🔽📅⏳🛫🔁]/
+            description regex matches /[⏫🔼🔽📅⏳🛫🔁]/u
+            description regex matches /^$/
+            description regex matches /^Log/i
+            description regex matches /waiting|waits|wartet/
+            description regex matches /waiting|waits|wartet/i
+            filename regex does not match /^Tasks User Support Kanban\\.md$/
+            folder regex matches /root/sub-folder/sub-sub-folder/
+            folder regex matches /root\\/sub-folder\\/sub-sub-folder/
+            heading regex matches /^Exactly Matched Heading$/
+            path regex does not match /^_meta/
+            path regex does not match /^_templates/
+            path regex does not match /sadfasfdafa/i
+            path regex does not match /sadfasfdafa/igm
+            path regex does not match /w.bble/
+            path regex matches /(george phone|exercise|100th|knee|TRVs|HWRC|2023-03-21 Python Pairing on the Hub with so-and-so|Epson|Fred's flat|1519 - feat - Theme-ability|McDermid)/i
+            path regex matches /(george)/i
+            path regex matches /Log/
+            path regex matches /clare/i
+            path regex matches /root/sub-folder/sub-sub-folder/index\\.md/
+            path regex matches /root\\/sub-folder\\/sub-sub-folder\\/index\\.md/
+            path regex matches /sadfasfdafa/i
+            recurrence regex does not match /\\d/
+            recurrence regex matches /\\d/
+            tag regex does not match /./
+            tag regex matches /#book$/
+            tag regex matches /#t$/
+            tags regex does not match /(home|town)/
+            tags regex matches /(home|pc_mac|town)/
+`;
+        const query = new Query({ source });
+
+        // Assert
+        verify(query.explainQuery() + `\nError: ${query.error ?? 'None'}`);
+    });
+});