Repository: obsidian-tasks-group/obsidian-tasks (ts)

Issue:
Allow long instruction lines to be split over several lines - perhaps a continuation character?
### ⚠️ Please check that this feature request hasn't been suggested before.

- [X] I searched previous [Ideas in Discussions](https://github.com/obsidian-tasks-group/obsidian-tasks/discussions/categories/ideas?discussions_q=category%3AIdeas+-label%3A%22status%3A+released%22+-label%3Aduplicate+sort%3Atop) didn't find any similar feature requests.
- [X] I searched previous [Issues](https://github.com/obsidian-tasks-group/obsidian-tasks/issues?q=is%3Aopen+is%3Aissue+label%3A%22type%3A+enhancement%22+sort%3Acomments-desc) didn't find any similar feature requests.

### 🔖 Feature description

With several filters combined by boolean operators, Tasks lines can get rather long.

A fictional example:

```text
(description includes 1) AND (description includes 2) AND (description includes 3) AND (description includes 4) AND (description includes 5) AND (description includes 6) AND (description includes 7)
```

As well as being hard to read, there are patterns in the filters, but they cannot be seen because of all being on one line. If there were an inconsistency in the filter, it would be hard to spot.


### ✔️ Solution

Add support for a 'continuation character' - a character that allows splitting of a long line, such as `\`


The above could then be written like this:

```text
(description includes 1) \
AND (description includes 2) \
AND (description includes 3) \
AND (description includes 4) \
AND (description includes 5) \
AND (description includes 6) \
AND (description includes 7)
```

Or:

```text
    (description includes 1) \
AND (description includes 2) \
AND (description includes 3) \
AND (description includes 4) \
AND (description includes 5) \
AND (description includes 6) \
AND (description includes 7)
```

I would restrict it to only being allowed at the joins in boolean filters, so there is no ambiguity as to whether a `\` character is part of the string being searched for.


### ❓ Alternatives

Can't think of any.

### 📝 Additional Context

_No response_

Pull request description:
feat: Long query lines can now be split with backslashes
<!--- Provide a general summary of your changes in the Title above -->

# Description

This implements line continuation using a trailing backslash, as discussed in #1351. 

    ```tasks
    (priority is highest) OR \
    (priority is lowest)
    ```

It is implemented as a global replace prior to line splitting, so this will work in any filter type. It does not trim whitespace around the newline; this is fine for the common case where the continuation occurs at a word boundary anyways, and could be useful in some edge cases where precise control over the whitespace is required.

The changes are implemented in a new `scan` function, which is responsible for splitting the input string into individual statements for further parsing. This leaves open the possibility for future enhancements to the parsing logic (for instance, if a more powerful parser gets implemented to support automatic line continuations after open parentheses).

## Motivation and Context

<!--- Why is this change required? What problem does it solve? -->
<!--- If it fixes an open issue, please link to the issue here. -->

Fixes #1351 (and duplicate #2326).

## How has this been tested?

<!--- Please describe in detail how you tested your changes. -->
<!--- Include details of your testing environment, tests ran to see how -->
<!--- your change affects other areas of the code, etc. -->

Includes 100% coverage of the new `Scanner.ts`. I have also tested in the sample Obsidian vault.

I added a documentation page called 'Line Continuation'. Let me know if this should be renamed or merged into another location.

## Screenshots (if appropriate)

## Types of changes

<!--- What types of changes does your code introduce? Put an `x` in all the boxes that apply: -->

Changes visible to users:

- [ ] **Bug fix** (prefix: `fix` - non-breaking change which fixes an issue)
- [x] **New feature** (prefix: `feat` - non-breaking change which adds functionality)
- [ ] **Breaking change** (prefix: `feat!!` or `fix!!` - fix or feature that would cause existing functionality to not work as expected)
- [x] **Documentation** (prefix: `docs` - improvements to any documentation content **for users**)
- [ ] **Sample vault** (prefix: `vault` - improvements to the [Tasks-Demo sample vault](https://github.com/obsidian-tasks-group/obsidian-tasks/tree/main/resources/sample_vaults/Tasks-Demo))
- [ ] **Contributing Guidelines** (prefix: `contrib` - any improvements to documentation content **for contributors** - see [Contributing to Tasks](https://publish.obsidian.md/tasks-contributing/))

Technically breaking if the users ended a filter with a backslash, but that's pretty unlikely.

Internal changes:

- [ ] **Refactor** (prefix: `refactor` - non-breaking change which only improves the design or structure of existing code, and making no changes to its external behaviour)
- [X] **Tests** (prefix: `test` - additions and improvements to unit tests and the smoke tests)
- [ ] **Infrastructure** (prefix: `chore` - examples include GitHub Actions, issue templates)

## Checklist

<!--- Go over all the following points, and put an `x` in all the boxes that apply. -->
<!--- If you're unsure about any of these, don't hesitate to ask. We're here to help! -->

- [X] My code follows the code style of this project and passes `yarn run lint`.
- [X] My change requires a change to the documentation.
- [X] I have [updated the documentation](https://publish.obsidian.md/tasks-contributing/Documentation/About+Documentation) accordingly.
- [X] My change has adequate [Unit Test coverage](https://publish.obsidian.md/tasks-contributing/Testing/About+Testing).

## Terms

<!--
By submitting this pull request, you must agree to follow our
[contributing guide](https://publish.obsidian.md/tasks-contributing) and
[Code of Conduct](https://github.com/obsidian-tasks-group/obsidian-tasks/blob/main/CODE_OF_CONDUCT.md).
Put an x in the boxes to confirm you agree.
-->

- [X] My contribution follow this project's [contributing guide](https://publish.obsidian.md/tasks-contributing)
- [X] I agree to follow this project's [Code of Conduct](https://github.com/obsidian-tasks-group/obsidian-tasks/blob/main/CODE_OF_CONDUCT.md)


Patch:
diff --git a/docs/Queries/About Queries.md b/docs/Queries/About Queries.md
index 93a7f428..136c547a 100644
--- a/docs/Queries/About Queries.md
+++ b/docs/Queries/About Queries.md
@@ -37,6 +37,7 @@ Therefore, Tasks allows you to set query options to filter the tasks that you wa
 - [[Global Query]]
 - [[Combining Filters]]
 - [[Regular Expressions]]
+- [[Line Continuations]]
 
 ### Viewing the results
 
diff --git a/docs/Queries/Combining Filters.md b/docs/Queries/Combining Filters.md
index d924362c..834d02c0 100644
--- a/docs/Queries/Combining Filters.md
+++ b/docs/Queries/Combining Filters.md
@@ -36,6 +36,7 @@ The following rules apply:
 - The operators are case-sensitive: they must be capitalised.
 - The operators must be surrounded by spaces.
 - Use more `(` and `)` to nest further filters together.
+- A trailing backslash may be used to break a long filter over several lines, as described in [[Line Continuations]].
 - There is no practical limit to the number of filters combined on each line, nor the level of nesting of parentheses.
 
 Recommendations:
@@ -226,7 +227,9 @@ I want to find tasks that are waiting for something else. But 'waiting' can be s
 ````text
 ```tasks
 not done
-(description includes waiting) OR (description includes waits) OR (description includes wartet)
+(description includes waiting) OR \
+  (description includes waits) OR \
+  (description includes wartet)
 ```
 ````
 
@@ -237,7 +240,10 @@ I want to see tasks from anywhere in my vault with the tag `#DailyNote` or tasks
 ````text
 ```tasks
 not done
-(tags include #DailyNote) OR ((path includes daily/Notes/Folder/) AND (path does not include 2022-07-11))
+(tags include #DailyNote) OR \
+( (path includes daily/Notes/Folder/) AND \
+  (path does not include 2022-07-11) \
+)
 ```
 ````
 
@@ -257,7 +263,9 @@ You could select any of the nearby locations with:
 
 ````text
 # Show all tasks I CAN do in this area:
-(tags include #context/loc1) OR (tags include #context/loc2) OR (tags include #context/loc3)
+(tags include #context/loc1) OR \
+  (tags include #context/loc2) OR \
+  (tags include #context/loc3)
 ````
 
 #### In None of Several Locations
@@ -268,7 +276,11 @@ An easy way to review all the _other_ tasks not possible in this area would be t
 
 ````text
 # Show all tasks I CANNOT do in this area - EASY WAY:
-NOT ( (tags include #context/loc1) OR (tags include #context/loc2) OR (tags include #context/loc3) )
+NOT ( \
+  (tags include #context/loc1) OR \
+  (tags include #context/loc2) OR \
+  (tags include #context/loc3) \
+)
 ````
 
 The nice thing about the above `NOT` use is that if a new context gets added to the group in the future, it can be added to both task blocks via a simple find-and-replace.
@@ -280,5 +292,7 @@ The above is much easier to maintain than the other option of:
 
 ````text
 # Show all tasks I CANNOT do in this area - HARDER WAY
-(tags do not include #context/loc1) AND (tags do not include #context/loc2) AND (tags do not include #context/loc3)
+(tags do not include #context/loc1) AND \
+  (tags do not include #context/loc2) AND \
+  (tags do not include #context/loc3)
 ````
diff --git a/docs/Queries/Comments.md b/docs/Queries/Comments.md
index b44e9b3f..61c44aaa 100644
--- a/docs/Queries/Comments.md
+++ b/docs/Queries/Comments.md
@@ -14,3 +14,19 @@ Example:
     # Uncomment the following line to enable short mode:
     # short mode
     ```
+
+## Inline comments
+
+> [!released]
+> Inline comments were introduced in Tasks 4.7.0.
+
+[Mustache.js](https://www.npmjs.com/package/mustache) comments can also be used within a
+line:
+
+    ```tasks
+    not done
+    short mode {{! This comment will be ignored }}
+    ```
+
+Any text between the `{{!` and `}}` of a line will be ignored. Multiline comments are
+not supported (except perhaps in combination with [line continuations](Line Continuations)).
diff --git a/docs/Queries/Line Continuations.md b/docs/Queries/Line Continuations.md
new file mode 100644
index 00000000..4edd818f
--- /dev/null
+++ b/docs/Queries/Line Continuations.md
@@ -0,0 +1,28 @@
+---
+publish: true
+---
+
+# Line Continuations
+> [!released]
+> Introduced in Tasks X.Y.Z.
+
+Long queries can be organized across multiple lines using a trailing backslash:
+
+    ```tasks
+    (priority is highest) OR \
+        (priority is lowest)
+    ```
+
+This can be helpful for long [[Combining Filters|combined filters]],  [[Custom
+Filters|custom filters]], and other queries that may be difficult to read on one line.
+
+Line continuations are treated as whitespace, and any extra indentation or whitespace around the backslash is compressed into a single space.
+
+In the rare case that a trailing backslash is needed for a query, the trailing backslash may be escaped by adding a space at the end of the line, or by adding an extra backslash and a blank line:
+
+    ```tasks
+    # Searches for a single backslash
+    description includes \\
+    
+    explain
+    ```
diff --git a/src/Query/Query.ts b/src/Query/Query.ts
index 0b2db1d9..30f0a117 100644
--- a/src/Query/Query.ts
+++ b/src/Query/Query.ts
@@ -12,6 +12,7 @@ import * as FilterParser from './FilterParser';
 import type { Grouper } from './Grouper';
 import type { Filter } from './Filter/Filter';
 import { QueryResult } from './QueryResult';
+import { scan } from './Scanner';
 
 export class Query implements IQuery {
     /** Note: source is the raw source, before expanding any placeholders */
@@ -41,47 +42,42 @@ export class Query implements IQuery {
         this.source = source;
         this.filePath = path;
 
-        source
-            .split('\n')
-            .map((rawLine: string) => rawLine.trim())
-            .forEach((rawLine: string) => {
-                const line = this.expandPlaceholders(rawLine, path);
-                if (this.error !== undefined) {
-                    // There was an error expanding placeholders.
-                    return;
-                }
+        scan(source).forEach((rawLine: string) => {
+            const line = this.expandPlaceholders(rawLine, path);
+            if (this.error !== undefined) {
+                // There was an error expanding placeholders.
+                return;
+            }
 
-                switch (true) {
-                    case line === '':
-                        break;
-                    case this.shortModeRegexp.test(line):
-                        this._layoutOptions.shortMode = true;
-                        break;
-                    case this.explainQueryRegexp.test(line):
-                        this._layoutOptions.explainQuery = true;
-                        break;
-                    case this.ignoreGlobalQueryRegexp.test(line):
-                        this._ignoreGlobalQuery = true;
-                        break;
-                    case this.limitRegexp.test(line):
-                        this.parseLimit(line);
-                        break;
-                    case this.parseSortBy(line):
-                        break;
-                    case this.parseGroupBy(line):
-                        break;
-                    case this.hideOptionsRegexp.test(line):
-                        this.parseHideOptions(line);
-                        break;
-                    case this.commentRegexp.test(line):
-                        // Comment lines are ignored
-                        break;
-                    case this.parseFilter(line):
-                        break;
-                    default:
-                        this.setError('do not understand query', line);
-                }
-            });
+            switch (true) {
+                case this.shortModeRegexp.test(line):
+                    this._layoutOptions.shortMode = true;
+                    break;
+                case this.explainQueryRegexp.test(line):
+                    this._layoutOptions.explainQuery = true;
+                    break;
+                case this.ignoreGlobalQueryRegexp.test(line):
+                    this._ignoreGlobalQuery = true;
+                    break;
+                case this.limitRegexp.test(line):
+                    this.parseLimit(line);
+                    break;
+                case this.parseSortBy(line):
+                    break;
+                case this.parseGroupBy(line):
+                    break;
+                case this.hideOptionsRegexp.test(line):
+                    this.parseHideOptions(line);
+                    break;
+                case this.commentRegexp.test(line):
+                    // Comment lines are ignored
+                    break;
+                case this.parseFilter(line):
+                    break;
+                default:
+                    this.setError('do not understand query', line);
+            }
+        });
     }
 
     private expandPlaceholders(source: string, path: string | undefined) {
diff --git a/src/Query/Scanner.ts b/src/Query/Scanner.ts
new file mode 100644
index 00000000..18ee520e
--- /dev/null
+++ b/src/Query/Scanner.ts
@@ -0,0 +1,25 @@
+/**
+ * Removes newlines escaped by a backslash.
+ * A trailing backslash at the end of a line can be escaped by doubling it.
+ *
+ * @param input input string
+ * @returns modified input
+ */
+export function continue_lines(input: string): string {
+    return input.replace(/[ \t]*\\\n[ \t]*/g, ' ');
+}
+/**
+ * Take an input string and split it into a list of statements.
+ *
+ * Generally this is similar to splitting the string into lines, but handles line
+ * continuations and escape sequences.
+ *
+ * @param input Input string
+ * @returns List of statements
+ */
+export function scan(input: string): string[] {
+    return continue_lines(input)
+        .split('\n')
+        .map((rawLine: string) => rawLine.trim())
+        .filter((line) => line !== '');
+}


Test patch:
diff --git a/tests/Query.test.ts b/tests/Query.test.ts
index 9d5698a3..1a9454fb 100644
--- a/tests/Query.test.ts
+++ b/tests/Query.test.ts
@@ -420,6 +420,9 @@ describe('Query parsing', () => {
             '(description includes wibble) OR (has due date)',
             '(has due date) OR ((has start date) AND (due after 2021-12-27))',
             '(is not recurring) XOR ((path includes ab/c) OR (happens before 2021-12-27))',
+            String.raw`(description includes line 1) OR \
+(description includes line 1 continued\
+ with \ backslash)`,
         ];
         test.concurrent.each<string>(filters)('recognises %j', (filter) => {
             // Arrange
@@ -1295,4 +1298,33 @@ At most 8 tasks per group (if any "group by" options are supplied).
             );
         });
     });
+
+    describe('line continuations', () => {
+        it('should work in group by functions', () => {
+            const source = String.raw`group by function \
+                const date = task.due.moment; \
+                const now = moment(); \
+                const label = (order, name) => '%%'+order+'%% =='+name+'=='; \
+                if (!date) return label(4, 'Undated'); \
+                if (date.isBefore(now, 'day')) return label(1, 'Overdue'); \
+                if (date.isSame(now, 'day')) return label(2, 'Today'); \
+                return label(3, 'Future');`;
+            const query = new Query(source);
+            expect(query.error).toBeUndefined();
+        });
+        it('should be explained correctly in boolean queries', () => {
+            const source = String.raw`explain
+(description includes line 1) OR        \
+  (description includes line 1 continued\
+with \ backslash)`;
+            const query = new Query(source);
+
+            const expectedDisplayText = String.raw`(description includes line 1) OR (description includes line 1 continued with \ backslash) =>
+  OR (At least one of):
+    description includes line 1
+    description includes line 1 continued with \ backslash
+`;
+            expect(query.explainQuery()).toEqual(expectedDisplayText);
+        });
+    });
 });
diff --git a/tests/Scanner.test.ts b/tests/Scanner.test.ts
new file mode 100644
index 00000000..fac9e793
--- /dev/null
+++ b/tests/Scanner.test.ts
@@ -0,0 +1,83 @@
+/* Note: some tests are sensitive to whitespace changes. Ensure your editor preserves
+ * trailing whitespace.
+ */
+import { continue_lines, scan } from '../src/Query/Scanner';
+
+describe('continue_lines', () => {
+    it('keeps non-continued text the same', () => {
+        const text = String.raw`not done
+due this week`;
+        expect(continue_lines(text)).toEqual(text);
+    });
+    it('removes backslashed newlines', () => {
+        const text = String.raw`line1 \
+continued`;
+        expect(continue_lines(text)).toEqual('line1 continued');
+    });
+    it('only consumes one backslash', () => {
+        const text = String.raw`line1 \\
+
+line2`;
+        expect(continue_lines(text)).toEqual(String.raw`line1 \ 
+line2`);
+    });
+    it('preserves non-final backslashes', () => {
+        const text = String.raw`line\1 \
+continued \\\
+
+line2`;
+        expect(continue_lines(text)).toEqual(String.raw`line\1 continued \\ 
+line2`);
+    });
+    it('ignores interleaved continuations', () => {
+        const text = String.raw`line1\\
+
+line2`;
+        expect(continue_lines(text)).toEqual(String.raw`line1\ 
+line2`);
+    });
+    it('compresses surrounding spaces', () => {
+        const text = String.raw`line1    \
+            continued`;
+        expect(continue_lines(text)).toEqual(String.raw`line1 continued`);
+    });
+    it('compresses surrounding tabs', () => {
+        const text = `line1\t\\
+\t\tcontinued\\
+
+line2`;
+        expect(continue_lines(text)).toEqual(String.raw`line1 continued 
+line2`);
+    });
+});
+
+describe('scan', () => {
+    it('works on an easy case', () => {
+        const text = String.raw`not done
+due this week`;
+        expect(scan(text)).toEqual(['not done', 'due this week']);
+    });
+    it('strips whitespace', () => {
+        const text = String.raw` not done   
+        due this week
+
+        
+        `;
+        expect(scan(text)).toEqual(['not done', 'due this week']);
+    });
+    it('supports line continuation', () => {
+        const text = String.raw`( property1 ) AND \
+ (property2)`;
+        expect(scan(text)).toEqual(['( property1 ) AND (property2)']);
+    });
+    it('drops empty lines', () => {
+        const text = String.raw`line1    
+ line2 
+
+ 
+  line3
+  
+  `;
+        expect(scan(text)).toEqual(['line1', 'line2', 'line3']);
+    });
+});