Repository: obsidian-tasks-group/obsidian-tasks (ts)

Issue:
urgency value not changing
### Please check that this issue hasn't been reported before.

- [X] I searched previous [Bug Reports](https://github.com/obsidian-tasks-group/obsidian-tasks/issues?q=is%3Aopen+is%3Aissue+label%3A%22type%3A+bug%22+sort%3Acomments-desc) didn't find any similar reports.

### Expected Behavior

Change due dates, priority, etc. leads to different urgency values and has influence on order of tasks in query

### Current behaviour

No matter how I change a task property, the urgency value remains 1.95 (for all my tasks).
I tried an example task as described in [Tasks Documentation](https://obsidian-tasks-group.github.io/obsidian-tasks/advanced/urgency/)


### Steps to reproduce

1. Create Task with due date, priority, etc.
2. Query tasks with
```tasks
not done
show urgency
```



### Which Operating Systems are you using?

- [ ] Android
- [ ] iPhone/iPad
- [ ] Linux
- [ ] macOS
- [ ] Windows

### Obsidian Version

1.1.16

### Tasks Plugin Version

3.2.0

### Checks

- [X] I have tried it with all other plugins disabled and the error still occurs

### Possible solution

_No response_

Pull request description:
fix: Enable queries like 'due last Wednesday' to search dates before today
<!--- Provide a general summary of your changes in the Title above -->

# Description

1. Fix queries of the form `due last wednesday` so that they actually search for the previous Wednesday instead of a future one.
2. Fix queries of the for `due in two weeks` which were broken by the above fix.


Co-Authored-By: Ilyas Landikov <93825870+ilandikov@users.noreply.github.com>

## Motivation and Context

Fixes #1892.

## How has this been tested?

By:

- adding a lot more queries to the tests, including all the examples in the date filtering docs section
- the running them in Tasks 1.25.0 to get their explanations
- and then fixing the current code so that it gives the same behaviour as the 1.25.0 code.

The outputs of these tests will eventually, I hope, go in the the Reference section of the documentation.

## Screenshots (if appropriate)

## Types of changes

<!--- What types of changes does your code introduce? Put an `x` in all the boxes that apply: -->

Changes visible to users:

- [x] **Bug fix** (prefix: `fix` - non-breaking change which fixes an issue)

Internal changes:

- [x] **Tests** (prefix: `test` - additions and improvements to unit tests and the smoke tests)

## Checklist

<!--- Go over all the following points, and put an `x` in all the boxes that apply. -->
<!--- If you're unsure about any of these, don't hesitate to ask. We're here to help! -->

- [x] My code follows the code style of this project and passes `yarn run lint`.
- [ ] My change requires a change to the documentation.
- [ ] I have [updated the documentation](https://publish.obsidian.md/tasks-contributing/Documentation/About+Documentation) accordingly.
- [x] My change has adequate [Unit Test coverage](https://publish.obsidian.md/tasks-contributing/Testing/About+Testing).

## Terms

<!--
By submitting this pull request, you must agree to follow our
[contributing guide](https://publish.obsidian.md/tasks-contributing) and
[Code of Conduct](https://github.com/obsidian-tasks-group/obsidian-tasks/blob/main/CODE_OF_CONDUCT.md).
Put an x in the boxes to confirm you agree.
-->

- [x] My contribution follow this project's [contributing guide](https://publish.obsidian.md/tasks-contributing)
- [x] I agree to follow this project's [Code of Conduct](https://github.com/obsidian-tasks-group/obsidian-tasks/blob/main/CODE_OF_CONDUCT.md)


Patch:
diff --git a/src/Query/DateParser.ts b/src/Query/DateParser.ts
index 56c18780..cb7c36e0 100644
--- a/src/Query/DateParser.ts
+++ b/src/Query/DateParser.ts
@@ -16,9 +16,10 @@ export class DateParser {
      * Parse a line and extract a pair of dates, returned in a tuple, sorted by date.
      * @param input - any pair of dates, separate by one or more spaces '17 August 2013 19 August 2013',
      *                or a single date.
-     * @return - A Tuple of dates. If both input dates are invalid, then both ouput dates will be invalid.
+     * @param forwardDate - if true, and date is ambiguous, chrono will return dates in the future
+     * @return - A Tuple of dates. If both input dates are invalid, then both output dates will be invalid.
      */
-    public static parseDateRange(input: string): DateRange {
+    public static parseDateRange(input: string, forwardDate: boolean = false): DateRange {
         const dateRangeParsers = [
             // Try parsing a relative date range like 'current month'
             DateParser.parseRelativeDateRange,
@@ -29,7 +30,7 @@ export class DateParser {
         ];
 
         for (const parser of dateRangeParsers) {
-            const parsedDateRange = parser(input);
+            const parsedDateRange = parser(input, forwardDate);
             if (parsedDateRange.isValid()) {
                 return parsedDateRange;
             }
@@ -39,9 +40,9 @@ export class DateParser {
         return DateRange.buildInvalid();
     }
 
-    private static parseAbsoluteDateRange(input: string): DateRange {
+    private static parseAbsoluteDateRange(input: string, forwardDate: boolean): DateRange {
         const result = chrono.parse(input, undefined, {
-            forwardDate: true,
+            forwardDate: forwardDate,
         });
 
         // Check chrono parsing
@@ -57,7 +58,7 @@ export class DateParser {
         return new DateRange(start, end);
     }
 
-    private static parseRelativeDateRange(input: string): DateRange {
+    private static parseRelativeDateRange(input: string, _forwardDate: boolean): DateRange {
         const relativeDateRangeRegexp = /(last|this|next) (week|month|quarter|year)/;
         const relativeDateRangeMatch = input.match(relativeDateRangeRegexp);
         if (relativeDateRangeMatch && relativeDateRangeMatch.length === 3) {
@@ -81,7 +82,7 @@ export class DateParser {
         return DateRange.buildInvalid();
     }
 
-    private static parseNumberedDateRange(input: string): DateRange {
+    private static parseNumberedDateRange(input: string, _forwardDate: boolean): DateRange {
         const parsingVectors: [RegExp, string, moment.unitOfTime.StartOf][] = [
             [/^\s*[0-9]{4}\s*$/, 'YYYY', 'year'],
             [/^\s*[0-9]{4}-Q[1-4]\s*$/, 'YYYY-Q', 'quarter'],
diff --git a/src/Query/Filter/DateField.ts b/src/Query/Filter/DateField.ts
index 12ce412a..76edb35e 100644
--- a/src/Query/Filter/DateField.ts
+++ b/src/Query/Filter/DateField.ts
@@ -1,5 +1,5 @@
 import type { Moment } from 'moment';
-import type { DateRange } from '../DateRange';
+import { DateRange } from '../DateRange';
 import type { Task } from '../../Task';
 import { DateParser } from '../DateParser';
 import { Explanation } from '../Explain/Explanation';
@@ -53,9 +53,24 @@ export abstract class DateField extends Field {
 
         const fieldNameKeywordDate = Field.getMatch(this.filterRegExp(), line);
         if (fieldNameKeywordDate !== null) {
-            const fieldKeyword = fieldNameKeywordDate[1];
-            const fieldDateString = fieldNameKeywordDate[2];
-            const fieldDates = DateParser.parseDateRange(fieldDateString);
+            const keywordAndDateString = fieldNameKeywordDate[1]; // Will contain the whole line except the field name
+            const fieldKeyword = fieldNameKeywordDate[2]; // Will be 'before', 'after', 'in', 'on' or undefined
+            const fieldDateString = fieldNameKeywordDate[3]; // Will contain the remainder of the instruction
+
+            // Try interpreting everything after the keyword as a date range:
+            let fieldDates = DateParser.parseDateRange(fieldDateString);
+
+            // If the date range parsing failed, try again to parse the whole line except the field name
+            // as a single date, using the pre-date-ranges parsing mechanism.
+            // This is needed to keep 'due in two weeks' working, as 'two weeks' is not actually a valid date range
+            // if the futureDates value passed in to chrono's parsing functions is false.
+            if (!fieldDates.isValid()) {
+                const date = DateParser.parseDate(keywordAndDateString);
+                if (date.isValid()) {
+                    fieldDates = new DateRange(date, date);
+                }
+            }
+
             if (!fieldDates.isValid()) {
                 result.error = 'do not understand ' + this.fieldName() + ' date';
             } else {
@@ -103,7 +118,7 @@ export abstract class DateField extends Field {
     }
 
     protected filterRegExp(): RegExp {
-        return new RegExp(`^${this.fieldNameForFilterInstruction()} (before|after|on|in)? ?(.*)`);
+        return new RegExp(`^${this.fieldNameForFilterInstruction()} ((before|after|on|in)? ?(.*))`);
     }
 
     /**


Test patch:
diff --git a/tests/Query/Explain/DocsSamplesForExplain.test.ts b/tests/Query/Explain/DocsSamplesForExplain.test.ts
index b6d4edb7..ce3ad6ad 100644
--- a/tests/Query/Explain/DocsSamplesForExplain.test.ts
+++ b/tests/Query/Explain/DocsSamplesForExplain.test.ts
@@ -1,44 +1,11 @@
 /**
  * @jest-environment jsdom
  */
-import { verify } from 'approvals/lib/Providers/Jest/JestApprovals';
 import moment from 'moment';
-import { Options } from 'approvals/lib/Core/Options';
-import { Query } from '../../../src/Query/Query';
+import { verifyExplanation, verifyQuery } from '../../TestingTools/ApprovalTestHelpers';
 
 window.moment = moment;
 
-/**
- * Save an instructions block to disc, so that it can be embedded in
- * to documentation, using a 'snippet' line.
- * @todo Figure out how to include the '```tasks' and '```' lines:
- *       see discussion in https://github.com/SimonCropp/MarkdownSnippets/issues/537
- * @param instructions
- * @param options
- */
-function verifyQuery(instructions: string, options?: Options): void {
-    options = options || new Options();
-    options = options.forFile().withFileExtention('query.text');
-    verify(instructions, options);
-}
-
-/**
- * Save an explanation of the instructions block to disk, so that it can be
- * embedded in to documentation, using a 'snippet' line.
- * @param instructions
- * @param options
- */
-function verifyExplanation(instructions: string, options?: Options): void {
-    const query = new Query({ source: instructions });
-    const explanation = query.explainQuery();
-
-    expect(query.error).toBeUndefined();
-
-    options = options || new Options();
-    options = options.forFile().withFileExtention('explanation.text');
-    verify(explanation, options);
-}
-
 describe('explain', () => {
     beforeAll(() => {
         jest.useFakeTimers();
diff --git a/tests/Query/Filter/ReferenceDocs/FilterReference/DateFieldReference.test.explain_date_reference_filters-date-examples.approved.explanation.text b/tests/Query/Filter/ReferenceDocs/FilterReference/DateFieldReference.test.explain_date_reference_filters-date-examples.approved.explanation.text
new file mode 100644
index 00000000..b821ace1
--- /dev/null
+++ b/tests/Query/Filter/ReferenceDocs/FilterReference/DateFieldReference.test.explain_date_reference_filters-date-examples.approved.explanation.text
@@ -0,0 +1,35 @@
+Explanation of this Tasks code block query:
+
+description includes ------------------------------
+
+due today =>
+  due date is on 2023-04-19 (Wednesday 19th April 2023)
+
+description includes ------------------------------
+
+due yesterday =>
+  due date is on 2023-04-18 (Tuesday 18th April 2023)
+
+due today =>
+  due date is on 2023-04-19 (Wednesday 19th April 2023)
+
+due tomorrow =>
+  due date is on 2023-04-20 (Thursday 20th April 2023)
+
+due next monday =>
+  due date is on 2023-04-24 (Monday 24th April 2023)
+
+due last friday =>
+  due date is on 2023-04-14 (Friday 14th April 2023)
+
+due 14 days ago =>
+  due date is on 2023-04-05 (Wednesday 5th April 2023)
+
+due in two weeks =>
+  due date is on 2023-05-03 (Wednesday 3rd May 2023)
+
+due 14 October =>
+  due date is on 2023-10-14 (Saturday 14th October 2023)
+
+due May =>
+  due date is on 2023-05-01 (Monday 1st May 2023)
diff --git a/tests/Query/Filter/ReferenceDocs/FilterReference/DateFieldReference.test.explain_date_reference_last-this-next-week-month-quarter-year.approved.explanation.text b/tests/Query/Filter/ReferenceDocs/FilterReference/DateFieldReference.test.explain_date_reference_last-this-next-week-month-quarter-year.approved.explanation.text
new file mode 100644
index 00000000..b02c3bec
--- /dev/null
+++ b/tests/Query/Filter/ReferenceDocs/FilterReference/DateFieldReference.test.explain_date_reference_last-this-next-week-month-quarter-year.approved.explanation.text
@@ -0,0 +1,76 @@
+Explanation of this Tasks code block query:
+
+description includes ------------------------------
+
+due today =>
+  due date is on 2023-04-19 (Wednesday 19th April 2023)
+
+description includes ------------------------------
+
+due last week =>
+  due date is between:
+    2023-04-10 (Monday 10th April 2023) and
+    2023-04-16 (Sunday 16th April 2023) inclusive
+
+due this week =>
+  due date is between:
+    2023-04-17 (Monday 17th April 2023) and
+    2023-04-23 (Sunday 23rd April 2023) inclusive
+
+due next week =>
+  due date is between:
+    2023-04-24 (Monday 24th April 2023) and
+    2023-04-30 (Sunday 30th April 2023) inclusive
+
+description includes ------------------------------
+
+due last month =>
+  due date is between:
+    2023-03-01 (Wednesday 1st March 2023) and
+    2023-03-31 (Friday 31st March 2023) inclusive
+
+due this month =>
+  due date is between:
+    2023-04-01 (Saturday 1st April 2023) and
+    2023-04-30 (Sunday 30th April 2023) inclusive
+
+due next month =>
+  due date is between:
+    2023-05-01 (Monday 1st May 2023) and
+    2023-05-31 (Wednesday 31st May 2023) inclusive
+
+description includes ------------------------------
+
+due last quarter =>
+  due date is between:
+    2023-01-01 (Sunday 1st January 2023) and
+    2023-03-31 (Friday 31st March 2023) inclusive
+
+due this quarter =>
+  due date is between:
+    2023-04-01 (Saturday 1st April 2023) and
+    2023-06-30 (Friday 30th June 2023) inclusive
+
+due next quarter =>
+  due date is between:
+    2023-07-01 (Saturday 1st July 2023) and
+    2023-09-30 (Saturday 30th September 2023) inclusive
+
+description includes ------------------------------
+
+due last year =>
+  due date is between:
+    2022-01-01 (Saturday 1st January 2022) and
+    2022-12-31 (Saturday 31st December 2022) inclusive
+
+due this year =>
+  due date is between:
+    2023-01-01 (Sunday 1st January 2023) and
+    2023-12-31 (Sunday 31st December 2023) inclusive
+
+due next year =>
+  due date is between:
+    2024-01-01 (Monday 1st January 2024) and
+    2024-12-31 (Tuesday 31st December 2024) inclusive
+
+description includes ------------------------------
diff --git a/tests/Query/Filter/ReferenceDocs/FilterReference/DateFieldReference.test.explain_date_reference_last-this-next-weekday.approved.explanation.text b/tests/Query/Filter/ReferenceDocs/FilterReference/DateFieldReference.test.explain_date_reference_last-this-next-weekday.approved.explanation.text
new file mode 100644
index 00000000..ab729156
--- /dev/null
+++ b/tests/Query/Filter/ReferenceDocs/FilterReference/DateFieldReference.test.explain_date_reference_last-this-next-weekday.approved.explanation.text
@@ -0,0 +1,106 @@
+Explanation of this Tasks code block query:
+
+description includes ------------------------------
+
+due today =>
+  due date is on 2023-04-19 (Wednesday 19th April 2023)
+
+description includes ------------------------------
+
+due monday =>
+  due date is on 2023-04-17 (Monday 17th April 2023)
+
+due last monday =>
+  due date is on 2023-04-10 (Monday 10th April 2023)
+
+due this monday =>
+  due date is on 2023-04-17 (Monday 17th April 2023)
+
+due next monday =>
+  due date is on 2023-04-24 (Monday 24th April 2023)
+
+description includes ------------------------------
+
+due tuesday =>
+  due date is on 2023-04-18 (Tuesday 18th April 2023)
+
+due last tuesday =>
+  due date is on 2023-04-11 (Tuesday 11th April 2023)
+
+due this tuesday =>
+  due date is on 2023-04-18 (Tuesday 18th April 2023)
+
+due next tuesday =>
+  due date is on 2023-04-25 (Tuesday 25th April 2023)
+
+description includes ------------------------------
+
+due wednesday =>
+  due date is on 2023-04-19 (Wednesday 19th April 2023)
+
+due last wednesday =>
+  due date is on 2023-04-12 (Wednesday 12th April 2023)
+
+due this wednesday =>
+  due date is on 2023-04-19 (Wednesday 19th April 2023)
+
+due next wednesday =>
+  due date is on 2023-04-26 (Wednesday 26th April 2023)
+
+description includes ------------------------------
+
+due thursday =>
+  due date is on 2023-04-20 (Thursday 20th April 2023)
+
+due last thursday =>
+  due date is on 2023-04-13 (Thursday 13th April 2023)
+
+due this thursday =>
+  due date is on 2023-04-20 (Thursday 20th April 2023)
+
+due next thursday =>
+  due date is on 2023-04-27 (Thursday 27th April 2023)
+
+description includes ------------------------------
+
+due friday =>
+  due date is on 2023-04-21 (Friday 21st April 2023)
+
+due last friday =>
+  due date is on 2023-04-14 (Friday 14th April 2023)
+
+due this friday =>
+  due date is on 2023-04-21 (Friday 21st April 2023)
+
+due next friday =>
+  due date is on 2023-04-28 (Friday 28th April 2023)
+
+description includes ------------------------------
+
+due saturday =>
+  due date is on 2023-04-22 (Saturday 22nd April 2023)
+
+due last saturday =>
+  due date is on 2023-04-15 (Saturday 15th April 2023)
+
+due this saturday =>
+  due date is on 2023-04-22 (Saturday 22nd April 2023)
+
+due next saturday =>
+  due date is on 2023-04-29 (Saturday 29th April 2023)
+
+description includes ------------------------------
+
+due sunday =>
+  due date is on 2023-04-16 (Sunday 16th April 2023)
+
+due last sunday =>
+  due date is on 2023-04-09 (Sunday 9th April 2023)
+
+due this sunday =>
+  due date is on 2023-04-16 (Sunday 16th April 2023)
+
+due next sunday =>
+  due date is on 2023-04-23 (Sunday 23rd April 2023)
+
+description includes ------------------------------
diff --git a/tests/Query/Filter/ReferenceDocs/FilterReference/DateFieldReference.test.ts b/tests/Query/Filter/ReferenceDocs/FilterReference/DateFieldReference.test.ts
new file mode 100644
index 00000000..53333f3c
--- /dev/null
+++ b/tests/Query/Filter/ReferenceDocs/FilterReference/DateFieldReference.test.ts
@@ -0,0 +1,33 @@
+/**
+ * @jest-environment jsdom
+ */
+
+import { readFileSync } from 'fs';
+import moment from 'moment';
+import { verifyExplanation } from '../../../../TestingTools/ApprovalTestHelpers';
+
+window.moment = moment;
+
+describe('explain', () => {
+    beforeEach(() => {
+        jest.useFakeTimers();
+        jest.setSystemTime(new Date('2023-04-19'));
+    });
+
+    afterEach(() => {
+        jest.useRealTimers();
+    });
+
+    it.each([
+        'filters-date-examples', // Alphabetical order by filename
+        'last-this-next-week-month-quarter-year',
+        'last-this-next-weekday',
+    ])('date reference %s', (queryFileBasename: string) => {
+        // Arrange
+        const inputFile = `tests/Query/Filter/ReferenceDocs/FilterReference/${queryFileBasename}.input.query`;
+        const instructions = readFileSync(inputFile, 'utf-8');
+
+        // Act, Assert
+        verifyExplanation(instructions);
+    });
+});
diff --git a/tests/Query/Filter/ReferenceDocs/FilterReference/filters-date-examples.input.query b/tests/Query/Filter/ReferenceDocs/FilterReference/filters-date-examples.input.query
new file mode 100644
index 00000000..037b74e9
--- /dev/null
+++ b/tests/Query/Filter/ReferenceDocs/FilterReference/filters-date-examples.input.query
@@ -0,0 +1,15 @@
+description includes ------------------------------
+
+due today
+
+description includes ------------------------------
+
+due yesterday
+due today
+due tomorrow
+due next monday
+due last friday
+due 14 days ago
+due in two weeks
+due 14 October
+due May
diff --git a/tests/Query/Filter/ReferenceDocs/FilterReference/last-this-next-week-month-quarter-year.input.query b/tests/Query/Filter/ReferenceDocs/FilterReference/last-this-next-week-month-quarter-year.input.query
new file mode 100644
index 00000000..7400c1c7
--- /dev/null
+++ b/tests/Query/Filter/ReferenceDocs/FilterReference/last-this-next-week-month-quarter-year.input.query
@@ -0,0 +1,29 @@
+description includes ------------------------------
+
+due today
+
+description includes ------------------------------
+
+due last week
+due this week
+due next week
+
+description includes ------------------------------
+
+due last month
+due this month
+due next month
+
+description includes ------------------------------
+
+due last quarter
+due this quarter
+due next quarter
+
+description includes ------------------------------
+
+due last year
+due this year
+due next year
+
+description includes ------------------------------
diff --git a/tests/Query/Filter/ReferenceDocs/FilterReference/last-this-next-weekday.input.query b/tests/Query/Filter/ReferenceDocs/FilterReference/last-this-next-weekday.input.query
new file mode 100644
index 00000000..f6e2648c
--- /dev/null
+++ b/tests/Query/Filter/ReferenceDocs/FilterReference/last-this-next-weekday.input.query
@@ -0,0 +1,54 @@
+description includes ------------------------------
+
+due today
+
+description includes ------------------------------
+
+due monday
+due last monday
+due this monday
+due next monday
+
+description includes ------------------------------
+
+due tuesday
+due last tuesday
+due this tuesday
+due next tuesday
+
+description includes ------------------------------
+
+due wednesday
+due last wednesday
+due this wednesday
+due next wednesday
+
+description includes ------------------------------
+
+due thursday
+due last thursday
+due this thursday
+due next thursday
+
+description includes ------------------------------
+
+due friday
+due last friday
+due this friday
+due next friday
+
+description includes ------------------------------
+
+due saturday
+due last saturday
+due this saturday
+due next saturday
+
+description includes ------------------------------
+
+due sunday
+due last sunday
+due this sunday
+due next sunday
+
+description includes ------------------------------
diff --git a/tests/TestingTools/ApprovalTestHelpers.ts b/tests/TestingTools/ApprovalTestHelpers.ts
new file mode 100644
index 00000000..ed4d6e4c
--- /dev/null
+++ b/tests/TestingTools/ApprovalTestHelpers.ts
@@ -0,0 +1,34 @@
+import { Options } from 'approvals/lib/Core/Options';
+import { verify } from 'approvals/lib/Providers/Jest/JestApprovals';
+import { Query } from '../../src/Query/Query';
+
+/**
+ * Save an instructions block to disc, so that it can be embedded in
+ * to documentation, using a 'snippet' line.
+ * @todo Figure out how to include the '```tasks' and '```' lines:
+ *       see discussion in https://github.com/SimonCropp/MarkdownSnippets/issues/537
+ * @param instructions
+ * @param options
+ */
+export function verifyQuery(instructions: string, options?: Options): void {
+    options = options || new Options();
+    options = options.forFile().withFileExtention('query.text');
+    verify(instructions, options);
+}
+
+/**
+ * Save an explanation of the instructions block to disk, so that it can be
+ * embedded in to documentation, using a 'snippet' line.
+ * @param instructions
+ * @param options
+ */
+export function verifyExplanation(instructions: string, options?: Options): void {
+    const query = new Query({ source: instructions });
+    const explanation = query.explainQuery();
+
+    expect(query.error).toBeUndefined();
+
+    options = options || new Options();
+    options = options.forFile().withFileExtention('explanation.text');
+    verify(explanation, options);
+}