Repository: blacksmithgu/obsidian-dataview
Pull request #1876 (2023-04-11 11:37:08)

New docs on comparing DQL and SQL


Patch:
diff --git a/docs/docs/api/code-reference.md b/docs/docs/api/code-reference.md
index 534d595..9b554c1 100644
--- a/docs/docs/api/code-reference.md
+++ b/docs/docs/api/code-reference.md
@@ -168,10 +168,26 @@ dv.taskList(dv.pages("#project").file.tasks
 
 ### `dv.table(headers, elements)`
 
-Render a dataview table with the given list of headers and 2D array of elements.
+Renders a dataview table. `headers` is an array of column headers. `elements` is an array of rows. Each row is itself an array of columns. Inside a row, every column which is an array will be rendered with bullet points.
+
+```js
+dv.table(
+	["Col1", "Col2", "Col3"],
+		[
+			["Row1", "Dummy", "Dummy"],
+			["Row2", 
+				["Bullet1",
+				 "Bullet2",
+				 "Bullet3"],
+			 "Dummy"],
+			["Row3", "Dummy", "Dummy"]
+		]
+	);
+```
+
+An example of how to render a simple table of book info sorted by rating.
 
 ```js
-// Render a simple table of book info sorted by rating.
 dv.table(["File", "Genre", "Time Read", "Rating"], dv.pages("#book")
     .sort(b => b.rating)
     .map(b => [b.file.link, b.genre, b["time-read"], b.rating]))
diff --git a/docs/docs/api/data-array.md b/docs/docs/api/data-array.md
index de97509..2f741f6 100644
--- a/docs/docs/api/data-array.md
+++ b/docs/docs/api/data-array.md
@@ -1,9 +1,9 @@
 # Data Arrays
 
-The general abstraction for lists of results in Dataview is the `DataArray`, which is a proxied array with additional
-functionality. Data arrays support indexing and iteration (via `for` and `for ... of` loops) as per normal arrays, but
+The general representation of result lists in Dataview is the `DataArray`, which is a [proxied](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Proxy) version of the JavaScript array with expanded
+functionality. Data arrays support indexing and iteration (via `for` and `for ... of` loops), like normal arrays do, but
 also include many data manipulation operators like `sort`, `groupBy`, `distinct`, `where`, and so on to make
-manipulating tabular data easy.
+manipulating tabular data easier.
 
 ## Creation
 
diff --git a/docs/docs/index.md b/docs/docs/index.md
index 7601569..19fa9ca 100644
--- a/docs/docs/index.md
+++ b/docs/docs/index.md
@@ -74,7 +74,7 @@ You can access **indexed data** with the help of **Queries**.
 
 There are **three different ways** you can write a Query: With help of the [Dataview Query Language](queries/dql-js-inline/#dataview-query-language-dql), as an [inline statement](queries/dql-js-inline#inline-dql) or in the most flexible but most complex way: as a [Javascript Query](queries/dql-js-inline#dataview-js). 
 
-The **Dataview Query Language** (**DQL**) gives you a broad and powerful toolbelt to query, display and operate on your data. An [**inline query**](queries/dql-js-inline#inline-dql) gives you the possibility to display exactly one indexed value anywhere in your note. You can also do calculations this way. With **DQL** at your hands, you'll be probably fine without any Javascript throurough your data journey.
+The **Dataview Query Language** (**DQL**) gives you a broad and powerful toolbelt to query, display and operate on your data. An [**inline query**](queries/dql-js-inline#inline-dql) gives you the possibility to display exactly one indexed value anywhere in your note. You can also do calculations this way. With **DQL** at your hands, you'll be probably fine without any Javascript through your data journey.
 
 A DQL Query consists of several parts:
 
@@ -150,4 +150,4 @@ As you can see, dataview doesn't only allow you to aggregate your data swiftly a
 
 ## Resources and Help
 
-This documentation is not the only place that can help you out on your data journey. Take a look at [Resources and Support](./resources/resources-and-support.md) for a list of helpful pages and videos.
\ No newline at end of file
+This documentation is not the only place that can help you out on your data journey. Take a look at [Resources and Support](./resources/resources-and-support.md) for a list of helpful pages and videos.
diff --git a/docs/docs/queries/data-commands.md b/docs/docs/queries/data-commands.md
index bbd4990..3f1fd88 100644
--- a/docs/docs/queries/data-commands.md
+++ b/docs/docs/queries/data-commands.md
@@ -71,7 +71,7 @@ GROUP BY (computed_field) AS name
 ```
 
 In order to make working with the `rows` array easier, Dataview supports field "swizzling". If you want the field `test` from every object in the `rows` array, then `rows.test` will automatically fetch the `test` field from every object in `rows`, yielding a new array.
-You can then apply aggregation operators like `sum()` over the resulting array.
+You can then apply aggregation operators like `sum()` or `flat()` over the resulting array.
 
 ## FLATTEN
 
diff --git a/docs/docs/queries/differences-to-sql.md b/docs/docs/queries/differences-to-sql.md
new file mode 100644
index 0000000..4983a53
--- /dev/null
+++ b/docs/docs/queries/differences-to-sql.md
@@ -0,0 +1,19 @@
+# Dataview Query Language (DQL) and SQL
+
+If you are familiar with SQL and experienced in writing SQL queries, you might approach writing a DQL query in a similar way. However, DQL is significantly different from SQL.
+
+A DQL query is **executed from top to bottom**, line-by-line. It is more like a computer program than a typical SQL query.
+
+When a line is evaluated, it produces a result set and **passes the whole set on to the next DQL line** which will manipulate the set that it received from the previous line. This is why in DQL it is possible, for example, to have multiple WHERE clauses. But in DQL it is not a 'clause' but a 'data command'. Every line of a DQL query (except the 1st and 2nd lines) is a 'data command'.
+
+## Anatomy of a DQL query
+
+Instead of starting with SELECT, a DQL query starts with a word determining the Query Type, which determines how your final result will be rendered on screen (a table, a list, a task list, or a calendar). Then follows the list of fields, which is actually very similar to the column list you put after a SELECT statement.
+
+The next line starts with FROM which is not followed by a table name but by a complex expression, similar to an SQL WHERE clause. Here you can filter on many things, like tags in files, file names, path names, etc. In the background, this command already produces a result set which will be our initial set for further data manipulation by 'data commands' on subsequent lines.
+
+You can have as many following lines as you want. Each will start with a [data command](../../queries/data-commands) and will re-shape the result set it received from the previous line. For example:
+
+- The WHERE data command will only keep those lines from the result set which match a given condition. This means that, unless all data in the result set matches the condition, this command will pass on a smaller result set to the next line than it received from the previous line. Unlike in SQL, you can have as many WHERE commands as you like.
+- The FLATTEN data command is not found in common SQL but in DQL you can use it to reduce the depth of the result set.
+- DQL, similarly to SQL, has a GROUP BY command but this can also be used multiple times, which is not possible in common SQL. You can even do several SORT or GROUP BY commands one after the other.
diff --git a/docs/docs/queries/dql-js-inline.md b/docs/docs/queries/dql-js-inline.md
index b310ac1..b10a1aa 100644
--- a/docs/docs/queries/dql-js-inline.md
+++ b/docs/docs/queries/dql-js-inline.md
@@ -103,7 +103,7 @@ create JS inline queries via inline code blocks:
 In inline DataviewJS, you have access to the `dv` variable, as in `dataviewjs` codeblocks, and can make all of the same calls. The result
 should be something which evaluates to a JavaScript value, which Dataview will automatically render appropriately.
 
-Unline Inline DQL queries, Inline JS queries do have access to everything a Dataview JS Query has available and can hence query and output multiple pages.
+Unlike Inline DQL queries, Inline JS queries do have access to everything a Dataview JS Query has available and can hence query and output multiple pages.
 
 !!! info "Change of Inline JS prefix"
-    You can change the `$=` to another token (like `dvjs:` or `$~`) in Dataviews' settings under "Codeblock Settings" > "Javascript Inline Query Prefix"
\ No newline at end of file
+    You can change the `$=` to another token (like `dvjs:` or `$~`) in Dataviews' settings under "Codeblock Settings" > "Javascript Inline Query Prefix"
diff --git a/docs/docs/queries/structure.md b/docs/docs/queries/structure.md
index 0f4661a..8c1ab4b 100644
--- a/docs/docs/queries/structure.md
+++ b/docs/docs/queries/structure.md
@@ -15,6 +15,9 @@ supports:
 - **Grouping** results based on fields
 - **Limiting** your result count
 
+!!! warning Differences to SQL
+    If you are familiar with SQL, please read [Differences to SQL](../../queries/differences-to-sql) to avoid confusing DQL with SQL.
+
 Let's have a look at how we can put DQL to use.
 
 ## General Format of a DQL Query
@@ -204,4 +207,4 @@ SORT length(contacts)
 FLATTEN contacts as c
 SORT link(c).age ASC
 ```
-~~~
\ No newline at end of file
+~~~
diff --git a/docs/docs/reference/functions.md b/docs/docs/reference/functions.md
index be3bf66..fce1f8b 100644
--- a/docs/docs/reference/functions.md
+++ b/docs/docs/reference/functions.md
@@ -62,6 +62,17 @@ date("2020-04-18") = <date object representing April 18th, 2020>
 date([[2021-04-16]]) = <date object for the given page, refering to file.day>
 ```
 
+### `date(text, format)`
+
+Parses a date from text to luxon `DateTime` with the specified format. Note localised formats might not work. 
+Uses [Luxon tokens](https://moment.github.io/luxon/#/formatting?id=table-of-tokens).
+
+```js
+date("12/31/2022", "MM/dd/yyyy") => DateTime for Decemeber 31th, 2022
+date("210313", "yyMMdd") => DateTime for March 13th, 2021
+date("946778645000", "x") => DateTime for "2000-01-02T03:04:05"
+```
+
 ### `dur(any)`
 
 Parses a duration from the provided string or duration, returning null on failure.
@@ -464,6 +475,18 @@ map([1, 2, 3], (x) => x + 2) = [3, 4, 5]
 map(["yes", "no"], (x) => x + "?") = ["yes?", "no?"]
 ```
 
+### `flat(array, [depth])`
+
+Concatenates sub-levels of the array to the desired depth. Default is 1 level, but it can
+concatenate multiple levels. E.g. Can be used to reduce array depth on `rows` lists after
+doing `GROUP BY`.
+
+```js
+flat(list(1, 2, 3, list(4, 5), 6)) => list(1, 2, 3, 4, 5, 6)
+flat(list(1, list(21, 22), list(list (311, 312, 313))), 4) => list(1, 21, 22, 311, 312, 313)
+flat(rows.file.outlinks)) => All the file outlinks at first level in output
+```
+
 ---
 
 ## String Operations
diff --git a/docs/mkdocs.yml b/docs/mkdocs.yml
index 6cfed3f..0da2228 100644
--- a/docs/mkdocs.yml
+++ b/docs/mkdocs.yml
@@ -27,6 +27,7 @@ nav:
     - Structure of a Query: 'queries/structure.md'
     - Query Types: 'queries/query-types.md'
     - Data Commands: 'queries/data-commands.md'
+    - Differences to SQL: 'queries/differences-to-sql.md'
     - Sources: 'reference/sources.md'
     - Expressions: 'reference/expressions.md'
     - Literals: 'reference/literals.md'
@@ -64,4 +65,4 @@ plugins:
         resources-and-support.md: resources/resources-and-support.md
         plugin/develop-against-dataview.md: resources/develop-against-dataview.md
         reference/faq.md: resources/faq.md
-        reference/examples.md: resources/examples.md
\ No newline at end of file
+        reference/examples.md: resources/examples.md
diff --git a/package-lock.json b/package-lock.json
index 6075663..50a1fb8 100644
--- a/package-lock.json
+++ b/package-lock.json
@@ -1,12 +1,12 @@
 {
   "name": "obsidian-dataview",
-  "version": "0.5.54",
+  "version": "0.5.55",
   "lockfileVersion": 2,
   "requires": true,
   "packages": {
     "": {
       "name": "obsidian-dataview",
-      "version": "0.5.54",
+      "version": "0.5.55",
       "license": "MIT",
       "dependencies": {
         "@codemirror/language": "https://github.com/lishid/cm-language",
@@ -32,7 +32,7 @@
         "@zerollup/ts-transform-paths": "^1.7.18",
         "compare-versions": "^4.1.1",
         "jest": "^27.1.0",
-        "obsidian": "^0.15.4",
+        "obsidian": "^1.2.8",
         "prettier": "2.3.2",
         "rollup": "^2.67.2",
         "rollup-jest": "^1.1.3",
@@ -5568,8 +5568,8 @@
       "dev": true
     },
     "node_modules/obsidian": {
-      "version": "0.15.9",
-      "resolved": "git+ssh://git@github.com/obsidianmd/obsidian-api.git#ff121cd409350d3afe39b08420f468d8aaee925e",
+      "version": "1.2.8",
+      "resolved": "git+ssh://git@github.com/obsidianmd/obsidian-api.git#583ba39e3f6c0546de5e5e8742256a60e2d78ebc",
       "license": "MIT",
       "dependencies": {
         "@types/codemirror": "0.0.108",
@@ -11373,8 +11373,8 @@
       "dev": true
     },
     "obsidian": {
-      "version": "git+ssh://git@github.com/obsidianmd/obsidian-api.git#ff121cd409350d3afe39b08420f468d8aaee925e",
-      "from": "obsidian@^0.15.4",
+      "version": "git+ssh://git@github.com/obsidianmd/obsidian-api.git#583ba39e3f6c0546de5e5e8742256a60e2d78ebc",
+      "from": "obsidian@^1.2.8",
       "requires": {
         "@types/codemirror": "0.0.108",
         "moment": "2.29.4"
diff --git a/package.json b/package.json
index 126d0e3..636be28 100644
--- a/package.json
+++ b/package.json
@@ -34,7 +34,7 @@
     "@zerollup/ts-transform-paths": "^1.7.18",
     "compare-versions": "^4.1.1",
     "jest": "^27.1.0",
-    "obsidian": "^0.15.4",
+    "obsidian": "^1.2.8",
     "prettier": "2.3.2",
     "rollup": "^2.67.2",
     "rollup-jest": "^1.1.3",
diff --git a/src/data-import/markdown-file.ts b/src/data-import/markdown-file.ts
index b0eb228..24abe2a 100644
--- a/src/data-import/markdown-file.ts
+++ b/src/data-import/markdown-file.ts
@@ -327,6 +327,9 @@ export function parseFrontmatter(value: any): Literal {
             }
 
             return result;
+        } else if (value instanceof Date) {
+            let dateParse = DateTime.fromJSDate(value)
+            return dateParse;
         } else {
             let object = value as Record<string, any>;
             let result: Record<string, Literal> = {};
diff --git a/src/expression/functions.ts b/src/expression/functions.ts
index ae9c1f2..6d2628f 100644
--- a/src/expression/functions.ts
+++ b/src/expression/functions.ts
@@ -276,6 +276,21 @@ export namespace DefaultFunctions {
 
             return null;
         })
+        .add2("string", "string", (d, f) => {
+            if (f === "x" || f==="X") {
+                let match = NUMBER_REGEX.exec(d)
+                if (match) return DateTime.fromMillis(Number.parseInt(match[0]) * ( f==="X" ? 1000 : 1))
+                else {
+                    throw Error("Not a number for format( (${ f }): ${ d }")
+                }
+            } else {
+                let parsedDate = DateTime.fromFormat(d, f)
+                if (parsedDate.isValid) return parsedDate
+                else { throw Error(`Can't handle format (${ f }) on date string (${ d })`) }    
+            } 
+
+            return null;
+        })
         .add1("null", () => null)
         .vectorize(1, [0])
         .build();
@@ -763,6 +778,18 @@ export namespace DefaultFunctions {
             type: link.type,
         }))
         .build();
+
+    // Concatenates sub-array elements into a new array
+    export const flat = new FunctionBuilder("flat")
+    .add1("array", a => {
+        return a.flat()
+    })
+    .add2("array", "number", (a, n) => {
+        // @ts-ignore
+        return a.flat(n)
+    })
+    .add1("null", () => null)
+    .build();
 }
 
 /** Default function implementations for the expression evaluator. */
@@ -815,6 +842,7 @@ export const DEFAULT_FUNCTIONS: Record<string, FunctionImpl> = {
     containsword: DefaultFunctions.containsword,
     reverse: DefaultFunctions.reverse,
     sort: DefaultFunctions.sort,
+    flat: DefaultFunctions.flat,
 
     // Aggregation operations like reduce.
     reduce: DefaultFunctions.reduce,
diff --git a/src/main.ts b/src/main.ts
index eaf0441..69209c7 100644
--- a/src/main.ts
+++ b/src/main.ts
@@ -86,8 +86,10 @@ export default class DataviewPlugin extends Plugin {
         });
 
         // editor extension for inline queries
-        this.cmExtension = [inlinePlugin(this.index, this.settings, this.api)];
-        this.registerEditorExtension(this.cmExtension);
+        if (this.settings.enableInlineDataview || this.settings.enableInlineDataviewJs) {
+            this.cmExtension = [inlinePlugin(this.app, this.index, this.settings, this.api)];
+            this.registerEditorExtension(this.cmExtension);
+        }
 
         // Dataview "force refresh" operation.
         this.addCommand({
diff --git a/src/ui/lp-render.ts b/src/ui/lp-render.ts
index 3954f9a..aec5759 100644
--- a/src/ui/lp-render.ts
+++ b/src/ui/lp-render.ts
@@ -29,10 +29,10 @@
 
 import { Decoration, DecorationSet, EditorView, ViewPlugin, ViewUpdate, WidgetType } from "@codemirror/view";
 import { EditorSelection, Range } from "@codemirror/state";
-import { syntaxTree } from "@codemirror/language";
+import { syntaxTree, tokenClassNodeProp } from "@codemirror/language";
 import { DataviewSettings } from "../settings";
 import { FullIndex } from "../data-index";
-import { Component, editorEditorField, editorLivePreviewField, editorViewField } from "obsidian";
+import { App, Component, editorInfoField, editorLivePreviewField, TFile } from "obsidian";
 import { DataviewApi } from "../api/plugin-api";
 import { tryOrPropogate } from "../util/normalize";
 import { parseField } from "../expression/parse";
@@ -40,6 +40,7 @@ import { executeInline } from "../query/engine";
 import { Literal } from "../data-model/value";
 import { DataviewInlineApi } from "../api/inline-api";
 import { renderValue } from "./render";
+import { SyntaxNode } from "@lezer/common";
 
 function selectionAndRangeOverlap(selection: EditorSelection, rangeFrom: number, rangeTo: number) {
     for (const range of selection.ranges) {
@@ -97,9 +98,10 @@ class InlineWidget extends WidgetType {
             if ((event as MouseEvent).shiftKey) {
                 // Set the cursor after the element so that it doesn't select starting from the last cursor position.
                 if (currentPos) {
-                    //@ts-ignore
-                    const { editor } = this.view.state.field(editorEditorField).state.field(editorViewField);
-                    editor.setCursor(editor.offsetToPos(currentPos));
+                    const { editor } = this.view.state.field(editorInfoField);
+                    if (editor) {
+                        editor.setCursor(editor.offsetToPos(currentPos));
+                    }
                 }
                 return false;
             }
@@ -108,65 +110,181 @@ class InlineWidget extends WidgetType {
     }
 }
 
-function getCssClasses(nodeName: string): string[] {
+function getCssClasses(props: Set<string>): string[] {
     const classes: string[] = [];
-    if (nodeName.includes("strong")) {
+    if (props.has("strong")) {
         classes.push("cm-strong");
     }
-    if (nodeName.includes("em")) {
+    if (props.has("em")) {
         classes.push("cm-em");
     }
-    if (nodeName.includes("highlight")) {
+    if (props.has("highlight")) {
         classes.push("cm-highlight");
     }
-    if (nodeName.includes("strikethrough")) {
+    if (props.has("strikethrough")) {
         classes.push("cm-strikethrough");
     }
-    if (nodeName.includes("comment")) {
+    if (props.has("comment")) {
         classes.push("cm-comment");
     }
     return classes;
 }
 
-function inlineRender(view: EditorView, index: FullIndex, dvSettings: DataviewSettings, api: DataviewApi) {
-    // still doesn't work as expected for tables and callouts
-    if (!index.initialized) return;
-    const currentFile = app.workspace.getActiveFile();
-    if (!currentFile) return;
+export function inlinePlugin(app: App, index: FullIndex, settings: DataviewSettings, api: DataviewApi) {
+    return ViewPlugin.fromClass(
+        class {
+            decorations: DecorationSet;
+            component: Component;
 
-    const widgets: Range<Decoration>[] = [];
-    const selection = view.state.selection;
-    /* before:
-     *     em for italics
-     *     highlight for highlight
-     * after:
-     *     strong for bold
-     *     strikethrough for strikethrough
-     */
-    const regex = new RegExp(".*?_?inline-code_?.*");
-    const PREAMBLE: string = "const dataview=this;const dv=this;";
+            constructor(view: EditorView) {
+                this.component = new Component();
+                this.component.load();
+                this.decorations = this.inlineRender(view) ?? Decoration.none;
+            }
 
-    for (const { from, to } of view.visibleRanges) {
-        syntaxTree(view.state).iterate({
-            from,
-            to,
-            enter: ({ node }) => {
+            update(update: ViewUpdate) {
+                // only activate in LP and not source mode
+                if (!update.state.field(editorLivePreviewField)) {
+                    this.decorations = Decoration.none;
+                    return;
+                }
+                if (update.docChanged) {
+                    this.decorations = this.decorations.map(update.changes);
+                    this.updateTree(update.view);
+                } else if (update.selectionSet) {
+                    this.updateTree(update.view);
+                } else if (update.viewportChanged /*|| update.selectionSet*/) {
+                    this.decorations = this.inlineRender(update.view) ?? Decoration.none;
+                }
+            }
+
+            updateTree(view: EditorView) {
+                for (const { from, to } of view.visibleRanges) {
+                    syntaxTree(view.state).iterate({
+                        from,
+                        to,
+                        enter: ({ node }) => {
+                            const { render, isQuery } = this.renderNode(view, node);
+                            if (!render && isQuery) {
+                                this.removeDeco(node, view);
+                                return;
+                            } else if (!render) {
+                                return;
+                            } else if (render) {
+                                this.addDeco(node, view);
+                            }
+                        },
+                    });
+                }
+            }
+
+            removeDeco(node: SyntaxNode, view: EditorView) {
+                this.decorations.between(node.from - 1, node.to + 1, (from, to, value) => {
+                    this.decorations = this.decorations.update({
+                        filterFrom: from,
+                        filterTo: to,
+                        filter: (from, to, value) => false,
+                    });
+                });
+            }
+
+            addDeco(node: SyntaxNode, view: EditorView) {
+                const from = node.from - 1;
+                const to = node.to + 1;
+                let exists = false;
+                this.decorations.between(from, to, (from, to, value) => {
+                    exists = true;
+                });
+                if (!exists) {
+                    const currentFile = app.workspace.getActiveFile();
+                    if (!currentFile) return;
+                    const newDeco = this.renderWidget(node, view, currentFile)?.value;
+                    if (newDeco) {
+                        this.decorations = this.decorations.update({
+                            add: [{ from: from, to: to, value: newDeco }],
+                        });
+                    }
+                }
+            }
+
+            // checks whether a node should get rendered/unrendered
+            renderNode(view: EditorView, node: SyntaxNode) {
                 const type = node.type;
-                // markdown formatting symbols
-                if (type.name.includes("formatting")) return;
-                // current node is not inline code
-                if (!regex.test(type.name)) return;
+                // current node is inline code
+                const tokenProps = type.prop<String>(tokenClassNodeProp);
+                const props = new Set(tokenProps?.split(" "));
+                if (props.has("inline-code") && !props.has("formatting")) {
+                    // contains the position of inline code
+                    const start = node.from;
+                    const end = node.to;
+                    // don't continue if current cursor position and inline code node (including formatting
+                    // symbols) overlap
+                    const selection = view.state.selection;
+                    if (selectionAndRangeOverlap(selection, start - 1, end + 1)) {
+                        if (this.isInlineQuery(view, start, end)) {
+                            return { render: false, isQuery: true };
+                        } else {
+                            return { render: false, isQuery: false };
+                        }
+                    } else if (this.isInlineQuery(view, start, end)) {
+                        return { render: true, isQuery: true };
+                    }
+                }
+                return { render: false, isQuery: false };
+            }
+
+            isInlineQuery(view: EditorView, start: number, end: number) {
+                const text = view.state.doc.sliceString(start, end);
+                const isInlineQuery =
+                    text.startsWith(settings.inlineQueryPrefix) || text.startsWith(settings.inlineJsQueryPrefix);
+                return isInlineQuery;
+            }
+
+            inlineRender(view: EditorView) {
+                // still doesn't work as expected for tables and callouts
+                if (!index.initialized) return;
+                const currentFile = app.workspace.getActiveFile();
+                if (!currentFile) return;
 
+                const widgets: Range<Decoration>[] = [];
+                /* before:
+                 *     em for italics
+                 *     highlight for highlight
+                 * after:
+                 *     strong for bold
+                 *     strikethrough for strikethrough
+                 */
+
+                for (const { from, to } of view.visibleRanges) {
+                    syntaxTree(view.state).iterate({
+                        from,
+                        to,
+                        enter: ({ node }) => {
+                            if (!this.renderNode(view, node).render) return;
+                            const widget = this.renderWidget(node, view, currentFile);
+                            if (widget) {
+                                widgets.push(widget);
+                            }
+                        },
+                    });
+                }
+
+                return Decoration.set(widgets, true);
+            }
+
+            renderWidget(node: SyntaxNode, view: EditorView, currentFile: TFile) {
+                const type = node.type;
                 // contains the position of inline code
                 const start = node.from;
                 const end = node.to;
-                // don't continue if current cursor position and inline code node (including formatting
-                // symbols) overlap
-                if (selectionAndRangeOverlap(selection, start - 1, end + 1)) return;
-
+                // safety net against unclosed inline code
+                if (view.state.doc.sliceString(end, end + 1) === "\n") {
+                    return;
+                }
                 const text = view.state.doc.sliceString(start, end);
                 let code: string = "";
                 let result: Literal = "";
+                const PREAMBLE: string = "const dataview=this;const dv=this;";
                 const el = createSpan({
                     cls: ["dataview", "dataview-inline"],
                 });
@@ -175,56 +293,47 @@ function inlineRender(view: EditorView, index: FullIndex, dvSettings: DataviewSe
                  * This is necessary because {@link InlineWidget.toDOM} is synchronous but some rendering
                  * asynchronous.
                  */
-                if (dvSettings.inlineQueryPrefix.length > 0 && text.startsWith(dvSettings.inlineQueryPrefix)) {
-                    code = text.substring(dvSettings.inlineQueryPrefix.length).trim();
-                    const field = tryOrPropogate(() => parseField(code));
-                    if (!field.successful) {
-                        result = `Dataview (inline field '${code}'): ${field.error}`;
-                        el.innerText = result;
-                    } else {
-                        const fieldValue = field.value;
-                        const intermediateResult = tryOrPropogate(() =>
-                            executeInline(fieldValue, currentFile.path, index, dvSettings)
-                        );
-                        if (!intermediateResult.successful) {
-                            result = `Dataview (for inline query '${fieldValue}'): ${intermediateResult.error}`;
+                if (text.startsWith(settings.inlineQueryPrefix)) {
+                    if (settings.enableInlineDataview) {
+                        code = text.substring(settings.inlineQueryPrefix.length).trim();
+                        const field = tryOrPropogate(() => parseField(code));
+                        if (!field.successful) {
+                            result = `Dataview (inline field '${code}'): ${field.error}`;
                             el.innerText = result;
                         } else {
-                            const { value } = intermediateResult;
-                            result = value;
-                            renderValue(result, el, currentFile.path, null as unknown as Component, dvSettings);
+                            const fieldValue = field.value;
+                            const intermediateResult = tryOrPropogate(() =>
+                                executeInline(fieldValue, currentFile.path, index, settings)
+                            );
+                            if (!intermediateResult.successful) {
+                                result = `Dataview (for inline query '${fieldValue}'): ${intermediateResult.error}`;
+                                el.innerText = result;
+                            } else {
+                                const { value } = intermediateResult;
+                                result = value;
+                                renderValue(result, el, currentFile.path, this.component, settings);
+                            }
                         }
+                    } else {
+                        result = "(disabled; enable in settings)";
+                        el.innerText = result;
                     }
-                } else if (
-                    dvSettings.inlineJsQueryPrefix.length > 0 &&
-                    text.startsWith(dvSettings.inlineJsQueryPrefix)
-                ) {
-                    if (dvSettings.enableInlineDataviewJs) {
-                        code = text.substring(dvSettings.inlineJsQueryPrefix.length).trim();
+                } else if (text.startsWith(settings.inlineJsQueryPrefix)) {
+                    if (settings.enableInlineDataviewJs) {
+                        code = text.substring(settings.inlineJsQueryPrefix.length).trim();
                         try {
                             // for setting the correct context for dv/dataview
                             const myEl = createDiv();
-                            const dvInlineApi = new DataviewInlineApi(
-                                api,
-                                null as unknown as Component,
-                                myEl,
-                                currentFile.path
-                            );
+                            const dvInlineApi = new DataviewInlineApi(api, this.component, myEl, currentFile.path);
                             if (code.includes("await")) {
                                 (evalInContext("(async () => { " + PREAMBLE + code + " })()") as Promise<any>).then(
                                     (result: any) => {
-                                        renderValue(
-                                            result,
-                                            el,
-                                            currentFile.path,
-                                            null as unknown as Component,
-                                            dvSettings
-                                        );
+                                        renderValue(result, el, currentFile.path, this.component, settings);
                                     }
                                 );
                             } else {
                                 result = evalInContext(PREAMBLE + code);
-                                renderValue(result, el, currentFile.path, null as unknown as Component, dvSettings);
+                                renderValue(result, el, currentFile.path, this.component, settings);
                             }
 
                             function evalInContext(script: string): any {
@@ -244,41 +353,19 @@ function inlineRender(view: EditorView, index: FullIndex, dvSettings: DataviewSe
                     return;
                 }
 
-                const classes = getCssClasses(type.name);
-
-                widgets.push(
-                    Decoration.replace({
-                        widget: new InlineWidget(classes, code, el, view),
-                        inclusive: false,
-                        block: false,
-                    }).range(start - 1, end + 1)
-                );
-            },
-        });
-    }
-
-    return Decoration.set(widgets, true);
-}
-
-export function inlinePlugin(index: FullIndex, settings: DataviewSettings, api: DataviewApi) {
-    return ViewPlugin.fromClass(
-        class {
-            decorations: DecorationSet;
+                const tokenProps = type.prop<String>(tokenClassNodeProp);
+                const props = new Set(tokenProps?.split(" "));
+                const classes = getCssClasses(props);
 
-            constructor(view: EditorView) {
-                this.decorations = inlineRender(view, index, settings, api) ?? Decoration.none;
+                return Decoration.replace({
+                    widget: new InlineWidget(classes, code, el, view),
+                    inclusive: false,
+                    block: false,
+                }).range(start - 1, end + 1);
             }
 
-            update(update: ViewUpdate) {
-                // only activate in LP and not source mode
-                //@ts-ignore
-                if (!update.state.field(editorLivePreviewField)) {
-                    this.decorations = Decoration.none;
-                    return;
-                }
-                if (update.docChanged || update.viewportChanged || update.selectionSet) {
-                    this.decorations = inlineRender(update.view, index, settings, api) ?? Decoration.none;
-                }
+            destroy() {
+                this.component.unload();
             }
         },
         { decorations: v => v.decorations }


Test patch:
diff --git a/src/test/function/coerce.test.ts b/src/test/function/coerce.test.ts
index 6ca95fa..688ab69 100644
--- a/src/test/function/coerce.test.ts
+++ b/src/test/function/coerce.test.ts
@@ -13,11 +13,6 @@ describe("string()", () => {
     test("number", () => expect(parseEval(`string(18)`)).toEqual("18"));
 });
 
-test("date()", () => {
-    expect(parseEval("date([[2020-04-18]])")).toEqual(DateTime.fromObject({ year: 2020, month: 4, day: 18 }));
-    expect(parseEval("date([[Place|2021-04]])")).toEqual(DateTime.fromObject({ year: 2021, month: 4, day: 1 }));
-});
-
 test("list()", () => {
     expectEvals("list(1, 2, 3)", [1, 2, 3]);
     expectEvals("list()", []);
diff --git a/src/test/function/functions.test.ts b/src/test/function/functions.test.ts
index dc627fb..e9197d8 100644
--- a/src/test/function/functions.test.ts
+++ b/src/test/function/functions.test.ts
@@ -1,6 +1,7 @@
 // <-- Functions -->
 // <-- Function vectorization -->
 
+import { DateTime } from "luxon";
 import { DefaultFunctions } from "expression/functions";
 import { parseEval, simpleContext } from "test/common";
 
@@ -58,6 +59,14 @@ test("Evaluate reverse(list)", () => {
     expect(parseEval('reverse(list("a", "b", "c"))')).toEqual(parseEval('list("c", "b", "a")'));
 });
 
+// <-- flat() -->
+
+test("Evaluate flat()", () => {
+    expect(parseEval("flat(list(1, 2, 3, list(11, 12)))")).toEqual(parseEval("list(1,2,3,11,12)"));
+    expect(parseEval("flat(list(1, list(21, list(221, 222)), 3, list(11, 12)))")).toEqual(parseEval("list(1,21,list(221,222),3,11,12)")); // flat(...)
+    expect(parseEval("flat(list(1, list(2, list(3, list(4, list(5))))), 3)")).toEqual(parseEval("list(1,2,3,4,list(5))")); // flat(..., 3)
+    expect(parseEval("flat(list(1, list(2, list(3, list(4, list(5))))), 10)")).toEqual(parseEval("list(1,2,3,4,5)")); // flat(..., 10)
+})
 // <-- sort() -->
 
 describe("sort()", () => {
@@ -112,3 +121,14 @@ test("Evaluate nonnull()", () => {
     expect(DefaultFunctions.nonnull(simpleContext(), null, null, 1)).toEqual([1]);
     expect(DefaultFunctions.nonnull(simpleContext(), "yes")).toEqual(["yes"]);
 });
+
+// <-- date() -->
+
+test("Evaluate date()", () => {
+    expect(parseEval("date([[2020-04-18]])")).toEqual(DateTime.fromObject({ year: 2020, month: 4, day: 18 }));
+    expect(parseEval("date([[Place|2021-04]])")).toEqual(DateTime.fromObject({ year: 2021, month: 4, day: 1 }));
+    expect(parseEval('date("12/31/2022", "MM/dd/yyyy")')).toEqual(DateTime.fromObject({ year: 2022, month: 12, day: 31 }));
+    expect(parseEval('date("210313", "yyMMdd")')).toEqual(DateTime.fromObject({ year: 2021, month: 3, day: 13 }));
+    expect(parseEval('date("946778645012","x")')).toEqual(DateTime.fromMillis(946778645012)) 
+    expect(parseEval('date("946778645000","X")')).toEqual(DateTime.fromMillis(946778645000)) 
+});