CVE-2020-9038 (published 2020-02-17T16:15Z)

Joplin through 1.0.184 allows Arbitrary File Read via XSS.

Severity: LOW
CVSS v2 base score: 3.5 (AV:N/AC:M/Au:S/C:N/I:P/A:N)
CVSS v3 base score: 5.4, MEDIUM (CVSS:3.1/AV:N/AC:L/PR:L/UI:R/S:C/C:L/I:L/A:N)

Weakness information:
cwe_id: CWE-79
cwe_name: Improper Neutralization of Input During Web Page Generation ('Cross-site Scripting')
description: The product does not neutralize or incorrectly neutralizes user-controllable input before it is placed in output that is used as a web page that is served to other users.
extended_description: 
            Cross-site scripting (XSS) vulnerabilities occur when:
            
               Untrusted data enters a web application, typically from a web request.
               The web application dynamically generates a web page that contains this untrusted data.
               During page generation, the application does not prevent the data from containing content that is executable by a web browser, such as JavaScript, HTML tags, HTML attributes, mouse events, Flash, ActiveX, etc.
               A victim visits the generated web page through a web browser, which contains malicious script that was injected using the untrusted data.
               Since the script comes from a web page that was sent by the web server, the victim's web browser executes the malicious script in the context of the web server's domain.
               This effectively violates the intention of the web browser's same-origin policy, which states that scripts in one domain should not be able to access resources or run code in a different domain.
            
            There are three main kinds of XSS:
            
               
                  Type 1: Reflected XSS (or Non-Persistent) - 
         			The server reads data directly from the HTTP request and reflects it back in the HTTP response. Reflected XSS exploits occur when an attacker causes a victim to supply dangerous content to a vulnerable web application, which is then reflected back to the victim and executed by the web browser. The most common mechanism for delivering malicious content is to include it as a parameter in a URL that is posted publicly or e-mailed directly to the victim. URLs constructed in this manner constitute the core of many phishing schemes, whereby an attacker convinces a victim to visit a URL that refers to a vulnerable site. After the site reflects the attacker's content back to the victim, the content is executed by the victim's browser.
               
                  Type 2: Stored XSS (or Persistent) - 
               The application stores dangerous data in a database, message forum, visitor log, or other trusted data store. At a later time, the dangerous data is subsequently read back into the application and included in dynamic content. From an attacker's perspective, the optimal place to inject malicious content is in an area that is displayed to either many users or particularly interesting users. Interesting users typically have elevated privileges in the application or interact with sensitive data that is valuable to the attacker. If one of these users executes malicious content, the attacker may be able to perform privileged operations on behalf of the user or gain access to sensitive data belonging to the user. For example, the attacker might inject XSS into a log message, which might not be handled properly when an administrator views the logs.
            
               
                  Type 0: DOM-Based XSS - 
               In DOM-based XSS, the client performs the injection of XSS into the page; in the other types, the server performs the injection. DOM-based XSS generally involves server-controlled, trusted script that is sent to the client, such as Javascript that performs sanity checks on a form before the user submits it. If the server-supplied script processes user-supplied data and then injects it back into the web page (such as with dynamic HTML), then DOM-based XSS is possible.
            
            
            Once the malicious script is injected, the attacker can perform a variety of malicious activities. The attacker could transfer private information, such as cookies that may include session information, from the victim's machine to the attacker. The attacker could send malicious requests to a web site on behalf of the victim, which could be especially dangerous to the site if the victim has administrator privileges to manage that site. Phishing attacks could be used to emulate trusted web sites and trick the victim into entering a password, allowing the attacker to compromise the victim's account on that web site. Finally, the script could exploit a vulnerability in the web browser itself possibly taking over the victim's machine, sometimes referred to as "drive-by hacking."
            In many cases, the attack can be launched without the victim even being aware of it. Even with careful users, attackers frequently use a variety of methods to encode the malicious portion of the attack, such as URL encoding or Unicode, so the request looks less suspicious.
         
         

Fix information:
repo_url: https://github.com/laurent22/joplin
commit_details: repo_url: https://github.com/laurent22/joplin
author: Laurent Cozic
author_date: 2020-02-13 23:59:23+00:00
msg: All: Security: Fixed potential Arbitrary File Read via XSS
repository: repo_url: https://github.com/laurent22/joplin
repo_name: laurent22/joplin
description: Joplin - the secure note taking and to-do app with synchronisation capabilities for Windows, macOS, Linux, Android and iOS.
repo_language: TypeScript
owner: laurent22
file_changes: filename: .eslintignore
change_type: ModificationType.MODIFY
diff: @@ -55,3 +55,4 @@ ElectronClient/app/gui/ShareNoteDialog.js
 ReactNativeClient/lib/JoplinServerApi.js
 ReactNativeClient/PluginAssetsLoader.js
 ReactNativeClient/lib/joplin-renderer/MdToHtml/rules/mermaid.js
+ReactNativeClient/lib/joplin-renderer/MdToHtml/rules/sanitize_html.js

programming_language: Shell
filename: .gitignore
change_type: ModificationType.MODIFY
diff: @@ -51,3 +51,4 @@ ElectronClient/app/gui/ShareNoteDialog.js
 ReactNativeClient/lib/JoplinServerApi.js
 ReactNativeClient/PluginAssetsLoader.js
 ReactNativeClient/lib/joplin-renderer/MdToHtml/rules/mermaid.js
+ReactNativeClient/lib/joplin-renderer/MdToHtml/rules/sanitize_html.js

programming_language: Shell
filename: package-lock.json
change_type: ModificationType.MODIFY
diff: @@ -2667,6 +2667,11 @@
       "resolved": "https://registry.npmjs.org/mdurl/-/mdurl-1.0.1.tgz",
       "integrity": "sha1-/oWy7HWlkDfyrf7BAP1sYBdhFS4="
     },
+    "memory-cache": {
+      "version": "0.2.0",
+      "resolved": "https://registry.npmjs.org/memory-cache/-/memory-cache-0.2.0.tgz",
+      "integrity": "sha1-eJCwHVLADI68nVM+H46xfjA0hxo="
+    },
     "micromatch": {
       "version": "3.1.10",
       "resolved": "https://registry.npmjs.org/micromatch/-/micromatch-3.1.10.tgz",
@@ -2920,6 +2925,15 @@
         "is-stream": "^1.0.1"
       }
     },
+    "node-html-parser": {
+      "version": "1.2.4",
+      "resolved": "https://registry.npmjs.org/node-html-parser/-/node-html-parser-1.2.4.tgz",
+      "integrity": "sha512-qHwPdGyGr9pOZBoSgUOuNPG20QYZVN00lFcxKQgjPUODSxVH7obQeLVVawa3B4cfSNtLIeczSzoy/xYA8XG5WQ==",
+      "dev": true,
+      "requires": {
+        "he": "1.1.1"
+      }
+    },
     "node-persist": {
       "version": "2.1.0",
       "resolved": "https://registry.npmjs.org/node-persist/-/node-persist-2.1.0.tgz",

programming_language: JavaScript
filename: package.json
change_type: ModificationType.MODIFY
diff: @@ -72,6 +72,7 @@
     "markdown-it-toc-done-right": "^4.1.0",
     "md5": "^2.2.1",
     "md5-file": "^4.0.0",
+    "memory-cache": "^0.2.0",
     "mime": "^2.0.3",
     "moment": "^2.24.0",
     "multiparty": "^4.2.1",
@@ -104,7 +105,8 @@
     "valid-url": "^1.0.9",
     "word-wrap": "^1.2.3",
     "xml2js": "^0.4.19",
-    "yargs-parser": "^7.0.0"
+    "yargs-parser": "^7.0.0",
+    "node-html-parser": "^1.2.4"
   },
   "devDependencies": {
     "jasmine": "^3.5.0"

programming_language: Python
filename: HtmlToHtml.js
change_type: ModificationType.ADD
diff: @@ -0,0 +1,79 @@
+/* eslint-disable no-unused-vars */
+
+require('app-module-path').addPath(__dirname);
+
+const os = require('os');
+const { time } = require('lib/time-utils.js');
+const { filename } = require('lib/path-utils.js');
+const { asyncTest, fileContentEqual, setupDatabase, setupDatabaseAndSynchronizer, db, synchronizer, fileApi, sleep, clearDatabase, switchClient, syncTargetId, objectsEqual, checkThrowAsync } = require('test-utils.js');
+const Folder = require('lib/models/Folder.js');
+const Note = require('lib/models/Note.js');
+const BaseModel = require('lib/BaseModel.js');
+const { shim } = require('lib/shim');
+const HtmlToHtml = require('lib/joplin-renderer/HtmlToHtml');
+const { enexXmlToMd } = require('lib/import-enex-md-gen.js');
+
+jasmine.DEFAULT_TIMEOUT_INTERVAL = 60 * 60 * 1000; // Can run for a while since everything is in the same test unit
+
+process.on('unhandledRejection', (reason, p) => {
+	console.log('Unhandled Rejection at: Promise', p, 'reason:', reason);
+});
+
+describe('HtmlToHtml', function() {
+
+	beforeEach(async (done) => {
+		await setupDatabaseAndSynchronizer(1);
+		await switchClient(1);
+		done();
+	});
+
+	it('should convert from Html to Html', asyncTest(async () => {
+		const basePath = `${__dirname}/html_to_html`;
+		const files = await shim.fsDriver().readDirStats(basePath);
+		const htmlToHtml = new HtmlToHtml();
+
+		for (let i = 0; i < files.length; i++) {
+			const htmlSourceFilename = files[i].path;
+			if (htmlSourceFilename.indexOf('.src.html') < 0) continue;
+
+			const htmlSourceFilePath = `${basePath}/${htmlSourceFilename}`;
+			const htmlDestPath = `${basePath}/${filename(filename(htmlSourceFilePath))}.dest.html`;
+
+			// if (htmlSourceFilename !== 'table_with_header.html') continue;
+
+			const htmlToHtmlOptions = {
+				bodyOnly: true,
+			};
+
+			const sourceHtml = await shim.fsDriver().readFile(htmlSourceFilePath);
+			let expectedHtml = await shim.fsDriver().readFile(htmlDestPath);
+
+			const result = await htmlToHtml.render(sourceHtml, null, htmlToHtmlOptions);
+			let actualHtml = result.html;
+
+			if (os.EOL === '\r\n') {
+				expectedHtml = expectedHtml.replace(/\r\n/g, '\n');
+				actualHtml = actualHtml.replace(/\r\n/g, '\n');
+			}
+
+			if (actualHtml !== expectedHtml) {
+				console.info('');
+				console.info(`Error converting file: ${htmlSourceFilename}`);
+				console.info('--------------------------------- Got:');
+				console.info(actualHtml);
+				console.info('--------------------------------- Raw:');
+				console.info(actualHtml.split('\n'));
+				console.info('--------------------------------- Expected:');
+				console.info(expectedHtml.split('\n'));
+				console.info('--------------------------------------------');
+				console.info('');
+
+				expect(false).toBe(true);
+				// return;
+			} else {
+				expect(true).toBe(true);
+			}
+		}
+	}));
+
+});

programming_language: JavaScript
filename: MdToHtml.js
change_type: ModificationType.ADD
diff: @@ -0,0 +1,79 @@
+/* eslint-disable no-unused-vars */
+
+require('app-module-path').addPath(__dirname);
+
+const os = require('os');
+const { time } = require('lib/time-utils.js');
+const { filename } = require('lib/path-utils.js');
+const { asyncTest, fileContentEqual, setupDatabase, setupDatabaseAndSynchronizer, db, synchronizer, fileApi, sleep, clearDatabase, switchClient, syncTargetId, objectsEqual, checkThrowAsync } = require('test-utils.js');
+const Folder = require('lib/models/Folder.js');
+const Note = require('lib/models/Note.js');
+const BaseModel = require('lib/BaseModel.js');
+const { shim } = require('lib/shim');
+const MdToHtml = require('lib/joplin-renderer/MdToHtml');
+const { enexXmlToMd } = require('lib/import-enex-md-gen.js');
+
+jasmine.DEFAULT_TIMEOUT_INTERVAL = 60 * 60 * 1000; // Can run for a while since everything is in the same test unit
+
+process.on('unhandledRejection', (reason, p) => {
+	console.log('Unhandled Rejection at: Promise', p, 'reason:', reason);
+});
+
+describe('MdToHtml', function() {
+
+	beforeEach(async (done) => {
+		await setupDatabaseAndSynchronizer(1);
+		await switchClient(1);
+		done();
+	});
+
+	it('should convert from Markdown to Html', asyncTest(async () => {
+		const basePath = `${__dirname}/md_to_html`;
+		const files = await shim.fsDriver().readDirStats(basePath);
+		const mdToHtml = new MdToHtml();
+
+		for (let i = 0; i < files.length; i++) {
+			const mdFilename = files[i].path;
+			if (mdFilename.indexOf('.md') < 0) continue;
+
+			const mdFilePath = `${basePath}/${mdFilename}`;
+			const htmlPath = `${basePath}/${filename(mdFilePath)}.html`;
+
+			// if (mdFilename !== 'table_with_header.html') continue;
+
+			const mdToHtmlOptions = {
+				bodyOnly: true,
+			};
+
+			const markdown = await shim.fsDriver().readFile(mdFilePath);
+			let expectedHtml = await shim.fsDriver().readFile(htmlPath);
+
+			const result = await mdToHtml.render(markdown, null, mdToHtmlOptions);
+			let actualHtml = result.html;
+
+			if (os.EOL === '\r\n') {
+				expectedHtml = expectedHtml.replace(/\r\n/g, '\n');
+				actualHtml = actualHtml.replace(/\r\n/g, '\n');
+			}
+
+			if (actualHtml !== expectedHtml) {
+				console.info('');
+				console.info(`Error converting file: ${mdFilename}`);
+				console.info('--------------------------------- Got:');
+				console.info(actualHtml);
+				console.info('--------------------------------- Raw:');
+				console.info(actualHtml.split('\n'));
+				console.info('--------------------------------- Expected:');
+				console.info(expectedHtml.split('\n'));
+				console.info('--------------------------------------------');
+				console.info('');
+
+				expect(false).toBe(true);
+				// return;
+			} else {
+				expect(true).toBe(true);
+			}
+		}
+	}));
+
+});

programming_language: JavaScript
filename: sanitize.dest.html
change_type: ModificationType.ADD
diff: @@ -0,0 +1,2 @@
+<img src onerror="" />
+<img src onerror="" />
\ No newline at end of file

programming_language: Batchfile
filename: sanitize.src.html
change_type: ModificationType.ADD
diff: @@ -0,0 +1,3 @@
+<img src="" onerror="alert('ohno')"/>
+<img src=""
+   onerror="alert('ohno')"/>
\ No newline at end of file

programming_language: CoffeeScript
filename: sanitize.html
change_type: ModificationType.ADD
diff: @@ -0,0 +1,2 @@
+<img src onerror="" />
+<img src onerror="" />
\ No newline at end of file

programming_language: Batchfile
filename: sanitize.md
change_type: ModificationType.ADD
diff: @@ -0,0 +1,3 @@
+<img src="" onerror="alert('ohno')"/>
+<img src=""
+   onerror="alert('ohno')"/>
\ No newline at end of file

programming_language: CoffeeScript
filename: index.js
change_type: ModificationType.MODIFY
diff: @@ -21,7 +21,7 @@
 	function absoluteUrl(url) {
 		if (!url) return url;
 		const protocol = url.toLowerCase().split(':')[0];
-		if (['http', 'https', 'file'].indexOf(protocol) >= 0) return url;
+		if (['http', 'https', 'file', 'data'].indexOf(protocol) >= 0) return url;
 
 		if (url.indexOf('//') === 0) {
 			return location.protocol + url;

programming_language: JavaScript
filename: package-lock.json
change_type: ModificationType.MODIFY
diff: @@ -2233,10 +2233,68 @@
         "minimist": "^1.1.1"
       },
       "dependencies": {
+        "domelementtype": {
+          "version": "1.3.1",
+          "resolved": "https://registry.npmjs.org/domelementtype/-/domelementtype-1.3.1.tgz",
+          "integrity": "sha512-BSKB+TSpMpFI/HOxCNr1O8aMOTZ8hT3pM3GQ0w/mWRmkhEDSFJkkyzz4XQsBV44BChwGkrDfMyjVD0eA2aFV3w=="
+        },
+        "domhandler": {
+          "version": "2.4.2",
+          "resolved": "https://registry.npmjs.org/domhandler/-/domhandler-2.4.2.tgz",
+          "integrity": "sha512-JiK04h0Ht5u/80fdLMCEmV4zkNh2BcoMFBmZ/91WtYZ8qVXSKjiw7fXMgFPnHcSZgOo3XdinHvmnDUeMf5R4wA==",
+          "requires": {
+            "domelementtype": "1"
+          }
+        },
+        "domutils": {
+          "version": "1.7.0",
+          "resolved": "https://registry.npmjs.org/domutils/-/domutils-1.7.0.tgz",
+          "integrity": "sha512-Lgd2XcJ/NjEw+7tFvfKxOzCYKZsdct5lczQ2ZaQY8Djz7pfAD3Gbp8ySJWtreII/vDlMVmxwa6pHmdxIYgttDg==",
+          "requires": {
+            "dom-serializer": "0",
+            "domelementtype": "1"
+          }
+        },
+        "htmlparser2": {
+          "version": "3.10.1",
+          "resolved": "https://registry.npmjs.org/htmlparser2/-/htmlparser2-3.10.1.tgz",
+          "integrity": "sha512-IgieNijUMbkDovyoKObU1DUhm1iwNYE/fuifEoEHfd1oZKZDaONBSkal7Y01shxsM49R4XaMdGez3WnF9UfiCQ==",
+          "requires": {
+            "domelementtype": "^1.3.1",
+            "domhandler": "^2.3.0",
+            "domutils": "^1.5.1",
+            "entities": "^1.1.1",
+            "inherits": "^2.0.1",
+            "readable-stream": "^3.1.1"
+          }
+        },
         "minimist": {
           "version": "1.2.0",
           "resolved": "https://registry.npmjs.org/minimist/-/minimist-1.2.0.tgz",
           "integrity": "sha1-o1AIsg9BOD7sH7kU9M1d95omQoQ="
+        },
+        "readable-stream": {
+          "version": "3.6.0",
+          "resolved": "https://registry.npmjs.org/readable-stream/-/readable-stream-3.6.0.tgz",
+          "integrity": "sha512-BViHy7LKeTz4oNnkcLJ+lVSL6vpiFeX6/d3oSH8zCW7UxP2onchk+vTGB143xuFjHS3deTgkKoXXymXqymiIdA==",
+          "requires": {
+            "inherits": "^2.0.3",
+            "string_decoder": "^1.1.1",
+            "util-deprecate": "^1.0.1"
+          }
+        },
+        "safe-buffer": {
+          "version": "5.2.0",
+          "resolved": "https://registry.npmjs.org/safe-buffer/-/safe-buffer-5.2.0.tgz",
+          "integrity": "sha512-fZEwUGbVl7kouZs1jCdMLdt95hdIv0ZeHg6L7qPeciMZhZ+/gdesW4wgTARkrFWEpspjEATAzUGPG8N2jJiwbg=="
+        },
+        "string_decoder": {
+          "version": "1.3.0",
+          "resolved": "https://registry.npmjs.org/string_decoder/-/string_decoder-1.3.0.tgz",
+          "integrity": "sha512-hkRX8U1WjJFd8LsDJ2yQ/wWWxaopEsABU1XfkM8A+j0+85JAGppt16cr1Whg6KIbb4okU6Mql6BOj+uup/wKeA==",
+          "requires": {
+            "safe-buffer": "~5.2.0"
+          }
         }
       }
     },
@@ -3158,19 +3216,14 @@
       }
     },
     "dom-serializer": {
-      "version": "0.2.1",
-      "resolved": "https://registry.npmjs.org/dom-serializer/-/dom-serializer-0.2.1.tgz",
-      "integrity": "sha512-sK3ujri04WyjwQXVoK4PU3y8ula1stq10GJZpqHIUgoGZdsGzAGu65BnU3d08aTVSvO7mGPZUc0wTEDL+qGE0Q==",
+      "version": "0.2.2",
+      "resolved": "https://registry.npmjs.org/dom-serializer/-/dom-serializer-0.2.2.tgz",
+      "integrity": "sha512-2/xPb3ORsQ42nHYiSunXkDjPLBaEj/xTwUO4B7XCZQTRk7EBtTOPaygh10YAAh2OI1Qrp6NWfpAhzswj0ydt9g==",
       "requires": {
         "domelementtype": "^2.0.1",
         "entities": "^2.0.0"
       },
       "dependencies": {
-        "domelementtype": {
-          "version": "2.0.1",
-          "resolved": "https://registry.npmjs.org/domelementtype/-/domelementtype-2.0.1.tgz",
-          "integrity": "sha512-5HOHUDsYZWV8FGWN0Njbr/Rn7f/eWSQi1v7+HsUVwXgn8nWWlL64zKDkS0n8ZmQ3mlWOMuXOnR+7Nx/5tMO5AQ=="
-        },
         "entities": {
           "version": "2.0.0",
           "resolved": "https://registry.npmjs.org/entities/-/entities-2.0.0.tgz",
@@ -3179,9 +3232,9 @@
       }
     },
     "domelementtype": {
-      "version": "1.3.1",
-      "resolved": "https://registry.npmjs.org/domelementtype/-/domelementtype-1.3.1.tgz",
-      "integrity": "sha512-BSKB+TSpMpFI/HOxCNr1O8aMOTZ8hT3pM3GQ0w/mWRmkhEDSFJkkyzz4XQsBV44BChwGkrDfMyjVD0eA2aFV3w=="
+      "version": "2.0.1",
+      "resolved": "https://registry.npmjs.org/domelementtype/-/domelementtype-2.0.1.tgz",
+      "integrity": "sha512-5HOHUDsYZWV8FGWN0Njbr/Rn7f/eWSQi1v7+HsUVwXgn8nWWlL64zKDkS0n8ZmQ3mlWOMuXOnR+7Nx/5tMO5AQ=="
     },
     "domexception": {
       "version": "1.0.1",
@@ -3192,20 +3245,21 @@
       }
     },
     "domhandler": {
-      "version": "2.4.2",
-      "resolved": "https://registry.npmjs.org/domhandler/-/domhandler-2.4.2.tgz",
-      "integrity": "sha512-JiK04h0Ht5u/80fdLMCEmV4zkNh2BcoMFBmZ/91WtYZ8qVXSKjiw7fXMgFPnHcSZgOo3XdinHvmnDUeMf5R4wA==",
+      "version": "3.0.0",
+      "resolved": "https://registry.npmjs.org/domhandler/-/domhandler-3.0.0.tgz",
+      "integrity": "sha512-eKLdI5v9m67kbXQbJSNn1zjh0SDzvzWVWtX+qEI3eMjZw8daH9k8rlj1FZY9memPwjiskQFbe7vHVVJIAqoEhw==",
       "requires": {
-        "domelementtype": "1"
+        "domelementtype": "^2.0.1"
       }
     },
     "domutils": {
-      "version": "1.7.0",
-      "resolved": "https://registry.npmjs.org/domutils/-/domutils-1.7.0.tgz",
-      "integrity": "sha512-Lgd2XcJ/NjEw+7tFvfKxOzCYKZsdct5lczQ2ZaQY8Djz7pfAD3Gbp8ySJWtreII/vDlMVmxwa6pHmdxIYgttDg==",
+      "version": "2.0.0",
+      "resolved": "https://registry.npmjs.org/domutils/-/domutils-2.0.0.tgz",
+      "integrity": "sha512-n5SelJ1axbO636c2yUtOGia/IcJtVtlhQbFiVDBZHKV5ReJO1ViX7sFEemtuyoAnBxk5meNSYgA8V4s0271efg==",
       "requires": {
-        "dom-serializer": "0",
-        "domelementtype": "1"
+        "dom-serializer": "^0.2.1",
+        "domelementtype": "^2.0.1",
+        "domhandler": "^3.0.0"
       }
     },
     "dot-prop": {
@@ -4640,40 +4694,20 @@
       }
     },
     "htmlparser2": {
-      "version": "3.10.1",
-      "resolved": "https://registry.npmjs.org/htmlparser2/-/htmlparser2-3.10.1.tgz",
-      "integrity": "sha512-IgieNijUMbkDovyoKObU1DUhm1iwNYE/fuifEoEHfd1oZKZDaONBSkal7Y01shxsM49R4XaMdGez3WnF9UfiCQ==",
-      "requires": {
-        "domelementtype": "^1.3.1",
-        "domhandler": "^2.3.0",
-        "domutils": "^1.5.1",
-        "entities": "^1.1.1",
-        "inherits": "^2.0.1",
-        "readable-stream": "^3.1.1"
+      "version": "4.0.0",
+      "resolved": "https://registry.npmjs.org/htmlparser2/-/htmlparser2-4.0.0.tgz",
+      "integrity": "sha512-cChwXn5Vam57fyXajDtPXL1wTYc8JtLbr2TN76FYu05itVVVealxLowe2B3IEznJG4p9HAYn/0tJaRlGuEglFQ==",
+      "requires": {
+        "domelementtype": "^2.0.1",
+        "domhandler": "^3.0.0",
+        "domutils": "^2.0.0",
+        "entities": "^2.0.0"
       },
       "dependencies": {
-        "readable-stream": {
-          "version": "3.4.0",
-          "resolved": "https://registry.npmjs.org/readable-stream/-/readable-stream-3.4.0.tgz",
-          "integrity": "sha512-jItXPLmrSR8jmTRmRWJXCnGJsfy85mB3Wd/uINMXA65yrnFo0cPClFIUWzo2najVNSl+mx7/4W8ttlLWJe99pQ==",
-          "requires": {
-            "inherits": "^2.0.3",
-            "string_decoder": "^1.1.1",
-            "util-deprecate": "^1.0.1"
-          }
-        },
-        "safe-buffer": {
-          "version": "5.2.0",
-          "resolved": "https://registry.npmjs.org/safe-buffer/-/safe-buffer-5.2.0.tgz",
-          "integrity": "sha512-fZEwUGbVl7kouZs1jCdMLdt95hdIv0ZeHg6L7qPeciMZhZ+/gdesW4wgTARkrFWEpspjEATAzUGPG8N2jJiwbg=="
-        },
-        "string_decoder": {
-          "version": "1.3.0",
-          "resolved": "https://registry.npmjs.org/string_decoder/-/string_decoder-1.3.0.tgz",
-          "integrity": "sha512-hkRX8U1WjJFd8LsDJ2yQ/wWWxaopEsABU1XfkM8A+j0+85JAGppt16cr1Whg6KIbb4okU6Mql6BOj+uup/wKeA==",
-          "requires": {
-            "safe-buffer": "~5.2.0"
-          }
+        "entities": {
+          "version": "2.0.0",
+          "resolved": "https://registry.npmjs.org/entities/-/entities-2.0.0.tgz",
+          "integrity": "sha512-D9f7V0JSRwIxlRI2mjMqufDrRDnx8p+eEOz7aUM9SuvF8gsBzra0/6tbjl1m8eQHrZlYj6PxqE00hZ1SAIKPLw=="
         }
       }
     },
@@ -5582,6 +5616,11 @@
       "resolved": "https://registry.npmjs.org/memoize-one/-/memoize-one-5.0.4.tgz",
       "integrity": "sha512-P0z5IeAH6qHHGkJIXWw0xC2HNEgkx/9uWWBQw64FJj3/ol14VYdfVGWWr0fXfjhhv3TKVIqUq65os6O4GUNksA=="
     },
+    "memory-cache": {
+      "version": "0.2.0",
+      "resolved": "https://registry.npmjs.org/memory-cache/-/memory-cache-0.2.0.tgz",
+      "integrity": "sha1-eJCwHVLADI68nVM+H46xfjA0hxo="
+    },
     "mermaid": {
       "version": "8.4.6",
       "resolved": "https://registry.npmjs.org/mermaid/-/mermaid-8.4.6.tgz",
@@ -6183,6 +6222,21 @@
         }
       }
     },
+    "node-html-parser": {
+      "version": "1.2.4",
+      "resolved": "https://registry.npmjs.org/node-html-parser/-/node-html-parser-1.2.4.tgz",
+      "integrity": "sha512-qHwPdGyGr9pOZBoSgUOuNPG20QYZVN00lFcxKQgjPUODSxVH7obQeLVVawa3B4cfSNtLIeczSzoy/xYA8XG5WQ==",
+      "requires": {
+        "he": "1.1.1"
+      },
+      "dependencies": {
+        "he": {
+          "version": "1.1.1",
+          "resolved": "https://registry.npmjs.org/he/-/he-1.1.1.tgz",
+          "integrity": "sha1-k0EP0hsAlzUVH4howvJx80J+I/0="
+        }
+      }
+    },
     "node-notifier": {
       "version": "6.0.0",
       "resolved": "https://registry.npmjs.org/node-notifier/-/node-notifier-6.0.0.tgz",

programming_language: TypeScript
filename: package.json
change_type: ModificationType.MODIFY
diff: @@ -131,11 +131,13 @@
     "markdown-it-toc-done-right": "^4.1.0",
     "md5": "^2.2.1",
     "md5-file": "^4.0.0",
+    "memory-cache": "^0.2.0",
     "mermaid": "^8.4.6",
     "moment": "^2.22.2",
     "multiparty": "^4.2.1",
     "mustache": "^3.0.1",
     "node-fetch": "^1.7.3",
+    "node-html-parser": "^1.2.4",
     "node-notifier": "^6.0.0",
     "promise": "^8.0.1",
     "query-string": "^5.1.1",

programming_language: JavaScript
filename: build.sh
change_type: ModificationType.MODIFY
diff: @@ -5,8 +5,8 @@ BUILD_DIR="$ROOT_DIR/app"
 
 rsync -a --delete "$ROOT_DIR/../ReactNativeClient/lib/" "$BUILD_DIR/lib/"
 
-cd "$ROOT_DIR/.."
-npm run tsc
+# cd "$ROOT_DIR/.."
+# npm run tsc
 
 cd "$BUILD_DIR"
 npm run compile

programming_language: Shell
filename: BaseApplication.js
change_type: ModificationType.MODIFY
diff: @@ -539,27 +539,6 @@ class BaseApplication {
 		return `${os.homedir()}/.config/${Setting.value('appName')}`;
 	}
 
-	async testing() {
-		const markdownUtils = require('lib/markdownUtils');
-		const ClipperServer = require('lib/ClipperServer');
-		const server = new ClipperServer();
-		const HtmlToMd = require('lib/HtmlToMd');
-		const service = new HtmlToMd();
-		const html = await shim.fsDriver().readFile('/mnt/d/test.html');
-		let markdown = service.parse(html, { baseUrl: 'https://duckduckgo.com/' });
-		console.info(markdown);
-		console.info('--------------------------------------------------');
-
-		const imageUrls = markdownUtils.extractImageUrls(markdown);
-		let result = await server.downloadImages_(imageUrls);
-		result = await server.createResourcesFromPaths_(result);
-		console.info(result);
-		markdown = server.replaceImageUrlsByResources_(markdown, result);
-		console.info('--------------------------------------------------');
-		console.info(markdown);
-		console.info('--------------------------------------------------');
-	}
-
 	async start(argv) {
 		let startFlags = await this.handleStartFlags_(argv);
 

programming_language: JavaScript
filename: HtmlToHtml.js
change_type: ModificationType.MODIFY
diff: @@ -1,33 +1,50 @@
 const htmlUtils = require('./htmlUtils');
 const utils = require('./utils');
 const noteStyle = require('./noteStyle');
+const memoryCache = require('memory-cache');
+const md5 = require('md5');
 
 class HtmlToHtml {
 	constructor(options) {
 		if (!options) options = {};
 		this.resourceBaseUrl_ = 'resourceBaseUrl' in options ? options.resourceBaseUrl : null;
 		this.ResourceModel_ = options.ResourceModel;
+		this.cache_ = new memoryCache.Cache();
 	}
 
-	render(markup, theme, options) {
-		const html = htmlUtils.processImageTags(markup, data => {
-			if (!data.src) return null;
-
-			const r = utils.imageReplacement(this.ResourceModel_, data.src, options.resources, this.resourceBaseUrl_);
-			if (!r) return null;
-
-			if (typeof r === 'string') {
-				return {
-					type: 'replaceElement',
-					html: r,
-				};
-			} else {
-				return {
-					type: 'setAttributes',
-					attrs: r,
-				};
-			}
-		});
+	async render(markup, theme, options) {
+		const cacheKey = md5(escape(markup));
+		let html = this.cache_.get(cacheKey);
+
+		if (!html) {
+			html = htmlUtils.sanitizeHtml(markup);
+
+			html = htmlUtils.processImageTags(html, data => {
+				if (!data.src) return null;
+
+				const r = utils.imageReplacement(this.ResourceModel_, data.src, options.resources, this.resourceBaseUrl_);
+				if (!r) return null;
+
+				if (typeof r === 'string') {
+					return {
+						type: 'replaceElement',
+						html: r,
+					};
+				} else {
+					return {
+						type: 'setAttributes',
+						attrs: r,
+					};
+				}
+			});
+		}
+
+		if (options.bodyOnly) return {
+			html: html,
+			pluginAssets: [],
+		};
+
+		this.cache_.put(cacheKey, html, 1000 * 60 * 10);
 
 		const cssStrings = noteStyle(theme, options);
 		const styleHtml = `<style>${cssStrings.join('\n')}</style>`;

programming_language: JavaScript
filename: MarkupToHtml.js
change_type: ModificationType.MODIFY
diff: @@ -33,7 +33,7 @@ class MarkupToHtml {
 		return '';
 	}
 
-	render(markupLanguage, markup, theme, options) {
+	async render(markupLanguage, markup, theme, options) {
 		return this.renderer(markupLanguage).render(markup, theme, options);
 	}
 }

programming_language: JavaScript
filename: MdToHtml.js
change_type: ModificationType.MODIFY
diff: @@ -2,6 +2,7 @@ const MarkdownIt = require('markdown-it');
 const md5 = require('md5');
 const noteStyle = require('./noteStyle');
 const { fileExtension } = require('./pathUtils');
+const memoryCache = require('memory-cache');
 const rules = {
 	image: require('./MdToHtml/rules/image'),
 	checkbox: require('./MdToHtml/rules/checkbox'),
@@ -12,6 +13,7 @@ const rules = {
 	code_inline: require('./MdToHtml/rules/code_inline'),
 	fountain: require('./MdToHtml/rules/fountain'),
 	mermaid: require('./MdToHtml/rules/mermaid').default,
+	sanitize_html: require('./MdToHtml/rules/sanitize_html').default,
 };
 const setupLinkify = require('./MdToHtml/setupLinkify');
 const hljs = require('highlight.js');
@@ -50,6 +52,7 @@ class MdToHtml {
 		this.cachedHighlightedCode_ = {};
 		this.ResourceModel_ = options.ResourceModel;
 		this.pluginOptions_ = options.pluginOptions ? options.pluginOptions : {};
+		this.contextCache_ = new memoryCache.Cache();
 	}
 
 	pluginOptions(name) {
@@ -106,6 +109,7 @@ class MdToHtml {
 
 	async render(body, style = null, options = null) {
 		if (!options) options = {};
+		if (!('bodyOnly' in options)) options.bodyOnly = false;
 		if (!options.postMessageSyntax) options.postMessageSyntax = 'postMessage';
 		if (!options.paddingBottom) options.paddingBottom = '0';
 		if (!options.highlightedKeywords) options.highlightedKeywords = [];
@@ -129,6 +133,7 @@ class MdToHtml {
 		const context = {
 			css: {},
 			pluginAssets: {},
+			cache: this.contextCache_,
 		};
 
 		const ruleOptions = Object.assign({}, options, {
@@ -203,6 +208,7 @@ class MdToHtml {
 		if (this.pluginEnabled('katex')) markdownIt.use(rules.katex(context, ruleOptions));
 		if (this.pluginEnabled('fountain')) markdownIt.use(rules.fountain(context, ruleOptions));
 		if (this.pluginEnabled('mermaid')) markdownIt.use(rules.mermaid(context, ruleOptions));
+		markdownIt.use(rules.sanitize_html(context, ruleOptions));
 		markdownIt.use(rules.highlight_keywords(context, ruleOptions));
 		markdownIt.use(rules.code_inline(context, ruleOptions));
 		markdownIt.use(markdownItAnchor, { slugify: uslugify });

programming_language: JavaScript
filename: sanitize_html.ts
change_type: ModificationType.ADD
diff: @@ -0,0 +1,40 @@
+const md5 = require('md5');
+const htmlUtils = require('../../htmlUtils');
+
+// @ts-ignore: Keep the function signature as-is despite unusued arguments
+function installRule(markdownIt:any, mdOptions:any, ruleOptions:any, context:any) {
+	markdownIt.core.ruler.push('sanitize_html', (state:any) => {
+		const tokens = state.tokens;
+
+		const walkHtmlTokens = (tokens:any[]) => {
+			if (!tokens || !tokens.length) return;
+
+			for (const token of tokens) {
+				if (!['html_block', 'html_inline'].includes(token.type)) {
+					walkHtmlTokens(token.children);
+					continue;
+				}
+
+				const cacheKey = md5(escape(token.content));
+				let sanitizedContent = context.cache.get(cacheKey);
+
+				if (!sanitizedContent) {
+					sanitizedContent = htmlUtils.sanitizeHtml(token.content);
+				}
+
+				token.content = sanitizedContent;
+
+				context.cache.put(cacheKey, sanitizedContent, 1000 * 60 * 60);
+				walkHtmlTokens(token.children);
+			}
+		};
+
+		walkHtmlTokens(tokens);
+	});
+}
+
+export default function(context:any, ruleOptions:any) {
+	return function(md:any, mdOptions:any) {
+		installRule(md, mdOptions, ruleOptions, context);
+	};
+}

programming_language: TypeScript
filename: htmlUtils.js
change_type: ModificationType.MODIFY
diff: @@ -2,8 +2,11 @@ const Entities = require('html-entities').AllHtmlEntities;
 const htmlentities = new Entities().encode;
 
 // [\s\S] instead of . for multiline matching
+const NodeHtmlParser = require('node-html-parser');
+
 // https://stackoverflow.com/a/16119722/561309
 const imageRegex = /<img([\s\S]*?)src=["']([\s\S]*?)["']([\s\S]*?)>/gi;
+const JS_EVENT_NAMES = ['onabort', 'onafterprint', 'onbeforeprint', 'onbeforeunload', 'onblur', 'oncanplay', 'oncanplaythrough', 'onchange', 'onclick', 'oncontextmenu', 'oncopy', 'oncuechange', 'oncut', 'ondblclick', 'ondrag', 'ondragend', 'ondragenter', 'ondragleave', 'ondragover', 'ondragstart', 'ondrop', 'ondurationchange', 'onemptied', 'onended', 'onerror', 'onfocus', 'onhashchange', 'oninput', 'oninvalid', 'onkeydown', 'onkeypress', 'onkeyup', 'onload', 'onloadeddata', 'onloadedmetadata', 'onloadstart', 'onmessage', 'onmousedown', 'onmousemove', 'onmouseout', 'onmouseover', 'onmouseup', 'onmousewheel', 'onoffline', 'ononline', 'onpagehide', 'onpageshow', 'onpaste', 'onpause', 'onplay', 'onplaying', 'onpopstate', 'onprogress', 'onratechange', 'onreset', 'onresize', 'onscroll', 'onsearch', 'onseeked', 'onseeking', 'onselect', 'onstalled', 'onstorage', 'onsubmit', 'onsuspend', 'ontimeupdate', 'ontoggle', 'onunload', 'onvolumechange', 'onwaiting', 'onwheel'];
 
 class HtmlUtils {
 
@@ -43,6 +46,34 @@ class HtmlUtils {
 		});
 	}
 
+	sanitizeHtml(html) {
+		const walkHtmlNodes = (nodes) => {
+			if (!nodes || !nodes.length) return;
+
+			for (const node of nodes) {
+				for (const attr in node.attributes) {
+					if (!node.attributes.hasOwnProperty(attr)) continue;
+					if (JS_EVENT_NAMES.includes(attr)) node.setAttribute(attr, '');
+				}
+				walkHtmlNodes(node.childNodes);
+			}
+		};
+
+		// Need to wrap in div, otherwise elements at the root will be skipped
+		// The DIV tags are removed below
+		const dom = NodeHtmlParser.parse(`<div>${html}</div>`, {
+			script: false,
+			style: true,
+			pre: true,
+			comment: false,
+		});
+
+		walkHtmlNodes([dom]);
+		const output = dom.toString();
+		return output.substr(5, output.length - 11);
+	}
+
+
 }
 
 const htmlUtils = new HtmlUtils();

programming_language: JavaScript
filename: package.json
change_type: ModificationType.MODIFY
diff: @@ -37,6 +37,8 @@
     "markdown-it-toc-done-right": "^4.1.0",
     "md5": "^2.2.1",
     "mermaid": "^8.4.6",
+    "memory-cache": "^0.2.0",
+    "node-html-parser": "^1.2.4",
     "uslug": "^1.0.4"
   }
 }

programming_language: JavaScript
filename: shim-init-node.js
change_type: ModificationType.MODIFY
diff: @@ -214,7 +214,7 @@ function shimInit() {
 		if (shim.isElectron()) {
 			const nativeImage = require('electron').nativeImage;
 			let image = nativeImage.createFromDataURL(imageDataUrl);
-			if (image.isEmpty()) throw new Error('Could not convert data URL to image'); // Would throw for example if the image format is no supported (eg. image/gif)
+			if (image.isEmpty()) throw new Error('Could not convert data URL to image - perhaps the format is not supported (eg. image/gif)'); // Would throw for example if the image format is no supported (eg. image/gif)
 			if (options.cropRect) {
 				// Crop rectangle values need to be rounded or the crop() call will fail
 				const c = options.cropRect;

programming_language: JavaScript
filename: package-lock.json
change_type: ModificationType.MODIFY
diff: @@ -5689,6 +5689,11 @@
         "mimic-fn": "^1.0.0"
       }
     },
+    "memory-cache": {
+      "version": "0.2.0",
+      "resolved": "https://registry.npmjs.org/memory-cache/-/memory-cache-0.2.0.tgz",
+      "integrity": "sha1-eJCwHVLADI68nVM+H46xfjA0hxo="
+    },
     "merge-stream": {
       "version": "1.0.1",
       "resolved": "https://registry.npmjs.org/merge-stream/-/merge-stream-1.0.1.tgz",
@@ -6380,6 +6385,21 @@
         "is-stream": "^1.0.1"
       }
     },
+    "node-html-parser": {
+      "version": "1.2.4",
+      "resolved": "https://registry.npmjs.org/node-html-parser/-/node-html-parser-1.2.4.tgz",
+      "integrity": "sha512-qHwPdGyGr9pOZBoSgUOuNPG20QYZVN00lFcxKQgjPUODSxVH7obQeLVVawa3B4cfSNtLIeczSzoy/xYA8XG5WQ==",
+      "requires": {
+        "he": "1.1.1"
+      },
+      "dependencies": {
+        "he": {
+          "version": "1.1.1",
+          "resolved": "https://registry.npmjs.org/he/-/he-1.1.1.tgz",
+          "integrity": "sha1-k0EP0hsAlzUVH4howvJx80J+I/0="
+        }
+      }
+    },
     "node-int64": {
       "version": "0.4.0",
       "resolved": "https://registry.npmjs.org/node-int64/-/node-int64-0.4.0.tgz",

programming_language: JavaScript
filename: package.json
change_type: ModificationType.MODIFY
diff: @@ -72,6 +72,8 @@
     "react-native-version-info": "^0.5.1",
     "react-native-webview": "^5.12.0",
     "react-redux": "5.0.7",
+    "memory-cache": "^0.2.0",
+    "node-html-parser": "^1.2.4",
     "redux": "4.0.0",
     "reselect": "^4.0.0",
     "rn-fetch-blob": "^0.12.0",

programming_language: JavaScript