CVE-2024-0763 (published 2024-02-27T22:15Z)

Any user can delete an arbitrary folder (recursively) on a remote server due to bad input sanitization leading to path traversal. The attacker would need access to the server at some privilege level since this endpoint is protected and requires authorization.


Weakness information:
cwe_id: CWE-20
cwe_name: Improper Input Validation
description: The product receives input or data, but it does
        not validate or incorrectly validates that the input has the
        properties that are required to process the data safely and
        correctly.
extended_description: 
	   Input validation is a frequently-used technique
	   for checking potentially dangerous inputs in order to
	   ensure that the inputs are safe for processing within the
	   code, or when communicating with other components.  When
	   software does not validate input properly, an attacker is
	   able to craft the input in a form that is not expected by
	   the rest of the application. This will lead to parts of the
	   system receiving unintended input, which may result in
	   altered control flow, arbitrary control of a resource, or
	   arbitrary code execution.
	   Input validation is not the only technique for
	   processing input, however.  Other techniques attempt to
	   transform potentially-dangerous input into something safe, such
	   as filtering (CWE-790) - which attempts to remove dangerous
	   inputs - or encoding/escaping (CWE-116), which attempts to
	   ensure that the input is not misinterpreted when it is included
	   in output to another component. Other techniques exist as well
	   (see CWE-138 for more examples.)
	   Input validation can be applied to:
	   
	     raw data - strings, numbers, parameters, file contents, etc.
	     metadata - information about the raw data, such as headers or size
	   
	   Data can be simple or structured.  Structured data
	   can be composed of many nested layers, composed of
	   combinations of metadata and raw data, with other simple or
	   structured data.
	   Many properties of raw data or metadata may need
	   to be validated upon entry into the code, such
	   as:
	   
	     specified quantities such as size, length, frequency, price, rate, number of operations, time, etc.
	     implied or derived quantities, such as the actual size of a file instead of a specified size
	     indexes, offsets, or positions into more complex data structures
	     symbolic keys or other elements into hash tables, associative arrays, etc.
	     well-formedness, i.e. syntactic correctness - compliance with expected syntax 
	     lexical token correctness - compliance with rules for what is treated as a token
	     specified or derived type - the actual type of the input (or what the input appears to be)
	     consistency - between individual data elements, between raw data and metadata, between references, etc.
	     conformance to domain-specific rules, e.g. business logic 
	     equivalence - ensuring that equivalent inputs are treated the same
	     authenticity, ownership, or other attestations about the input, e.g. a cryptographic signature to prove the source of the data
	   
	   Implied or derived properties of data must often
	   be calculated or inferred by the code itself.  Errors in
	   deriving properties may be considered a contributing factor
	   to improper input validation.
	   
	   Note that "input validation" has very different
	   meanings to different people, or within different
	   classification schemes.  Caution must be used when
	   referencing this CWE entry or mapping to it.  For example,
	   some weaknesses might involve inadvertently giving control
	   to an attacker over an input when they should not be able
	   to provide an input at all, but sometimes this is referred
	   to as input validation.
	   Finally, it is important to emphasize that the
	   distinctions between input validation and output escaping
	   are often blurred, and developers must be careful to
	   understand the difference, including how input validation
	   is not always sufficient to prevent vulnerabilities,
	   especially when less stringent data types must be
	   supported, such as free-form text. Consider a SQL injection
	   scenario in which a person's last name is inserted into a
	   query. The name "O'Reilly" would likely pass the validation
	   step since it is a common last name in the English
	   language. However, this valid name cannot be directly
	   inserted into the database because it contains the "'"
	   apostrophe character, which would need to be escaped or
	   otherwise transformed. In this case, removing the
	   apostrophe might reduce the risk of SQL injection, but it
	   would produce incorrect behavior because the wrong name
	   would be recorded.
       
         

Fix information:
repo_url: https://github.com/mintplex-labs/anything-llm
commit_details: repo_url: https://github.com/mintplex-labs/anything-llm
author: Timothy Carambat
author_date: 2024-01-19 12:56:00-08:00
msg: Employ strict validations on document pathing (#627)

* Employ strict validations on document pathing

* add comment

* update validSubfolder var
repository: repo_url: https://github.com/mintplex-labs/anything-llm
repo_name: Mintplex-Labs/anything-llm
description: The all-in-one Desktop & Docker AI application with full RAG and AI Agent capabilities.
repo_language: JavaScript
owner: mintplex-labs
file_changes: filename: index.js
change_type: ModificationType.MODIFY
diff: @@ -1,37 +1,29 @@
 const fs = require("fs");
 const path = require("path");
 const { v5: uuidv5 } = require("uuid");
+const documentsPath =
+  process.env.NODE_ENV === "development"
+    ? path.resolve(__dirname, `../../storage/documents`)
+    : path.resolve(process.env.STORAGE_DIR, `documents`);
+const vectorCachePath =
+  process.env.NODE_ENV === "development"
+    ? path.resolve(__dirname, `../../storage/vector-cache`)
+    : path.resolve(process.env.STORAGE_DIR, `vector-cache`);
 
 // Should take in a folder that is a subfolder of documents
 // eg: youtube-subject/video-123.json
 async function fileData(filePath = null) {
   if (!filePath) throw new Error("No docPath provided in request");
+  const fullFilePath = path.resolve(documentsPath, normalizePath(filePath));
+  if (!fs.existsSync(fullFilePath) || !isWithin(documentsPath, fullFilePath))
+    return null;
 
-  const fullPath =
-    process.env.NODE_ENV === "development"
-      ? path.resolve(
-          __dirname,
-          `../../storage/documents/${normalizePath(filePath)}`
-        )
-      : path.resolve(
-          process.env.STORAGE_DIR,
-          `documents/${normalizePath(filePath)}`
-        );
-
-  const fileExists = fs.existsSync(fullPath);
-  if (!fileExists) return null;
-
-  const data = fs.readFileSync(fullPath, "utf8");
+  const data = fs.readFileSync(fullFilePath, "utf8");
   return JSON.parse(data);
 }
 
 async function viewLocalFiles() {
-  const folder =
-    process.env.NODE_ENV === "development"
-      ? path.resolve(__dirname, `../../storage/documents`)
-      : path.resolve(process.env.STORAGE_DIR, `documents`);
-  const dirExists = fs.existsSync(folder);
-  if (!dirExists) fs.mkdirSync(folder);
+  if (!fs.existsSync(documentsPath)) fs.mkdirSync(documentsPath);
 
   const directory = {
     name: "documents",
@@ -39,14 +31,9 @@ async function viewLocalFiles() {
     items: [],
   };
 
-  for (const file of fs.readdirSync(folder)) {
+  for (const file of fs.readdirSync(documentsPath)) {
     if (path.extname(file) === ".md") continue;
-
-    const folderPath =
-      process.env.NODE_ENV === "development"
-        ? path.resolve(__dirname, `../../storage/documents/${file}`)
-        : path.resolve(process.env.STORAGE_DIR, `documents/${file}`);
-
+    const folderPath = path.resolve(documentsPath, file);
     const isFolder = fs.lstatSync(folderPath).isDirectory();
     if (isFolder) {
       const subdocs = {
@@ -83,10 +70,7 @@ async function cachedVectorInformation(filename = null, checkOnly = false) {
   if (!filename) return checkOnly ? false : { exists: false, chunks: [] };
 
   const digest = uuidv5(filename, uuidv5.URL);
-  const file =
-    process.env.NODE_ENV === "development"
-      ? path.resolve(__dirname, `../../storage/vector-cache/${digest}.json`)
-      : path.resolve(process.env.STORAGE_DIR, `vector-cache/${digest}.json`);
+  const file = path.resolve(vectorCachePath, `${digest}.json`);
   const exists = fs.existsSync(file);
 
   if (checkOnly) return exists;
@@ -106,15 +90,10 @@ async function storeVectorResult(vectorData = [], filename = null) {
   console.log(
     `Caching vectorized results of ${filename} to prevent duplicated embedding.`
   );
-  const folder =
-    process.env.NODE_ENV === "development"
-      ? path.resolve(__dirname, `../../storage/vector-cache`)
-      : path.resolve(process.env.STORAGE_DIR, `vector-cache`);
-
-  if (!fs.existsSync(folder)) fs.mkdirSync(folder);
+  if (!fs.existsSync(vectorCachePath)) fs.mkdirSync(vectorCachePath);
 
   const digest = uuidv5(filename, uuidv5.URL);
-  const writeTo = path.resolve(folder, `${digest}.json`);
+  const writeTo = path.resolve(vectorCachePath, `${digest}.json`);
   fs.writeFileSync(writeTo, JSON.stringify(vectorData), "utf8");
   return;
 }
@@ -122,21 +101,16 @@ async function storeVectorResult(vectorData = [], filename = null) {
 // Purges a file from the documents/ folder.
 async function purgeSourceDocument(filename = null) {
   if (!filename) return;
+  const filePath = path.resolve(documentsPath, normalizePath(filename));
+
+  if (
+    !fs.existsSync(filePath) ||
+    !isWithin(documentsPath, filePath) ||
+    !fs.lstatSync(filePath).isFile()
+  )
+    return;
+
   console.log(`Purging source document of ${filename}.`);
-  const filePath =
-    process.env.NODE_ENV === "development"
-      ? path.resolve(
-          __dirname,
-          `../../storage/documents`,
-          normalizePath(filename)
-        )
-      : path.resolve(
-          process.env.STORAGE_DIR,
-          `documents`,
-          normalizePath(filename)
-        );
-
-  if (!fs.existsSync(filePath)) return;
   fs.rmSync(filePath);
   return;
 }
@@ -144,15 +118,11 @@ async function purgeSourceDocument(filename = null) {
 // Purges a vector-cache file from the vector-cache/ folder.
 async function purgeVectorCache(filename = null) {
   if (!filename) return;
-  console.log(`Purging vector-cache of ${filename}.`);
-
   const digest = uuidv5(filename, uuidv5.URL);
-  const filePath =
-    process.env.NODE_ENV === "development"
-      ? path.resolve(__dirname, `../../storage/vector-cache`, `${digest}.json`)
-      : path.resolve(process.env.STORAGE_DIR, `vector-cache`, `${digest}.json`);
+  const filePath = path.resolve(vectorCachePath, `${digest}.json`);
 
-  if (!fs.existsSync(filePath)) return;
+  if (!fs.existsSync(filePath) || !fs.lstatSync(filePath).isFile()) return;
+  console.log(`Purging vector-cache of ${filename}.`);
   fs.rmSync(filePath);
   return;
 }
@@ -161,24 +131,20 @@ async function purgeVectorCache(filename = null) {
 // folder via iteration of all folders and checking if the expected file exists.
 async function findDocumentInDocuments(documentName = null) {
   if (!documentName) return null;
-  const documentsFolder =
-    process.env.NODE_ENV === "development"
-      ? path.resolve(__dirname, `../../storage/documents`)
-      : path.resolve(process.env.STORAGE_DIR, `documents`);
-
-  for (const folder of fs.readdirSync(documentsFolder)) {
+  for (const folder of fs.readdirSync(documentsPath)) {
     const isFolder = fs
-      .lstatSync(path.join(documentsFolder, folder))
+      .lstatSync(path.join(documentsPath, folder))
       .isDirectory();
     if (!isFolder) continue;
 
     const targetFilename = normalizePath(documentName);
-    const targetFileLocation = path.join(
-      documentsFolder,
-      folder,
-      targetFilename
-    );
-    if (!fs.existsSync(targetFileLocation)) continue;
+    const targetFileLocation = path.join(documentsPath, folder, targetFilename);
+
+    if (
+      !fs.existsSync(targetFileLocation) ||
+      !isWithin(documentsPath, targetFileLocation)
+    )
+      continue;
 
     const fileData = fs.readFileSync(targetFileLocation, "utf8");
     const cachefilename = `${folder}/${targetFilename}`;
@@ -194,8 +160,25 @@ async function findDocumentInDocuments(documentName = null) {
   return null;
 }
 
+/**
+ * Checks if a given path is within another path.
+ * @param {string} outer - The outer path (should be resolved).
+ * @param {string} inner - The inner path (should be resolved).
+ * @returns {boolean} - Returns true if the inner path is within the outer path, false otherwise.
+ */
+function isWithin(outer, inner) {
+  if (outer === inner) return false;
+  const rel = path.relative(outer, inner);
+  return !rel.startsWith("../") && rel !== "..";
+}
+
 function normalizePath(filepath = "") {
-  return path.normalize(filepath).replace(/^(\.\.(\/|\\|$))+/, "");
+  const result = path
+    .normalize(filepath.trim())
+    .replace(/^(\.\.(\/|\\|$))+/, "")
+    .trim();
+  if (["..", ".", "/"].includes(result)) throw new Error("Invalid path.");
+  return result;
 }
 
 module.exports = {
@@ -207,4 +190,6 @@ module.exports = {
   storeVectorResult,
   fileData,
   normalizePath,
+  isWithin,
+  documentsPath,
 };

programming_language: TypeScript
filename: purgeDocument.js
change_type: ModificationType.MODIFY
diff: @@ -1,30 +1,53 @@
 const fs = require("fs");
 const path = require("path");
-const { purgeVectorCache, purgeSourceDocument, normalizePath } = require(".");
+const {
+  purgeVectorCache,
+  purgeSourceDocument,
+  normalizePath,
+  isWithin,
+  documentsPath,
+} = require(".");
 const { Document } = require("../../models/documents");
 const { Workspace } = require("../../models/workspace");
 
-async function purgeDocument(filename) {
+async function purgeDocument(filename = null) {
+  if (!filename || !normalizePath(filename)) return;
+
+  await purgeVectorCache(filename);
+  await purgeSourceDocument(filename);
   const workspaces = await Workspace.where();
   for (const workspace of workspaces) {
     await Document.removeDocuments(workspace, [filename]);
   }
-  await purgeVectorCache(filename);
-  await purgeSourceDocument(filename);
   return;
 }
 
-async function purgeFolder(folderName) {
-  if (folderName === "custom-documents") return;
-  const documentsFolder =
-    process.env.NODE_ENV === "development"
-      ? path.resolve(__dirname, `../../storage/documents`)
-      : path.resolve(process.env.STORAGE_DIR, `documents`);
+async function purgeFolder(folderName = null) {
+  if (!folderName) return;
+  const subFolder = normalizePath(folderName);
+  const subFolderPath = path.resolve(documentsPath, subFolder);
+  const validRemovableSubFolders = fs
+    .readdirSync(documentsPath)
+    .map((folder) => {
+      // Filter out any results which are not folders or
+      // are the protected custom-documents folder.
+      if (folder === "custom-documents") return null;
+      const subfolderPath = path.resolve(documentsPath, folder);
+      if (!fs.lstatSync(subfolderPath).isDirectory()) return null;
+      return folder;
+    })
+    .filter((subFolder) => !!subFolder);
+
+  if (
+    !validRemovableSubFolders.includes(subFolder) ||
+    !fs.existsSync(subFolderPath) ||
+    !isWithin(documentsPath, subFolderPath)
+  )
+    return;
 
-  const folderPath = path.resolve(documentsFolder, normalizePath(folderName));
   const filenames = fs
-    .readdirSync(folderPath)
-    .map((file) => path.join(folderPath, file));
+    .readdirSync(subFolderPath)
+    .map((file) => path.join(subFolderPath, file));
   const workspaces = await Workspace.where();
 
   const purgePromises = [];
@@ -47,7 +70,7 @@ async function purgeFolder(folderName) {
   }
 
   await Promise.all(purgePromises.flat().map((f) => f()));
-  fs.rmSync(folderPath, { recursive: true }); // Delete root document and source files.
+  fs.rmSync(subFolderPath, { recursive: true }); // Delete target document-folder and source files.
 
   return;
 }

programming_language: TypeScript