Repository: frappe/frappe

Issue:
failing bench data-import with file from server's local file system
CSV data import fails when started via frappe-bench on server with file from local file system in Frappe Framework Version 14.29.1:

Already tracked down the issue to `def read_file(self, file_path: str):` at
https://github.com/frappe/frappe/blob/d0711e1436f58073fc6c94c0aec6de8b5efc5457/frappe/core/doctype/data_import/importer.py#L577

There the import data file is looked up from the database instead of reading it from path in the server's local file system.

Test/Reproduction procedure:
- (activate development mode in frappe site)
- go on as Administrator in Desk
- create a DocType 
- add an import for that DocType
- open import and download CVS template
- fill in some valid data into the template file
- go on as frappe user on server computer 
- save template to file system, e.g. to`.../frappe-bench/sites/yoursite/private/files/csv-filename.csv`
- cd into `.../frappe-bench`
- run `bench data-import --doctype TheDocType --file /absolute/path/to/csv-filename.csv` (absolute path is required)
  - (BTW: validation fails with relative path telling the template is invalid -- another issue, resulting from https://github.com/frappe/frappe/blob/d0711e1436f58073fc6c94c0aec6de8b5efc5457/frappe/core/doctype/data_import/importer.py#L409)
- => This Issue: ValidationError: Invalid or corrupted content for import

Tested also the import file: Import as Administrator on Desk works with the same file.

I actually used a site with frappe and an own app in development. I do not expect any difference to the above given procedure.

This import variant is required when importing existing data from old systems, which are to be replaced (german: Bestandsdaten-Import). Import of to many data via Desk ran into timeouts (experienced in Version 13). `bench data-import` doesn't.



Discussion:
same issue
Is there any work already on this? If not I may have a solution ready to submit a PR
@Ahuahuachi send PR :+1: 


I think the issue here is that bench automatically ALWAYS switches current working directory to root of bench so your relative path will not work. 

IDK of any elegant solution to this. 
reference:

https://github.com/frappe/bench/blob/480e35c1c7ca4e9c74de473e6071cc862302d2f4/bench/cli.py#L226-L234



Resolving patch:
diff --git a/frappe/commands/utils.py b/frappe/commands/utils.py
--- a/frappe/commands/utils.py
+++ b/frappe/commands/utils.py
@@ -386,7 +386,14 @@ def import_doc(context, path, force=False):
 
 @click.command("data-import")
 @click.option(
-	"--file", "file_path", type=click.Path(), required=True, help="Path to import file (.csv, .xlsx)"
+	"--file",
+	"file_path",
+	type=click.Path(exists=True, dir_okay=False, resolve_path=True),
+	required=True,
+	help=(
+		"Path to import file (.csv, .xlsx)."
+		"Consider that relative paths will resolve from 'sites' directory"
+	),
 )
 @click.option("--doctype", type=str, required=True)
 @click.option(
diff --git a/frappe/core/doctype/data_import/importer.py b/frappe/core/doctype/data_import/importer.py
--- a/frappe/core/doctype/data_import/importer.py
+++ b/frappe/core/doctype/data_import/importer.py
@@ -579,6 +579,10 @@ def read_file(self, file_path: str):
 
 		file_content = None
 
+		if self.console:
+			file_content = frappe.read_file(file_path, True)
+			return file_content, extn
+
 		file_name = frappe.db.get_value("File", {"file_url": file_path})
 		if file_name:
 			file = frappe.get_doc("File", file_name)


Test patch: