Repository: mindsdb/mindsdb

Issue:
Permissions error on /datasources/data
This is reported from the user while uploading the data through the Scout. This happens on a Windows machine [permissions error line](https://github.com/mindsdb/mindsdb/blob/stable/mindsdb/interfaces/datastore/datastore.py#L81)
```
Exception on /datasources/Somedata [PUT] Traceback (most recent call last): File "C:\Users\jp299\AppData\Roaming\mindsdb_gui\mindsdb_server\env\lib\site-packages\flask\app.py", line 1950, in full_dispatch_request rv = self.dispatch_request() File "C:\Users\jp299\AppData\Roaming\mindsdb_gui\mindsdb_server\env\lib\site-packages\flask\app.py", line 1936, in dispatch_request return self.view_functions[rule.endpoint](**req.view_args) File "C:\Users\jp299\AppData\Roaming\mindsdb_gui\mindsdb_server\env\lib\site-packages\flask_restx\api.py", line 375, in wrapper resp = resource(*args, **kwargs) File "C:\Users\jp299\AppData\Roaming\mindsdb_gui\mindsdb_server\env\lib\site-packages\flask\views.py", line 89, in view return self.dispatch_request(*args, **kwargs) File "C:\Users\jp299\AppData\Roaming\mindsdb_gui\mindsdb_server\env\lib\site-packages\flask_restx\resource.py", line 44, in dispatch_request resp = meth(*args, **kwargs) File "C:\Users\jp299\AppData\Roaming\mindsdb_gui\mindsdb_server\env\lib\site-packages\flask_restx\marshalling.py", line 248, in wrapper resp = f(*args, **kwargs) File "C:\Users\jp299\AppData\Roaming\mindsdb_gui\mindsdb_server\env\lib\site-packages\mindsdb\api\http\namespaces\datasource.py", line 114, in put ca.default_store.save_datasource(ds_name, source_type, source, file_path) File "C:\Users\jp299\AppData\Roaming\mindsdb_gui\mindsdb_server\env\lib\site-packages\mindsdb\interfaces\datastore\datastore.py", line 81, in save_datasource os.replace(file_path, source) PermissionError: [WinError 32] El proceso no tiene acceso al archivo porque est� siendo utilizado por otro proceso: 'C:\\Users\\jp299\\AppData\\Local\\Temp\\datasource_file_pzpfjtzc\\Somedata.xlsx' -> 'C:\\Users\\jp299\\AppData\\Roaming\\mindsdb_gui\\mindsdb_server\\env\\lib\\site-packages/var/datastore\\Somedata\datasource\\Somedata.xlsx'
```


Discussion:
Quite similar error, it looks like different processes are accessing the same location and  we got PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 
```
$terminal: 127.0.0.1 - - [19/Aug/2020 12:46:15] "GET /datasources/ HTTP/1.1" 200 -
$terminal: Field(field_name=b'name', value=b'c2') Field(field_name=b'source_type', value=b'file')
$terminal: Field(field_name=b'source', value=b'processed.cleveland.data')
$terminal: [Errno 2] No such file or directory: 'C:\\Users\\BCP\\AppData\\Roaming\\mindsdb_gui\\mindsdb_server\\env\\lib\\site-packages/var/datastore\\cleveland\\datasource\\metadata.json'
$terminal: Exception on /datasources/c2 [PUT] Traceback (most recent call last): File "C:\Users\BCP\AppData\Roaming\mindsdb_gui\mindsdb_server\env\lib\site-packages\flask\app.py", line 1950, in full_dispatch_request rv = self.dispatch_request() File "C:\Users\BCP\AppData\Roaming\mindsdb_gui\mindsdb_server\env\lib\site-packages\flask\app.py", line 1936, in dispatch_request return self.view_functions[rule.endpoint](**req.view_args) File "C:\Users\BCP\AppData\Roaming\mindsdb_gui\mindsdb_server\env\lib\site-packages\flask_restx\api.py", line 375, in wrapper resp = resource(*args, **kwargs) File "C:\Users\BCP\AppData\Roaming\mindsdb_gui\mindsdb_server\env\lib\site-packages\flask\views.py", line 89, in view return self.dispatch_request(*args, **kwargs) File "C:\Users\BCP\AppData\Roaming\mindsdb_gui\mindsdb_server\env\lib\site-packages\flask_restx\resource.py", line 44, in dispatch_request resp = meth(*args, **kwargs) File "C:\Users\BCP\AppData\Roaming\mindsdb_gui\mindsdb_server\env\lib\site-packages\flask_restx\marshalling.py", line 248, in wrapper resp = f(*args, **kwargs) File "C:\Users\BCP\AppData\Roaming\mindsdb_gui\mindsdb_server\env\lib\site-packages\mindsdb\api\http\namespaces\datasource.py", line 122, in put ca.default_store.save_datasource(ds_name, source_type, source, file_path) File "C:\Users\BCP\AppData\Roaming\mindsdb_gui\mindsdb_server\env\lib\site-packages\mindsdb\interfaces\datastore\datastore.py", line 81, in save_datasource os.replace(file_path, source) PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Users\\BCP\\AppData\\Local\\Temp\\datasource_file_uteci2nj\\processed.cleveland.data' -> 'C:\\Users\\BCP\\AppData\\Roaming\\mindsdb_gui\\mindsdb_server\\env\\lib\\site-packages/var/datastore\\c2\\datasource\\processed.cleveland.data'
$terminal: 127.0.0.1 - - [19/Aug/2020 12:46:16] "PUT /datasources/c2 HTTP/1.1" 500 -
$terminal: [Errno 2] No such file or directory: 'C:\\Users\\BCP\\AppData\\Roaming\\mindsdb_gui\\mindsdb_server\\env\\lib\\site-packages/var/datastore\\c2\\datasource\\metadata.json' [Errno 2] No such file or directory: 'C:\\Users\\BCP\\AppData\\Roaming\\mindsdb_gui\\mindsdb_server\\env\\lib\\site-packages/var/datastore\\cleveland\\datasource\\metadata.json'
$terminal: 127.0.0.1 - - [19/Aug/2020 12:46:20] "GET /datasources/ HTTP/1.1" 200 -
$terminal: 127.0.0.1 - - [19/Aug/2020 12:46:20] "GET /util/ping HTTP/1.1" 200 -
```
I did some tests and now at 90% sure that the reason is that 'python-multipart' dependency doesn't close downloaded file properly. I will make fork with fix and switch our dependency to it until It will fixed in main repository.

Resolving patch:
diff --git a/mindsdb/api/http/namespaces/datasource.py b/mindsdb/api/http/namespaces/datasource.py
--- a/mindsdb/api/http/namespaces/datasource.py
+++ b/mindsdb/api/http/namespaces/datasource.py
@@ -1,12 +1,9 @@
 import datetime
-import json
 import os
-import re
 import threading
-
 import tempfile
+
 import multipart
-import csv
 
 import mindsdb
 from dateutil.parser import parse
@@ -69,14 +66,19 @@ def delete(self, name):
     def put(self, name):
         '''add new datasource'''
         data = {}
+
         def on_field(field):
             print(f'\n\n{field}\n\n')
             name = field.field_name.decode()
             value = field.value.decode()
             data[name] = value
 
+        file_object = None
+
         def on_file(file):
+            nonlocal file_object
             data['file'] = file.file_name.decode()
+            file_object = file.file_object
 
         temp_dir_path = tempfile.mkdtemp(prefix='datasource_file_')
 
@@ -100,6 +102,9 @@ def on_file(file):
                 parser.write(chunk)
             parser.finalize()
             parser.close()
+
+            if file_object is not None and not file_object.closed:
+                file_object.close()
         else:
             data = request.json
 
diff --git a/mindsdb/interfaces/datastore/datastore.py b/mindsdb/interfaces/datastore/datastore.py
--- a/mindsdb/interfaces/datastore/datastore.py
+++ b/mindsdb/interfaces/datastore/datastore.py
@@ -73,7 +73,7 @@ def save_datasource(self, name, source_type, source, file_path=None):
         if source_type == 'file':
             try:
                 source = os.path.join(ds_dir, source)
-                os.replace(file_path, source)
+                shutil.move(file_path, source)
                 ds = FileDS(source)
             except Exception:
                 shutil.rmtree(ds_meta_dir)


Test patch: