Repository: spectrochempy/spectrochempy

Issue:
tests after installation of spectrochempy_data 
**Describe you problem**

I have installed spectrochempy_data but testing does not work. On  WIN , `preferences.datadir`  returns `OSError: symbolic link privilege not held`

 for instance:

```
C:\Miniconda3\envs\scpy-dev\python.exe "C:\Program Files\JetBrains\PyCharm 2020.2.3\plugins\python\helpers\pycharm\_jb_pytest_runner.py" --path C:/Users/traverta/Dropbox/Programmes/spectrochempy/tests/test_plotters
Testing started at 16:58 ...
Launching pytest with arguments C:/Users/traverta/Dropbox/Programmes/spectrochempy/tests/test_plotters in C:\Users\traverta\Dropbox\Programmes\spectrochempy\tests\test_plotters

SpectroChemPy's API - v.0.2.11.dev2+g6ceb47f0
Loading SpectroChemPy API: 100%|██████████| 1211/1211 [00:02<00:00, 548.16it/s]
© Copyright 2014-2021 - A.Travert & C.Fernandez @ LCS
ImportError while loading conftest 'C:\Users\traverta\Dropbox\Programmes\spectrochempy\tests\conftest.py'.
..\conftest.py:18: in <module>
    import spectrochempy
..\..\spectrochempy\__init__.py:74: in <module>
    from spectrochempy import api
..\..\spectrochempy\api.py:126: in <module>
    DATADIR = pathclean(preferences.datadir)
C:\Miniconda3\envs\scpy-dev\lib\site-packages\traitlets\traitlets.py:575: in __get__
    return self.get(obj, cls)
C:\Miniconda3\envs\scpy-dev\lib\site-packages\traitlets\traitlets.py:538: in get
    default = obj.trait_defaults(self.name)
C:\Miniconda3\envs\scpy-dev\lib\site-packages\traitlets\traitlets.py:1578: in trait_defaults
    return self._get_trait_default_generator(names[0])(self)
C:\Miniconda3\envs\scpy-dev\lib\site-packages\traitlets\traitlets.py:975: in __call__
    return self.func(*args, **kwargs)
..\..\spectrochempy\application.py:478: in _get_default_datadir
    return self.parent.datadir.path
C:\Miniconda3\envs\scpy-dev\lib\site-packages\traitlets\traitlets.py:575: in __get__
    return self.get(obj, cls)
C:\Miniconda3\envs\scpy-dev\lib\site-packages\traitlets\traitlets.py:538: in get
    default = obj.trait_defaults(self.name)
C:\Miniconda3\envs\scpy-dev\lib\site-packages\traitlets\traitlets.py:1578: in trait_defaults
    return self._get_trait_default_generator(names[0])(self)
C:\Miniconda3\envs\scpy-dev\lib\site-packages\traitlets\traitlets.py:975: in __call__
    return self.func(*args, **kwargs)
..\..\spectrochempy\application.py:418: in _get_path_default
    path.symlink_to(testdata, target_is_directory=True)
C:\Miniconda3\envs\scpy-dev\lib\pathlib.py:1352: in symlink_to
    self._accessor.symlink(target, self, target_is_directory)
E   OSError: symbolic link privilege not held

Process finished with exit code 4
```



Discussion:
I have seen a similar problem here:

https://github.com/explosion/spaCy/issues/895#issuecomment-289198926
Arnaud Can you check if you have a testdata directory (normally a symlink) in   $HOME/.spectrochempy   (spectrochempy with a starting dot) 

Anyway I think it should work without this but I will check again.
This link should point on the actual data: in   <your_env>/share/spectrochempy_data/testdata

(Use` where python `. to know the path to your env. For me it is: /Users/christian/opt/miniconda3/envs/scpy_dev
so I can check the presence of th files:
```
ls /Users/christian/opt/miniconda3/envs/scpy_dev/share/spectrochempy_data/tesdata
```

I think it is a problem with the installation of the symlink on windows. But I thought I had made a work around on this. I'll check again. 
It seems that the symlink is created only when spectrochempy_data is installed with admin privileges. But then the link is in my admin home/.spectrochempy  dir,  not my user directory.   

I will fix it manually on my system.
Indeed, you can you create the link yourself for the moment. May be the simpler to solve this problem( I already spend to much time on this) would be to modified the setup.py to install the data in the .spectrochempy dir.  This is what I do in Colab, if you look at the documentation. This is probably the simpler (or may be setup.py could create this link). I will look at this.
There is something quite strange:

I don't see why IF the symlink is not installed you have this error:
```
    import spectrochempy
..\..\spectrochempy\__init__.py:74: in <module>
    from spectrochempy import api
..\..\spectrochempy\api.py:126: in <module>
    DATADIR = pathclean(preferences.datadir)         
C:\Miniconda3\envs\scpy-dev\lib\site-packages\traitlets\traitlets.py:575: in __get__
    return self.get(obj, cls)
C:\Miniconda3\envs\scpy-dev\lib\site-packages\traitlets\traitlets.py:538: in get
    default = obj.trait_defaults(self.name)
```

``DATADIR = pathclean(preferences.datadir)``   do not try to find the file: it just set a path. 

on my Mac, if I remove the symlink from .spectrochempy: 
I got the error when try to read file, because of course they are not present.

```
Loading SpectroChemPy API: 100%|██████████| 1211/1211 [00:01<00:00, 820.53it/s]
 WARNING | No file with name `/Users/christian/.spectrochempy/testdata/irdata/nh4y-activation.spg` could be found. Sorry!
ERROR: file or directory no found....
```

But you said that it is not installed! 




OK I see the problem is here;
418 in application.py 
```path.symlink_to(testdata, target_is_directory=True)```

I confirm: if the symlink is not present (or renamed) I have this error `OSError: symbolic link privilege not held`
Actually The symlink should be created with this instruction, but it doesn't work ON WINDOWS!
The reason is probably here: https://security.stackexchange.com/questions/10194/why-do-you-have-to-be-an-admin-to-create-a-symlink-in-windows

(the kids are back, I am leaving for some time from now ... :-).
For the moment I tried to solve the problem by just copying files instead of doing a symlink . I am not sure this is the best solution, I will think about another way to install directly in this folder. For this I need to modify spectrochempy_data installation.
So for the moment, the pull request that I will merge is just a workaround 
Let me know if it works for you, and we close this issue. I should also tag it as soon it is ok, so that the windows users can load a working version.
It is working  but it seems that data files are duplicated in the .spectrochempy folder: I have all the subfolders that were in testdata + testdata and the same subfolders, i.e. 
```
.spectrochempy/
    |- agirdata/
    |- irdata/
    |- matladata/
   (....)
    |- testdata/
        |- agirdata/
        |- irdata/
        |- matlabdata/
       (...)
        |- wodger.spg
   |- wodger.spg
```
I can't check but I suspect I had also something like that with the symlink)
yes, there is still some problems. I am right now trying to install directly the spectrochempy_data in the .spectrochempy/testdata directory, so there will be no needs anymore to symlink or to copy data. 

Resolving patch:
diff --git a/spectrochempy/__init__.py b/spectrochempy/__init__.py
--- a/spectrochempy/__init__.py
+++ b/spectrochempy/__init__.py
@@ -54,22 +54,6 @@
 
 __all__ = ['api']
 
-# if '/bin/scpy_update' not in sys.argv[0]:  # avoid indefinite loop of calls this process
-#
-#     p = Path('~/.spectrochempy/tmp')
-#     p = p.expanduser()
-#     p.mkdir(parents=True, exist_ok=True)
-#     q = p / 'updated'
-#     q.touch(exist_ok=True)
-#     text = q.read_text()
-#     if not text:
-#         text = str(time() - 90000)
-#     if time() - float(text) > 90000:  # check every 24 hours at the maximum
-#         import subprocess
-#
-#         subprocess.Popen(["scpy_update"])
-#         q.write_text(str(time()))
-
 # import the main api
 from spectrochempy import api
 from spectrochempy.api import *  # noqa: F401
diff --git a/spectrochempy/application.py b/spectrochempy/application.py
--- a/spectrochempy/application.py
+++ b/spectrochempy/application.py
@@ -21,7 +21,6 @@
 import warnings
 import pprint
 import json
-import time
 from os import environ
 from pathlib import Path
 import threading
@@ -359,6 +358,26 @@ class DataDir(HasTraits):
 
     path = Instance(Path)
 
+    @default('path')
+    def _get_path_default(self, **kwargs):
+
+        super().__init__(**kwargs)
+
+        # create a directory testdata in .spectrochempy to avoid an error if the following do not work
+        path = _find_or_create_spectrochempy_dir() / 'testdata'
+        path.mkdir(exist_ok=True)
+
+        # try to use the conda installed testdata (spectrochempy_data package)
+        try:
+            conda_env = environ['CONDA_PREFIX']
+            path = Path(conda_env) / 'share' / 'spectrochempy_data' / 'testdata'
+            if not path.exists():
+                path = Path(conda_env) / 'share' / 'spectrochempy_data'   # depending on the version of spectrochempy_data
+        except KeyError:
+            pass
+
+        return path
+
     def listing(self):
         """
         Create a str representing a listing of the testdata folder.
@@ -396,55 +415,6 @@ def _repr_html_(self):
         # _repr_html is needed to output in notebooks
         return self.listing().replace('\n', '<br/>').replace(" ", "&nbsp;")
 
-    @default('path')
-    def _get_path_default(self):
-        # the spectra path in package data
-        # OLD: return Path(get_pkg_path('testdata', 'scp_data'))
-        # now installed with spectrochempy_data
-        # we need to fing the share directory
-        path = _find_or_create_spectrochempy_dir() / 'testdata'
-
-        must_copy = False
-        if not path.exists() or (not path.is_symlink() and not list(path.iterdir())):
-            # try to use the conda installed tesdata (spectrochempy_data package)
-            try:
-                conda_env = environ['CONDA_PREFIX']
-                testdata = Path(conda_env) / 'share' / 'spectrochempy_data'
-                if testdata.exists():
-                    # create a symbolic link to this tesdata directory
-                    # However this work locally or on Colab, BUT not on Travis
-                    if not environ.get('TRAVIS_BRANCH', None):
-                        if path.exists():
-                            path.rmdir()
-                        path.symlink_to(testdata, target_is_directory=True)
-                    else:
-                        must_copy=True
-            except KeyError:
-                pass
-
-            except OSError:
-                must_copy = True
-
-            if must_copy:
-                # we need to copy file so it will work
-                from spectrochempy.utils import copytree
-                if path.exists():
-                    path.rmdir()
-                copytree(testdata, path.parent)
-
-
-        if path.exists() and path.is_file():
-            msg = 'Intended Data directory `{0}` is ' \
-                  'actually a file.'
-            raise IOError(msg.format(path))
-
-        # OK but what if like in colab we found nothing
-        if not path.exists():
-            # create a directory to avoid an error
-            path.mkdir()
-
-        return path
-
 
 # ======================================================================================================================
 # General Preferences


Test patch: