Repository: explosion/spaCy

Issue:
Provide compat that works with on Windows with Py 3.7.1
## Feature description
The script that creates the symbolic link doesn't work by default  when running `python -m spacy link en_core_web_sm en` -- normally if Dos is the default shell a call to `mklink` just works. However, in via this module, it does not and it is not a permission issues nor a system policy restriction as just running `mklink /d ...` from the same shell/command line works..

This fails regardless if under a virtualenv as well.

Instead a user is presented with the following error message.

```
(venv) C:\g\py\spacy> python -m spacy link en_core_web_sm en
C:\Program Files\Python37\lib\importlib\_bootstrap.py:219: RuntimeWarning: cymem.cymem.Pool size changed, may indicate binary incompatibility. Expected 48 from C header, got 64 from PyObject
  return f(*args, **kwds)
C:\Program Files\Python37\lib\importlib\_bootstrap.py:219: RuntimeWarning: cymem.cymem.Address size changed, may indicate binary incompatibility. Expected 24 from C header, got 40 from PyObject
  return f(*args, **kwds)

    Error: Couldn't link model to 'en'
    Creating a symlink in spacy/data failed. Make sure you have the required
    permissions and try re-running the command as admin, or use a
    virtualenv. You can still import the model as a module and call its
    load() method, or create the symlink manually.

    C:\g\py\spacy\venv\lib\site-packages\en_core_web_sm -->
    C:\g\py\spacy\venv\lib\site-packages\spacy\data\en

Traceback (most recent call last):
  File "C:\Program Files\Python37\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "C:\Program Files\Python37\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\g\py\spacy\venv\lib\site-packages\spacy\__main__.py", line 31, in <module>
    plac.call(commands[command], sys.argv[1:])
  File "C:\g\py\spacy\venv\lib\site-packages\plac_core.py", line 328, in call
    cmd, result = parser.consume(arglist)
  File "C:\g\py\spacy\venv\lib\site-packages\plac_core.py", line 207, in consume
    return cmd, self.func(*(args + varargs + extraopts), **kwargs)
  File "C:\g\py\spacy\venv\lib\site-packages\spacy\cli\link.py", line 48, in link
    symlink_to(link_path, model_path)
  File "C:\g\py\spacy\venv\lib\site-packages\spacy\compat.py", line 87, in symlink_to
    orig.symlink_to(dest)
  File "C:\Program Files\Python37\lib\pathlib.py", line 1320, in symlink_to
    self._accessor.symlink(target, self, target_is_directory)
OSError: symbolic link privilege not held
```

The following command works regardless and has no issue with permissions nor policy. The error message indicating symbolic link privilege not held is misleading.

```
cmd /c mklink /k c:\path\to\symlink c:\target\file
```

A suggestion is changing the respective lines in `venv\Lib\site-packages\spacy\compat.py (symlink_to)`

```
def symlink_to(orig, dest):
    if is_python2 and is_windows:
        import subprocess
        subprocess.call(['mklink', '/d', path2str(orig), path2str(dest)], shell=True)
    else:
        orig.symlink_to(dest)
```

to 

```
def symlink_to(orig, dest):
    if (is_python2 or is_python3) and is_windows:
        import subprocess
        subprocess.call(['mklink', '/d', path2str(orig), path2str(dest)], shell=True)
    else:
        orig.symlink_to(dest)
````





Discussion:

Resolving patch:
diff --git a/spacy/compat.py b/spacy/compat.py
--- a/spacy/compat.py
+++ b/spacy/compat.py
@@ -1,6 +1,7 @@
 # coding: utf8
 from __future__ import unicode_literals
 
+import os
 import sys
 import ujson
 import itertools
@@ -80,12 +81,18 @@ def getattr_(obj, name, *default):
 
 
 def symlink_to(orig, dest):
-    if is_python2 and is_windows:
+    if is_windows:
         import subprocess
         subprocess.call(['mklink', '/d', path2str(orig), path2str(dest)], shell=True)
     else:
         orig.symlink_to(dest)
 
+def symlink_remove(link):
+    # https://stackoverflow.com/questions/26554135/cant-delete-unlink-a-symlink-to-directory-in-python-windows
+    if( os.path.isdir(path2str(link)) and is_windows ): # this should only be on Py2.7 and windows
+        os.rmdir(path2str(link))
+    else:
+        os.unlink(path2str(link))
 
 def is_config(python2=None, python3=None, windows=None, linux=None, osx=None):
     return (python2 in (None, is_python2) and


Test patch:
diff --git a/spacy/tests/test_symlink_windows.py b/spacy/tests/test_symlink_windows.py
new file mode 100644
--- /dev/null
+++ b/spacy/tests/test_symlink_windows.py
@@ -0,0 +1,36 @@
+import os
+from pathlib import Path
+
+from ..compat import symlink_to, symlink_remove, path2str
+
+import pytest
+
+def target_local_path():
+  return './foo-target'
+
+def link_local_path():
+  return './foo-symlink'
+
+
+@pytest.fixture(scope='function')
+def setup_target(request):
+  target = Path(target_local_path())
+  if not target.exists():
+    os.mkdir( path2str(target) )
+
+  # yield -- need to cleanup even if assertion fails
+  # https://github.com/pytest-dev/pytest/issues/2508#issuecomment-309934240
+  def cleanup():  
+    symlink_remove( Path(link_local_path() ) )
+    os.rmdir( target_local_path()  )
+
+  request.addfinalizer(cleanup)
+
+
+def test_create_symlink_windows(setup_target):
+  target = Path(target_local_path())
+  link = Path(link_local_path())
+  assert target.exists()
+
+  symlink_to(link, target)
+  assert link.exists()