Repository: nipy/nipype

Issue:
Installation error
### Summary
I am trying to install nipype, but am encountering an error. I am using Python 2.7 on Windows 8.1. I ran the install as administrator with: 

`python -m pip install nipype
`
### Actual behavior
The installation runs at first but produces the following error:

```
error: can't copy 'nipype\testing\data\brukerdir\pdata': doesn't exist or not a regular file. 
         --------------------------------------------------------------------------
Command "C:\Python27\python.exe -u -c "import setuptools, tokenize;__file__='c:\
\users\\joseph\\appdata\\local\\temp\\pip-build-2bngm1\\nipype\\setup.py';f=geta
ttr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.clos
e();exec(compile(code, __file__, 'exec'))" install --record c:\users\joseph\appd
ata\local\temp\pip-yvbrtb-record\install-record.txt --single-version-externally-
managed --compile" failed with error code 1 in c:\users\joseph\appdata\local\tem
p\pip-build-2bngm1\nipype\
```

### Execution environment

Windows 8.1 



Discussion:
Same on linux.

```
 error: can't copy 'nipype/testing/data/brukerdir/pdata': doesn't exist or not a regular file
```
@gdevenyi and @jcgriffis - i would like to get to the bottom of this.

on centos 7.3 i could do this without any error:

```
conda create -n test python=2 pip
source activate test
which python
python -m pip install nipype
```

i suspect this will work on windows 8.1 as well using conda. could either of you test and post?

this is my environment list.

```
$ conda list
# packages in environment at /home/satra/miniconda3/envs/test:
#
ca-certificates           2017.4.17                     0    conda-forge
certifi                   2017.4.17                py27_0    conda-forge
click                     6.7                       <pip>
configparser              3.5.0                     <pip>
decorator                 4.0.11                    <pip>
funcsigs                  1.0.2                     <pip>
future                    0.16.0                    <pip>
isodate                   0.5.4                     <pip>
lxml                      3.8.0                     <pip>
mock                      2.0.0                     <pip>
ncurses                   5.9                          10    conda-forge
networkx                  1.11                      <pip>
nibabel                   2.1.0                     <pip>
nipype                    0.13.1                    <pip>
numpy                     1.13.0                    <pip>
openssl                   1.0.2k                        0    conda-forge
pbr                       3.1.1                     <pip>
pip                       9.0.1                    py27_0    conda-forge
prov                      1.5.0                     <pip>
py                        1.4.34                    <pip>
pydotplus                 2.0.2                     <pip>
pyparsing                 2.2.0                     <pip>
pytest                    3.1.3                     <pip>
python                    2.7.13                        1    conda-forge
python-dateutil           2.6.0                     <pip>
rdflib                    4.2.2                     <pip>
readline                  6.2                           0    conda-forge
scipy                     0.19.1                    <pip>
setuptools                33.1.1                   py27_0    conda-forge
simplejson                3.11.1                    <pip>
six                       1.10.0                    <pip>
sqlite                    3.13.0                        1    conda-forge
tk                        8.5.19                        1    conda-forge
traits                    4.6.0                     <pip>
wheel                     0.29.0                   py27_0    conda-forge
zlib                      1.2.11                        0    conda-forge
```


I tried running the commands posted above, but the install still didn't work. Here is the output: 

`(C:\Users\Joseph\Anaconda2) C:\Users\Joseph> conda create -n test python=2 pip
Fetching package metadata ...........
Solving package specifications: .

Package plan for installation in environment C:\Users\Joseph\Anaconda2\envs\test
:

The following NEW packages will be INSTALLED:

    pip:            9.0.1-py27_1
    python:         2.7.13-1
    setuptools:     27.2.0-py27_1
    vs2008_runtime: 9.00.30729.5054-0
    wheel:          0.29.0-py27_0

Proceed ([y]/n)? y


 To activate this environment, use:
> activate test

To deactivate this environment, use:
> deactivate test

for power-users using bash, you must source

(C:\Users\Joseph\Anaconda2) C:\Users\Joseph>activate test

(test) C:\Users\Joseph>which python
'which' is not recognized as an internal or external command,
operable program or batch file.

(test) C:\Users\Joseph>python -m pip install nipype
Collecting nipype
  Using cached nipype-0.13.1.tar.gz
Collecting nibabel>=2.1.0 (from nipype)
Collecting networkx>=1.9 (from nipype)
  Using cached networkx-1.11-py2.py3-none-any.whl
Collecting numpy>=1.8.2 (from nipype)
  Downloading numpy-1.13.0-cp27-none-win_amd64.whl (7.6MB)
    100% |################################| 7.6MB 57kB/s
Collecting python-dateutil>=2.2 (from nipype)
  Downloading python_dateutil-2.6.0-py2.py3-none-any.whl (194kB)
    100% |################################| 194kB 898kB/s
Collecting scipy>=0.14 (from nipype)
  Downloading scipy-0.19.1.tar.gz (14.1MB)
    100% |################################| 14.1MB 33kB/s
    Complete output from command python setup.py egg_info:
    non-existing path in 'numpy\\distutils': 'site.cfg'
    Running from numpy source directory.
    c:\users\joseph\appdata\local\temp\easy_install-rfantp\numpy-1.13.0\setup.py
:367: UserWarning: Unrecognized setuptools command, proceeding with generating C
ython sources and expanding templates
      run_build = parse_setuppy_commands()
    c:\users\joseph\appdata\local\temp\easy_install-rfantp\numpy-1.13.0\numpy\di
stutils\system_info.py:572: UserWarning:
        Atlas (http://math-atlas.sourceforge.net/) libraries not found.
        Directories to search for the libraries can be specified in the
        numpy/distutils/site.cfg file (section [atlas]) or by setting
        the ATLAS environment variable.
      self.calc_info()
    c:\users\joseph\appdata\local\temp\easy_install-rfantp\numpy-1.13.0\numpy\di
stutils\system_info.py:572: UserWarning:
        Blas (http://www.netlib.org/blas/) libraries not found.
        Directories to search for the libraries can be specified in the
        numpy/distutils/site.cfg file (section [blas]) or by setting
        the BLAS environment variable.
      self.calc_info()
    c:\users\joseph\appdata\local\temp\easy_install-rfantp\numpy-1.13.0\numpy\di
stutils\system_info.py:572: UserWarning:
        Blas (http://www.netlib.org/blas/) sources not found.
        Directories to search for the sources can be specified in the
        numpy/distutils/site.cfg file (section [blas_src]) or by setting
        the BLAS_SRC environment variable.
      self.calc_info()
    c:\users\joseph\appdata\local\temp\easy_install-rfantp\numpy-1.13.0\numpy\di
stutils\system_info.py:572: UserWarning:
        Lapack (http://www.netlib.org/lapack/) libraries not found.
        Directories to search for the libraries can be specified in the
        numpy/distutils/site.cfg file (section [lapack]) or by setting
        the LAPACK environment variable.
      self.calc_info()
    c:\users\joseph\appdata\local\temp\easy_install-rfantp\numpy-1.13.0\numpy\di
stutils\system_info.py:572: UserWarning:
        Lapack (http://www.netlib.org/lapack/) sources not found.
        Directories to search for the sources can be specified in the
        numpy/distutils/site.cfg file (section [lapack_src]) or by setting
        the LAPACK_SRC environment variable.
      self.calc_info()
    C:\Users\Joseph\Anaconda2\envs\test\lib\distutils\dist.py:267: UserWarning:
Unknown distribution option: 'define_macros'
      warnings.warn(msg)
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "c:\users\joseph\appdata\local\temp\pip-build-ddzazg\scipy\setup.py",
 line 416, in <module>
        setup_package()
      File "c:\users\joseph\appdata\local\temp\pip-build-ddzazg\scipy\setup.py",
 line 412, in setup_package
        setup(**metadata)
      File "C:\Users\Joseph\Anaconda2\envs\test\lib\distutils\core.py", line 111
, in setup
        _setup_distribution = dist = klass(attrs)
      File "C:\Users\Joseph\Anaconda2\envs\test\lib\site-packages\setuptools-27.
2.0-py2.7.egg\setuptools\dist.py", line 315, in __init__
      File "C:\Users\Joseph\Anaconda2\envs\test\lib\site-packages\setuptools-27.
2.0-py2.7.egg\setuptools\dist.py", line 361, in fetch_build_eggs
      File "C:\Users\Joseph\Anaconda2\envs\test\lib\site-packages\setuptools-27.
2.0-py2.7.egg\pkg_resources\__init__.py", line 851, in resolve
      File "C:\Users\Joseph\Anaconda2\envs\test\lib\site-packages\setuptools-27.
2.0-py2.7.egg\pkg_resources\__init__.py", line 1123, in best_match
      File "C:\Users\Joseph\Anaconda2\envs\test\lib\site-packages\setuptools-27.
2.0-py2.7.egg\pkg_resources\__init__.py", line 1135, in obtain
      File "C:\Users\Joseph\Anaconda2\envs\test\lib\site-packages\setuptools-27.
2.0-py2.7.egg\setuptools\dist.py", line 428, in fetch_build_egg
      File "C:\Users\Joseph\Anaconda2\envs\test\lib\site-packages\setuptools-27.
2.0-py2.7.egg\setuptools\command\easy_install.py", line 664, in easy_install
      File "C:\Users\Joseph\Anaconda2\envs\test\lib\site-packages\setuptools-27.
2.0-py2.7.egg\setuptools\command\easy_install.py", line 694, in install_item
      File "C:\Users\Joseph\Anaconda2\envs\test\lib\site-packages\setuptools-27.
2.0-py2.7.egg\setuptools\command\easy_install.py", line 875, in install_eggs
      File "C:\Users\Joseph\Anaconda2\envs\test\lib\site-packages\setuptools-27.
2.0-py2.7.egg\setuptools\command\easy_install.py", line 1114, in build_and_insta
ll
      File "C:\Users\Joseph\Anaconda2\envs\test\lib\site-packages\setuptools-27.
2.0-py2.7.egg\setuptools\command\easy_install.py", line 1102, in run_setup
    distutils.errors.DistutilsError: Setup script exited with error: Microsoft V
isual C++ 9.0 is required. Get it from http://aka.ms/vcpython27

    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in c:\users\joseph\a
ppdata\local\temp\pip-build-ddzazg\scipy\
`
@jcgriffis can you retry after downloading the [C++ compiler for Python 2.7](https://www.microsoft.com/en-us/download/details.aspx?id=44266)
@jcgriffis - i would recommend the conda-forge install on windows. note that we do not test nipype on windows. we normally recommend using nipype through docker on windows, but with windows 8.1 that may be difficult.

@mgxd - the c++ compiler is required for traits, but if installed via conda it should work fine. a c++ compiler is not a requirement for nipype.


@jcgriffis did conda install fix your error? If you are still having problems, feel free to reopen this
Hi !

I am having the same issue, under mac OSX 10.10.5.
I did `sudo python setup.py install`
 and got, those last lines:
> creating build/lib/nipype/testing/data/brukerdir
> copying nipype/testing/data/brukerdir/fid -> build/lib/nipype/testing/data/brukerdir
> error: can't copy 'nipype/testing/data/brukerdir/pdata': doesn't exist or not a regular file

I don't have conda, I use pip to install python packages. And I use python2.7

Any suggestions ? 

Many thanks for your time and help.

Claire.

@cclairec - the issue is likely with an older version of setuptools. i would suggest updating pip first and then doing `pip install nipype` or installing the current master using pip.

i would also recommend learning about and creating virtual environments so that you do not have to use sudo.

finally, conda is something that can be installed by users themselves (without sudo) on all platforms.  you may want to consider trying miniconda for your python environment.
This is with an up-to-date Ubuntu 16.04 on my end.
This now triggers a "pip is out of date report". I guess this is an ubuntu problem.
Yes, that should go away with `pip install pip --upgrade`
Upgrading pip does not solve this.
@gdevenyi - did you update setuptools as well? could you please provide us a report of your python environment packages?
I've moved to a miniconda environment to solve this. I'm building an automated VM container in cloud environment so its hard to poke around.

To reproduce,
Ubuntu 16.04, fully up-to-date, pip install nipype

16.04 is a long term release so I would hope this could be worked around so that LTS users can install it.
Also an issue in Python 3 on Ubuntu.


```
pip3 install nipype
...
    error: can't copy 'nipype/testing/data/brukerdir/pdata': doesn't exist or not a regular file
    
    ----------------------------------------
Command "/usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-puamnggj/nipype/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-2y8phpp7-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-build-puamnggj/nipype/
```

thanks @gdevenyi @ashgillman , I was able to replicate as well. This will be fixed for our upcoming release - til then I would recommend using the conda installation.
@ashgillman, @gdevenyi - i tested it via docker (`docker run -it --rm ubuntu:16.04`). 

```
apt-get -y update
apt-get -y upgrade
apt-get install -y python-pip

root@afeaec958e1f:~# pip --version
pip 8.1.1 from /usr/lib/python2.7/dist-packages (python 2.7)
root@afeaec958e1f:~# python -c "import setuptools; print(setuptools.__version__)"
20.7.0
```

this fails
```
pip install nipype
```

but then this succeeds after upgrading setuptools.
```
pip install --upgrade setuptools
root@afeaec958e1f:~# pip install --upgrade setuptools
Collecting setuptools
  Downloading setuptools-36.6.0-py2.py3-none-any.whl (481kB)
    100% |################################| 481kB 1.4MB/s 
Installing collected packages: setuptools
  Found existing installation: setuptools 20.7.0
    Uninstalling setuptools-20.7.0:
      Successfully uninstalled setuptools-20.7.0
Successfully installed setuptools-36.6.0

pip install nipype
```

so the workaround is really just to upgrade `setuptools`, which didn't behave properly. i hope this is not too much of a workaround. 

on the nipype side we can set a version requirement for setuptools and distribute a wheel.
@satra, we should then add `'setuptools>=XX.x.x'` here 

https://github.com/nipy/nipype/blob/557aad3370f820e3765454ea6774f95826ed8fc1/setup.py#L122

That should make sure setuptools is upgraded before running the installation.
@satra Fixes for me :)

And I believe a better solution than having to use conda.

Hopefully @oesteban solution fixes permanently
@satra fixes for me too! Thanks

(Ubuntu 16.04, non-conda python - perhaps remove "windows"?)

Resolving patch:
diff --git a/conftest.py b/nipype/conftest.py
similarity index 77%
rename from conftest.py
rename to nipype/conftest.py
--- a/conftest.py
+++ b/nipype/conftest.py
@@ -8,5 +8,5 @@ def add_np(doctest_namespace):
 
 
     filepath = os.path.dirname(os.path.realpath(__file__))
-    datadir = os.path.realpath(os.path.join(filepath, 'nipype/testing/data'))
+    datadir = os.path.realpath(os.path.join(filepath, 'testing/data'))
     doctest_namespace["datadir"] = datadir
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -100,15 +100,17 @@ def main():
         pjoin('testing', 'data', 'dicomdir', '*'),
         pjoin('testing', 'data', 'bedpostxout', '*'),
         pjoin('testing', 'data', 'tbss_dir', '*'),
-        pjoin('testing', 'data', 'brukerdir', '*'),
-        pjoin('testing', 'data', 'brukerdir', 'pdata', '*'),
+        pjoin('testing', 'data', 'brukerdir', 'fid'),
         pjoin('testing', 'data', 'brukerdir', 'pdata', '1', '*'),
+        pjoin('testing', 'data', 'ds005', '*'),
         pjoin('workflows', 'data', '*'),
         pjoin('pipeline', 'engine', 'report_template.html'),
         pjoin('external', 'd3.js'),
         pjoin('interfaces', 'script_templates', '*'),
         pjoin('interfaces', 'tests', 'realign_json.json'),
         pjoin('interfaces', 'tests', 'use_resources'),
+        'pytest.ini',
+        'conftest.py',
     ]
 
     # Python 3: use a locals dictionary


Test patch: