Repository: statsmodels/statsmodels

Issue:
patsy is a hard dependency?
setup.py checks for the presence of numpy,scipy and pandas, but not patsy.
This is with current git master.

```
$ python -c "import statsmodels.api as sm"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/travis/virtualenv/python2.7_with_system_site_packages/local/lib/python2.7/site-packages/statsmodels/api.py", line 3, in <module>
    import regression
  File "/home/travis/virtualenv/python2.7_with_system_site_packages/local/lib/python2.7/site-packages/statsmodels/regression/__init__.py", line 1, in <module>
    from linear_model import yule_walker
  File "/home/travis/virtualenv/python2.7_with_system_site_packages/local/lib/python2.7/site-packages/statsmodels/regression/linear_model.py", line 41, in <module>
    import statsmodels.base.model as base
  File "/home/travis/virtualenv/python2.7_with_system_site_packages/local/lib/python2.7/site-packages/statsmodels/base/model.py", line 10, in <module>
    from statsmodels.formula import handle_formula_data
  File "/home/travis/virtualenv/python2.7_with_system_site_packages/local/lib/python2.7/site-packages/statsmodels/formula/__init__.py", line 4, in <module>
    from formulatools import handle_formula_data
  File "/home/travis/virtualenv/python2.7_with_system_site_packages/local/lib/python2.7/site-packages/statsmodels/formula/formulatools.py", line 2, in <module>
    from patsy import dmatrices
ImportError: No module named patsy
```



Discussion:
I also ran into this with a virtualenv last week.

I would add it as a requirement that can be automatically installed, however, we will most likely include patsy directly in a source or binary distribution, and link it in the repository.

Related to #549.

I installed statsmodels using pip:

```
$ pip install statsmodels
Downloading/unpacking statsmodels
  Downloading statsmodels-0.5.0-tutorial-beta.tar.gz (4.4Mb): 4.4Mb downloaded
```

when running it fails with ``patsy` not installed.

```
  File "/.virtualenvs/.env/options/local/lib/python2.7/site-packages/statsmodels/formula/__init__.py", line 4, in <module>
    from formulatools import handle_formula_data
  File "/.virtualenvs/.env/options/local/lib/python2.7/site-packages/statsmodels/formula/formulatools.py", line 2, in <module>
    from patsy import dmatrices
ImportError: No module named patsy
```

It seems that `patsy` is indeed a required dependency.

Thanks

Hmm, I'm considering adding this to install_requires. It's not quite so bad if something goes wrong with a patsy install as it is with numpy/scipy. Thoughts?

Patsy install_requires numpy though (and Nathaniel doesn't want to change that). So pip/easy_install `-U` will still upgrade numpy.

Patsy doesn't require a specific version of numpy. 

easy_install doesn't upgrade dependencies, at least when I looked at it a long time ago

What does pip install -U do?  (I'm not sure and I thought it behaves the same as easy_install)
From a quick search it really sucks. (Uninstalling and reinstalling everything ?)
http://stackoverflow.com/questions/2861183/pip-upgrade-package-without-upgrading-dependencies
https://github.com/pypa/pip/issues/search?q=upgrade

Last time I worked with the virtualenv and pip install, I thought it just didn't recognize my manually installed numpy, and it reinstalled numpy for that reason.

the patsy setup.py can detect an installed numpy version and specify it as 
a ver requirement in install_requires. That would hemstring pip -U.

I'm not sure what the suggestion is for us in these comments / if there is one. 

I looked into git submodules some more last night, and that's not the way to go here (I don't think).

So, AFAIK, we have two choices.

1) just specify patsy as a dependencies in the places it's missing and check for it just like we're checking for numpy/scipy/etc. 
2) we put it in install_requires and don't worry about whatever it may do / assume users know what they're doing

My suggestion was to go ahead and use install_requires, with the suggested 
workaround for  the "pip -U" problem raised by @rgommers. 

The way I see it now, if pip install -U reinstalls all dependencies, the 2) is out, unless patsy changes.

2) would mean if my tox testing uses `pip install -U statsmodels`, then it would wipe my virtualenv (as it had before) but it's too stupid to rebuild it again from the binaries. `-U statsmodels` triggers `-U patsy` triggers `-U numpy` which is definitely useless for version testing.
So it essentially kills my tox version testing

2/2 means my explanation wasn't clear. Let me try again.
If patsy's setup.py contained something along the lines of:

```
np_ver = ""
try:
   import numpy as np
   np_ver = "==" + str(np.__version__)
except:
   pass

...
setup(
install_requires = ['numpy'+np_ver ]
}
```

then when you did

```
pip install -U statsmodels
```

numpy would not be upgraded when patsy was.

Yes, but this is a patsy issue not something we can fix on our end right? I can make a PR there.

That's right. It was my understanding that the two projects are siblings.
If that's not the case, my suggestion might not be a good way to go.

And this would have to be contained in the patsy version that is currently released right? I have to relearn everything about setuptools/distribute/distutils every time I want to get any of this right, and I just don't have time.

True as well. and yes, (python) packaging is often dirty work.

I notice that you check all the other dependencies and complain if they are missing.
why not just do that for patsy as well?

Having originally opened this ticket, I would have been fine with that.

Then that's a vote for 1. ;) (And it was mostly oversight)

And I overlooked 1. yes, that's +1 for 1

I'm also +1 for 1)   given the current behavior of patsy and pip


Resolving patch:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -51,6 +51,11 @@ def check_dependency_versions(min_versions):
         from pandas.version import version as pversion
     except ImportError:
         raise ImportError("statsmodels requires pandas")
+    try:
+        from patsy import __version__ as patsy_version
+    except ImportError:
+        raise ImportError("statsmodels requires patsy")
+
     try:
         assert StrictVersion(strip_rc(npversion)) >= min_versions['numpy']
     except AssertionError:
@@ -70,6 +75,13 @@ def check_dependency_versions(min_versions):
         raise ImportError("Pandas version is %s. Requires >= %s" %
                 (pversion, min_versions['pandas']))
 
+    try: # patsy dev looks like 0.1.0+dev
+        pversion = re.match("\d*\.\d*\.\d*", patsy_version).group()
+        assert StrictVersion(pversion) >= min_versions['patsy']
+    except AssertionError:
+        raise ImportError("Patsy version is %s. Requires >= %s" %
+                (pversion, min_versions["patsy"]))
+
 
 MAJ = 0
 MIN = 5
@@ -192,6 +204,7 @@ def configuration(parent_package='', top_path=None, package_name=DISTNAME):
         'numpy' : '1.4.0',
         'scipy' : '0.7.0',
         'pandas' : '0.7.1',
+        'patsy' : '0.1.0',
                    }
 
     check_dependency_versions(min_versions)


Test patch: