Alter imports in pandas/__init__.py to be explicit
State: closed

Also adds an `__all__` to all the api files as well as a test case that
checks that everything in the api files ends up in the toplevel pandas
namespace. I personally find it hard to trace back the top level pandas
functions because of the `from xyz import *` calls. I changed everything
to be explicit. I think it's better, but it's your call.

The only large change with this is that the `pandas` module/package no
longer exports `numpy/np` at the top level.


Labels:

Comments:
let me think about this 

the problem is you have to then explicity add to this file (which is not a bad thing),
but something else to remember to do and keep in sync.....

@jreback on the flip side, it means that new features explicitly have to be added to the top level namespace. I think that's a net positive.  It _may_ also make it easier for static code analyzers to pick up the toplevel pandas methods. (not sure if that's true...I know that Spyder already does it just fine).

Regardless, I made this up because I wanted to suggest it and see what others thought.

I'm not sure, I think this way looks more confusing... atm, things are ~~only~~ mostly in the top-level if there are in a `something/api.py`, which I kind of like/seemed kind of neat...

Tracing back functions is usually easy (or am I misunderstanding what you mean):

```
In [1]: pd.ExcelWriter
Out[1]: pandas.io.excel.ExcelWriter  <- now I know where to find it
```

It would be good to find some way to avoid polluting the namespace with `numpy` and others. Maybe just keeping `__all__` in the api modules and extending `__all__` in `pandas/__init__.py`?

(to be clear, that would mean that `__init__.py` reverts to previous version except for the set of `__all__.extend(...)` statements at the end of the doc)

refresh on what does putting something in **all** do? (in an API module
which for the most part explicitly import things)

as opposed to import \* of the API modules?

`__all__` determines which things are imported when you do `from xyz import
*`. Not that big of a deal, but there are just a few random elements (like
numpy) that shouldn't be included with `from pandas import *` BUT it's
clearly not a big deal.

On Fri, Jun 7, 2013 at 10:19 PM, jreback notifications@github.com wrote:

> refresh on what does putting something in _all_ do? (in an API module
> which for the most part explicitly import things)
> 
> as opposed to import \* of the API modules?
> 
> —
> Reply to this email directly or view it on GitHubhttps://github.com/pydata/pandas/pull/3806#issuecomment-19141474
> .

Ah, so currently you do:

```
In [1]: from pandas import *

In [2]: io  # same with np, etc.
Out[2]: <module 'pandas.io' from 'pandas/io/__init__.pyc'>
```

but not with this pr.

ok them I would be ok doing that on the individual API imports

eg when I add say json I would add to the API import (which I did already) but also to the **all** in the same io/API. py file?

I can be reached on my cell 917-971-6387

On Jun 7, 2013, at 10:22 PM, Jeff Tratner notifications@github.com wrote:

> `__all__` determines which things are imported when you do `from xyz import 
> *`. Not that big of a deal, but there are just a few random elements (like 
> numpy) that shouldn't be included with `from pandas import *` BUT it's 
> clearly not a big deal. 
> 
> On Fri, Jun 7, 2013 at 10:19 PM, jreback notifications@github.com wrote: 
> 
> > refresh on what does putting something in _all_ do? (in an API module 
> > which for the most part explicitly import things) 
> > 
> > as opposed to import \* of the API modules? 
> > 
> > — 
> > Reply to this email directly or view it on GitHubhttps://github.com/pydata/pandas/pull/3806#issuecomment-19141474 
> > . 
> > 
> > —
> > Reply to this email directly or view it on GitHub.

_If you use `import *` you deserve everything you get..._

Yes, exactly.

On Fri, Jun 7, 2013 at 10:27 PM, Andy Hayden notifications@github.comwrote:

> _If you use import \* you deserve everything you get..._
> 
> —
> Reply to this email directly or view it on GitHubhttps://github.com/pydata/pandas/pull/3806#issuecomment-19141569
> .

Rather, yes to @jreback

On Fri, Jun 7, 2013 at 10:39 PM, Jeffrey Tratner
jeffrey.tratner@gmail.comwrote:

> Yes, exactly.
> 
> On Fri, Jun 7, 2013 at 10:27 PM, Andy Hayden notifications@github.comwrote:
> 
> > _If you use import \* you deserve everything you get..._
> > 
> > —
> > Reply to this email directly or view it on GitHubhttps://github.com/pydata/pandas/pull/3806#issuecomment-19141569
> > .

Since most of the module `api.py` are essentially a list of `from module import a, b, c`, isn't this a bit redundant? 

Also, you could just delete the np reference, except that it was put there explicitly. https://github.com/pydata/pandas/commit/8efb3a6e7ec022f699f4a061e99cf65da2eca130

Okay, reasonable objections, I'm closing this.

@dalejung By not including it in the `__all__` it means the `np` doesn't get imported when using

```
from pandas import *
```

At the moment it dumps all the modules into namespace, e.g. io, core, compat, option, tests, tools, stats, util, version...

_I don't think anyone is suggesting we lose `pd.np` :)_

Can we leave this open for a bit and see if anyone has any strong opinions (I confess that I don't but I'm thinking it _could_ be a good idea, I don't think it hurts any).

@hayd btw - this does remove `pd.np`. Is the pandas namespace really supposed to include numpy?

Oh does it?? I thought I tested this branch and it didn't... ! (when doing `import pandas as pd`) ...just tested again and still works, maybe I'm missing something.

Certainly conscious decision by wes to include np as a convenience, he's mentioned it is some talks (and maybe in the book).

It doesn't pull in np with `from pandas import *`, but if you do `import pandas as pd` you have access to `pd.np` ?

yeah, sorry ...haha thinking about it wrong.

On Sat, Jun 8, 2013 at 7:29 AM, Andy Hayden notifications@github.comwrote:

> It doesn't pull in np with from pandas import *, but if you do import
> pandas as pd you have access to pd.np ?
> 
> —
> Reply to this email directly or view it on GitHubhttps://github.com/pydata/pandas/pull/3806#issuecomment-19147302
> .

Is `pd.np` something that people use? I always assumed the import line was there explicitly for `from pandas import *`, putting a commonly aliased module into the current global scope.

i never use it. i import pylab in my ipython config and the pandas stuff that i want in my ipython startup file seems not that useful, but i can imagine for demos it might be...

what did we decide to do with this?

Not sure there's a consensus... I don't think anyone has any strong feelings either way.

I think it seems reasonable (having `io` and friends in the namespace seems a little weird).

@hayd I didn't think about it in terms of demos initially.  It's very useful to be able to do `from pandas import *` and then be able to immediately use `np`. Just don't want to accidentally interfere with the ability to run good demos and work quickly.

I agree with np being there :) (strongly)

so IIRC, then we are trying to protect `from pandas import *` people from themselves?

even though we have many explicity places where we say `import pandas as pd`

we are already listing the imported function in the api modules (e.g. io.api), which is fine, and then they get
auto-propogated to the top-level

I say leave it as is

(unless we need to have the `__all__` set at the api level for some reason?)

slightly separate but related issue but can we get rid of all _relative_ imports? frankly i find them misleading whenever i successfully compile pandas but for whatever reason i cannot import it...eg, accidental install instead of develop  

Do you mean absolute? where are there any relative imports? (i.e., change
all `from pandas import XYZ` to `from . import XYZ`) [or maybe I'm confused
about what relative import means :P]

On Sun, Jun 9, 2013 at 7:21 PM, Phillip Cloud notifications@github.comwrote:

> slightly separate but related issue but can we get rid of all _relative_imports? frankly i find them misleading whenever i successfully compile
> pandas but for whatever reason i cannot import it...eg, accidental install
> instead of develop
> 
> —
> Reply to this email directly or view it on GitHubhttps://github.com/pydata/pandas/pull/3806#issuecomment-19176015
> .

Oh sorry, yeah, [docs on relative imports](http://docs.python.org/2/tutorial/modules.html#intra-package-references). Basically statements of the form `'from . import awesome_module'`; there's a few scattered about:

```
$ grin 'from \.+ import\s+\w'
./pandas/__init__.py:
    6 :     from . import hashtable, tslib, lib
./pandas/algos.pyx:
   60 : from . import lib
./pandas/index.pyx:
   18 : from . import algos, tslib, hashtable as _hash
```