BLD: Install scripts tests only during inplace
State: closed

The `scripts` directory is not visible if the installation is not inplace.

Follow-up to #20061.

cc @WillAyd @datapythonista 

Labels:
name: Bug
name: Build
description: Library building on various platforms

Comments:
Would the other option be to just skip these tests if the scripts folder isn't included? I actually wonder if that would be preferable especially in cases where users install a binary and try to run `pd.test()` in the REPL
@WillAyd : I don't see why we should be packaging these tests when we do a release of `pandas` to the general public (the `scripts` directory won't be present at all).  That's another reason why I would prefer exclusion vs. just skipping tests.
@jreback : See my answer [above](https://github.com/pandas-dev/pandas/pull/22413#issuecomment-414094889).  The only time these tests should be running is if we do an inplace install, since that's when the `scripts` directory will be visible.  Otherwise, why should we be packaging those tests at all in the distribution (e.g. the public releases)?
# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/22413?src=pr&el=h1) Report
> Merging [#22413](https://codecov.io/gh/pandas-dev/pandas/pull/22413?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/4f11d1a9a9b02a37dbe109d8413cc75d73b92853?src=pr&el=desc) will **not change** coverage.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/22413/graphs/tree.svg?token=eZ4WkYLtcO&width=650&src=pr&height=150)](https://codecov.io/gh/pandas-dev/pandas/pull/22413?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master   #22413   +/-   ##
=======================================
  Coverage   92.05%   92.05%           
=======================================
  Files         169      169           
  Lines       50733    50733           
=======================================
  Hits        46702    46702           
  Misses       4031     4031
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `90.46% <ø> (ø)` | :arrow_up: |
| #single | `42.24% <ø> (ø)` | :arrow_up: |


------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/22413?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/22413?src=pr&el=footer). Last update [4f11d1a...c14c540](https://codecov.io/gh/pandas-dev/pandas/pull/22413?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

> Would the other option be to just skip these tests if the scripts folder isn't included?

\+1
@gfyoung I understand your argument, but I think the added complexity (I would expect to find logic about whether the tests are ran/skipped in the tests itself, our `setup.py` is already very complex) is not worth the few kb's for those tests. 
agree with @jorisvandenbossche here, pls simply skip these test. We don't want multiple paths for doing things like skips.
Fair enough.  IMHO, these validation tests should not have been located in the `pandas/tests` directory, but oh well. 🤷‍♂️ 
Not sure what it involves in the CI configuration, but I'm +1 on moving the tests for the scripts to `scripts/tests` too.
@jreback @jorisvandenbossche : Thoughts on moving the tests?  As much as I know both of you don't want the extra complexity in `setup.py` of "skipping" the copying of certain files during distribution, I am also not a fan of seeing tests fail locally just because the `scripts` isn't packaged.

We can skip the tests for now, but the long-term solution I think is just moving them elsewhere.
> I am also not a fan of seeing tests fail locally just because the scripts isn't packaged.

They wouldn't fail, the idea is that they would be skipped.

That said, I am also fine with having the tests live in `scripts/tests/, if that doesn't complicate the travis set-up too much to also run those (I suppose now we everywhere only test `pandas` directory?)
I'm -1 on moving to `scripts/tests`. If we are worried about the added size of the new module then we are making micro-optimizations whose benefit would pale in comparison to the effort making sure that we can account for tests that live outside of the `tests` directory
> whose benefit would pale in comparison to the effort making sure that we can account for tests that live outside

@WillAyd : I'm not sure we can make this assertion yet without giving it a shot first. 😉 
let's see if this fixes the wheels
ok wheels fixed: https://travis-ci.org/MacPython/pandas-wheels/builds/418513881