BUG: Travis building on container-based infrastructure
State: closed

- [ ] closes #10598
- [x] passes `git diff upstream/master | flake8 --diff`
- [ ] whatsnew entry

Travis is not always detecting sudo use and is running forks using container-based infrastructure. `JOB_NAME=27_slow_nnet_LOCALE` fails when using containers because of setlocale. This PR adds `sudo: required` to make sure travis does not use containers while sudo is needed in the travis set up scripts. 

Example: 
- https://travis-ci.org/nparley/pandas/jobs/124695152 (container)
- https://travis-ci.org/nparley/pandas/jobs/124778492 (non container)


Labels:
name: Build
description: Library building on various platforms

Comments:
does this cause anything to fail (on all runs)? e.g. so it seems that we are NOT testing the locale stuff then ? (w/o the sudo)

These are all the runs of a clean fork running on containers https://travis-ci.org/nparley/pandas/builds/124695150 (vs not https://travis-ci.org/nparley/pandas/builds/124778490). Most of the runs pass fine it seems. `27_nslow_nnet_COMPAT` also has the problem. `35_numpy_dev` also fails with:

```
ImportError: libatlas.so.3gf: cannot open shared object file: No such file or directory
```

you may need to change the `ci/install-3.5_NUMPY_DEV.sh` to have that work, it does an `sudo apt-et`......to install the system libraries (as its pulling down wheels, not conda envs for testing)

Would the preference be to try and get travis working with containers over setting to sudo required?

IIUC travis migrated us to a 'new' container infrastructure, but we are still doing it the 'old' way? is that accurate (IOW, the old warning has disappeared but they are doing some sort of magic to still put us on the containers). or is that completely wrong?

ideally we want to be doing this the most preferred way on travis. we don't care too much are actually using sudo as eveything lives in a conda env anyhow.

From what I understand if you were using travis before 2015-01-01 you are still using standard infrastructure.

> - For repos we recognize before 2015-01-01, linux builds are sent to our standard infrastructure.
> - For repos we recognize on or after 2015-01-01, linux builds are sent to our container-based 

(from https://docs.travis-ci.com/user/workers/container-based-infrastructure/)

When you are running on containers you get the message:

```
This job is running on container-based infrastructure, which does not allow use of 'sudo', setuid and setguid executables.
If you require sudo, add 'sudo: required' to your .travis.yml
See https://docs.travis-ci.com/user/workers/container-based-infrastructure/ for details.
```

which does not appear for `pydata / pandas` builds. So I don't think pandas has been migrated. Or it was but now isn't again. 

`hostname: travis-worker-gce-org-prod4-4:8e9fb454-c5a1-4d27-bb67-cb0494ed1029`

so we are on gce. is that the container? 

No they are just ubuntu VMs on gce I think. It's a container if it has docker worker:

```
Using worker: worker-linux-docker-d4bd0a53.prod.travis-ci.org:travis-linux-14
```

They changed their VMs to gce as well as adding containers and then it was a bit confusing if you were being migrated to gce or containers.

@nparley ok I guess we are not on containers. I _would_ like to change. The main reason is that we can then use Travis own caching system, rather then our out-sourced version (with `iron cache`), which while working is prob not as robust and integrated etc.

@nparley does this now run on containers? can you rebase and repush

@jreback no this PR actually is just forcing Travis _not_ to run on containers to make sure it builds. I have just moved house the last two weeks so haven't had time to look at it again but will have a go at seeing if the locale stuff can be done at containers soon.

This is now building on containers ok,  https://travis-ci.org/nparley/pandas/builds/128974759. I changed the local override to zh_CN.UTF-8 from zh_CN.GB18030. The PR needs a clean up and a squash I think. The thing I have not looked at yet is the caching part.

can we print the local in `pandas/util/print_versions.py`

```
In [2]: import locale

In [3]: locale.getlocale()
Out[3]: (None, None)
```

```
    try:
        (sysname, nodename, release,
         version, machine, processor) = platform.uname()
        blob.extend([
            ("python", "%d.%d.%d.%s.%s" % sys.version_info[:]),
            ("python-bits", struct.calcsize("P") * 8),
            ("OS", "%s" % (sysname)),
            ("OS-release", "%s" % (release)),
            # ("Version", "%s" % (version)),
            ("machine", "%s" % (machine)),
            ("processor", "%s" % (processor)),
            ("byteorder", "%s" % sys.byteorder),
            ("LC_ALL", "%s" % os.environ.get('LC_ALL', "None")),
            ("LANG", "%s" % os.environ.get('LANG', "None")),

### add here
```

I _think_ this should print on the 2 non-default locales.

## [Current coverage](https://codecov.io/gh/pydata/pandas/pull/12946?src=pr) is **84.33%**

> Merging [#12946](https://codecov.io/gh/pydata/pandas/pull/12946?src=pr) into [master](https://codecov.io/gh/pydata/pandas/branch/master?src=pr) will increase coverage by **<.01%**

``` diff
@@             master     #12946   diff @@
==========================================
  Files           138        138          
  Lines         51106      51107     +1   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits          43102      43103     +1   
  Misses         8004       8004          
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last updated by [1a12ead...bbc7b35](https://codecov.io/gh/pydata/pandas/compare/1a12eadaf92eb1ee05446d24e5d7a12cde971ce3...bbc7b353372bef18d79c68e8a4725dbf6131565c)

@nparley looking good. is there any speed diff?

I think they spin up faster but don't really build faster. Might need to investigate caching. 

There is a problem with blosc creating a stack trace. It seems to be happening in the main travis builds as well when `ci/print_version.py` is called. I have managed to get the problem in a docker container outside of travis too. I have commented out blosc version in the lastest commit to see if it goes away. The question is though does this mean blosc if miss behaving somewhere else?

not sure exactly where blosc is problematic
prob a threading issue - u might have to play with it and see

you can file a separate bug report on that

rebase this and ping when green.

squash too

I wonder if `source activate pandas && ci/print_versions.py` should be removed from the after script. I think it better now the versions info being printed in the `ci/script.sh` file as that reflects the locale that has been set. As the after script is run in a new shell the locale does not reflect the locale that was set in `ci/script.sh`

I think the local setting needs to be in a separate scripts that you can call in `scripts.sh` AND before the post script print_versions. We need to have that separate as it makes a distinct section in travis.

you could also change the after_script to `ci/after_print_versions.sh` or something and put the locale, then source, then run print_versions.py there

@jreback  I have created this branch https://github.com/nparley/pandas/compare/travis...nparley:travis_cache which adds travis caching to this PR and removes the old caching. It shaves some time off the builds and almost gets the builds down below 3 hours. 

The logic for caching is:

cache `$home/miniconda` for each build

cache `$home/.cccache` for each Linux build and use the ccache that's installed on travis. 

cache `$home/.cache` for each build with the cython files. 

This is the build https://travis-ci.org/nparley/pandas/builds/134152663

Is it ok to merge that into here?

take out the old caching stuff as well. `prep_cache/submit_cache`.

Further is this cached somehow by the branch-name-user? e.g. we only really want to cache pydata/pandas master (everything else should be fresh).

Each brach has it's own cache. I.e. there is a master cache, then for the first branch push off master the build will use the master cache but then create it's own cache and use that for the next push. For PRs they use the cache of the branch that they will be merged into but will not write to the cache, only pushes to the branches or master do that. This means that feature branches have there own build cache separate from the master. 

As for if it should be clean off the master:
- For the miniconda bit I think that's ok if it's coming from cache. The script runs a conda update and then if any of the versions have changed in the requirements they will be updated. So I don't see any need for a fresh miniconda install. 
- The cython bit checks if any cython files have changed in the git diff of the commit and if so rebuilds the cython files, should be ok but maybe should rebuild. 
- The compiler cache should maybe not be used incase it hides a bug. 

The solution is to check the travis variables like `$TRAVIS_PULL_REQUEST` and `$TRAVIS_BRANCH` in the scripts. For example if we don't want any compiler or cython caching outside of the master I will just add `if $TRAVIS_BRANCH == 'master'` around the cache parts. If we are happy about caching in feature branches but not in PRs I will use `$TRAVIS_PULL_REQUEST == false`

I am actually ok with leaving caching on for all (not sure how travis actually ages caching, but that's there issue). Though maybe add a variable CACHE=True (or something), or have a single build (in the allowed failures section), that _doesn't_ cache.

Which is the best build not to cache?

the 3.5 - last of required ones -

Right I think this is mostly done. Maybe I'll keep it up to date for upstream/master for a few commits to make sure it behaves, then I'll squash it.

ok, we need a way to invalidate the caching. IOW say a file is changed in `ci/*` compared to the cache. I would turn off caching. Is it possible to do that? The main issue is that caching could last forever otherwise, and will 'work' even though we updated dependencies.

If `use_cache` is not set in the travis build the job will not use the cache but will write to the cache after the build. (Maybe it would be a good idea to actually delete all the files in the cache if `use_cache` is not set as well, to be extra sure it's clean). I could unset use_cache if anything has been touched in ci/\* in the last commit?