CI debug
State: closed

Closes https://github.com/pandas-dev/pandas/issues/27541

Labels:
name: CI
description: Continuous Integration

Comments:
Different error this time right?
We did have a timeout earlier, but I think it looked a bit different than the last one. I restarted the build.
I don't know if directly relevant here since current build complies but there is a note for pytest >5.0.0 that if used with pytest-xdist that should at least be 1.29.0 or later in the changeling:

https://docs.pytest.org/en/latest/changelog.html#features

Might be related to some of the issues (this one seems suspiciously related to the one we saw last week on Azure)
Though it does look like the issue Jeff has in the OP of #27541 has pytest-xdist 1.28.0 and pytest 5.0.1 so issue might just boil down to that
Ignore comments above - I should have looked at diff. But yea this seems like the right change
Huh that sounds extremely relevant. In a meeting now, but will update everything to be pytest >5, pytest-xdist >=1.2.9 (unless someone beats me).
Current build here meets those specs but just failed again so I think it's still something else
Trying to pin that build to `pytest<5`. Won't have more time for this today unfortunately.
Well this one failed with out of memory errors when we tried to open a subprocess

```
=================================== FAILURES ===================================
_____________________________ test_initial_warning _____________________________
[gw0] linux -- Python 3.6.8 /home/travis/miniconda3/envs/pandas-dev/bin/python
    def test_initial_warning():
        code = (
            "import pandas as pd; import matplotlib.pyplot as plt; "
            "s = pd.Series(1, pd.date_range('2000', periods=12)); "
            "fig, ax = plt.subplots(); "
            "ax.plot(s.index, s.values)"
        )
        call = [sys.executable, "-c", code]
>       out = subprocess.check_output(call, stderr=subprocess.STDOUT).decode()

```
Yea was about to comment. I have the feeling this might be the root issue and the original issues we saw were problems with pytest / pytest-xdist compat and actually reporting out the error.
Is it this same test failing repeatedly?  We could wrap the subprocess call in something like:

```
try:
     subprocess...
except OSError as err:
     if err.errno == errno.ENOMEM:
          pytest.xfail(...)
     raise
```
We do seem to have less *disk space*, which indicates we may be running on a smaller VM. From https://travis-ci.org/pandas-dev/pandas/jobs/561486224 (3 days ago)

```
[0K$ df -h
Filesystem      Size  Used Avail Use% Mounted on
udev            3.7G  4.0K  3.7G   1% /dev
tmpfs           748M  292K  748M   1% /run
/dev/sda1        69G   11G   55G  17% /
none            4.0K     0  4.0K   0% /sys/fs/cgroup
none            5.0M     0  5.0M   0% /run/lock
none            3.7G     0  3.7G   0% /run/shm
none            100M     0  100M   0% /run/user
none            768M  179M  590M  24% /var/ramfs
```

And from today

```
[0K$ df -h
Filesystem      Size  Used Avail Use% Mounted on
udev            1.8G  4.0K  1.8G   1% /dev
tmpfs           370M  292K  370M   1% /run
/dev/sda1        69G   12G   55G  17% /
none            4.0K     0  4.0K   0% /sys/fs/cgroup
none            5.0M     0  5.0M   0% /run/lock
none            1.9G     0  1.9G   0% /run/shm
none            100M     0  100M   0% /run/user
none            768M  179M  590M  24% /var/ramfs
```

I'm not sure if we print our RAM in the logs.
FYI, we may want to keep an eye on https://travis-ci.community/t/builds-often-fail-because-of-lack-of-resources-memory-cpu/4331 and https://github.com/travis-ci/worker/issues/604. Seems like the worker images are being given less resources.

For now, we might lighten the load on that build by removing some optional deps. I'd propose removing matplotlib for a start.
Another OOM crash. I'm removing matplotlib from those two environments.
OK, that passed. Any objections to merging as is, and cleaning up the CI files in a followup PR? I suspect the pytest & pytest-xdist pins are no longer necessary, but it'd be good to get other PRs going again.
Ok by me
hmm I suspect we need to backport this to 0.25.1