CI: Fix clipboard problems
State: closed

- [X] closes #29676

Fixes the clipboard problems in the CI. With this PR we're sure they are being run, and they work as expected.

Labels:
name: CI
description: Continuous Integration
name: Unreliable Test
description: Unit tests that occasionally fail

Comments:
First run successful (problems uploading the coverage, but all tests passed in all builds). Restarting...
Or I don't think this should harm, if you prefer to merge and see in the other builds... Otherwise I'll restart the job couple times more to see if this looks like it's fixing the unreliable test.
Looks like this doesn't fix the problem. Re-reading again the docs, I think the new option would help errors if there are multiple `xvfb` processes, not if the wrapped command starts multiple processes.

I think it's still worth merging this, but still need to find out what's making that test fail.
@datapythonista maybe try https://github.com/pandas-dev/pandas/pull/29695/commits/6557e9b10f4e4138f674bff1788a56d109c66617 out? I didn't look into the failures for that run. Turns out they were unrelated.

I'm not sure how pytest-xdist splits up modules, but if it's done after importing the file, that first one would have already run.
> @datapythonista maybe try [6557e9b](https://github.com/pandas-dev/pandas/commit/6557e9b10f4e4138f674bff1788a56d109c66617) out?

Will have a look. `pytest-xdist` should allocate whole files to workers, I guess that will happen in the worker already, but not sure.
The whole file is definitely run on one worker (you can check with `-v`).
I'm just unsure whether or not the top-level of the module is executed
before being assigned to a worker.

On Tue, Nov 19, 2019 at 1:03 PM Marc Garcia <notifications@github.com>
wrote:

> @datapythonista <https://github.com/datapythonista> maybe try 6557e9b
> <https://github.com/pandas-dev/pandas/commit/6557e9b10f4e4138f674bff1788a56d109c66617>
> out?
>
> Will have a look. pytest-xdist should allocate whole files to workers, I
> guess that will happen in the worker already, but not sure.
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> <https://github.com/pandas-dev/pandas/pull/29712?email_source=notifications&email_token=AAKAOIQQMK4AK6HGJCH5OITQUQ2ALA5CNFSM4JPGDLK2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEEPK64A#issuecomment-555659120>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AAKAOIW3EEZZBWFXS66BGX3QUQ2ALANCNFSM4JPGDLKQ>
> .
>

could we put this into an allowed_fail or something so this doesn't hang up other PRs?  or would that make it too likely to slip through the cracks?
Pushed up the deferred check. I think if that's failing and @datapythonista is out of ideas then we can xfail these tests.
got green here.  does that mean this is fixed or just a lucky run?
Lucky run so far, let's try again...
Yep, you're right. Not sure why I thought only the methods using `mock_clipboard` as a parameter. A bit strange that the only test that actually tests the clipboard is that last one, that seems to be added later to test the long unicode characters.

What I see is that `xsel` is only installed in an Azure build. In the failing build `qt` is indirectly installed by `conda`. I want to see were tests fail if they are not skipped. They are skipped if the clipboard doesn't work, and I guess it works when `xsel` or `qt` are found, but it's difficult to really tell (even more when conda can be installing them because of an unrelated dependency).

I'll continue tomorrow, it's 2am here, but once we know where they are running we may want to do one of:
- Install `xsel` there
- Remove `qt`

I'd also remove the skip, and add to the patterns `(not slow and not network and not clipboard)` in the builds where we don't want them to run. Not beautiful, but understanding what is going on with the magic of skips is very difficult. And I think it's a good fix enough for now, considering that we should be simplifying the tests soon in #29685.
@datapythonista I've xfailed those tests on CI for now, so less urgency on this now. We'll want to get it sorted out before the release though.
@jreback @TomAugspurger this should fix the problems with the clipboard, and stop xfailing the tricky test.

What I'm doing here:
- Be explicit on when we run the tests (previously, we ignored the clipboard tests if the pandas `to_clipboard()` method was failing)
- Use `xvfb` when a X server is not available in the CI
- Install `xsel` in all the builds where we test the clipboard
- Uninstall `qt`, which was causing problems
Thanks for tracking this down. A bit unfortunate about qt, but uninstalling is probably the right move. I don’t expect this to cause issues elsewhere. 

> On Jan 14, 2020, at 19:47, Marc Garcia <notifications@github.com> wrote:
> 
> ﻿
> @jreback @TomAugspurger this should fix the problems with the clipboard, and stop xfailing the tricky test.
> 
> What I'm doing here:
> 
> Be explicit on when we run the tests (previously, we ignored the clipboard tests if the pandas to_clipboard() method was failing)
> Use xvfb when a X server is not available in the CI
> Install xsel in all the builds where we test the clipboard
> Uninstall qt, which was causing problems
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub, or unsubscribe.

lgtm. I moved to the 1.0.0 milestone. @TomAugspurger if you are ok, merge away.
Thanks @datapythonista!
The strange thing is that it seems this has not been fixed on i586. In openSUSE, test suite passes with x86_64, but fails with i586 on test_raw_rountrip:
```
[ 1652s] ______________________ test_raw_roundtrip[\U0001f44d...] _______________________
[ 1652s] [gw6] linux -- Python 3.8.3 /usr/bin/python3
[ 1652s] 
[ 1652s] data = '👍...'
[ 1652s] 
[ 1652s]     @pytest.mark.single
[ 1652s]     @pytest.mark.clipboard
[ 1652s]     @pytest.mark.parametrize("data", ["\U0001f44d...", "Ωœ∑´...", "abcd..."])
[ 1652s]     def test_raw_roundtrip(data):
[ 1652s]         # PR #25040 wide unicode wasn't copied correctly on PY3 on windows
[ 1652s]         clipboard_set(data)
[ 1652s] >       assert data == clipboard_get()
[ 1652s] E       AssertionError: assert '👍...' == ''
[ 1652s] E         + 👍...
[ 1652s] 
[ 1652s] ../../BUILDROOT/python-pandas-1.0.5-44.4.i386/usr/lib/python3.8/site-packages/pandas/tests/io/test_clipboard.py:256: AssertionError
```

[Complete logs of the failing build](https://github.com/pandas-dev/pandas/files/4851944/_log.zip)

Can you verify that the clipboard is set up properly on that system? If not then the clipboard tests need to be skipped.
> Can you verify that the clipboard is set up properly on that system? If not then the clipboard tests need to be skipped.

1. What do you mean exactly by “properly set up clipboard”? This is chroot-based build system for openSUSE. We use ``xvfb-run`` to run tests in the fake X11 environment, but it is quite possible there is something wrong with it.

2. Couldn’t the test system somehow identify problems with clipboard settings?
We were erroneously skipping the clipboard tests so we decided to make these opt-out.