TST: XFAIL Travis read_html tests
State: closed

- [x] xref #29622

Labels:
name: Testing
description: pandas testing functions or related to the test suite

Comments:
can't make it strict?
looks like there was an xpass on the old_np build.  possibly related: i think bs4 had a release yesterday
> looks like there was an xpass on the old_np build. possibly related: i think bs4 had a release yesterday

Due to the fact we pin bs4 to 4.6.0 in the `py36_locale_slow_old_np` build. I've unpinned - let me know if this is not appropriate?

 Other option is to make this not strict?
> I've unpinned - let me know if this is not appropriate?

@datapythonista preference for this pin?

> Other option is to make this not strict?

If all else fails, yah.  I appreciate you putting in the effort to avoid this last resort.
Can confirm travis-36-slow build is now passing - https://travis-ci.org/pandas-dev/pandas/jobs/631033419?utm_medium=notification&utm_source=github_status

We can remove this from allowed failures in `.travis.yml` if desired
> @datapythonista preference for this pin?

We've got bs4 also pinned at 4.6.0 in the minimum versions build. I think ideally we should have a minimum versions build for the slow tests too. I guess if we had it, it'd fail like the old np build. So I guess unpinning will avoid the failure, but won't really fix the problem.

Do we want to increase the minimum required version instead?
> We've got bs4 also pinned at 4.6.0 in the minimum versions build. I think ideally we should have a > minimum versions build for the slow tests too. I guess if we had it, it'd fail like the old np build. So I > guess unpinning will avoid the failure, but won't really fix the problem.

Thanks for the info!

Should the minimum version build just also run the slow tests opposed to a sep build?

> Do we want to increase the minimum required version instead?

Okay shall we bump to min beautifulsoup4 == 4.8.0 ? (`azure-36-minimum_versions.yml` and `azure-36-locale_slow.yaml`) Therefore this xfail can be strict across all builds?


> Should the minimum version build just also run the slow tests opposed to a sep build?

My understanding is that a build with both the "normal" and the slow tests would be too slow. That's why we have the slow marker, so they are called in separate builds.



> Okay shall we bump to min beautifulsoup4 == 4.8.0 ?

This is from just few months ago. I think we're usually compatible with dependencies that are at least couple of years old.
> > Should the minimum version build just also run the slow tests opposed to a sep build?
> 
> My understanding is that a build with both the "normal" and the slow tests would be too slow. That's why we have the slow marker, so they are called in separate builds.
> 
> > Okay shall we bump to min beautifulsoup4 == 4.8.0 ?
> 
> This is from just few months ago. I think we're usually compatible with dependencies that are at least couple of years old.

Thanks @datapythonista for the info!

Okay so i've adjust the xfail to take into account the version of bs4 installed + removed the travis 36 slow build from the allowed failures.

Happy to create a min version build for the slow tests too as you suggested (sep PR). If that's a useful for us?

> Happy to create a min version build for the slow tests too as you suggested (sep PR). If that's a useful for us?

I think it makes more sense to discuss and add this as part of #29685. I don't think we really understand well what's being tested. I think creating builds from zero in an organized way is better than add this individually. I'm on it.
All green - @jbrockmendel / @datapythonista mind reviewing? 
Thanks @alimcmaster1