BrandonFors/pandas-github-issues huggingface dataset:
TST: return pytest MarkDecorator from td.skip_if_no #26735
State: closed

- [x] xref https://github.com/pandas-dev/pandas/pull/26735#issuecomment-500156276
- [x] tests added / passed
- [ ] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] I've replaced `safe_import` function with a corresponding test decorator. I've also investigated whether `safe_import` could be replaced with `_safe_import` and the answer is negative (see `pandas/tests/io/excel/test_writers.py`).


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Clean

Comments:
Thanks @ladyyvii !
CI: 3.8 build
State: closed

Travis config is a mystery to me, so this is trial-and-error until I get the hang of it.

Closes https://github.com/pandas-dev/pandas/issues/26626

Labels:
name: CI
description: Continuous Integration

Comments:
see #27266 as well here
For a moment I was amazed that the last "restore everything disabled during debugging" commit came back green.

Then I saw that I had mangled the Travis config so badly that Travis didn't run at all.
FYI, on the NumPy mailing list they announced that they built 3.8 wheels. Looks like https://github.com/MacPython/numpy-wheels/blob/master/.travis.yml 

Maybe https://github.com/MacPython/numpy-wheels/commit/9d8a0adf9ef8ba0c42c2221b5584a9460ffe1e50 + an update of multibuild in the pandas-wheels repo would do the trick.
> Maybe MacPython/numpy-wheels@9d8a0ad + an update of multibuild in the pandas-wheels repo would do the trick.

Does this replace the approach being taken in this PR? Or supplement?
@TomAugspurger this approach is now working.  I'll take a look at the MacPython/multibuild approach to see if its significantly cleaner, but we have this as a fallback.
Hmm, in the output I see

```
Looking in links: https://7933911d6844c6c53a7d-47bd50c35cd79bd838daf386af554a83.ssl.cf2.rackcdn.com/
Requirement already satisfied: numpy in /home/travis/virtualenv/python3.8-dev/lib/python3.8/site-packages (1.18.0.dev0+ea66b1d)
```

but it isn't clear to me how we got NumPy previously.
Reverted to use git clone, with comment about possible pip alternative
@jreback thoughts on https://github.com/pandas-dev/pandas/pull/28730#discussion_r334575150?

3.8 is out now, so I'd like to get this in and 0.25.2 released as soon as possible (ideally by sometime tomorrow).
https://github.com/pandas-dev/pandas/issues/29001 for followup.

Thanks @jbrockmendel.
TST: Fix broken test cases where Timedelta/Timestamp raise
State: closed

Orthogonal to #28725, but will need to be rebased after that is merged.

Labels:
name: Testing
description: pandas testing functions or related to the test suite

Comments:
see my comment on #28725 
can you rebase
rebased+green
These are user facing changes right? Need a whatsnew?
> These are user facing changes right? Need a whatsnew?

not unless you have an example of what is changed. these are broken behavior in an internal method, likely there is a lot of duplicated / unsused code (except by the tests).
thanks @jbrockmendel
DOC: fix pr09,pr08 errors in frame.py
State: closed

fixing methods defined in this file
this is part of a larger clean up of
the dataframe class

this fixes a subset of #28702


- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`


Labels:
name: Docs

Comments:
Thanks @josibake
Change maybe_promote fill_value to dt64/td64 NaT instead of iNaT
State: closed

Unambiguous, preliminary for fixing more xfails.

Labels:
name: Missing-data
description: np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate
name: Compat
description: pandas objects compatability with Numpy or Python functions

Comments:
ok thanks @jbrockmendel 

yeah this for sure needs refactoring, but ok to un-xfail tests first
DOC: Fixed PR06 docstrings errors in pandas.ExcelWriter
State: closed

- [x] xref #28724
- [ ] tests added / passed
- [x] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Docs

Comments:
Thanks @LauraCollard !
API: Change matplotlib formatter registration
State: closed

This PR does two things

1. Enforces the deprecation of pandas auto registering our formatters.
2. Provides a new option, "auto", which will only apply our formatters to our plots. By default, the value will be "auto".

Background: pandas used to register formatters for Period, datetime, date, and time. This formatters affected plots created using matplotlib only, not pandas. For various reasons (including import time) we don't want to do this on an `import pandas`.

A minimal change would be to just not do the registration on `import pandas`. Users would either manually do it, or pandas does it automatically when we do DataFrame.plot. The downside of pandas doing it automatically is then in the following

```
fig1 = plt.plot(x, y)
DataFrame.plot(...)  # registers formatters
fig2 = plt.plot(x, y)
```

fig1 and fig2 will be different because pandas has implicilty registered formatters.

The `auto` option will automatically register the formatters, make the plot, then deregister the formatters.

Closes https://github.com/pandas-dev/pandas/issues/18720

Labels:
name: Visualization
description: plotting

Comments:
All green here now.
Should be good to go.
Thanks!
DOC: Fixed PR06 docstrings errors in pandas.arrays.IntervalArray
State: closed

- [x] xref #28724
- [ ] tests added / passed
- [x] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Docs

Comments:
Thanks @LauraCollard !
DOC: Fixed PR06 docstrings errors in pandas.timedelta_range
State: closed

- [x] xref #28724
- [ ] tests added / passed
- [x] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Docs

Comments:
LGTM
Thanks @LauraCollard !
DOC: Fixed PR06 docstring errors in pandas.DataFrame
State: closed

- [x] xref #28724
- [ ] tests added / passed
- [x] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Docs

Comments:
LGTM.  @LauraCollard making these bite-size and easy to review is great!
Thanks @LauraCollard !
Backport PR #28710 on branch 0.25.x (CI Failing: TestReadHtml.test_spam_url #28708)
State: closed

Backport PR #28710: CI Failing: TestReadHtml.test_spam_url #28708

Labels:
name: CI
description: Continuous Integration
name: IO HTML
description: read_html, to_html, Styler.apply, Styler.applymap

Comments:
Remaining failure will be fixed by backporting https://github.com/pandas-dev/pandas/pull/28400
precursor to Split out test_pytables.py to sub-module of tests
State: closed

- precursor to #18498
- All tests in test_pytables are passing
@simonjayhawkins as discussed this is the pre work to splitting this large test file into smaller files. So far, I have fixturised the class base class and updated methods using self.path from base to use the setup_path fixture instead.

Next I plan to create a test file for TestTimezones class and make the test functional so the class itself is no longer required.
Once this is running fine, I will then apply similar approach to the other bigger classes.

Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: IO HDF5
description: read_hdf, HDFStore

Comments:
Hello @tolaa001! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:



There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2019-10-01 13:14:18 UTC
thanks @tolaa001 for the PR!

just to be clear, there is a non-trivial amount of effort for #18498 and will require several PRs.

this PR is to add a couple of fixtures in order to eliminate the Base class. In this PR, the fixtures replicate the current unittest setup and teardown methods only and improving these fixtures and adding additional fixtures for some of the assertions will come later.

the next PR would be to split on complete classes. There are 3 test classes in `pandas/tests/io/pytables/test_pytables.py`, `TestHDFStore`, `TestHDFComplexValues` and `TestTimezones`.

so the first split would split `TestHDFComplexValues` and `TestTimezones` from test_pytables.py into test_complex.py and test_timezones.py (and move the two fixtures created here to a conftest.py) but this would need to be a seperate PR with code moved and miniamal code changes to make the diff simpler to review.

This PR is preparing for that split.

`TestHDFStore` could then be split further once the classes have been separated.

> next steps would be to move test_pytables to a sub-dir

already a sub-dir so no need to nest further.
Update contributing.rst
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Docs

Comments:
this is duplicating #28705 
@mimikaan can you address comments
was already covered by https://github.com/pandas-dev/pandas/pull/28907

thanks
Backport PR #28671 on branch 0.25.x (BUG: restore limit in RangeIndex.get_indexer)
State: closed

Backport PR #28671: BUG: restore limit in RangeIndex.get_indexer

Labels:
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: Regression
description: Functionality that used to work in a prior pandas version

Comments:
Pushed a commit renaming `any_not_none` --> `_any_not_none` since it was deprivatized on master in #27741 but hasn't been deprivatized on the 0.25.x branch.
The docs build failure should be fixed by #28737
Thanks for the fix @jschendel !
CI/TST: temporary xfail spam url tests
State: closed

xref https://github.com/pandas-dev/pandas/issues/28708

Labels:

Comments:
This temporary xfails the tests to get CI green again. 

But, since the web page listing the html pages is gone, we can maybe also just remove the test? (I don't know if it was testing something specifically, that we might need to replace with another test, though)
@jorisvandenbossche Hopefully #28710 should pass (passes locally) and this won't be necessary.
Yes, also just saw that, perfect!
CI Failing: TestReadHtml.test_spam_url #28708
State: closed

- [x] closes #28708 
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] fixed test_spam_url test by changing a url address to the same html page stored in the pandas github repo


Labels:
name: CI
description: Continuous Integration
name: IO HTML
description: read_html, to_html, Styler.apply, Styler.applymap

Comments:
lgtm merge on green.
is this a new failure? https://travis-ci.org/pandas-dev/pandas/jobs/591987317
@jorisvandenbossche @simonjayhawkins 
```
_______________ TestReadHtml.test_thousands_macau_index_col[bs4] _______________
[gw0] linux -- Python 3.6.9 /home/travis/miniconda3/envs/pandas-dev/bin/python
self = <pandas.tests.io.test_html.TestReadHtml object at 0x7fa2b70d77f0>
datapath = <function datapath.<locals>.deco at 0x7fa2bbdc30d0>
    @pytest.mark.slow
    def test_thousands_macau_index_col(self, datapath):
        all_non_nan_table_index = -2
        macau_data = datapath("io", "data", "macau.html")
>       dfs = self.read_html(macau_data, index_col=0, header=0)
pandas/tests/io/test_html.py:390: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
pandas/io/html.py:1103: in read_html
    displayed_only=displayed_only,
pandas/io/html.py:919: in _parse
    raise_with_traceback(retained)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
exc = AttributeError("'NoneType' object has no attribute 'items'",)
traceback = None
    def raise_with_traceback(exc, traceback=Ellipsis):
        """
        Raise exception with existing traceback.
        If traceback is not passed, uses sys.exc_info() to get traceback.
        """
        if traceback == Ellipsis:
            _, _, traceback = sys.exc_info()
>       raise exc.with_traceback(traceback)
E       AttributeError: 'NoneType' object has no attribute 'items'
pandas/compat/__init__.py:47: AttributeError
```
Thanks @ladyyvii !

@jreback no, that is already failing on master as well. 
@MeeseeksDev backport 0.25.x
> is this a new failure? https://travis-ci.org/pandas-dev/pandas/jobs/591987317

see #28125, tests pass with beautifulsoup4 4.7.1, fails with 4.8.0.
thanks, guys! It was my first PR to open source :-)
> thanks, guys! It was my first PR to open source :-)

And already more PRs open I see ! ;)
DOC: Fixed PR09 docstring errors in pandas.tseries (#27977)
State: closed

- [ ] xref #27977
- [ ] tests added / passed
- [ ] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x ] fixed minor issues in the couple of docstrings (ErrorCode: PR09) in pandas.tseries.offsets.py, namely added dots at the end of parameters {param_name} descriptions. Some of the ErrorCodes: PR09 still appear when running the validation script. This is due the script not covering some of the special cases (such as extra info in the bullet points format in the descriptions of parameters {param_name}), see #25786 and #25461.


Labels:
name: Docs

Comments:
Thanks a lot for the contribution! The failing CI are related to https://github.com/pandas-dev/pandas/issues/28708, and the doc and checks CI builds are passing, so going to merge this.
read_hdf closes HDF5 stores that it didn't open.
State: closed

- [ ] closes #28699
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry

This commit fixes #28699. I added a test for the bug.

Labels:
name: IO HDF5
description: read_hdf, HDFStore

Comments:
I'm not entirely sure why these checks failed. It looks like there is some major reorganization of test_pytables.py happening right now?
Can you rebase?

I'm not aware of a major reorganization of the file, but there was a recent change that removed an `except Exception:` that may be causing the merge conflict.
Looks like how paths are created was changed in test_pytables.py yesterday which created some conflicts. I updated my test to the new format and rebased.
Got it sorted out. The previous error was due to me using single quotes somewhere rather than double quotes, and black pandas didn't like that.
lgtm. can you add a whatsnew note in bugfix of 1.0.0 for IO section. note that I am about to merge a refactor of the pytables tests, so will have to rebase on master (but shouldn't be a conflict i think).
I updated the whatsnew for 1.0.0. Let me know if I need to change anything for the refactor of the pytables tests.
can you merge master, we moved some tests around
I merged master and resolved the conflicts.
Thanks @kboone !
BUG: Fix Series.sort_values descending & mergesort
State: closed

The solution uses the `nargsort` function, which was created to handle
in one place the intricacies of sorting. Not adapting Series.sort_values
to use may have been an oversight.

- [ ] closes #28697
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:

Comments:
> it’s possible some existing test are off because this path is not exactly the same as nanargsort (but we do want to unify)

Yes that is the case. Now for quicksort, the ascending and descending sort are not guaranteed to be reversed versions of each other.

```python
n = 100000
np.random.seed(123)
values = np.random.randint(0, 4, n)
idx = np.arange(n)
s = pd.Series(values, index=idx)
asc = s.sort_values(ascending=True)
desc = s.sort_values(ascending=False)
asc.index.equals(desc.index[::-1])  # False ( without nanargsort it is True)
```

Many tests for `Series.value_counts` -- which are failing -- rely on checking the index, which is arbitrary (https://github.com/pandas-dev/pandas/pull/24302).
@has2k1 can you rebase
Closing as this looks stale, but certainly @has2k1 ping if you'd like to pick back up
feature wanted: pd.DataFrame.info() should show line numbers #17304
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Output-Formatting
description: __repr__ of pandas objects, to_string

Comments:
Thanks for the PR. This is a continuation of #17332 right?
Yes. The function is working but for some reason only a single test passed, can you give me some advice on that?
At least one of the repr tests is failing. You can see that in CI but here's a link to it as well:

https://dev.azure.com/pandas-dev/pandas/_build/results?buildId=18136&view=logs&jobId=3a03f79d-0b41-5610-1aa4-b4a014d0bc70&taskId=264dbb61-dd41-5ada-cfaa-25751f4c2dca&lineStart=61&lineEnd=62&colStart=1&colEnd=1
Superseded by #28876
TST: Add test for categorical with str and tuples
State: closed

The bug is not present any more.

- [x] closes #21416
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- ~[ ] whatsnew entry~


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Categorical
description: Categorical Data Type

Comments:
Hello @has2k1! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:



There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2019-10-06 18:02:10 UTC
This bug affects  numpy versions *1.13.3* to *1.15.4*. Versions greater than *1.16.0* are fine. However, since it was a regression in numpy (e.g. *1.11.0* is okay), I do not know if it will not show up again.

The tests are failing for lower versions of numpy.
To check for different versions of numpy, you need to skipif using the ``compat.numpy._np_version_under1p*` bools.
Thanks,  @has2k1!
To html encoding add
State: closed

- [x ] closes https://github.com/pandas-dev/pandas/issues/28663
- [x ] tests added / passed
- [x ] passes `black pandas`
- [x ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x ] whatsnew entry

added encoding argument to DataFrame.to_html()


Labels:
name: IO HTML
description: read_html, to_html, Styler.apply, Styler.applymap

Comments:
hi @josibake , I found two checks failed but I just got `[error]Bash exited with code '1'` in azure log, could you give me some tips to fix this?
hey @ailurus1991 , if you look further up in the logs, it says that running `black` changed two files.

try running `black pandas/core` commit those changes.
@WillAyd if `encoding` is specified and not applicable we should presumably raise? Although I don't think it should be part of this PR. Something to think about so we're consistent across the IO methods.
> @WillAyd if `encoding` is specified and not applicable we should presumably raise? Although I don't think it should be part of this PR. Something to think about so we're consistent across the IO methods.

https://github.com/pandas-dev/pandas/issues/28766#issuecomment-538413490  "We should raise when encoding is specified and it's buf isn't a string path."
> Although I don't think it should be part of this PR.

maybe it should. Since this is effectively an enhancement then it may be worth getting it right off the bat.
@ailurus1991 looks ok, can you add a whtasnew note in other enhancements section
> @ailurus1991 looks ok, can you add a whtasnew note in other enhancements section

@jreback Hi Jeff, I added a simple whatsnew in doc, could you review it? thx
thanks
To html encoding add
State: closed

- [x ] closes #xxxx
- [x ] tests added / passed
- [x ] passes `black pandas`
- [x ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x ] whatsnew entry


Labels:

Comments:
Hello @ailurus1991! Thanks for opening this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:

* In the file [`pandas/core/frame.py`](https://github.com/pandas-dev/pandas/blob/12fb0c5c5191d2141cbc529ca2d5a0382de048b8/pandas/core/frame.py):

> [Line 2258:89](https://github.com/pandas-dev/pandas/blob/12fb0c5c5191d2141cbc529ca2d5a0382de048b8/pandas/core/frame.py#L2258): [E501](https://duckduckgo.com/?q=pep8%20E501) line too long (89 > 88 characters)
DOC: fix PR09,PR08 docstring errors in pandas.plotting
State: closed

fixes the errors along with minor changes to standardize kwds -> **kwargs
closes #28687

verified this fixes PR09,PR08 errors with:

```
./scripts/validate_docstrings.py --errors=PR09,PR08 | grep "pandas.plotting"
```

- [x] closes #xxxx
- [x] tests added / passed

Labels:
name: Docs

Comments:
this is related to #27977 and #28602 (adding for visibility)
Thanks @josibake !
DOC: fix formatting in the ExtensionArray docstrings
State: closed

- [x] closes #28685 
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Docs

Comments:
Hello @HughKelley! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:



There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2019-10-14 21:05:31 UTC
@datapythonista, seen above, there was an existing style error in the file modified for this pull request. Should I just go ahead and fix the line that is too long? 
@datapythonista  
> 
> lgtm, thanks @HughKelley

First contribution to a project. Thanks for the help and direction!
> First contribution to a project. Thanks for the help and direction!

You're welcome. Hopefully not the last one. :)
@WillAyd thanks!
Thanks @HughKelley
Deprecate is copy in take
State: closed

- [ X ] closes #27357
- [X  ] tests added / passed
- [ X ] passes `black pandas`
- [ X ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:

Comments:
this cannot be against 0.25x
@chrisstpierre thanks for the PR. As mentioned though this will need to target master and not the `0.25.x` branch. I *think* you'll need to open a new PR to do that, so closing this one as is.

Certainly appreciate the contribution though and can look in more detail when targeting master
BUG: Fix RangeIndex.get_indexer for decreasing RangeIndex
State: closed

- [X] closes #28678
- [X] tests added / passed
- [X] passes `black pandas`
- [X] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [X] whatsnew entry


Labels:
name: Bug
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: Regression
description: Functionality that used to work in a prior pandas version

Comments:
@jschendel you will need to fix the conflicts now I merged the other PR
lgtm.
@jreback : green
Thanks!