BrandonFors/pandas-github-issues huggingface dataset:
Revert "TST: Bare pytest raises" in base extension tests
State: closed

Reverts pandas-dev/pandas#38576, see discussion there. We should not add detailed error message asserts, as those tests are also used by external packages, that might have different error messages.

Labels:
name: Testing
description: pandas testing functions or related to the test suite

Comments:
@MarcoGorelli on the PR you mentioned "` ^pandas/tests/extension/` excluded from when the "disallow bare pytest raises" will be applied to CI"; but so I understand it correctly: at the moment we don't yet test it in CI? (it seems "bare_pytest_raises" validation type is not yet used in practice). So this is not yet actionable right now?
> So this is not yet actionable right now?

Yes, that's right - I was just asking about what should be done in the future about these tests once it is checked in CI.

Also, does `pandas/tests/arrays/boolean/test_arithmetic.py` need to be reverted too?
I don't have a strong preference. I can also add `match=None`, whatever you prefer
thanks @jorisvandenbossche
ENH: support downcasting of nullable EAs in pd.to_numeric
State: closed

- [x] closes #33013
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry

Picking up #33435.

In `pd.to_numeric` when encountering an `IntegerArray` and `FloatingArray` (or `Series` built from them) we downcast `_data` then reconstruct the array using the downcast `_data` and the `_mask` from the original array.

Labels:
name: Numeric Operations
description: Arithmetic, Comparison, and Logical operations
name: ExtensionArray
description: Extending pandas with custom dtypes or arrays.
name: NA - MaskedArrays
description: Related to pd.NA and nullable extension arrays

Comments:
Green + addressed comments
Thanks @arw2019 !
BUG: IntervalIndex, PeriodIndex, DatetimeIndex symmetric_difference with Categorical
State: closed

- [ ] closes #xxxx
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode
name: Index
description: Related to the Index class or subclasses

Comments:
looks fine, can you add a whatsnew (e.g. symmetric difference giving upcasting?)
great ping on green
ping
DEPR: Hide deprecated attrs _AXIS_NAMES & _AXIS_NUMBERS
State: closed

- [x] closes #38722
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry

Users that do e.g. `[getattr(pd.DataFrame, x) for x in dir(pd.DataFrame())]` or `inspect.getmembers(pd.DataFrame())` currently get an unfriendly deprecation warning. This fixes that by adding `_AXIS_NAMES` & `_AXIS_NUMBERS` to `_hidden_attrs`.

@JSunRae, is it possible for you to add this to your pandas source code and see if this fixes your problem?

xref: #33637

Labels:
name: Deprecate
description: Functionality to remove in pandas

Comments:
Updated.
thanks @topper-123 very nice!
@meeseeksdev backport 1.2.x
CLN: remove duplicate banklist.html file
State: closed

Wasn't quite a duplicate — had a missing `<tr>`. Also updated the `.gitignore` to more accurately only exclude HTML files from `doc/source/_static/`, since other files are checked in.

- [ ] ~~closes #xxxx~~
- [ ] tests added / passed
- [ ] ~~passes `black pandas`~~
- [ ] ~~passes `git diff upstream/master -u -- "*.py" | flake8 --diff`~~
- [ ] ~~whatsnew entry~~


Labels:
name: Docs

Comments:
thanks @afeld
WIP/REF: setitem_with_indexer always use same path for 2D
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry

@phofl I'm hoping you're interested in picking up the torch to get this across the finish line.  There are two main places where I think this still needs work

1) being systematic about whether/when we issue a SettingWithCopyWarning
2) perf, as operating column-wise is going to take a toll.  We need to operate block-wise.

Labels:
name: Refactor
description: Internal refactoring of code
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves

Comments:
Oh great. Yes for sure. Will look into this
Closing in favor of an upcoming approach.
DOC: make shared includes for describing the datasets in the Getting Started tutorials
State: closed

Just a refactor; the net result should be the same.

- [ ] ~~closes #xxxx~~
- [x] tests added / passed
- [ ] ~~passes `black pandas`~~
- [ ] ~~passes `git diff upstream/master -u -- "*.py" | flake8 --diff`~~
- [ ] ~~whatsnew entry~~

Labels:
name: Docs

Comments:
There's also overlap in the data set descriptions between the [combining data](https://pandas.pydata.org/pandas-docs/stable/getting_started/intro_tutorials/08_combine_dataframes.html) and [time series](https://pandas.pydata.org/pandas-docs/stable/getting_started/intro_tutorials/09_timeseries.html) tutorials. I wasn't sure which was correct, so didn't touch those.
thanks @afeld. pls check that this renders nicely in the built docs:    https://pandas.pydata.org/pandas-docs/dev/
BUG: GroupBy.idxmax/idxmin with EA dtypes
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry

@jorisvandenbossche thoughts on how/where to handle the skipna kwarg?

Labels:
name: Groupby
name: ExtensionArray
description: Extending pandas with custom dtypes or arrays.

Comments:
> thoughts on how/where to handle the skipna kwarg?

We don't yet handle that -> https://github.com/pandas-dev/pandas/issues/35178, https://github.com/pandas-dev/pandas/issues/33941, https://github.com/pandas-dev/pandas/issues/33942. 
Also https://github.com/pandas-dev/pandas/pull/37924 fixing idxmin/max for Series with EAs is not yet tackling it. 

So I think this is something we first need to decide upon / fix for the Series case, before doing this for groupby.


To be clear, the open issues are mainly about the behaviour for argmin/max with skipna=False, not for idxmin/max (but since idxmin/max is typically implemented based on argmin/max, it's still impacted)
@jorisvandenbossche do the blockers you mentioned have some momentum to them?  i.e. if I want to get this fixed, do i need to go work on those directly or can i just be patient for a bit?
gentle ping @jorisvandenbossche thoughts on how to make this actionable?
By looking into those linked issues, and trying to move them forward ..

Now, I think this PR can already fix the non-skipna case in the meantime?
> Add a whatsnew note?

updated
updated + green
updated docstrings, not sure about testing for the NotImplementedError cases since that isnt actually desired behavior
> updated docstrings, not sure about testing for the NotImplementedError cases since that isnt actually desired behavior

sure but we want to lock it down to make sure that it is not actually changed w/o doing anything about it.
test added + greenish
BUG: Ensure series/frame mode() keeps int index
State: closed

- [x] closes #33321
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry


Labels:
name: Bug
name: Dtype Conversions
description: Unexpected or buggy dtype conversions

Comments:
green
thanks @mzeitlin11
DOC: reference the visualization page from the ecosystem one
State: closed

The link already exists in the other direction, so want to make sure readers who come across the latter understand the built-in option.

- [ ] ~~closes #xxxx~~
- [ ] ~~tests added / passed~~
- [ ] ~~passes `black pandas`~~
- [ ] ~~passes `git diff upstream/master -u -- "*.py" | flake8 --diff`~~
- [ ] ~~whatsnew entry~~


Labels:
name: Docs

Comments:
Good idea. Thanks @afeld
REGR: to_csv created corrupt ZIP files when chunksize<rows
State: closed

- [x] closes #38714
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry

When `ZipFile`'s `write` is called multiple times, it will create multiple files within the zip file (with the same filename).

Edit: This also happens independently of `chunksize` as https://github.com/pandas-dev/pandas/blob/master/pandas/_libs/writers.pyx#L14 calls `writerows` multiple times.

Labels:
name: IO CSV
description: read_csv, to_csv

Comments:
thanks @twoertwein very nice
@meeseeksdev backport 1.2.x
Something went wrong ... Please have  a look at my logs.
BUG: pd.read_json May Not Maintain Numeric String Index
State: closed

- [x] closes #28556
- [x] tests added / passed
- [ ] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry


Labels:
name: Bug
name: Dtype Conversions
description: Unexpected or buggy dtype conversions
name: IO JSON
description: read_json, to_json, json_normalize

Comments:
Hello @theoniko! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:





There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2021-01-04 22:32:49 UTC
All requested changes were made. Please re-review the pr.
merge master as well
lgtm pushed a small doc change and can merge on green
TST: GH30999 Add match=msg to all but two pytest.raises in tests/io
State: closed

This pull request partially addresses xref #30999 to remove bare `pytest.raises` by adding `match=msg`. It doesn't close that issue as I have only addressed test modules in the pandas/tests/io/ directory.

I thought this was going to be relatively small because there were only 25 such instances, but it turned out some of them were in methods that were reused several times, so this ended up larger than I expected.

There were a couple of cases where I found the structure of the tests quite confusing, with an internal function definition and two nested asserts. This structure also made it difficult to add the assertion about the error message. I restructured the tests to be, in my opinion, more straightforward.

There's one file, pandas/tests/io/test_sql.py, with two instances of `pytest.raises` in tests which were skipped on my machine and which I couldn't figure out how to run, so those are left as is.

- [ ] closes #xxxx
- [ ] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Error Reporting
description: Incorrect or improved errors from pandas

Comments:
> There were a couple of cases where I found the structure of the tests quite confusing, with an internal function definition and two nested asserts. This structure also made it difficult to add the assertion about the error message. I restructured the tests to be, in my opinion, more straightforward.

yep your changes are fine. general restructuring of tests should be a dedicated PR, but moderinzation like this is ok.
thanks @moink
BUG: inconsistency between frame.any/all with dt64 vs dt64tz
State: closed

- [ ] closes #xxxx
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry


Labels:
name: Bug
name: Datetime
description: Datetime data dtype
name: Reduction Operations
description: sum, mean, min, max, etc.

Comments:
> What is the inconsistency this is fixing? (how is it behaving differently?)

```
dti = pd.date_range("2016-01-01", periods=3)
dti2 = dti.tz_localize("UTC")

df = pd.DataFrame(dti)
df2 = pd.DataFrame(dti2)

>>> df.all()
0    True
dtype: bool

>>> df2.all()
Series([], dtype: bool)
```
Thanks for that example, that helps ;) 
(in general, it would help if you try give a bit more context when opening PRs)

That's indeed certainly an inconsistency to fix. But quickly checking, this actually seems to have worked fine before, so might be a regression? (and thus should potentially target this for 1.2.1)
> this actually seems to have worked fine before, so might be a regression?

first bad commit: [50ae0bfc383b5cefa39e82c0446dce11d4ce10e9] BUG: DataFrame reductions inconsistent with Series counterparts (#37827)
@meeseeksdev backport 1.2.x
Something went wrong ... Please have  a look at my logs.
TST: GH30999 Add match=msg to all pytest.raises in tests/reductions and add an error message to nanmedian
State: closed

This pull request partially addresses xref #30999 to remove bare `pytest.raises` by adding `match=msg`. It doesn't close that issue as I have only addressed test modules in the pandas/tests/reductions directory.

When going through the tests I found one case where the error message was the empty string. I decided that wasn't correct so I modified the `TypeError` to inherit the error message from the `ValueError` it was raising from. This made the error message similar to other messages in the same test.

- [ ] closes #xxxx
- [ ] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Error Reporting
description: Incorrect or improved errors from pandas

Comments:
thanks @moink
TST: GH38718 Tests for casting an interval range from float to int
State: closed

xrefs #38697 , #15832

- [x] closes #38718
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Dtype Conversions
description: Unexpected or buggy dtype conversions
name: Interval
description: Interval data type

Comments:
cc @jbrockmendel 
LGTM
thanks @moink
Slicing columns with mixed types <str>,<int> fails with ValueError #20975
State: closed

- [ ] closes #20975 
- [x] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves

Comments:
Hello @hungyiwu! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:



There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2020-12-27 13:43:36 UTC
I saw only the travis-ci check failed but when going through its log I couldn't identify the cause. Any suggestions will be appreciated!
thanks @hungyiwu
BUG: loc with empty multiindex raises exception
State: closed

- [x] closes #36936 
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: MultiIndex

Comments:
Added test case for the changes made in GH#36936
thanks @kasim95
ENH: Add xlsb auto detection to read_excel and respect default options
State: closed

- [x] closes #35416 , closes #34252
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry

Note: Even though most of the reader options for read_excel only have 1 option, I think it is still useful to respect pandas.set_option. 


Also, if some people have older xlrd(e.g. 1.2.0) and don't have openpyxl, they can use options to force pandas to read xlsx files with xlrd. (this is possible now, but xlrd's maintainer has made it clear he doesn't support for things like this)

Labels:
name: IO Excel
description: read_excel, to_excel

Comments:
cc @WillAyd @TomAugspurger 
Hello @lithomas1! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:









There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2021-01-01 21:48:02 UTC
FYI @rhshadrach, I'm going to be changing the behavior in this PR if both xlrd and openpyxl aren't installed.
Instead of raising, ``ImportError: Missing optional dependency 'xlrd'. Install xlrd >= 1.0.0 for Excel support Use pip or conda to install xlrd.`` (which is wrong anyways, we require xlrd>=1.2.0, this will need to be fixed in another PR)
We now raise, ``ImportError: Missing optional dependency 'openpyxl'.  Use pip or conda to install openpyxl.``

This way, we encourage users to use openpyxl not xlrd. 
is there a test for this https://github.com/pandas-dev/pandas/issues/34252, e.g. that the set option works correctly? (i see one added for an invalid value), and am pretty sure we are testing this, but just want to confirm.
@jreback, 
I don't think there is a way to know which engine read_excel uses, so I didn't add a test. 
@lithomas1 `read_excel` is monkey patched in the tests for this purpose. Use `pd.read_excel.keywords["engine"]`.
Ack - my previous comment was incorrect. That is of course only for which engine is passed to read_excel.
thanks @lithomas1 very nice!
TST: Remove broken test for deprecated functionality
State: closed

Registering converters by default on import is deprecated since #28722.

The test did not fail as the subprocess call contains errors in the python code. Specifically, the code is in double and single quotes. This means that the python code passed to the subprocess call is never actually excecuted. Python in the subprocess is called to "excecute" a string. This is will simply finish with exit code 0 as there is nothing to do basically.
Additonally, the test code contains a stray closing bracket at the end which would make the code fail if it was excecuted. 

- [ ] closes #xxxx
- [ ] tests added / passed
- [x] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Visualization
description: plotting
name: Deprecate
description: Functionality to remove in pandas

Comments:
can u fix the test instead 
How would you want this test fixed? There is a test for registering the converters explicitly. I don't know what case still needs to be tested with this after converters are no longer automatically registered.
this is a test that is using a sub process so it's different - it makes sure that we are not corrupting things in import 
As far as my understanding went, there is nothing converter related done on import anymore. That's why I thought the test is no longer needed. But you are suggesting that something about the imports still needs to have a test. 
I probably don't have a deep enough understanding of what is done there then. So somebody who knows this better should probably look at this to fix it properly.
thanks @theOehrly
TST: Add match=msg to all pytest.raises in pandas/tests/io/json
State: closed

This pull request partially addresses xref #30999 to remove bare `pytest.raises` by adding `match=msg`. It doesn't close that issue as I have only addressed test modules in tje pandas/tests/io/json directory.

I also moved a couple of lines out of the `pytest.raises` context to make clear which pandas method was raising the error. And I deleted a couple of unreachable asserts in tests.

- [ ] closes #xxxx
- [ ] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Error Reporting
description: Incorrect or improved errors from pandas
name: IO JSON
description: read_json, to_json, json_normalize

Comments:
thanks @moink the error messages really do help in reading the test code.
DEPR MultiIndex.is_lexsorted and MultiIndex.lexsort_depth
State: closed

- [x] closes #32259
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry

The issue is that `is_lexsorted` only looks at whether the _codes_ are sorted - the order of the codes doesn't necessarily reflect the sorting of the index values, e.g. if the index is created from a `.groupby(sort=False)` operation (see the linked issue for an example).

So, instead of `is_lexsorted`, users should use `is_monotonic_increasing`. There isn't an equivalent function to point them to for `lexsort_depth`, right?

Labels:
name: MultiIndex
name: Deprecate
description: Functionality to remove in pandas

Comments:
`lexsort_depth` also probably needs to be made private, although there's already a `_lexsort_depth`...will think about how to rename things
Some of the CI failures caused by #38703
> Some of the CI failures caused by #38703

I haven't checked yet, but thanks for looking into it!
thanks @MarcoGorelli
Backport PR #38679 on branch 1.2.x (Revert "Deprecated casting an object-dtype index of datetime objects to DatetimeIndex in the Series constructor (23598)")
State: closed

Backport PR #38679: Revert "Deprecated casting an object-dtype index of datetime objects to DatetimeIndex in the Series constructor (23598)"

Labels:
name: Deprecate
description: Functionality to remove in pandas

Comments:
same failure seen on another PR (before PR this is backporting was merged to master)

```
=========================== short test summary info ============================
FAILED pandas/tests/arrays/categorical/test_warnings.py::TestCategoricalWarnings::test_tab_complete_warning
FAILED pandas/tests/frame/test_api.py::TestDataFrameMisc::test_tab_complete_warning[DataFrame]
FAILED pandas/tests/frame/test_api.py::TestDataFrameMisc::test_tab_complete_warning[Series]
FAILED pandas/tests/indexes/test_base.py::TestIndex::test_tab_complete_warning
FAILED pandas/tests/resample/test_resampler_grouper.py::test_tab_complete_ipython6_warning
= 5 failed, 145317 passed, 4090 skipped, 1091 xfailed, 6 warnings in 874.50s (0:14:34) =
```