BrandonFors/pandas-github-issues huggingface dataset:
TST: changed behavior of df.loc on multiindex
State: closed

-added unittest  for issue #43599 
-error occurs due to different versions of pandas



Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: MultiIndex
name: Stale

Comments:
Hello @gamenerd457! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:

* In the file [`pandas/tests/indexing/multiindex/test_loc.py`](https://github.com/pandas-dev/pandas/blob/0f772a6504ce7f0cc715549fcda650e861e76cb9/pandas/tests/indexing/multiindex/test_loc.py):

> [Line 915:1](https://github.com/pandas-dev/pandas/blob/0f772a6504ce7f0cc715549fcda650e861e76cb9/pandas/tests/indexing/multiindex/test_loc.py#L915): [E302](https://duckduckgo.com/?q=pep8%20E302) expected 2 blank lines, found 0
> [Line 917:77](https://github.com/pandas-dev/pandas/blob/0f772a6504ce7f0cc715549fcda650e861e76cb9/pandas/tests/indexing/multiindex/test_loc.py#L917): [W291](https://duckduckgo.com/?q=pep8%20W291) trailing whitespace
> [Line 922:76](https://github.com/pandas-dev/pandas/blob/0f772a6504ce7f0cc715549fcda650e861e76cb9/pandas/tests/indexing/multiindex/test_loc.py#L922): [W291](https://duckduckgo.com/?q=pep8%20W291) trailing whitespace
> [Line 924:9](https://github.com/pandas-dev/pandas/blob/0f772a6504ce7f0cc715549fcda650e861e76cb9/pandas/tests/indexing/multiindex/test_loc.py#L924): [E123](https://duckduckgo.com/?q=pep8%20E123) closing bracket does not match indentation of opening bracket's line

* In the file [`pandas/tests/indexing/test_loc.py`](https://github.com/pandas-dev/pandas/blob/0f772a6504ce7f0cc715549fcda650e861e76cb9/pandas/tests/indexing/test_loc.py):

> [Line 2864:1](https://github.com/pandas-dev/pandas/blob/0f772a6504ce7f0cc715549fcda650e861e76cb9/pandas/tests/indexing/test_loc.py#L2864): [W391](https://duckduckgo.com/?q=pep8%20W391) blank line at end of file



##### Comment last updated at 2021-10-12 07:12:18 UTC



> Thanks for the pr @gamenerd457. Please look at the contributing guide (https://pandas.pydata.org/docs/development/index.html#development), especially the portions about testing (we don't use `unittest`) and style (this is currently failing checks)

I have made the changes
ca you merge master and update to comments
This pull request is stale because it has been open for thirty days with no activity. Please update or respond to this comment if you're still interested in working on this.
Thanks for the PR but appears to have gone stale. If interested in continuing please merge the main branch, address the review, and we can reopen.
PERF: tighter cython declarations, faster __iter__
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [ ] whatsnew entry


Labels:
name: Performance
description: Memory or execution speed performance
name: Internals
description: Related to non-user accessible pandas implementation

Comments:
can you post benchmark changes in the top
> Are the ndim=1 additions for readability or performance? Have always just assumed (but never checked) that cython infers it.

When I was working on this I convinced myself that it made a difference, but I can no longer remember why.  Maybe @da-woods can weigh in?

> can you post benchmark changes in the top

It's all going to be micro-perf that asvs don't pick up well
> > Are the ndim=1 additions for readability or performance? Have always just assumed (but never checked) that cython infers it.
> 
> When I was working on this I convinced myself that it made a difference, but I can no longer remember why. Maybe @da-woods can weigh in?

https://cython.readthedocs.io/en/latest/src/tutorial/numpy.html#efficient-indexing

> “ndim” keyword-only argument, if not provided then one-dimensional is assumed

I had a quick test of a simple example (looking at the annotated source) and the behaviour looks to match what the docs say. So I think there's no performance advantage to specifying `ndim=1` but you may prefer to be explicit for readability reasons.
Thanks @da-woods.

If the `ndim=1` is extraneous, I'm OK with either the more concise or more explicit versions.
ENH: ExponentialMovingWindow.sum
State: closed

- [x] closes #13297
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [x] whatsnew entry


Labels:
name: Enhancement
name: Window
description: rolling, ewma, expanding

Comments:
thanks @mroeschke nice!
BUG: Series[object].__setitem__(ndarray[M8ns]_with_matching_size)
State: closed

- [ ] closes #xxxx
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [x] whatsnew entry


Labels:
name: Bug
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves

Comments:
wow
this looks user facing can you add a note (and rebase)
whatsnew added + greenish
Add clarifications to the docs regarding `to_feather`
State: closed

- [X] closes https://github.com/pandas-dev/pandas/issues/28208
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [ ] whatsnew entry

As discussed in https://github.com/pandas-dev/pandas/issues/28208, the docs should be more precise about in which cases `to_feather()` will fail.

Labels:
name: Docs

Comments:
@jreback can this be merged?
thanks @jmakov
BUG/ERR: sparse array cmp methods mismatched len
State: closed

- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [x] whatsnew entry


Labels:
name: Bug
name: Error Reporting
description: Incorrect or improved errors from pandas
name: Numeric Operations
description: Arithmetic, Comparison, and Logical operations
name: Sparse
description: Sparse Data Type

Comments:
thanks @mzeitlin11
Update README.md
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [ ] whatsnew entry


Labels:

Comments:
Thank you for the pull request, but this change appears redundant with the first sentence in the `README.md` and doesn't add much value.

If you are interested in contributing, please browse the issues labeled `good first issue`. Closing
TYP: make IOHandles generic
State: closed

Towards #41610

This doesn't help with any ignore statements, as `Buffer` is defined as `Buffer = Union[IO[AnyStr], RawIOBase, BufferedIOBase, TextIOBase, TextIOWrapper, mmap]`: `IO[AnyStr]` will be replaced with `IO[str]` or `IO[bytes]`.

btw., `Buffer` is a "bad" union as it contains one generic and many actual types (I think, I'm the one to blame for that). It is often used without an actual type (`Buffer` instead of `Buffer[str]` or `Buffer[bytes]`).

Labels:
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
Hello @twoertwein! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:

















There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2021-11-14 03:38:45 UTC
#43951 is a follow-up to this PR that also contains this PR. If #43951 isn't too large to review (it is large), I wouldn't mind simply skipping this PR.
lgtm @simonjayhawkins if any comments
can you merge merge master and can get this in
@jreback  green'ish
thanks @twoertwein
REGR: RollingGroupby.corr producing incorrect results
State: closed

- [x] closes #43386
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [x] whatsnew entry

The regression is caused by the groupby paths requiring the data to be sorted by groups.

cc @mroeschke and @simonjayhawkins 

Labels:
name: Bug
name: Groupby
name: Regression
description: Functionality that used to work in a prior pandas version
name: Window
description: rolling, ewma, expanding

Comments:
docs build failure unrelated:

> WARNING: failed to reach any of the inventories with the following issues:
intersphinx inventory 'https://www.statsmodels.org/devel/objects.inv' not fetchable due to <class 'requests.exceptions.ConnectionError'>: HTTPSConnectionPool(host='www.statsmodels.org', port=443): Max retries exceeded with url: /devel/objects.inv (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7fab49a33f70>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution'))
building [mo]: targets for 0 po files that are out of date
building [html]: targets for 158 source files that are out of date
updating environment: [new config] 2550 added, 0 changed, 0 removed
reading sources... [  4%] development/code_style .. reference/api/pandas.DataFrame.dot
there is a fail on the warnings built but i think already fixed on master

thanks @rhshadrach 
@meeseeksdev backport 1.3.x
Something went wrong ... Please have  a look at my logs.
TYP: enable reportPropertyTypeMismatch
State: closed

This might be controversial. The setter allows any `Mapping` but the getter returns a `dict`.

[reportPropertyTypeMismatch](https://github.com/microsoft/pyright/blob/main/docs/configuration.md):

> [...] type of the value passed to the setter is not assignable to the value returned by the getter. Such mismatches violate the intended use of properties, which are meant to act like variables. The default value for this setting is 'error'.

Labels:
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
This looks like a case where 1) pyright is technically incorrect (passing a non-dict Mapping looks benign) and 2) its not that big a deal (if using "dict" will allow us to enable pyright in a way that will prevent bugs, then its fine)
I think the three options are:

1. Add a pyright ignore comment to the top of the file and enable the pyright rule or
2. Keep the pyright rule disabled or
3. Change Mapping to dict and enable the pyright rule

I'm slightly inclined to 1 as it seems to be a deliberate choice to support setting any dict-like object.
I updated the PR to not change the type annotation, but instead add a pyright-ignore comment.
BUG: json_normalize prefixed multi-char sep to all keys
State: closed

- [x] closes #43831
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [x] whatsnew entry


Labels:
name: Bug
name: IO JSON
description: read_json, to_json, json_normalize

Comments:
thanks @debnathshoham
TST: added test for cut when labels are tuples of Timestamps (#40661)
State: closed

- [x] closes #40661 
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [ ] whatsnew entry

Added a test for the cut method which uses Timestamp tuples as labels. Using cut with such labels raised error in v1.2.3 as reported in the issue, but it runs correctly after v1.3.0.

Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: cut
description: cut, qcut

Comments:
thanks @Svanazar
CI: Unpin fsspec
State: closed

- [ ] closes #42026
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [ ] whatsnew entry

Testing a change out to see if it fixes it. Not sure if this is a bug on our side our on fsspec side. 

It seems the change is that fastparquet apends a slash in front of paths now.

cc @martindurant 

Labels:
name: IO Data
description: IO issues that don't fit into a more specific label
name: CI
description: Continuous Integration
name: Dependencies
description: Required and optional dependencies

Comments:
thanks @lithomas1 prob not worth to backport this.
TST: Test Series' settitem with Interval and NaN
State: closed

Fixes #27937. As described in that issue, assigning a mixture of NaN and
Interval values into a slice of a categorical-typed series would raise
an exception.  This is no longer the case.

- [x] closes #27937
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [ ] whatsnew entry


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: Missing-data
description: np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate
name: Categorical
description: Categorical Data Type
name: Interval
description: Interval data type

Comments:
thanks @ElDeveloper very nice!
Thanks so much for your feedback!
TST: test selection methods after assignment with at
State: closed

- [X] closes #29282
- [X] tests added / passed
- [X] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves

Comments:
thanks @saehuihwang very nice!
TYP: Fix typing of frame.explode
State: closed

cc @simonjayhawkins 

Labels:
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
@meeseeksdev backport 1.3.x
Something went wrong ... Please have  a look at my logs.
Thanks @phofl for the quick follow-up. backporting typing PRs is not necessary.
update fix ignored sort in api.py and add test
State: closed

- [x] closes #43375 
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [x] whatsnew entry

As mentioned in the ticket, the function `api.union_indexes` does not handle the sort parameter. I added support for it, but we explicitly check that sort can only take the values `False` and `None` in [here](https://github.com/pandas-dev/pandas/blob/62d1f5c0fb16b6990aceabb8491d61de63cd042a/pandas/core/indexes/base.py#L2898)

Labels:
name: Bug
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode

Comments:
Hello @CloseChoice! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:











There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2021-10-08 14:56:28 UTC
> might have to adjust some other tests

Can we just adjust other test results? That would be technically be a breaking change, wouldn't it?
my point about adjusting test is that this case might be only internal or not - will have to see
@github-actions pre-commit
@github-actions pre-commit
@jreback all lights on green. I also added a whatsnew entry.
thanks @CloseChoice
CLN: Add comment on catch_warnings in agg_list_like
State: closed

- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them

Follow up from #43741

Labels:
name: Clean
name: Apply
description: Apply, Aggregate, Transform, Map

Comments:
merge when ready
Thanks @rhshadrach
REF: unnecessary method sharing with PeriodIndex
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [ ] whatsnew entry


Labels:
name: Refactor
description: Internal refactoring of code
name: Period
description: Period data type
name: Index
description: Related to the Index class or subclasses

Comments:
this is just moving things around (in the same file) + typing?
yes, and remove a handful of is_period_dtype checks
CLN: unify some fused types
State: closed

Change summary:
- `numeric` -> `numeric_t`
    - This replaces usages of `asof_t`
- `numeric` + `object` -> `numeric_object_t`
    - This replaces usages of `algos_t`, `join_t`, `reshape_t`
- `i64 + u64 + f32 + f64` -> `iu_64_floating_t` (don't love this name, suggestions welcome)
    - This replaces `groupby_t`
- `iu_64_floating_t` + `object` -> `iu_64_floating_obj_t`
   - This replaces `rank_t` (which was renamed to `numeric_object_t` in #43789, so it replaces `numeric_object_t` in this pr)


Labels:
name: Internals
description: Related to non-user accessible pandas implementation
name: Clean

Comments:
@jbrockmendel if comments.
LGTM
PERF: tolist
State: closed

The parent class implementation is faster.

```
dti = pd.date_range("2016-01-01", periods=10**5, freq="S")

%timeit dti.tolist()
286 ms ± 3 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)  # <- master
62.8 ms ± 765 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)  # <- PR
```

Labels:
name: Performance
description: Memory or execution speed performance

Comments:
wow
DOC: Typo(s?) in pandas.DataFrame.isin
State: closed

- [🗸] closes #43755 
- [🗸] tests added / passed
- [🗸] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [🗸] whatsnew entry


Labels:
name: Docs

Comments:
@jreback I did the changes you suggested, you can review them.
thanks @Varun270
Multiindex drop doc improvement
State: closed

- [ ] closes #43793 
- [ ] tests added / passed - N/A
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them - N/A
- [ ] Improved documentation for ``DataFrame.drop``


Labels:
name: Docs
name: MultiIndex

Comments:
thanks @juliandwain
CI/TYP: set python_version for mypy
State: closed

- [x] closes #43807
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [ ] whatsnew entry


Labels:
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
cc @simonjayhawkins 
thanks @twoertwein do we need to mention this in the docs?
DOC: Add link to previous versions documentation
State: closed

Add a "Previous versions" block with link to https://pandas.pydata.org where links to the documentation of prior versions are available. See preview below.

![image](https://user-images.githubusercontent.com/22901938/135364568-9f22f930-b7b4-42c7-ba39-c57f6290afc2.png)

- [x] closes #43782 
- [ ] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [ ] whatsnew entry


Labels:
name: Docs

Comments:
looks reasonable.

@pandas-dev/pandas-core if any comments.
thanks @DrGFreeman 
@meeseeksdev backport 1.3.x
Something went wrong ... Please have  a look at my logs.
BTW, for a longer term / better solution: there is active work in the upstream theme to add a version switcher (https://github.com/pydata/pydata-sphinx-theme/pull/436). So that will make it more accessible to get to the older versions.
TYP: GroupBy
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [ ] whatsnew entry


Labels:
name: Groupby
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
some failures on 32-bit
REF: share _get_bool_indexer
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [ ] whatsnew entry


Labels:
name: Refactor
description: Internal refactoring of code
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves

Comments:
cool
REG: Regression in explode when column is non string
State: closed

- [x] closes #43314
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [x] whatsnew entry

This should cover the issue since the other pr did not make any progress.

Is there a fixture providing all scalar dtypes?


Labels:
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode
name: Regression
description: Functionality that used to work in a prior pandas version

Comments:
> pandas/core/frame.py:8298: error: List item 0 has incompatible type "Union[str, Tuple[Any, ...], List[Union[str, Tuple[Any, ...]]]]"; expected "Union[str, Tuple[Any, ...]]"  [list-item]

otherwise lgtm
Now I rememeber why the assertion was there
> Is there a fixture providing all scalar dtypes?
ea_scalar_and_dtype

prob could add one for all scalar types
thanks @phofl 
@meeseeksdev backport 1.3.x
Something went wrong ... Please have  a look at my logs.
Will do as a follow up
CI/TYP: run pyright at manual stage in pre-commit
State: closed

with numpy<1.21.0 pyright (and mypy?) cannot determine some types which leads to overlapping overloads. See https://github.com/pandas-dev/pandas/pull/43747#issuecomment-930343454

@attack68 


Labels:
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
we cannot bump this now. needs to be a while. (you *could* do it for the pyright checking i suppose) but that might be complicated.
> we cannot bump this now. needs to be a while.

Definitly not for a release but not even for development? 

> (you _could_ do it for the pyright checking i suppose) but that might be complicated.

Pre-commit allows specifying additional dependencies for a hook, but it seems they need to be from the same language as the hook (pyright: node, numpy: python).

I think the best options is to run pyright at the "manual" stage to avoid annoying contributors. This would still make it possible to easily run pyright.



Mypy also needs 1.21.0, otherwise it reports many errors.
thanks @twoertwein yep having to manualish now is fine