BrandonFors/pandas-github-issues huggingface dataset:
Temporarily add back Index._get_attributes_dict for dask compat
State: closed

See https://github.com/pandas-dev/pandas/pull/43895#issuecomment-938472299. There was of course no need to revert the full PR, just to add this method again. 

Labels:
name: Compat
description: pandas objects compatability with Numpy or Python functions

Comments:
(failure is an unrelated "unexpected ResourceWarning")
IIUC the motivation is for dask.  Are dask users affected or just their CI?  If the latter, seems easier for them to fix than for revert-fix-unrevert?

If the former, can you add a FutureWarning on _get_attributes_dict
It's also dask users/developers, _if_ they are using a development version of pandas. The current situation basically makes you can't test/develop in a dask context with pandas master at the moment. Adding this back temporarily is an easy way to make this process a bit more smooth.

> If the former, can you add a FutureWarning on _get_attributes_dict

I don't think that's needed. Users are not directly use this, but through dask. 
(but of course easy to add)
> (but of course easy to add)

yah this is the easiest way to get me to "OK sure"
Added a warning.
Bug multiple css selectors GH44011
State: closed

- [x ] closes #44011
- [x ] tests added / passed
- [x ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [x ] whatsnew entry


Labels:
name: IO HTML
description: read_html, to_html, Styler.apply, Styler.applymap
name: Styler
description: conditional formatting using DataFrame.style

Comments:
I need some orientation about pandas-dev check
> I need some orientation about pandas-dev check

this should work, I think the checks are flaky sometimes. I hav re run, if it fails, merge master and then push again.
LGTM - thanks @mabittar
ENH: Ability to coerce floats when reading csv files GH2570
State: closed

- [x] closes #2570
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [x] whatsnew entry

Ran the `io.csv.Read*` benchmarks which resulted in `BENCHMARKS NOT SIGNIFICANTLY CHANGED.`

I believe that `Dict[Hashable, Tuple[Dtype, str]]` should be added to `DtypeArg` to fix the failing typing check. However, that results in 20 additional errors in io\sql.py and io\parsers\base_parser.py with the additional tuple type. Perhaps there should be a distinction and an additional `CSVDtypeArg`. Looking for advice on how best to approach fixing this.

Labels:
name: Stale

Comments:
This pull request is stale because it has been open for thirty days with no activity. Please update or respond to this comment if you're still interested in working on this.
this needs an api discussion first, we are not going to add yet another argument. pls comment on the referenced issue. thanks.
DOC: DatetimeArray.std
State: closed

`DatetimeIndex.std` is missing a doc-string. Add doc-string to `DatetimeArray.std` as `DatetimeIndex` is re-using `DatetimeArray.std`.


There are a few more keywords, but they seem to be not implemented (numpy-compatiblity?)

```py
pd.DatetimeIndex([1, 2]).std(out=[])
ValueError: the 'out' parameter is not supported in the pandas implementation of std()

pd.DatetimeIndex([1, 2]).std(keepdims=True)
ValueError: the 'keepdims' parameter is not supported in the pandas implementation of std()

pd.DatetimeIndex([1, 2]).std(dtype=np.dtype("datetime64[ns]"))
ValueError: the 'dtype' parameter is not supported in the pandas implementation of std()
```

For some reason, this doc-string is needed for #43828.

Labels:
name: Docs

Comments:
Getting

> /home/runner/work/pandas/pandas/pandas/core/indexes/datetimes.py:docstring of pandas.core.indexes.datetimes.DatetimeIndex:124: WARNING: autosummary: stub file not found 'pandas.DatetimeIndex.std'. Check your autosummary_generate setting.


even though `pd.core.indexes.datetimes.DatetimeIndex([1]).std.__doc__` returns the doc-string.


edit: Running `python doc/make.py --single "pandas.DatetimeIndex.std" --num-jobs 1` also picks up on the doc-string.
I think it wants an 'std' entry in doc/source/reference/indexing.rst DatetimeIndex.std
> I think it wants an 'std' entry in doc/source/reference/indexing.rst DatetimeIndex.std

Thank you @jbrockmendel , now it seems to be happy :)
DOC: Upload cheatsheets to site
State: closed

- [x] closes #40949
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [ ] whatsnew entry


I went ahead and modified the `.github/workflows/ci.yml` so that the cheatsheets are copied to the docs build directory once the Sphinx site is built

Labels:
name: Docs

Comments:
cc @Dr-Irv @datapythonista over to you
Thanks @trallard, really nice.

The CI for this commit (PR merge) will update the website, and the cheatsheet in https://pandas.pydata.org/Pandas_Cheat_Sheet.pdf should be updated with the new version. The new cheatsheet contains 3 columns instead of two for the "Subset" sections, in the bottom right part of the first page. In 30 minutes or so we should see the change. If the job fails, the cheatseet is not replaced, or whatever, we'll follow up with another PR. There is no easy way to test this further without merging the PR.

Note that the copying of the cheatsheet to the directory to upload was already being run in this PR. So the job shouldn't be failing. It's just the upload that hasn't been tested already.
Ups, just realized now that we're copying the cheatsheets to the directory after the directory is uploaded, so the cheatsheets won't be pushed, and will be delete from the server, my bad. I'll open a PR to fix it.
TST: More type checking during .(i)loc
State: closed

- [X] Related to #43996
- [X] tests added to `pandas/tests/indexing/test_iloc.py`
- [X] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [X] During assignment to a .iloc or a .loc indexer, depending on whether the data frame is with mixed types or singular, the typing of the resulting assignment is not consistent.

As explained in issue #43996, the update of v1.3.0 causes a change in dtype casting when assigning a series. The issue contains a minimal working example of this rather inconsistent behaviour. In short, the type of a column changes when the assignment happens on a data frame only containing one column. With multiple columns, assigning the same series retains the original type. 

This contribution includes a test case which performs an assignment operation on a single column data frame. Naturally, because of this inconsistency of typing in the current pandas code, this test case fails. Its aim is to serve the bug fix for this inconsistency.

Labels:
name: Regression
description: Functionality that used to work in a prior pandas version
name: Categorical
description: Categorical Data Type
name: Stale

Comments:
oh i c you are just testing this, needs a patch.
Yes. I'm unsure how to patch this just yet.
@DamianBarabonkovQC wanna trade?  I'll figure out the patch for this and ask you to take a look at a less esoteric issue?
This pull request is stale because it has been open for thirty days with no activity. Please update or respond to this comment if you're still interested in working on this.
closing as stale if you want to continue, pls ping to reopen.
TST: Adding test to test_numeric, #37348
State: closed

- [x] closes #37348
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them

This makes sure that comparing int to empty strings doesn't return an error.


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Dtype Conversions
description: Unexpected or buggy dtype conversions

Comments:
thanks @gabrieldi95
[BUG] Fix DataFrameGroupBy.boxplot with subplots=False fails for object columns
State: closed

- [x] closes #43480
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [x] whatsnew entry


Labels:
name: Visualization
description: plotting

Comments:
thanks @brendandrury
ENH: Support observed keyword argument in Categorical.value_counts
State: closed

- [x] closes #43498
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [x] whatsnew entry


Labels:
name: Enhancement
name: Categorical
description: Categorical Data Type
name: Stale

Comments:
Thanks for the PR - this will also need a whatsnew since its an API change.
Think this needs abit of discussion before merging - there are lots of other methods this could be applied to. Doesn't make sense for the API to just support value_counts
> Think this needs abit of discussion before merging - there are lots of other methods this could be applied to. Doesn't make sense for the API to just support value_counts

@alimcmaster1 Any specifics?
This pull request is stale because it has been open for thirty days with no activity. Please update or respond to this comment if you're still interested in working on this.
Thanks for the pull request but appears to have gone stale. Additionally, this enhancement may need more discussion in the original issue before proceeding with a PR. Closing but happy to have more discussion in the issue.
BUG: Fix for rolling with uneven nanosecond windows have wrong results
State: closed

- [x] closes #43997
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [x] whatsnew entry


Labels:
name: Window
description: rolling, ewma, expanding

Comments:
Appears there's another merge conflict, sorry. 
Thanks @rosagold
MAINT: Correct small cast issues on Windows
State: closed

Use correct types to avoid downcasts on Windows

- [X] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them


Labels:
name: Compat
description: pandas objects compatability with Numpy or Python functions

Comments:
avoids build warnings?
Yes for some casts. Others are places where 54 but it's are being truncated to 32 bits. Would only ever show up on windows 64 in a frame with more then 2**32 rows.
BUG: DataFrame.drop fails when columns argument given tuple
State: closed

- [x] closes #43978 
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [ ] whatsnew entry


Labels:
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves

Comments:
I dont like allowing tuples as sequence input. See: https://github.com/pandas-dev/pandas/pull/42329#issuecomment-872249189
Closed this for now since it is not clear how to proceed and there is ambiguity around using tuples for indexing.
BUG: Fix `skipna` default value in method signatures
State: closed

Set `skipna` default values to `True` so that the value matches the
purpose of the variable and its docstring.

- [X] closes #34063
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [ ] whatsnew entry


Labels:
name: Docs

Comments:
I did not change the [mad](https://github.com/pandas-dev/pandas/blob/master/pandas/core/generic.py#L10585) method, since its signature matches the docstring and [the implementation](https://github.com/pandas-dev/pandas/blob/master/pandas/core/generic.py#L10605). Changing that one would require a deprecation cycle I guess; if not, let me know and I will amend the commit.
> I did not change the [mad](https://github.com/pandas-dev/pandas/blob/master/pandas/core/generic.py#L10585) method, since its signature matches the docstring and [the implementation](https://github.com/pandas-dev/pandas/blob/master/pandas/core/generic.py#L10605). Changing that one would require a deprecation cycle I guess; if not, let me know and I will amend the commit.

yeah this is fine
looks fine. @jbrockmendel ok here?
could remove the `if skipna is None` in _stat_function
> could remove the if skipna is None in _stat_function

Realizing here that I may have left behind many of them in private functions, e.g.

https://github.com/pandas-dev/pandas/pull/43980/files#diff-1a2e3df0db7dd8bddc2ec4bff9de8a7a55e328e6c32e2cecde761dc9549fcd46R10390
https://github.com/pandas-dev/pandas/pull/43980/files#diff-1a2e3df0db7dd8bddc2ec4bff9de8a7a55e328e6c32e2cecde761dc9549fcd46R10443
https://github.com/pandas-dev/pandas/pull/43980/files#diff-1a2e3df0db7dd8bddc2ec4bff9de8a7a55e328e6c32e2cecde761dc9549fcd46R10509
https://github.com/pandas-dev/pandas/pull/43980/files#diff-1a2e3df0db7dd8bddc2ec4bff9de8a7a55e328e6c32e2cecde761dc9549fcd46R10553

etc. There are still 10 (+2 corresponding to the `sem` method) appearances.

Are these 10 supposed to be changed as well ?
@jreback @jbrockmendel any suggestion as to what I should do concerning https://github.com/pandas-dev/pandas/pull/43980#issuecomment-941407410 ? Thanks.
> @jreback @jbrockmendel any suggestion as to what I should do concerning [#43980 (comment)](https://github.com/pandas-dev/pandas/pull/43980#issuecomment-941407410) ? Thanks.

see if you break anything by changing them. IIRC we did this for some backwards compat but that was quite a while ago.
> see if you break anything by changing them. IIRC we did this for some backwards compat but that was quite a while ago.

:+1: Push-forced in e15a1ce. Let's see what the CIs say.
thanks @jhlegarreta very nice!
BUG/TST: fixed so various methods call `__finalize__`, issue #28283
State: closed

### Progress towards #28283

After this PD, the following methods should propagate metadata using `__finalize__`:
- `DataFrame.combine, DataFrame.combine_first`
- `DataFrame.explode`
- `DataFrame.isin`
- `DataFrame.merge`
- `DataFrame.melt`
- `DataFrame.mode`
- `DataFrame.quantile`

<br/>

- [ ] closes #xxxx
- [x] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [ ] whatsnew entry

Contributors:
- [Casper Lundberg](https://github.com/casperlundberg)
- [Rickard Bemm](https://github.com/Rickos99)
- [Simon Lundberg](https://github.com/SimonHotDog)


Labels:

Comments:
prob best to do these in smaller chunks
prefer that you do this in smaller chunks.
STYLE ban np.testing
State: closed

xref https://github.com/pandas-dev/pandas/pull/43674#discussion_r720831806

There used to be a linting rule for this, but I missed it when moving linting rules over to `pandas-dev-flaker` in https://github.com/pandas-dev/pandas/pull/40906 - sorry about that, I thought I'd checked that everything had been preserved

Would be good to use this one from `pandas-dev-flaker` directly, but without a `--per-file-selects` option it'd be rather cumbersome (https://github.com/PyCQA/flake8/issues/344#issuecomment-940054283)

Labels:
name: Code Style
description: Code style, linting, code_checks

Comments:
nice!
Example for pandas.DataFrame.to_period docs
State: closed

- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [x] whatsnew entry
Doc improvement - added the example section for pandas.DataFrame.to_period


Labels:
name: Docs

Comments:
@MarcoGorelli Please review
Restarting CI - ping on green @oovk 
@alimcmaster1 
Thanks for the PR @oovk
CI: Add Cache task on Windows
State: closed

- [ ] closes #42580
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [ ] whatsnew entry


Labels:
name: CI
description: Continuous Integration

Comments:
how much does this help with build performance (look at master vs this PR).

cc @lithomas1 @fangchenli 
I don't see the Azure builds here. Is it actually running?
> how much does this help with build performance (look at master vs this PR).
> 
> cc @lithomas1 @fangchenli

Not sure. I never got cache working in #42581
I think I'm going the wrong way. Maybe I misunderstood the goal of the issue.

When I read builds, I could see at least two steps that run often that can be improved:
- Install dependencies
- run conda-incubator/setup-miniconda

`run conda-incubator/setup-miniconda` already has a cache but maybe we could improve it ?


similar to #44029
DOC: Fix Series nlargest and nsmallest doc strings (#43964)
State: closed

- [x] closes #43937
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [x] Cancel nlargest and nsmallest's doc list item text bold effect, detail could see below.

From
![Clip_20211011_094600](https://user-images.githubusercontent.com/25895405/136727253-30d20e21-ba1c-4be0-b1ad-87239b66d59e.png)
To
![Clip_20211011_105541](https://user-images.githubusercontent.com/25895405/136727262-f28a75e0-48e8-478b-a460-7690181d97d0.png)

Labels:
name: Docs

Comments:
Thanks @Zeroto521  - LGTM.

Could you also fix or create a follow up issue for `DataFrame.nlargest` - look inconsistent with others backticks are not correct around the first args: https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.nlargest.html 
https://github.com/pandas-dev/pandas/blob/accf59c8d1c581a056d8202d1fed95fdfae1d897/pandas/core/frame.py#L6597
> Could you also fix or create a follow up issue for `DataFrame.nlargest` - look inconsistent with others backticks are not correct around the first args: https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.nlargest.html

Do you mean to add double backticks for `first` and `last`, like the following?

```diff
-           - `first` : prioritize the first occurrence(s)
-           - `last` : prioritize the last occurrence(s)
+           - ``first`` : prioritize the first occurrence(s)
+           - ``last`` : prioritize the last occurrence(s)
            - ``all`` : do not drop any duplicates, even it means
              selecting more than `n` items.
```
> > Could you also fix or create a follow up issue for `DataFrame.nlargest` - look inconsistent with others backticks are not correct around the first args: https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.nlargest.html
> 
> Do you mean to add double backticks for `first` and `last`, like the following?
> 
> ```diff
> -           - `first` : prioritize the first occurrence(s)
> -           - `last` : prioritize the last occurrence(s)
> +           - ``first`` : prioritize the first occurrence(s)
> +           - ``last`` : prioritize the last occurrence(s)
>             - ``all`` : do not drop any duplicates, even it means
>               selecting more than `n` items.
> ```

Yes exactly so they are consistent with the docs here :) @Zeroto521 
@alimcmaster1 see #44040
Test that using a `tuple` or `list` for parameter `ascending` of `sort_index` is the same
State: closed

- [x] closes #43884 
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [x] whatsnew entry

I could not find a suitable place to add a `whatsnew` entry, if there is one I will push another commit.

Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode

Comments:
thanks @fAndreuzzi
DOC: cancel replace's doc list item text bold effect
State: closed

- [x] closes #43937
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [x] Cancel replace's doc list item text bold effect, detail could see below.

From 
![image](https://user-images.githubusercontent.com/25895405/136691591-6aae6597-99e7-4ae4-bca6-1e7380b1cad4.png)
To
![image](https://user-images.githubusercontent.com/25895405/136691594-c2b14ad8-c761-49c9-9ba3-b1a472280df7.png)



Labels:
name: Docs

Comments:
thanks @Zeroto521
PERF: Index.insert
State: closed

```
import pandas as pd

idx = pd.Index([4, 5, 6])

%timeit idx.insert(1, 19)
26.7 µs ± 299 ns per loop (mean ± std. dev. of 7 runs, 10000 loops each)  # <- master
15.5 µs ± 185 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)  # <- PR
```

Labels:
name: Performance
description: Memory or execution speed performance

Comments:
some tests failing
REF: share RangeIndex methods
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [ ] whatsnew entry


Labels:
name: Refactor
description: Internal refactoring of code
name: Index
description: Related to the Index class or subclasses

Comments:
assume this preserves range index perf asv's
> assume this preserves range index perf asv's

haven't measured, but logic-wise the only difference i can think of is function call overhead, and even that could go either way
TYP: Use Protocols for file-like objects in read/to_*
State: closed

Fixes #41610, rebased on top of #43855.

This PR does a few things:

1. break `FilePathOrBuffer` apart to not mix basic types and generics
2. use protocols instead of union of specific classes
3. define many fine-grained protocols for the to/read methods/functions

I tested that the protocols are sufficient (need no additional attributes/methods) using mock classes with:

- read_csv (python/c/pyarrow; w/wo compression) and to_csv (w/wo compression)
- to_json and read_json (each w/wo compression)
- to_pickle, read_pickle
- to_excel (openpyxl/xlsxwriter) and read_excel (openpyxl)
- to_stata and read_stata

Future: use many overloads for `get_handle` to return the (wrapped) fine-grained protocols.

Labels:
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
Hello @twoertwein! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:

































































There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2021-11-16 21:51:31 UTC
sorry, I squashed - thought it would make the rebasing easier.
> sorry, I squashed - thought it would make the rebasing easier.

you can or not - doesn't matter as squashes in merge anyways


thanks @twoertwein