BrandonFors/pandas-github-issues huggingface dataset:
fix #39556 (infer_freq not working with freq="H" and DST
State: closed

- check that the delta are unique before checking if they are day multiples
- add test with freq="H" that raises the bug

- [x] closes #39556 
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [x] whatsnew entry => not sure where to enter this ...


Labels:
name: Datetime
description: Datetime data dtype
name: Timezones
description: Timezone data dtype

Comments:
I have added the whatsnew entry
There is some check that fails after having added the what's new entry. Any clue why?
@sdementen The CI failures are due to #39688. They should be gone if you merge master.
cc @mroeschke i think this is an hour-based analogue of the problem of freq=Day vs freq=DayDST.

i.e. this fixes one problem but will introduce others.  im hesitant to do that, but the long-term fix has been stuck in limbo for a while
> @sdementen The CI failures are due to #39688. They should be gone if you merge master.

Do I need to do this myself? Or will this automatically be solved once someone merge my PR into master?
> > @sdementen The CI failures are due to #39688. They should be gone if you merge master.
> 
> Do I need to do this myself? Or will this automatically be solved once someone merge my PR into master?

you wanna merge master yourself yeah
I haven't used `infer_freq` much, but if `infer_freq` = "infer the wall time frequency" then yes this the same issue as freq=Day vs freq=DayDST.
merged with master, all tests pass, ready to merge ;-)
@arw2019 @mroeschke , do I need to do something more re this PR for it to be merged ?
> can u also add an explicit test that is similar to the OP

The test `test_infer_freq_tz_transition` is similar to my OP in the issue. The bug was not revealing itself because the frequency "H" was not used (only "3H") and it is this frequency "H" that I uses in my OP.
cc @mroeschke if any comments, merge when good
Thanks @sdementen
Revert "Backport PR #39526 on branch 1.2.x (CI: pin numpy for CI / Checks github action)"
State: closed

Reverts pandas-dev/pandas#39529

will be merging this just prior to tagging 1.2.2 see https://github.com/pandas-dev/pandas/issues/39513#issuecomment-771006563 and then re-appling after release so that ci on 12.x is green

Labels:
name: CI
description: Continuous Integration
name: Blocker
description: Blocking issue or pull request for an upcoming release
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
CI / Checks is expected to fail and this PR will be merged only after all other PRs for 1.2.2 have been backported

https://github.com/pandas-dev/pandas/pulls?q=is%3Aopen+is%3Apr+milestone%3A1.2.2
REGR: fix transform of empty DataFrame/Series
State: closed

Closes #39636

Labels:
name: Regression
description: Functionality that used to work in a prior pandas version
name: Apply
description: Apply, Aggregate, Transform, Map

Comments:
Thanks @jorisvandenbossche if you can add a release note we can get this into 1.2.2
thanks @jorisvandenbossche the backport may note be trivial here as things have moved around
@meeseeksdev backport 1.2.x
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
$ git checkout 1.2.x
$ git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
$ git cherry-pick -m1 6db0b55a88a3f9b83136f0735b9a01c003af41ea
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
$ git commit -am 'Backport PR #39639: REGR: fix transform of empty DataFrame/Series'
```

4. Push to a named branch :

```
git push YOURFORK 1.2.x:auto-backport-of-pr-39639-on-1.2.x
```

5. Create a PR against branch 1.2.x, I would have named this PR:

> "Backport PR #39639 on branch 1.2.x"

And apply the correct labels and milestones.

Congratulation you did some good work ! Hopefully your backport PR will be tested by the continuous integration and merged soon!

If these instruction are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
                
cc @simonjayhawkins
API: transform behaves differently with 'ffill' on DataFrameGroupBy and SeriesGroupBy
State: closed

- [ ] closes #24211
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] whatsnew entry


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Missing-data
description: np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate

Comments:
Thanks @ftrihardjo - could you make the title the same as the issue please, and put the issue number in the pull request body (near `closes #`) ?
thanks @ftrihardjo
Revert "Fixed comment for pandas.unique"
State: closed

Reverts pandas-dev/pandas#39557

Labels:
name: Docs

Comments:
xref https://github.com/pandas-dev/pandas/pull/39557#issuecomment-774667542

but maybe no harm keeping on 1.2.x
thanks @simonjayhawkins 
@meeseeksdev backport 1.2.x
Can't Dooooo.... It seem like this is already backported (commit is empty).I won't do anything. MrMeeseeks out.
CLN: Styler HTML output adopt structured code standard
State: closed

- [x] closes #39626 
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [x] whatsnew entry

Labels:
name: IO HTML
description: read_html, to_html, Styler.apply, Styler.applymap
name: Styler
description: conditional formatting using DataFrame.style

Comments:
this is ready for review...
thanks @attack68
CLN: Use kwargs instead of kwds in apply functions
State: closed

- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them

I've applied the change here to the API function `apply` but I don't believe this is an API change as you cannot pass to `**kwds` (or `**kwargs`) by name.

Labels:
name: Clean
name: Apply
description: Apply, Aggregate, Transform, Map

Comments:
thanks yep, this is not user facing. we *could* have a rule for this to prevent recurrences (but prob no big deal)
CI/TST: Disallow usage of pytest.xfail
State: closed

- [x] closes #38902
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them




Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: CI
description: Continuous Integration
name: Code Style
description: Code style, linting, code_checks

Comments:
looks fine, some doc issue in ci / checks. ping on green.
@jreback green
thanks @rhshadrach
BUG: Series.loc raising KeyError for Iterator indexer in case of setitem
State: closed

- [x] closes #39614
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [x] whatsnew entry


Labels:
name: Bug
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves

Comments:
can you add any relevant cases to SetitemCastingEquivalents
The functions in there don't work with a generator.

Are you referring to adding a new function or to add a parametrization? Parametrization won't work with a generator, while adding a new function would cause a lot of special casing I think
> Are you referring to adding a new function or to add a parametrization? Parametrization won't work with a generator, while adding a new function would cause a lot of special casing I think

i was thinking adding to test_slice_key a check with `(x for x in indkey)` and in test_int_key one for `(x for x in [key])`
At thx. I though you were talking about adding another function. 

Should a Generator work for iloc?

```
IndexError: .iloc requires numeric indexers, got <generator object <genexpr> at 0x7f86ac39a890>
```

iloc.getitem raises this, so I assume no?
> Should a Generator work for iloc?

i dont know if we have a specific policy for that.  my only opinion here is that loc/iloc/`__getitem__`/`__setitem__` should be consistent
> At thx. I though you were talking about adding another function.
> 
> Should a Generator work for iloc?
> 
> ```
> IndexError: .iloc requires numeric indexers, got <generator object <genexpr> at 0x7f86ac39a890>
> ```
> 
> iloc.getitem raises this, so I assume no?

I don't think we accept iterator / generators for indexers, though we do in the constructors. I think its ok that we do, but agree that we should generally test these. Can you open an issue for this.
err rather, I agree with @jbrockmendel that we should systemcially test all setter & getter indexing methods in order to close this (rather than just adding in a specific place).
This would support Iterators for iloc and loc consistently
can you merge master
Done
I have added support for more dimensional iloc indexers. This should cover all cases now
can you rebase
Done
probably needs another rebase, but i think this was ready
rebased
thanks @phofl very nice
BUG: quantile for ExtensionArray
State: closed

- [ ] closes #xxxx
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] whatsnew entry

I _think_ this should work for arbitrary EAs, but we only have tests for sparse, dt64tz, and this adds Period (and some untested cases for dt64tz)

Part of the idea is to get the code out of the Block method for ArrayManager compat.

Labels:
name: Algos
description: Non-arithmetic algos: value_counts, factorize, sorting, isin, clip, shift, diff
name: ExtensionArray
description: Extending pandas with custom dtypes or arrays.

Comments:
hmm Interval breaks.

im inclined to leave testing the masked dtypes to @jorisvandenbossche 
I think it is important to add *some* basic testing for the different EAs we have internally. Because right now `quantile` works for eg Float64 (by coercing to a numpy array internally, I assume). And thus we need to ensure we don't break that behaviour (I think the current PR does so) 
BTW, I think it is perfectly fine to start with something that "works for us", without having it a fully general mechanism for external ExtensionArrays)
lgtm. thanks.
As mentioned above, this actually breaks it for nullable float/integer (which were working before)
REGR: appending to existing excel file created corrupt files
State: closed

- [x] closes #39576
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [x] whatsnew entry

Tentatively ready: I added a working test but I don't have MS Excel to make sure it is happy now. Waiting for someone to test it https://github.com/pandas-dev/pandas/issues/39576#issuecomment-774102850

Labels:
name: IO Excel
description: read_excel, to_excel

Comments:
cc @simonjayhawkins ok to merge on green
thanks @twoertwein 
@meeseeksdev backport 1.2.x
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
$ git checkout 1.2.x
$ git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
$ git cherry-pick -m1 098b9707f244e1ad8c3f2f9b94c050fe7c22bb9e
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
$ git commit -am 'Backport PR #39605: REGR: appending to existing excel file created corrupt files'
```

4. Push to a named branch :

```
git push YOURFORK 1.2.x:auto-backport-of-pr-39605-on-1.2.x
```

5. Create a PR against branch 1.2.x, I would have named this PR:

> "Backport PR #39605 on branch 1.2.x"

And apply the correct labels and milestones.

Congratulation you did some good work ! Hopefully your backport PR will be tested by the continuous integration and merged soon!

If these instruction are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
                
cc @twoertwein or @simonjayhawkins for backport
REGR: Rolling.count setting min_periods after call
State: closed

- [x] closes #39554
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [x] whatsnew entry


Labels:
name: Regression
description: Functionality that used to work in a prior pandas version
name: Window
description: rolling, ewma, expanding

Comments:
@meeseeksdev backport 1.2.x
thanks @mroeschke 
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
$ git checkout 1.2.x
$ git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
$ git cherry-pick -m1 d558bce8e9d5d4adfb0ab587be20b8a231dd1eea
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
$ git commit -am 'Backport PR #39604: REGR: Rolling.count setting min_periods after call'
```

4. Push to a named branch :

```
git push YOURFORK 1.2.x:auto-backport-of-pr-39604-on-1.2.x
```

5. Create a PR against branch 1.2.x, I would have named this PR:

> "Backport PR #39604 on branch 1.2.x"

And apply the correct labels and milestones.

Congratulation you did some good work ! Hopefully your backport PR will be tested by the continuous integration and merged soon!

If these instruction are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
GH38482 : added warning in docs to show -1 will be returned for not found values
State: closed

- [ ] closes #38482
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] whatsnew: added warning in docs to show -1 will be returned for not found values, with a suitable example.


Labels:
name: Docs
name: Stale

Comments:
Hello @shuaggar-sys! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:



There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2021-02-04 18:05:51 UTC
on running the below i get stale output, i ran it before i added the first commit, the message is not updated in the generated html file.
`python make.py --single pandas.Index.get_indexer`
did you do `python make.py clean`?
Sorry for the rookie mistake, i did not do that. Yes everything looks fine after the build.


![pandas_pr_ss](https://user-images.githubusercontent.com/67254493/106975072-d8bf8300-677b-11eb-9155-c163c12e424a.png)

This pull request is stale because it has been open for thirty days with no activity. Please update or respond to this comment if you're still interested in working on this.
closing as stale,  please open a new PR if interested in continuing
PERF: Rolling/Expanding.cov/corr
State: closed

- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [x] whatsnew entry

Also I think I stumbled on some bugs along the way; added whatsnew and modified the existing tests where necessary.

```
$ asv_bench % asv continuous -f 1.1 upstream/master HEAD -b rolling.Pairwise.time_groupby

       before           after         ratio
     [f79468b8]       [6a1e8592]
     <master>         <ref/groupby_rolling_pairwise>
-        299±50ms         178±20ms     0.59  rolling.Pairwise.time_groupby(1000, 'corr', True)
-         272±7ms         157±20ms     0.58  rolling.Pairwise.time_groupby(1000, 'cov', True)
-         262±1ms          146±4ms     0.56  rolling.Pairwise.time_groupby(None, 'cov', True)
-        303±10ms          158±6ms     0.52  rolling.Pairwise.time_groupby(10, 'cov', True)
-        279±20ms        139±0.7ms     0.50  rolling.Pairwise.time_groupby(None, 'corr', True)
-        315±10ms         149±10ms     0.47  rolling.Pairwise.time_groupby(10, 'corr', True)
-         137±2ms       34.0±0.8ms     0.25  rolling.Pairwise.time_groupby(10, 'cov', False)
-         127±5ms         28.2±1ms     0.22  rolling.Pairwise.time_groupby(None, 'cov', False)
-         134±5ms       28.1±0.4ms     0.21  rolling.Pairwise.time_groupby(1000, 'cov', False)
-         135±2ms         27.7±1ms     0.21  rolling.Pairwise.time_groupby(None, 'corr', False)
-        178±10ms         33.7±2ms     0.19  rolling.Pairwise.time_groupby(10, 'corr', False)
-        159±20ms         29.9±2ms     0.19  rolling.Pairwise.time_groupby(1000, 'corr', False)

SOME BENCHMARKS HAVE CHANGED SIGNIFICANTLY.
PERFORMANCE INCREASED.
```

Labels:
name: Performance
description: Memory or execution speed performance
name: Window
description: rolling, ewma, expanding

Comments:
thanks @mroeschke
TST: Remove duplicate invalid sheet tests from io.excel
State: closed

- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them

Followup from #39482 - removed tests are duplicates of `test_readers.test_bad_sheetname_raises`


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: IO Excel
description: read_excel, to_excel

Comments:
thanks @rhshadrach
REGR: do not try to infer the excel type from a workbook object
State: closed

- [ ] closes #39528
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] whatsnew entry


Labels:

Comments:
@rhshadrach you beat me by half an hour :) and your PR #39586 seems to be more thorough.
Ah, which was really thanks to your discussion in the issue.
REG: read_excel with engine specified raises on non-path/non-buffer
State: closed

- [x] closes #39528
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [x] whatsnew entry


Labels:
name: Regression
description: Functionality that used to work in a prior pandas version
name: IO Excel
description: read_excel, to_excel

Comments:
looks good to me!
if you can merge master and ping on greem
This PR enables the possibility that the sheet in `get_sheet_data` is not read-only. We only want to apply the fix from #39486 in the case the sheet is read-only (which is always the case when reading from a path or buffer). Unfortunately it seems to me that openpyxl provides no way of telling if a sheet is read-only or not; for this I've opened [an issue upstream](https://foss.heptapod.net/openpyxl/openpyxl/-/issues/1605).

The method `reset_dimensions` only exists on read-only sheets, and so it can be used for this purpose (even if not ideal).

I've added tests for this and to ensure the fix from #39486 works on non-read-only workbooks.
@jreback - green
thanks @rhshadrach 
@meeseeksdev backport 1.2.x
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
$ git checkout 1.2.x
$ git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
$ git cherry-pick -m1 95ee0b662b77e2e5f3b9f152b47a3adb7ecae12c
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
$ git commit -am 'Backport PR #39586: REG: read_excel with engine specified raises on non-path/non-buffer'
```

4. Push to a named branch :

```
git push YOURFORK 1.2.x:auto-backport-of-pr-39586-on-1.2.x
```

5. Create a PR against branch 1.2.x, I would have named this PR:

> "Backport PR #39586 on branch 1.2.x"

And apply the correct labels and milestones.

Congratulation you did some good work ! Hopefully your backport PR will be tested by the continuous integration and merged soon!

If these instruction are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
                
cc @simonjayhawkins 
@rhshadrach There have been some changes to master since 1.2 that were not backported (#38710 and  #39008) and trying to apply these changes, I'm getting

```
=================================================== short test summary info ===================================================
FAILED pandas/tests/io/excel/test_readers.py::TestReaders::test_corrupt_bytes_raises[engine_and_read_ext0] - AttributeError:...
FAILED pandas/tests/io/excel/test_readers.py::TestExcelFileRead::test_excel_read_binary[engine_and_read_ext0] - AttributeErr...
================================== 2 failed, 1398 passed, 146 skipped, 10 xfailed in 25.65s ===================================
```


ahh , needed a `content=`
BUG: at/iat __setitem__ failing to cast
State: closed

- [ ] closes #xxxx
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [x] whatsnew entry


Labels:
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: Dtype Conversions
description: Unexpected or buggy dtype conversions

Comments:
thanks. we might have some open issues that this would close if you can search at some point.
ENH add warning to Series.drop column noting that it does nothing
State: closed

Partial solution to #39509

Labels:
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: Compat
description: pandas objects compatability with Numpy or Python functions
name: Deprecate
description: Functionality to remove in pandas
name: Stale

Comments:
This pull request is stale because it has been open for thirty days with no activity. Please update or respond to this comment if you're still interested in working on this.
Hello,
I don't really know how to fix this to the point the maintainers want(haven't contributed before).

If you still feel this is worth fixing(I do), someone else should take over. It seems like a pretty simple change - adding a warning or a raise.

Thx,
Fealth


Thanks for taking a stab at this @Fealthas. Agreed this behavior needs improving, but appears this PR has gotten stale, so closing due to that.