BrandonFors/pandas-github-issues huggingface dataset:
deprecate nonkeyword args for bfill
State: closed

- [ ] xref #41485
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] whatsnew entry


Labels:
name: Deprecate
description: Functionality to remove in pandas

Comments:
Maybe can include this in https://github.com/pandas-dev/pandas/pull/41508 to somewhat reduce the number of PRs
sure
Deprecate passing args as positional DataFrame/Series.ffill
State: closed

- [ ] xref #41485
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] whatsnew entry


Labels:
name: Deprecate
description: Functionality to remove in pandas

Comments:
can you merge master
Deprecate non-keyword arguments for drop_duplicates.
State: closed

- [ ] xref #41485
- [ ] tests added / passed
- [ ] Ensure all linting tests pass
- [ ] whatsnew entry

Labels:
name: Deprecate
description: Functionality to remove in pandas

Comments:
Hello @jmholzer! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:

















There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2021-05-24 20:25:12 UTC
@MarcoGorelli I made the changes to drop_duplicates that we discussed in the comments of #41551, they are in the latest commit.
@MarcoGorelli regarding the warning message generation for `MultiIndex`, I think we need to remove the `@final` decorator from `Index.drop_duplicates`, is this acceptable?

https://docs.python.org/3.8/library/typing.html#typing.final


> @MarcoGorelli regarding the warning message generation for `MultiIndex`, I think we need to remove the `@final` decorator from `Index.drop_duplicates`, is this acceptable?
> 
> https://docs.python.org/3.8/library/typing.html#typing.final

sure. we tend to use @final to identify methods that aren't overloaded rather than methods that shouldn't be overloaded.
ENH: groupby rank supports object dtype
State: closed

- [x] closes #38278
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [x] whatsnew entry

This was fixed way back in #38744 and looks thoroughly tested in #41387, so just a whatsnew here. 

I think the other string types could also be supported with tweaks to `ea_wrap_cython_operation` to pull object data out of those types if we want to support those too.

Labels:
name: Docs
name: Groupby

Comments:
thanks @mzeitlin11
BUG: columns name retention in groupby methods
State: closed

- [x] closes #41564
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [x] whatsnew entry


Labels:
name: Bug
name: Groupby

Comments:
is there an issue for this?
> is there an issue for this?

No, though I _think_ it fixes #41564
checked and this does close #41564, will update above
@meeseeksdev backport 1.3.x
Something went wrong ... Please have  a look at my logs.
Deprecate passing default arguments as positional in reset_index
State: closed

- [ ] xref #41485
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] whatsnew entry


Labels:
name: Deprecate
description: Functionality to remove in pandas

Comments:
looks like theres a usage in the docs that needs updating
this does not close #41485
TST: Add tests for old issues 2
State: closed

- [x] closes #14467
- [x] closes #14564
- [x] closes #14773
- [x] closes #15036
- [x] closes #15106
- [x] closes #15239
- [x] closes #15273
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them



Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Groupby
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode
name: IO JSON
description: read_json, to_json, json_normalize
name: Nuisance Columns
description: Identifying/Dropping nuisance columns in reductions, groupby.add, DataFrame.apply

Comments:
This does not yet close #14564, https://github.com/pandas-dev/pandas/issues/14564#issuecomment-609116549?

should maybe change that one to an xref
> This does not yet close #14564, #14564 (comment)?

Sorry my comment was a little unclear. Our aggregation ops now support returning `datetime64[ns]` now so returning `NaT` should be expected here as mentioned in the original issue

> 3. Empty frame with datetime values
> 
> df = DataFrame(columns=['a', 'b'], dtype='datetime64')
> df.quantile(0.5, numeric_only=False)
> 
> On 0.18.1 / master, it gives a series of NaNs, where this should be NaTs:
> 
> In [13]: df.quantile(0.5, numeric_only=False)
> Out[13]: 
> a   NaN
> b   NaN
> Name: 0.5, dtype: float64


thanks @mroeschke
Deprecate nonkeyword args set axis
State: closed

- [ ] xref #41485
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] whatsnew entry


Labels:
name: Deprecate
description: Functionality to remove in pandas

Comments:
can u rebase
[ArrowStringArray] PERF: str.extract object fallback
State: closed

```
       before           after         ratio
     [40482273]       [1788ffe6]
     <master>         <extract-single-group>
-         134±2ms          106±1ms     0.79  strings.Extract.time_extract_single_group('string', False)
-         359±5ms        110±0.8ms     0.31  strings.Extract.time_extract_single_group('arrow_string', False)

SOME BENCHMARKS HAVE CHANGED SIGNIFICANTLY.
PERFORMANCE INCREASED.
```

Labels:
name: Performance
description: Memory or execution speed performance
name: Strings
description: String extension data type and string data

Comments:
Have dumbed down the changes to just the materialization of the array. Iteration on a pyarow array is the main bottleneck here.

```
       before           after         ratio
     [40482273]       [79ff0d3e]
     <master>         <extract-single-group>
-         140±2ms          114±2ms     0.82  strings.Methods.time_extract('string')
-       134±0.9ms          107±2ms     0.80  strings.Extract.time_extract_single_group('string', False)
-         139±2ms        108±0.9ms     0.77  strings.Extract.time_extract_single_group('string', True)
-         356±3ms          118±2ms     0.33  strings.Extract.time_extract_single_group('arrow_string', False)
-        367±10ms          120±1ms     0.33  strings.Methods.time_extract('arrow_string')
-         360±2ms          112±1ms     0.31  strings.Extract.time_extract_single_group('arrow_string', True)

SOME BENCHMARKS HAVE CHANGED SIGNIFICANTLY.
PERFORMANCE INCREASED.
```
very nice
STYLE: Add pre-commit hook for debug statements
State: closed

Added https://github.com/pre-commit/pre-commit-hooks#debug-statements as a pre-commit hook.

- [ ] closes #xxxx
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] whatsnew entry


Labels:
name: Code Style
description: Code style, linting, code_checks

Comments:
Thanks @klssmith !
BUG: CustomBusinessMonthBegin(End) sometimes ignores extra offset (GH41356)
State: closed

- [x] closes #41356 #14869
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [x] whatsnew entry

-----------------
**Problem description**: _CustomBusinessMonth does not apply extra offset when initially rolled month begin `new` is already a business day. This happens because we currently use CustomBusinessDay.rollforward (rollback) to both 1) roll to a business day and 2) apply the extra offset. 
**Fix**: Separating the logic to 1) roll to a business day and 2) apply the extra offset.

Labels:
name: Frequency
description: DateOffsets

Comments:
@jackzyliu can you merge master to resolve conflicts
Hi @simonjayhawkins, I just rebased after master, but some checks started failing and I can't replicate the failed tests offline. Examples: 
- `pandas/tests/tseries/offsets/test_offsets_properties.py::test_on_offset_implementations` with actions-38.yaml
- `pandas/tests/arithmetic/test_datetime64.py::TestDatetimeIndexArithmetic::test_sub_dti_dti` with actions-39-numpydev.yaml
 
These checks were successful before rebasing without any code change. What could be the issue here? 
The test_on_offset_implementations failure is a hypothesis failure so that maybe the reason it didn't fail before. may need to investigate that one further. The numpydev failures are a known issue.
Re-rebased after master. @simonjayhawkins 



> The test_on_offset_implementations failure is a hypothesis failure so that maybe the reason it didn't fail before. may need to investigate that one further. The numpydev failures are a known issue.

Previously, `pandas/tests/tseries/offsets/test_offsets_properties.py::test_on_offset_implementations` unrelatedly failed with the following pair of randomly generated test inputs,:
```
dt = datetime.datetime(1900, 1, 1, 0, 0, tzinfo=pytz.timezone('Africa/Kinshasa'))
offset = pd.offsets.MonthBegin(66)
``` 
I created a separate issue and pull request for this. 
Bumping up this PR for further review. Since we just released `v1.3.0`, I have moved the whatsnew entry to `1.3.1` for now. Happy to move it to `1.4.0` instead. 

The failed test seem to come from master?

Please let me know of further comments. Thanks.
@jreback @jbrockmendel @simonjayhawkins 

- Moved whatsnew entry to `1.4.0`
- Rebased after master
- All checks green

Please let me know of further comments. Thanks!
Resolved conflict with master. Green.
thanks @jackzyliu, sorry about the wait, lots of PRs :->

keep em coming!
[ArrowStringArray] REF: _str_startswith/_str_endswith
State: closed

using pyarrow native functions for _str_startswith/_str_endswith #41222 were merged before _str_contains #41217 so we can de-duplication some logic.

and also a change to _str_match for perf gain on object fallback

```
-      32.8±0.2ms       25.5±0.1ms     0.78  strings.Methods.time_match('arrow_string')
```

Labels:
name: Refactor
description: Internal refactoring of code
name: Strings
description: String extension data type and string data

Comments:
AFAICT the coverage build uses pyarrow 4.0.0 so the extra loc for the fallback won't be hit but are tested in other envs
thanks!
Deprecate non-keyword arguments in drop
State: closed

- [ ] xref #41485
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] whatsnew entry


Labels:
name: Deprecate
description: Functionality to remove in pandas

Comments:
FYI @MarcoGorelli a way to avoid the conflicts as we merge is to put the notes slightly randomly in the whatsnew deprecation section (e.g. not one after the other) and/or put blank lines to separate the conflicts, but nbd.
can u rebase
Thanks @simonjayhawkins - Index.drop doesn't have inplace and the return type is always the same so I didn't prioritise it here
TST/CLN: tests/strings/test_strings.py
State: closed

variable and test renaming, test splitting and parametrization of existing as a pre-cursor to further parametrization on dtype and moving.

Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Strings
description: String extension data type and string data
name: Clean

Comments:
can you rebase. lgtm. ping on green.
green
thanks @simonjayhawkins
TST: Add tests for old issues
State: closed

- [x] closes #13427
- [x] closes #13432
- [x] closes #13501
- [x] closes #13569
- [x] closes #13872
- [x] closes #13912
- [x] closes #13934
- [x] closes #14018
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them



Labels:
name: Testing
description: pandas testing functions or related to the test suite

Comments:
thanks @mroeschke
CI: Revert disable of some window test on Windows
State: closed

- [x] xref #37535
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them



Labels:
name: CI
description: Continuous Integration

Comments:
thanks!
DEPR: dropping nuisance columns in DataFrame reductions
State: closed

- [ ] closes #xxxx
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [x] whatsnew entry

Discussed on this week's call

Labels:
name: Deprecate
description: Functionality to remove in pandas
name: Reduction Operations
description: sum, mean, min, max, etc.
name: Nuisance Columns
description: Identifying/Dropping nuisance columns in reductions, groupby.add, DataFrame.apply

Comments:
added a whatsnew subsection.  this is actually just the first half of the note im about to push for #41475.  

the smart money says ive made a mess of the rst conventions regarding `code-block:: ipython` vs `ipython:: python` vs [...]
ping
REF: Various things preparing instantiable NumericIndex
State: closed

Separating various stuff from #41153, making that PR easier to review after this is merged, Similar idea as #41472.

Nothing in this PR changes anything, but just makes it e.g. simpler to update test fixtures after this is merged.

Labels:
name: Index
description: Related to the Index class or subclasses

Comments:
thanks @topper-123
Backport PR #41462: CI: Fix changed flake8 error message after upgrade (#41462)
State: closed

cc @jreback We could avoid the version changes in the pre commits if you prefer

Labels:
name: Code Style
description: Code style, linting, code_checks

Comments:
Thanks @phofl probably makes sense to update pre-commit as well to be consistent so that anyone doing manual backports can use pre-commit to make sure that the ci will pass
DEPR: dropping nuisance columns in DataFrameGroupby apply, agg, transform
State: closed

- [x] closes #21664
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [x] whatsnew entry

Discussed on this week's call.

Labels:
name: Groupby
name: Deprecate
description: Functionality to remove in pandas
name: Nuisance Columns
description: Identifying/Dropping nuisance columns in reductions, groupby.add, DataFrame.apply

Comments:
In `groupby._transform`, I think we can always use `_wrap_transform_fast_result` in the case of a string aggregator instead of falling back to `_transform_general` for certain cases. The only thing stopping this now is it removes the warnings added here. Digging into it, I'm finding some inconsistencies:

```
df = DataFrame(
    {
        "A": ["foo", "bar", "foo", "bar", "foo", "bar", "foo", "foo"],
        "B": ["one", "one", "two", "three", "two", "two", "one", "three"],
        "C": np.random.randn(8),
        "D": np.random.randn(8),
    }
)
df.groupby("A").mean()
df.groupby("A").agg("mean")
df.groupby("A").agg(np.mean)
df.groupby("A").transform("mean")  # <--- warns
df.groupby("A").transform(np.mean)  # <--- warns
df.groupby("A").apply("mean")
df.groupby("A").apply(np.mean)  # <--- warns
```

Should all, just the `np.mean`, or none of these warn? From the [whatsnew note](https://pandas.pydata.org/docs/whatsnew/v1.3.0.html#deprecated-dropping-nuisance-columns-in-dataframe-reductions-and-dataframegroupby-operations), it seemed to me that all of these should warn.
> Should all, just the np.mean, or none of these warn? From the [whatsnew note](https://pandas.pydata.org/docs/whatsnew/v1.3.0.html#deprecated-dropping-nuisance-columns-in-dataframe-reductions-and-dataframegroupby-operations), it seemed to me that all of these should warn.

I think you're right.  IRC getting consistency across all these cases was something we punted on.
Thanks, I'll take a deeper look into emitting the warning in the right places.
I missed that the default of `numeric_only` is still currently True, so many of the above cases are currently correct. With #46072, all of the `.mean()` and `"mean"` forms will warn. 

~The `np.mean` should warn in all cases, but doesn't for `agg`. I plan to be putting up a PR for this.~
I'm no longer certain on this. `agg` intercepts np.mean and replaces it with 'mean', which is why it doesn't warn. `transform` tries to do the same, but ends up having to call `transform_item_by_item` instead. `apply` also intercepts and replaces with 'mean', but then uses `DataFrame._reduce` which warns.
TST/CLN: move some tests
State: closed

no changes, only moved. maybe could split pandas/tests/strings/test_strings.py by method

Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Strings
description: String extension data type and string data
name: Clean

Comments:
> no changes, only moved. maybe could split pandas/tests/strings/test_strings.py by method

yes pls :->
[ArrowStringArray] TST: move/combine a couple of tests
State: closed

separate commits to simplify review (we may be able to remove pandas/tests/strings/test_string_array.py eventually)

Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Strings
description: String extension data type and string data

Comments:
thanks @simonjayhawkins small targeted prs are great!
REF: collect methods in NumericIndex
State: closed

Seperates the refactoring of existing methods out of #41153. Will make it easier to see what #41153 actually brings in of new things.

This PR changes no functionality, but only makes the exisiting numeric indexes into thin subclasses of the existing `NumericIndex`.

Labels:
name: Refactor
description: Internal refactoring of code
name: Index
description: Related to the Index class or subclasses

Comments:
Updated.
LGTM; if you find anywhere where `@final` can be correctly added, pls do
Wrt. @final, I’m not sure I like it in library code.

Couldn’t a third party subclass an index and change one of methods in an unanticipated way, but the @final be in the way of that?
Updated type hints.
thanks @topper-123
CI: Fix changed flake8 error message after upgrade
State: closed

Upgrade caused ci on 1.2.x to fail. Should backport probably

Labels:
name: CI
description: Continuous Integration
name: Code Style
description: Code style, linting, code_checks

Comments:
ok i suppose to backport, though we usually don't do the pre-commit 
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
$ git checkout 1.2.x
$ git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
$ git cherry-pick -m1 bfa75255849ed1473679948a1e4197886a252bdb
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
$ git commit -am 'Backport PR #41462: CI: Fix changed flake8 error message after upgrade'
```

4. Push to a named branch :

```
git push YOURFORK 1.2.x:auto-backport-of-pr-41462-on-1.2.x
```

5. Create a PR against branch 1.2.x, I would have named this PR:

> "Backport PR #41462 on branch 1.2.x"

And apply the correct labels and milestones.

Congratulation you did some good work ! Hopefully your backport PR will be tested by the continuous integration and merged soon!

If these instruction are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
                
@meeseeksdev backport 1.2.x
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
$ git checkout 1.2.x
$ git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
$ git cherry-pick -m1 bfa75255849ed1473679948a1e4197886a252bdb
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
$ git commit -am 'Backport PR #41462: CI: Fix changed flake8 error message after upgrade'
```

4. Push to a named branch :

```
git push YOURFORK 1.2.x:auto-backport-of-pr-41462-on-1.2.x
```

5. Create a PR against branch 1.2.x, I would have named this PR:

> "Backport PR #41462 on branch 1.2.x"

And apply the correct labels and milestones.

Congratulation you did some good work ! Hopefully your backport PR will be tested by the continuous integration and merged soon!

If these instruction are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
Backport PR #41452 on branch 1.2.x (CI: Pin jinja2 to version lower than 3.0)
State: closed

Backport PR #41452: CI: Pin jinja2 to version lower than 3.0

Labels:
name: CI
description: Continuous Integration
name: Dependencies
description: Required and optional dependencies

Comments:
Fastparquet errors and flake8 package inconsistency. So unrelated here
DEPR: DatetimeIndex.union with mixed timezones
State: closed

- [x] closes #39328
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [x] whatsnew entry


Labels:
name: Timezones
description: Timezone data dtype
name: Deprecate
description: Functionality to remove in pandas
name: Index
description: Related to the Index class or subclasses

Comments:
is this user facing anywhere aside from a direct index .union op? e.g. does this bubble to a concat?
> is this user facing anywhere aside from a direct index .union op? e.g. does this bubble to a concat?

not concat, _might_ be used in merge, will look into this
confirmed that merge is not affected (though we dont have any tests for this case with DTI, so ill add some in a separate PR)
thanks @jbrockmendel