BrandonFors/pandas-github-issues huggingface dataset:
PERF: DataFrame construction
State: closed

```
In [1]: from asv_bench.benchmarks.frame_ctor import *

In [2]: cls = FromArrays

In [3]: self = cls()

In [4]: self.setup()

In [5]: %timeit self.time_frame_from_arrays_int()
6.21 ms ± 184 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)  # <- master
3.27 ms ± 182 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)  # <- PR

```

Labels:
name: Performance
description: Memory or execution speed performance
name: Constructors
description: Series/DataFrame/Index/pd.array Constructors

Comments:
cool i would actually add a release note about this as this is a significant perf improvement. does this hold more generally with other construction asv's?

also doe you think its the removing asserts or the changes in not using is_sparse that are the big wins here? (guessing its the asserts)
> also doe you think its the removing asserts or the changes in not using is_sparse that are the big wins here? (guessing its the asserts)

I forget the exact numbers, but remember being surprised how big a difference the is_sparse change made
I'm seeing failure for the groupby.apply doctest on master. The condition that leads to raising an error is

    if newb.shape != self.shape:

Looks like shape is being cached here - might be related?
good catch, that caching should be reverted (and ideally a non-doctest test implemented)
BUG: `Styler.set_sticky` not handling display of `index names`. 1/2
State: closed

This addresses part of the bug identified in #42537, where column/index objects are not `MultiIndexes`.

Will need a follow-on to fully correct for the `MultiIndex` case.


Labels:
name: Bug
name: Styler
description: conditional formatting using DataFrame.style

Comments:
@simonjayhawkins can this make it into 1.3.1 release?
although the fix is not complete, this PR does provide a fix for common cases so is better than nothing.
thanks @attack68 

yeah if its a bug fix / regression we can certainly backport (and likey to have a number of 1.3.x releases).

@meeseeksdev backport 1.3.x
Something went wrong ... Please have  a look at my logs.
DOC: fix small issues
State: closed

read_clipboard passes the text to read_csv not read_table. See here for example: https://pandas.pydata.org/docs/dev/reference/api/pandas.read_clipboard.html

also replace "OS X" -> macOS

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] whatsnew entry


Labels:
name: Docs

Comments:
Thanks @partev
CI/TYP: remove unneeded stata ignore
State: closed

Should fix typing failure on master


Labels:
name: CI
description: Continuous Integration
name: IO Stata
description: read_stata, to_stata
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
> environment.yml is the env used for mypy type checking on ci. so if pinned this PR is not needed?

oh numpy has bumped as well as fastparquet on ci. so not related.
> oh numpy has bumped

I assume this will happen on 1.3.x too, so milestone 1.3.1 to trigger backport
Thanks @mzeitlin11
Pin fastparquet
State: closed

- [ ] xref #42588
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] whatsnew entry

Get CI to green.

Labels:
name: CI
description: Continuous Integration
name: IO Parquet
description: parquet, feather

Comments:
Should probably pin in requirements-dev.txt as well until it is ready to move to 0.7.0.
The fastparquet tests also fail on 1.3.x, so milestone 1.3.1 to trigger backport
@lithomas1 can you open an issue to unpin this.
> @lithomas1 can you open an issue to unpin this.

added xref to the issue to the op, so no need to open a separate issue.
@github-actions pre-commit
Green, can someone from @pandas-dev/pandas-core merge?

Sorry about the noise in this PR, was using the Github Web UI, which pre-commit apparently doesn't like.

Windows py39 is unrelated timeout. Windows py38 & Database are the only builds that matter in this case.
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
$ git checkout 1.3.x
$ git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
$ git cherry-pick -m1 46ed60a2cb007276b38f0bcdead312d95cfc4d54
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
$ git commit -am 'Backport PR #42612: Pin fastparquet'
```

4. Push to a named branch :

```
git push YOURFORK 1.3.x:auto-backport-of-pr-42612-on-1.3.x
```

5. Create a PR against branch 1.3.x, I would have named this PR:

> "Backport PR #42612 on branch 1.3.x"

And apply the correct labels and milestones.

Congratulation you did some good work ! Hopefully your backport PR will be tested by the continuous integration and merged soon!

If these instruction are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
                
thx @lithomas1 
backported manually
thx @phofl.
PERF: select_dtypes
State: closed

```
In [1]: from asv_bench.benchmarks.dtypes import *

In [2]: cls = SelectDtypes

In [3]: self = cls()

In [4]: dtype = "Int32"

In [5]: self.setup(dtype)

In [6]: %timeit self.time_select_dtype_string_exclude(dtype)
1.79 ms ± 50.7 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)   # <-- master
47.7 µs ± 461 ns per loop (mean ± std. dev. of 7 runs, 10000 loops each)  # <-- PR
```

Labels:
name: Performance
description: Memory or execution speed performance
name: Dtype Conversions
description: Unexpected or buggy dtype conversions

Comments:
nice, can you rebase (I think the typing issue is resolved on master)
DOC: use the commonly accepted capitalization of LaTeX
State: closed

Latex -> LaTeX

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] whatsnew entry


Labels:

Comments:
look like heading have a special capitalization rule:
doc/source/reference/io.rst:102:Heading capitalization formatted incorrectly. Please correctly capitalize "LaTeX" to "Latex"
BUG: SeriesGroupBy.nlargest/smallest inconsistent shape
State: closed

- [x] Closes #15272
- [x] Closes #16345
- [x] Closes #29129
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [x] whatsnew entry

Labels:
name: Bug
name: Groupby

Comments:
very nice merge on green
thanks @rhshadrach
STYLE bump pandas-dev-flaker
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] whatsnew entry


Labels:
name: Code Style
description: Code style, linting, code_checks

Comments:
on another thought, this isn't worth bumping people's pre-commit envs for, I'll bump when there's a few more checks
BUG: pivot_table with margins=True and numeric columns is buggy
State: closed

- [x] closes #26568
- [x] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [x] Added test for issue #26568


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode

Comments:
thanks @Alex-Gregory-1
TST: added test for issue #26568
State: closed

- [x] closes #26568
- [x] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [x] Added test for issue #26568


Labels:

Comments:
@github-actions pre-commit
DEPR: Raise FutureWarning when creating empty series without dtype
State: closed

- [x] closes #30017
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [x] whatsnew entry

Note, this would make the next release 1.4 for sure.


Labels:
name: Deprecate
description: Functionality to remove in pandas
name: Series
description: Series data structure

Comments:
thanks @lithomas1 very nice! yeah i guess we should have done this a while back, but better late than not!
POC: disallow mutating in groupby ops
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] whatsnew entry


Labels:
name: Groupby

Comments:
Love the idea. Out of curiosity why do you think this is better to manage in internals than in the groupby itself though? I am somewhat wary of adding more state management internally
> Out of curiosity why do you think this is better to manage in internals than in the groupby itself though? I am somewhat wary of adding more state management internally

It isn't clear to me how you would do this in the groupby object.  It's fundamentally the Series/DataFrame that we're trying to disallow mutating, so seems that is where we need to freeze the state.
BUG: DataFrame.copy not consolidating
State: closed

- [x] closes #42477
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] whatsnew entry

@mzeitlin11 did you want to suggest an improved warning message for .insert?

Labels:
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: Regression
description: Functionality that used to work in a prior pandas version

Comments:
> @mzeitlin11 did you want to suggest an improved warning message for .insert?

Thanks for the heads up.  Since this fix should be independent to that and don't want to hold up this pr, probably easiest as a separate pr
@jbrockmendel tests are failing. also needs release note for 1.3.1
> also needs release note for 1.3.1

bugfix section
hmm these failures unrelated? maybe merge master again
can you run some asv's as well to make sure this doesn't change perf drastically. maybe insert / concat / copy ones.
@meeseeksdev backport 1.3.x
Something went wrong ... Please have  a look at my logs.
I'm seeing this warning with Python 3.8.12; pandas 1.3.4 using this function to combine a list of data frames (from parquet files)
`pd.concat(objs=[pd.read_parquet(path=i, use_nullable_dtypes=True) for i in local_files], axis=0, ignore_index=True).copy()`
BUG: Series(dask.array) GH#38645
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] whatsnew entry

This should close #38645 in conjunction with dask/dask#7888, will hold off on closing the issue until @jsignell confirms.

Labels:
name: Regression
description: Functionality that used to work in a prior pandas version
name: Constructors
description: Series/DataFrame/Index/pd.array Constructors

Comments:
do we want to include in 1.3.1? and include a release note so that users know that constructors now support the numpy array protocol? 
> and include a release note so that users know that constructors now support the numpy array protocol?

maybe also update the docs, although the discussion on what is array-like is ongoing #41807
Thanks! I can confirm that this works when paired with https://github.com/dask/dask/pull/7905 which implements a lazy version of ``Array.__iter__``. 
i suppose this is fine for 1.3.1 and can rearrange in a follow on

if u can add a release note
@meeseeksdev backport 1.3.x
Something went wrong ... Please have  a look at my logs.
BUG: concat bool+int with ArrayManager
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] whatsnew entry


Labels:
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode

Comments:
(this is whats needed to get the CI back to green)
@meeseeksdev backport 1.3.x
Something went wrong ... Please have  a look at my logs.
Revert "PERF: tighten _should_compare for MultiIndex"
State: closed

Reverts pandas-dev/pandas#42231

This introduced a bug discussed in #42231.  The long-term fix is in MultiIndex.get_indexer, but that's going to take me a little while to get done.

Labels:
name: Regression
description: Functionality that used to work in a prior pandas version

Comments:
@simonjayhawkins @jreback should we try to get this in for 1.3.1?
#42231 (source of regression) is not on 1.3.x?
woops, assumed bc we got a bug report it must have been released, but not so
thanks @jbrockmendel
DOC: Correct singular "die"/plural "dice"
State: closed

Minor change to DataFrame.plot.hist documentation: "Dice" is traditionally plural (one die, two dice). Nowadays it can also be singular, but "die" is clearer. "Dices" is simply not a thing.


Labels:
name: Docs

Comments:
I would argue "roll a die" is the correct language for this physical simulation, whereas "draw a number from 1 to 6" is more appropriate for a computational simulation. "draw a die" just seems weird. 
looks fine.
Thanks @drewlevitt
commenting import modules purpose
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] whatsnew entry


Labels:

Comments:
Hello @mushahidmehdi! Thanks for opening this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:

* In the file [`pandas/core/apply.py`](https://github.com/pandas-dev/pandas/blob/dd8a8808e5558349191c5228b0faadff6801b427/pandas/core/apply.py):

> [Line 17:89](https://github.com/pandas-dev/pandas/blob/dd8a8808e5558349191c5228b0faadff6801b427/pandas/core/apply.py#L17): [E501](https://duckduckgo.com/?q=pep8%20E501) line too long (94 > 88 characters)
> [Line 17:95](https://github.com/pandas-dev/pandas/blob/dd8a8808e5558349191c5228b0faadff6801b427/pandas/core/apply.py#L17): [W291](https://duckduckgo.com/?q=pep8%20W291) trailing whitespace


Hi @mushahidmehdi and thanks for contributing.
But I don't consider this PR to be of any value. Firstly it only impacts one file, but otherwise the comments added are implied by the code segments themselves which are ubiquitous. It is akin to commenting a `for` loop with the comment `do a for loop`, and that simply diminishes the value of other code comments which actually explain the more complex areas.

Please do have a look at the `good first issues` in the Issues section since these are items where PRs are more likely to be accepted and welcomed.
Thanks for the feedback,

I know what I added to the file doesn't have any value to it, I was just at
begging my open Source contributions therefore I was trying to be as
careful as possible without touching the Source code.

As soon as I will understand the project in better detail,  I definitely
will add some value to pandas Library other issues.






On Fri, Jul 16, 2021, 23:27 attack68 ***@***.***> wrote:

> Hi @mushahidmehdi <https://github.com/mushahidmehdi> and thanks for
> contributing.
> But I don't consider this PR to be of any value. Firstly it only impacts
> one file, but otherwise the comments added are implied by the code segments
> themselves which are ubiquitous. It is akin to commenting a for loop with
> the comment do a for loop, and that simply diminishes the value of other
> code comments which actually explain the more complex areas.
>
> Please do have a look at the good first issues in the Issues section
> since these are items where PRs are more likely to be accepted and welcomed.
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> <https://github.com/pandas-dev/pandas/pull/42572#issuecomment-881699614>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AP2XK43FHGLVXTJWIVHDE7LTYCI2BANCNFSM5AQDFTIA>
> .
>
CI: Fixed a typo in Web and Docs
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] whatsnew entry

I accidentally made a typo in #42556. 
Hopefully works this time. The docs weren't getting uploaded, b/c of this typo.

Labels:
name: CI
description: Continuous Integration
name: Web
description: pandas website

Comments:
cool, ping on greenish

@jreback Green.
@meeseeksdev backport 1.3.x
Something went wrong ... Please have  a look at my logs.
BUG: Groupby min/max with nullable dtypes
State: closed

- [x] closes #41743
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [x] whatsnew entry

asvs show no change for non-nullable dtypes

Labels:
name: Bug
name: Groupby
name: NA - MaskedArrays
description: Related to pd.NA and nullable extension arrays

Comments:
> not really thrilled with this expansion, but i guess its needed.

I'd be OK with _not_ special-casing MaskedArray all throughout the codebase.
can u refresh with some places where we special case now?
> can u refresh with some places where we special case now?

array_algos.quantile, GroupBy._bool_agg, GroupBy._get_cythonized_result, WrappedCythonOp._masked_ea_wrap_cython_operation, and a bunch prospectively in #42838, #39133, #37493
small naming comments & if you can rebase
just rebased; what did you think of the compromise naming of `result_mask`?
> just rebased; what did you think of the compromise naming of `result_mask`?

cool with that

one place where need to update i think
> one place where need to update i think

my thought with "result_mask" was to make a change in "mask" unnecessary.  im not wild about mask_in bc it is awkward in cases without mask_out
> > one place where need to update i think
> 
> my thought with "result_mask" was to make a change in "mask" unnecessary. im not wild about mask_in bc it is awkward in cases without mask_out

right result_mask is fine, its the plain mask (when we also have mask_out) that bothers me. just need consistent naming.
> right result_mask is fine, its the plain mask (when we also have mask_out) that bothers me. just need consistent naming.

"result_mask" is the alternative to "mask_out", so there is no more "mask_out".  I don't like "mask_in" bc we'll end up with some functions that call it "mask_in" and others that call the same thing "mask" (bc those functions dont have result_mask/mask_out)
> but it might be good to add a benchmark for this

good idea.  will do as follow-up, as im hoping we can get this in quick-ish
> I agree with Brock on having consistent use of mask being better than some mask and some mask_in. I think having a function here with both mask and result_mask, I think it's clear that mask is about the input.


I am cool with the names, just want to be consistent across the code base on these names / meaning
> I am cool with the names, just want to be consistent across the code base on these names / meaning

What Joris and I are saying is that this _is_ the best way to be consistent w/r/t naming
also in followon should validate first/last (and maybe others) are correctly handling these dtypes, can you create an issue.
ENH: Add support for nullable boolean and integers in Stata writers
State: closed

Add code that allows nullable arrays to be written

closes #40855

- [X] closes #40855
- [X] tests added / passed
- [X] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [X] whatsnew entry


Labels:
name: Enhancement
name: IO Stata
description: read_stata, to_stata

Comments:
Seeing a lot of random failures unrelated to this.
This pull request is stale because it has been open for thirty days with no activity. Please update or respond to this comment if you're still interested in working on this.
@jreback I think this is ready, and all green now.

pls merge master as well
thanks @bashtage
CI: Remove xfail for xpassing test
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] whatsnew entry


Labels:
name: CI
description: Continuous Integration

Comments:
Seems to work. Would someone on the core team mind restarting the posix jobs? The minimum versions job failed to download conda and I would really like to see that one run, just to make sure everything works.
close and reopen the PR will force the ci to rerun 
Thanks @lithomas1 
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
$ git checkout 1.3.x
$ git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
$ git cherry-pick -m1 ac68115b3beb0f1c3ce6dba4800ebccf34e1cfdf
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
$ git commit -am 'Backport PR #42558: CI: Remove xfail for xpassing test'
```

4. Push to a named branch :

```
git push YOURFORK 1.3.x:auto-backport-of-pr-42558-on-1.3.x
```

5. Create a PR against branch 1.3.x, I would have named this PR:

> "Backport PR #42558 on branch 1.3.x"

And apply the correct labels and milestones.

Congratulation you did some good work ! Hopefully your backport PR will be tested by the continuous integration and merged soon!

If these instruction are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).