BrandonFors/pandas-github-issues huggingface dataset:
REF: unwrap PandasArray earlier
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [ ] whatsnew entry


Labels:
name: Refactor
description: Internal refactoring of code
name: ExtensionArray
description: Extending pandas with custom dtypes or arrays.

Comments:
is this user facing in any way? any perf change?
> is this user facing in any way? 

no

> any perf change?

we'll take a hit from the isinstance check in sanitize_array, but a pretty good improvement from removing two `maybe_coerce_values` (though i forget which was the motivating asv for this)
DOC: Fix introduction of DataFrame.rename
State: closed

- [x] closes #43708
- [ ] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [ ] whatsnew entry


Labels:
name: Docs
name: DataFrame
description: DataFrame data structure

Comments:
there is a discussion somewhere, that `inplace` is considered being deprecated becuase it doesn't really perform the calculation in place (and therefore save memory) it actually creates a copy and disregards it. Therefore the original docs  here are actually more informative than what you have changed it to.
closing as this change makes the docs different that all other functions where this arg is.
BUG: Index.get_loc(np.nan) non-unique non-monotonic
State: closed

- [ ] closes #xxxx
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [x] whatsnew entry

The tests file will need to be rebased following #43705.

Labels:
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves

Comments:
looks fine, user facing?
> looks fine, user facing?

yah, will add whatsnew
Whatsnew added + green
looks good. can you rebase again and ping on green.
ping
ENH: Add equivalence test for float in to_numeric() GH43693
State: closed

- [x] closes #43693 
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [x] whatsnew entry

This PR prevents issues where float64 arrays would be downcast to float32 arrays even if the values were out of range for float32. An equivalence check is performed to ensure all values are equal to 7 significant digits. If this fails, the original float64 is retained. This behavior is consistent with int where downcasting is not performed if the integer is out of bounds.

Pre Change:
```python
>>> pd.to_numeric(pd.Series([608202549.0]), downcast='float')
0    608202560.0
```
Post Change:
```python
>>> pd.to_numeric(pd.Series([608202549.0]), downcast='float')
0    608202549.0
```

Labels:
name: Numeric Operations
description: Arithmetic, Comparison, and Logical operations

Comments:
This pull request is stale because it has been open for thirty days with no activity. Please update or respond to this comment if you're still interested in working on this.
@jbencina can you move the note, this looks good
@jreback Moved the change to V1.5.0. Not sure why this PR has trouble with the CI checks, but I don't think it's related to my code (eg. Azure timing out)
thanks @jbencina
REGR: Passing args/kwargs to Series.agg fails
State: closed

- [x] closes #43357
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [x] whatsnew entry


Labels:
name: Bug
name: Regression
description: Functionality that used to work in a prior pandas version
name: Apply
description: Apply, Aggregate, Transform, Map

Comments:
thanks @rhshadrach 
@meeseeksdev backport 1.3.x
Something went wrong ... Please have  a look at my logs.
BUG: Fix some index_col tests for pyarrow read_csv
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [ ] whatsnew entry


Labels:
name: IO CSV
description: read_csv, to_csv
name: Arrow
description: pyarrow functionality

Comments:
add this PR number on the same release note for the arrow reader
> add this PR number on the same release note for the arrow reader

This shouldn't be necessary, as its unreleased. I'm planning a lot more changes to the Arrow CSV reader to bring up somewhat up to par, and this would add clutter to the release notes.
> > add this PR number on the same release note for the arrow reader
> 
> This shouldn't be necessary, as its unreleased. I'm planning a lot more changes to the Arrow CSV reader to bring up somewhat up to par, and this would add clutter to the release notes.

it doesn't matter. the arrow change is now composed of multiple PRs which is totally fine (an alternative if you want is to link these follow PRs to the original at the top). The point is in theory we can point to a single issue / PR to have the full change.
thanks @lithomas1
GH36236 - Additional Test for df.join()
State: closed

TST

- [x] closes #36263 
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [ ] whatsnew entry


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode
name: Stale

Comments:
Hello @kasmith11! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:



There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2021-09-27 14:22:52 UTC
@github-actions pre-commit
Hi @kasmith11. Thanks for the PR!
Is there a particular issue you're trying to address?
Hello @debnathshoham I wanted to contribute a test for the [following issue](https://github.com/pandas-dev/pandas/issues/36263).
Awesome! Could you please make the title to more informative.
And mark closes #36263 in the OP.
@debnathshoham done!
@github-actions pre-commit
This pull request is stale because it has been open for thirty days with no activity. Please update or respond to this comment if you're still interested in working on this.
Thanks for the pull request, but it appears to have gone stale and the added test is still not passing the CI. If you'd like to continue, please merge master and we can reopen.
API: change ExtensionArray.astype to always return an ExtensionArray
State: closed

Closes #24877

This is a work-in-progress to see if it's realistic to require `ExtensionArray.astype` to always return an ExtensionArray (in practice this means returning a PandasArray if now a numpy array would be returned). 

Further we will also need to see how this impacts external EAs, and how this change can be managed.

(datetimelike and categorical arrays were not yet updated)

Labels:
name: API Design
name: ExtensionArray
description: Extending pandas with custom dtypes or arrays.

Comments:
This pull request is stale because it has been open for thirty days with no activity. Please update or respond to this comment if you're still interested in working on this.
https://github.com/pandas-dev/pandas/issues/24877#issuecomment-925134203
@jorisvandenbossche planning to do anything else here?
Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen.
PERF: nsmallest
State: closed

```
from asv_bench.benchmarks.series_methods import *
self6 = NSort()
self6.setup("last")

%timeit self6.time_nsmallest('last')
10.6 ms ± 43.4 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)  # <- master
1.5 ms ± 17.2 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)  # <- PR
```

Labels:
name: Performance
description: Memory or execution speed performance

Comments:
wow!
Backport PR #43647 on branch 1.3.x (BUG: REGR: read_csv with memory_map=True raises UnicodeDecodeError: 'utf-8' codec can't decode byte 0xc4 in position 262143: unexpected end of data )
State: closed

Backport PR #43647: BUG: REGR: read_csv with memory_map=True raises UnicodeDecodeError: 'utf-8' codec can't decode byte 0xc4 in position 262143: unexpected end of data 

Labels:
name: Regression
description: Functionality that used to work in a prior pandas version
name: IO CSV
description: read_csv, to_csv

Comments:
ahh in the backport we need to remove the pyarrow skip (as pyarrow is not in this version).

cc @michal-gh if you wouldn't mind pushing to this PR (or can open a different one if you'd like)
CLN: simplify `Styler._translate`
State: closed

Bit of a pre cursor to some other cleaning in #43649.

Prevents needing to pass variables around in worker functions, and has the added benefit of making the CSS classes user editable, by storing as a class attribute instead of module constants.

I ran the updated styler ASVs for this and "BENCHMARKS NOT SIGNIFICANTLY CHANGED"

Labels:
name: Refactor
description: Internal refactoring of code
name: Styler
description: conditional formatting using DataFrame.style

Comments:
@jreback got this to green. would appreciate pushing it through as a dependency to other prs.
Hello @attack68! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:









There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2021-10-15 16:48:05 UTC
@jreback ping green
@jreback can I bump this up. Other PRs depend on it. Will change if you have strong opinion.
ping @jreback. removed both and tested green. just noticed a doc update was also needed now.
thanks @attack68 
very nice, just a comment for followup.
PERF: dropna
State: closed

```
from asv_bench.benchmarks.frame_methods import *
self3 = Dropna()
self3.setup("all", 0)

%timeit self3.time_dropna('all', 0)
125 ms ± 3.03 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)  # <- master
34.8 ms ± 269 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)  # <- PR
```

Labels:
name: Missing-data
description: np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate
name: Performance
description: Memory or execution speed performance

Comments:
wow!
REF: unify i8 casting in DatetimeEngine
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [ ] whatsnew entry


Labels:
name: Refactor
description: Internal refactoring of code
name: Datetime
description: Datetime data dtype
name: Dtype Conversions
description: Unexpected or buggy dtype conversions

Comments:
precommit failing
greenish, docbuild looks unrelated
yep https://github.com/pandas-dev/pandas/issues/43685
POC: New UDF methods option
State: closed

Start of #41112

- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [ ] whatsnew entry

The goal is to provide an implementation for apply on lists. See https://github.com/pandas-dev/pandas/issues/41112#issuecomment-902366002 for why this necessitates a change to agg behavior for lists. It is possible to break this up into several PRs, but wanted to give a view as to what it would appear in totality.

Attached is a notebook demonstrating the differences in behavior (github doesn't allow attaching the notebook directly).

[udfs.pdf](https://github.com/pandas-dev/pandas/files/7199652/udfs.pdf)



Labels:
name: API Design
name: Apply
description: Apply, Aggregate, Transform, Map
name: Stale

Comments:
This pull request is stale because it has been open for thirty days with no activity. Please update or respond to this comment if you're still interested in working on this.
PERF: Groupby.any/all object dtype
State: closed

```
from asv_bench.benchmarks.groupby import *
self = GroupByMethods()
self.setup("object", "any", "direct", 10)
 

%timeit self.time_dtype_as_field("object", "any", "direct", 10)
1.12 ms ± 60.7 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)  # <- PR
11.4 ms ± 49.8 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)   # <- master
```

Labels:
name: Groupby
name: Performance
description: Memory or execution speed performance

Comments:
nice!
TST: dropping of nuisance columns for groupby ops #38815
State: closed

Add more aggregation functions for nuisance tests

- [ x] closes #38815
- [x ] tests added / passed
- [x ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [ ] whatsnew entry


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Groupby

Comments:
also pls merge master
https://github.com/pandas-dev/pandas/pull/43674/checks?check_run_id=3771738681
need sto assert this warning happens (this is pretty new btw)
thanks @horaceklai
ENH: latex styler bool pandas options
State: closed

- [x] adds the last  remaining options for #41395 for the existing args in Styler.to_latex.


Labels:
name: API Design
name: IO LaTeX
description: to_latex
name: Styler
description: conditional formatting using DataFrame.style

Comments:
can we add only the most important options here, e.g. it is perfectly fine to have keyword args that are not set by a global option. the namespace is getting cluttered.
sure, do you want to review some of the other ones that went in, and, to be consistent in terms of importance, remove some others also? 

Always easier to add something back than to remove it after a release.
> sure, do you want to review some of the other ones that went in, and, to be consistent in terms of importance, remove some others also?
> 
> Always easier to add something back than to remove it after a release.

yeah i think let's keep the options to the current exposed ones in prior versions e.g. don't add non-super-common options (i know vague)
> > sure, do you want to review some of the other ones that went in, and, to be consistent in terms of importance, remove some others also?
> > Always easier to add something back than to remove it after a release.
> 
> yeah i think let's keep the options to the current exposed ones in prior versions e.g. don't add non-super-common options (i know vague)

I removed ``css_convert`` and ``siunitx`` since they are probably more rarely used, but ``hrules`` I think should stick becuase DataFrame.to_html and Styler.to_html, have different defaults, the DataFrame version can only use True and Styler can render both with a False default.
@jreback ping green
cool
BUG: rename fails when using level
State: closed

- [x] closes #43659 
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [x] whatsnew entry


Labels:
name: Bug
name: MultiIndex

Comments:
Hello @wojtek2kdev! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:





There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2021-09-23 15:45:42 UTC
Also need whatsnew
@phofl Sorry, it's my first PR to this project. What is "whatsnew entry"?
Check doc/source/whatsnew. These are our release notes. Yours should go into 1.4
@phofl I'm not sure, but all of failed checks are my fault? For example: https://github.com/pandas-dev/pandas/pull/43667/checks?check_run_id=3655863372. I checked this test locally and it passes.
thanks @wojtek2kdev very nice!
ENH: RangeIndex.sort_values now avoids materialization of values
State: closed

- [x] closes #43584
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [x] whatsnew entry
What's new: On any instance of RangeIndex, sort_values will not sort as it is already sorted.

Labels:
name: Enhancement
name: Performance
description: Memory or execution speed performance
name: Index
description: Related to the Index class or subclasses

Comments:
also pls add asv's that catch this case.
also some pre-commit issues with typing: https://github.com/pandas-dev/pandas/pull/43666/checks?check_run_id=3659974388
@usersblock if you can address comments and merge master
I still need to know whether I should remove the @final since it's blocking my override.

> I still need to know whether I should remove the @Final since it's blocking my override.

yes you should
Hello @usersblock! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:











There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2021-10-19 22:06:41 UTC
cc @jbrockmendel if comments
i think in test_algos.TestFactorize.test_factorize with this PR you could pass `exact=True` to `assert_index_equal`
thanks @usersblock very nice!
DEPR: rolling/expanding/ewm.validate
State: closed

- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [x] whatsnew entry

An method with validation logic that should be in `__init__` ideally and should not be called by users. 

Labels:
name: Deprecate
description: Functionality to remove in pandas
name: Window
description: rolling, ewma, expanding

Comments:
thanks @mroeschke
ENH: `Styler.bar` accepts matplotlib colormap
State: closed

No one has requested this but I think its nice none-the-less, especially in finance / science. Kind of like background_gradient with slightly different vizualisations.

![Screen Shot 2021-09-19 at 21 25 07](https://user-images.githubusercontent.com/24256554/133940391-b2a8157e-8346-4cb4-8af2-8e71146e7c90.png)


![Screen Shot 2021-09-19 at 21 32 01](https://user-images.githubusercontent.com/24256554/133940536-8cc11271-0564-4123-a79e-b715d61356d8.png)




Labels:
name: Enhancement
name: Styler
description: conditional formatting using DataFrame.style

Comments:
ok this is fine, thanks @attack68
REF: simplify IndexEngine signature
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [ ] whatsnew entry


Labels:
name: Index
description: Related to the Index class or subclasses

Comments:
> does this make the engine non lazy?

Populating the hashtable is still lazy
CI: adding cython coverage
State: closed

- [x] closes #43652

Trying to include cython codes in codecov reports.
Working with the CI for the first time.


Labels:
name: CI
description: Continuous Integration
name: Stale

Comments:
@debnathshoham - do you still want to work on this? See comments above - H2H if needed
This pull request is stale because it has been open for thirty days with no activity. Please update or respond to this comment if you're still interested in working on this.
closing as stale
BUG: add css conversion for new `Styler.apply_index` styles
State: closed

- [x] closes #43648

no whats new needed since `apply_index` is new to 1.4.0 and this just makes it more complete.


Labels:
name: Bug
name: IO LaTeX
description: to_latex
name: Styler
description: conditional formatting using DataFrame.style

Comments:
sure
PERF: sparse take
State: closed

This helps some on #41023 because `__getitem__` uses `take` internally for `SparseArray`. However, the big gain there can follow after this to move towards not densifying on `__getitem__` with a slice

Benchmarks:
```
       before           after         ratio
     [e7e7b407]       [4de1fe27]
     <master>         <perf_sparse_take>
-        72.7±6μs         38.8±4μs     0.53  sparse.Take.time_take(array([0]), False)
-     1.00±0.06ms          350±4μs     0.35  sparse.Take.time_take(array([-1, -1, -1, ..., -1, -1, -1]), False)
-        737±20μs          206±6μs     0.28  sparse.Take.time_take(array([    0,     1,     2, ..., 99997, 99998, 99999]), False)
```

Labels:
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: Performance
description: Memory or execution speed performance
name: Sparse
description: Sparse Data Type

Comments:
lgtm. cc @jbrockmendel 
thanks @mzeitlin11
REF: remove JoinUnit.shape
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [ ] whatsnew entry


Labels:
name: Refactor
description: Internal refactoring of code
name: Internals
description: Related to non-user accessible pandas implementation

Comments:
Can you please wait with further PRs to JoinUnit until the discussion in https://github.com/pandas-dev/pandas/pull/43507 is resolved? (since those might make it harder to revert the change)
> Can you please wait with further PRs to JoinUnit until the discussion in #43507 is resolved? (since those might make it harder to revert the change)

@jorisvandenbossche if you don't like the new behavior then pls open an issue
@jorisvandenbossche discussion is not happening. please open an issue if you disagree with current behavior.
Can you respond to my comment at https://github.com/pandas-dev/pandas/pull/43507#issuecomment-920375856?
This pull request is stale because it has been open for thirty days with no activity. Please update or respond to this comment if you're still interested in working on this.
moving off 1.4
@jbrockmendel pls rebase, let's just do this