BrandonFors/pandas-github-issues huggingface dataset:
BUG: Unary pos/neg ops on IntegerArrays failing with TypeError
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Numeric Operations
description: Arithmetic, Comparison, and Logical operations
name: NA - MaskedArrays
description: Related to pd.NA and nullable extension arrays

Comments:
This is a follow-up to #36078, which aimed to address #36063
> does this have a linked issue? can you add a whatsnew note (you can put in bug fixes for ExtensionArrays) for 1.2

marking as 1.1.2 for now to match #36063
sorry. I made an attempt to fix the issue (#36063 ) but then I realized I was trying to do stuff without fully understanding it, I'm not sure if this is even in the right direction at the moment. I can close this PR (put it in draft anyway because I was unsure) if someone else wants a crack at this.
Thanks @asishm no problem, no pressure. will remove milestone.
BUG: add py39 compat check for ast.slice #32766
State: closed

- [x] closes #32766
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`



Labels:
name: Compat
description: pandas objects compatability with Numpy or Python functions

Comments:
Should this only be done for py39+?
does this need a test?
> does this need a test?

Not sure about this. Might need other people's opinion.
thanks @VirosaLi and thanks for pushing on the 3.9 stuff. note we *might* need to backport this.
> thanks @VirosaLi and thanks for pushing on the 3.9 stuff. note we _might_ need to backport this.

You are welcome. Let me know if you need me to backport this.
BUG: Unary pos/neg ops on IntegerArrays failing with TypeError
State: closed

- [ ] closes #36063
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Bug
name: ExtensionArray
description: Extending pandas with custom dtypes or arrays.

Comments:
What behavior do we want for unary minus on a uint array?  Wrap around?
`0 - uint_series` currently wraps arround. probably makes sense to keep the unary neg to exhibit the same behavior.

Also I'm not sure if I wrote the tests correctly. I think I'd have to xfail test_neg and test_pos for BooleanArrays or move those methods over to test_integer.py specifically. thoughts?
there's a simpler fix here

by adding `__neg__` and `__pos__` implementations to BaseMaskedArray (`__invert__` is already implemented) and changing the `__neg__` implementation of BooleanArray to call `__invert__`

That passes the current tests that I added.

thoughts?
#36081 for implementation
DOC: Fix typo of `=!` to `!=` in docstring
State: closed

This fixes GH36075.

- [x] closes #36075 

I'm marking the following list items as not-applicable because I'm simply swapping two characters in a docstring that had been in the incorrect order.

- [NA] tests added / passed
- [NA] passes `black pandas`
- [NA] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [NA] whatsnew entry


Labels:
name: Docs

Comments:
Thanks @daytonb !
Thanks! It's been a pleasure. Thanks for the quick merge.
So this change actually breaks building docs, at least using sphinx. Leads to an error

```
/usr/local/lib/python3.8/site-packages/docutils/nodes.py", line 567, in __getitem__
    return self.attributes[key]
KeyError: 'reftarget'
```

whereas simply swapping them back fixes the problem. Any idea what might be going on here?
Is no one else gets to it before me I could look into it, but probably not
until tomorrow.


On Fri, Apr 2, 2021 at 7:34 PM Sami Jawhar ***@***.***> wrote:

> So this change actually breaks building docs, at least using sphinx. Leads
> to an error
>
> /usr/local/lib/python3.8/site-packages/docutils/nodes.py", line 567, in __getitem__
>     return self.attributes[key]
> KeyError: 'reftarget'
>
> whereas simply swapping them back fixes the problem. Any idea what might
> be going on here?
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> <https://github.com/pandas-dev/pandas/pull/36077#issuecomment-812754556>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/ABJTWI3UNDN2GEQOTMUPOK3TGZH7HANCNFSM4QTOIJ3A>
> .
>

@sjawhar, can you give me the steps you ran to reproduce this issue? It seams unlikely that my text change could lead to the error you posted. Also, when I my docstring change is already part of the current (v1.2.3) release and the documentation is rendered properly online, all of which seems to indicate that this docstring change isn't the cause of the issue you're reporting. If you can give me some steps to reproduce, I'd be happy to try to figure out what's going on.
Backport PR #35852 on branch 1.1.x (API: replace dropna=False option with na_sentinel=None in factorize)
State: closed

Backport PR #35852: API: replace dropna=False option with na_sentinel=None in factorize

Labels:
name: API Design

Comments:
@charlesdong1991 and can merge this when green as well
> @charlesdong1991 and can merge this when green as well

and when merging on the backport branch ensure that Co-authored-by: Kaiqi Dong <kaiqi@kth.se> or whoever the author is is included otherwise they won't appear in the contributors list 
may I kindly ask how to check if the author is included in this case? 😅  @simonjayhawkins 
when you click on confirm squash and merge you'll see

![image](https://user-images.githubusercontent.com/13159005/92010074-42f17900-ed41-11ea-80c2-22f287c0055f.png)

so the the co-author is included automatically. just need to ensure that if cleaning the description that you don't remove the co-author (i.e. the original author)
yep, get it! thanks very much! @simonjayhawkins
TYP: statically define attributes in plotting._matplotlib.core
State: closed

continuation of changes in #36016

pandas\plotting\_matplotlib\core.py:231: error: "MPLPlot" has no attribute "style"  [attr-defined]
pandas\plotting\_matplotlib\core.py:232: error: "MPLPlot" has no attribute "style"  [attr-defined]
pandas\plotting\_matplotlib\core.py:233: error: "MPLPlot" has no attribute "style"  [attr-defined]
pandas\plotting\_matplotlib\core.py:235: error: "MPLPlot" has no attribute "style"  [attr-defined]
pandas\plotting\_matplotlib\core.py:385: error: "MPLPlot" has no attribute "label"; maybe "ylabel" or "xlabel"?  [attr-defined]
pandas\plotting\_matplotlib\core.py:553: error: "MPLPlot" has no attribute "mark_right"  [attr-defined]
pandas\plotting\_matplotlib\core.py:732: error: "MPLPlot" has no attribute "style"  [attr-defined]
pandas\plotting\_matplotlib\core.py:733: error: "MPLPlot" has no attribute "style"  [attr-defined]
pandas\plotting\_matplotlib\core.py:735: error: "MPLPlot" has no attribute "style"  [attr-defined]
pandas\plotting\_matplotlib\core.py:738: error: "MPLPlot" has no attribute "style"  [attr-defined]
pandas\plotting\_matplotlib\core.py:739: error: "MPLPlot" has no attribute "style"  [attr-defined]
pandas\plotting\_matplotlib\core.py:741: error: "MPLPlot" has no attribute "style"  [attr-defined]
pandas\plotting\_matplotlib\core.py:1008: error: "ScatterPlot" has no attribute "label"  [attr-defined]
pandas\plotting\_matplotlib\core.py:1075: error: "LinePlot" has no attribute "stacked"  [attr-defined]
pandas\plotting\_matplotlib\core.py:1180: error: "LinePlot" has no attribute "stacked"  [attr-defined]
pandas\plotting\_matplotlib\core.py:1269: error: "AreaPlot" has no attribute "stacked"  [attr-defined]
pandas\plotting\_matplotlib\core.py:1351: error: "BarPlot" has no attribute "stacked"  [attr-defined]
pandas\plotting\_matplotlib\core.py:1427: error: "BarPlot" has no attribute "stacked"  [attr-defined]



Labels:
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
thanks
BUG: groupby and agg on read-only array gives ValueError: buffer source array is read-only
State: closed

- [x] closes #36014 
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry

I found multiple aggregate functions that were failing for a read-only array. I applied the change suggested in #36014 to all of them.

A couple of questions.

1. I had to add the Py_ssize_t cdefs to suppress the error below for `len(values) != len(labels)`. Let me know if there is another preferred way of doing it.

```
error: comparison of integer expressions of different signedness: ‘Py_ssize_t’ {aka ‘long int’} and ‘size_t’ {aka ‘long unsigned int’} [-Werror=sign-compare]
```

2. I see some functions don't have the `len(values) != len(labels)` check. Should it be there in all the functions?

Labels:
name: Groupby
name: Compat
description: pandas objects compatability with Numpy or Python functions

Comments:
LGTM pending Jeff's test request
@jreback,
> also you are referencing 2 issues. can you add tests that are in these issues.

Tests for #35436 and #34857? Those issues are not fixed by this PR. Even if I add tests, they'd fail. Unless you mean something else which I am missing.

I tried @jbrockmendel's suggestion. `if values.shape[0] != labels.shape[0]` works. What do we want to use? Py_ssize_t cdefs, or `.shape[0]`?

Also, should the `len(values) != len(labels)` check be there in all functions?
> Tests for #35436 and #34857? Those issues are not fixed by this PR. Even if I add tests, they'd fail. Unless you mean something else which I am missing

hmm thought i saw this, ok no prob then.
thanks @jeet-parekh
CLN remove unnecessary trailing commas in groupby tests
State: closed

xref #35925


Labels:
name: Clean

Comments:
Hello @xcz011! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:

















There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2020-09-02 12:13:40 UTC
Thanks @xcz011
Comma cleanup for #35925
State: closed

- [x] pandas/tests/generic/test_finalize.py
- [x] pandas/tests/generic/test_to_xarray.py
- [x] pandas/tests/groupby/aggregate/test_numba.py
- [x] pandas/tests/groupby/test_apply.py
- [x] pandas/tests/groupby/test_categorical.py
- [x] pandas/tests/groupby/test_groupby.py
- [x] pandas/tests/groupby/test_groupby_dropna.py
- [x] pandas/tests/groupby/test_groupby_subclass.py


Labels:
name: Code Style
description: Code style, linting, code_checks
name: Clean

Comments:
Thanks @JonathanShrek
CLN remove unnecessary trailing commas
State: closed

#35925

- pandas/tests/arithmetic/test_interval.py
- pandas/tests/arithmetic/test_numeric.py
- pandas/tests/arrays/boolean/test_logical.py

I am not including "pandas/io/sas/sas_xport.py" and "pandas/io/stata.py" as there is a PR already open for those.

Labels:
name: Clean

Comments:
Thanks @tiagohonorato !
STY/WIP: check for private imports/lookups
State: closed

Mostly this is an amalgam of #33479, #33394, and #33393.

Labels:
name: Code Style
description: Code style, linting, code_checks

Comments:
looks good.
We're down to 4 things that need to either be renamed or add to the ignore list: _GroupBy, _Rolling, _Rolling_and_Expanding, _IntegerDtype.

@mroeschke any preferences for the Rolling names?
No preference for those.
green, booyah
very nice! (hopefully recent merges don't break on this :->)
> hopefully recent merges don't break on this

`make check` passes on master
BUG: Don't raise when constructing Series from ordered set
State: closed

- [x] closes #36044
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry

This may or may not be a regression, since before the change that caused the error to be raised the output was still wrong (same bug that motivated the initial patch):

```python
In [2]: keys = {'a': 1, 'b': 2}.keys()

In [3]: pd.Series(keys)
Out[3]: 
0    (a, b)
1    (a, b)
dtype: object
```

Labels:
name: Bug
name: Constructors
description: Series/DataFrame/Index/pd.array Constructors

Comments:
@dsaxton merge conflicts (we should get a Github Action to notify us when this occurs)
@meeseeksdev backport 1.1.x
CLN: _wrap_applied_output
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry

2nd step toward #35412. In this, the order of the largest if-else is switched with the condition negated. We can then drop the else entirely, resulting in much of the function having one less level of nesting.

Labels:
name: Groupby
name: Clean
name: Apply
description: Apply, Aggregate, Transform, Map

Comments:
Hello @rhshadrach! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:



There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2020-09-01 22:06:57 UTC
nice!
CLN remove unnecessary trailing commas in pandas/io
State: closed

@MarcoGorelli can you review this PR this is related to issue #35925, if this looks good to you I can open another PR
- [x] pandas/io/sas/sas_xport.py
- [x] pandas/io/stata.py
  
- [x] passes `black pandas`



Labels:
name: Code Style
description: Code style, linting, code_checks
name: Clean

Comments:
Hello @sarthakchaudhary13! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:





There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2020-09-03 11:12:51 UTC
Thanks for reviewing this PR @dsaxton, this is the output after re-running black on pandas/io.
```
$ black pandas/io
All done! ✨ 🍰 ✨ 
47 files left unchanged. 
```
> Thanks for reviewing this PR @dsaxton, this is the output after re-running black on pandas/io.
> 
> ```
> $ black pandas/io
> All done! ✨ 🍰 ✨ 
> 47 files left unchanged. 
> ```

Can you check this with `black` version 19.10b0 as well, as that's the version that's run during CI? Ideally, you would:

- upgrade `black`
- run `black` on some files
- check what's changed, and try removing the trailing commas
- run `black` again
- downgrade `black` to version 19.10b0 and check everything is still fine

> Can you check this with `black` version 19.10b0 as well, as that's the version that's run during CI? Ideally, you would:
> 
>     * upgrade `black`
> 
>     * run `black` on some files
> 
>     * check what's changed, and try removing the trailing commas
> 
>     * run `black` again
> 
>     * downgrade `black` to version 19.10b0 and check everything is still fine

@MarcoGorelli Thanks for the tip, both black versions checks are passing
I did a git revert for my previous commit but somehow other files also got changed, can you help me to fix this 😞 
@sarthakchaudhary13 I think I would do something like
```
git fetch upstream master
git rebase -i upstream/master  # choose which commits you want to keep
git push origin <your branch name> -f
```
> @sarthakchaudhary13 I think I would do something like
> 
> ```
> git fetch upstream master
> git rebase -i upstream/master  # choose which commits you want to keep
> git push origin <your branch name> -f
> ```

@MarcoGorelli Thanks, if this looks good to you I can open another PR with fixing more files 😄 
@sarthakchaudhary13 looks like there's a merge conflict, could you fetch and merge upstream/master and then push again please?
> @sarthakchaudhary13 looks like there's a merge conflict, could you fetch and merge upstream/master and then push again please?

Yeah sure.
Thanks @sarthakchaudhary13
BUG: frame._item_cache not cleared when Series is altered
State: closed

- [x] closes #33675
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Bug
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves

Comments:
lgtm. ok for 1.1.2 i think. any perf implications?
> any perf implications?

I dont think so.  It's just a dict.pop, and any case when its hit is one where we're doing the wrong thing now.
thanks @jbrockmendel 
@meeseeksdev backport 1.1.x
BUG: incorrect year returned in isocalendar for certain dates
State: closed

- [x] closes #36032
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry


Labels:
name: Bug
name: Datetime
description: Datetime data dtype

Comments:
I'm not sure if a new test is needed here. I added the failing case to parametrize.
cc @jbrockmendel 
LGTM
thanks @asishm
CI: pin setuptools on 1.1.x
State: closed

top answer on stack overflow is to pin https://stackoverflow.com/questions/63663362/django-python3-on-install-i-get-parent-module-setuptools-not-loaded

also see https://github.com/MacPython/pandas-wheels/pull/97#issuecomment-684938715

Note: 50.0.1 released a hour ago but still the same. https://dev.azure.com/pandas-dev/pandas/_build/results?buildId=41325&view=logs&j=a67b4c4c-cd2e-5e3c-a361-de73ac9c05f9&t=9a6bfc0f-544f-57f9-291f-bf4b75b05642

Labels:
name: CI
description: Continuous Integration

Comments:
sure
REF: simplify CSVFormatter
State: closed

- [ ] closes #xxxx
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry

Refactor CSVFormatter
----------------------
1. Put data validation in setters
2. Extract helper methods and properties


Labels:
name: Refactor
description: Internal refactoring of code
name: IO CSV
description: read_csv, to_csv

Comments:
@ivanovmg: Welcome to `pandas` !  I like the general direction of this PR.

If you can address all of @jreback's comments, I think we should be good.
I addressed the issues raised.

* 46f593d87 (HEAD -> refactor/csvs, origin/refactor/csvs) REF: refactor number formatting
* 6041666d0 TYP: add type annotations as reviewed
* 3da7207bb REF: rename regular -> flat as reviewed

For some reason the changes are not yet reflected here in PR page.
@jreback, is there anything else to be done on this PR? Or good to go?
thanks @ivanovmg very nice
BUG: NDFrame.replace wrong exception type, wrong return when size==0
State: closed

- [ ] closes #xxxx
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry


Labels:
name: Error Reporting
description: Incorrect or improved errors from pandas

Comments:
ping
DOC: add semicolons to suppress text repr of matplotlib objects in visualization.rst
State: closed

add semicolons (";") after the plotting function to suppress unnecessary debug messages similar to:

<matplotlib.axes._subplots.AxesSubplot at 0x7fe278bb5160>

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Docs
name: Visualization
description: plotting
name: Stale

Comments:
Is there a more generic way to do this? I think this is distracting for end users

@jorisvandenbossche 
> I think this is distracting for end users

FWIW, I've seen this quite frequently in tutorials
We indeed already do it in several places in visualization.rst itself (see https://pandas.pydata.org/docs/dev/user_guide/visualization.html), so this is making it more consistent.
This pull request is stale because it has been open for thirty days with no activity. Please update or respond to this comment if you're still interested in working on this.
@partev can you merge master and we can get this in
@jreback done
thanks @partev 

@jorisvandenbossche has a comment from above if you can have a look
BUG: 35977 Adding regex support for ExtensionBlock replace method
State: closed

- [x] closes #35977 
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry

Overrides replace method to add support of regex in ExtensionBlock . If input to_replace is a regex we convert data type of block to Object and calls object replace method which supports regex otherwise we call replace method of super class Block.

```ipython
In [2]: b.replace(r'^\s*$', pd.NA, regex=True, inplace=True)
   ...: print(b)
   a     b
0  a     d
1  b  <NA>
2  c  <NA>

In [3]: b.replace(r'^\s*$', pd.NA, regex=True, inplace=False)
Out[3]: 
   a     b
0  a     d
1  b  <NA>
2  c  <NA>


```

Labels:
name: Internals
description: Related to non-user accessible pandas implementation
name: ExtensionArray
description: Extending pandas with custom dtypes or arrays.
name: replace
description: replace method
name: Stale

Comments:
pls merge master as we just merged a change in how replace works in internals.
@jreback synced with upstream master also verified that this bug is still present in upstream master
@cgangwar11

Conflicting files
pandas/tests/arrays/categorical/test_replace.py
Also will need to merge master
Hello @cgangwar11! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:





There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2020-11-15 07:14:13 UTC
This pull request is stale because it has been open for thirty days with no activity. Please update or respond to this comment if you're still interested in working on this.
closing as stale. if you want to continue, pls ping and can re-open.
TYP: Postponed Evaluation of Annotations (PEP 563)
State: closed

a few files as POC/for discussion

from https://www.python.org/dev/peps/pep-0563/

> PEP 3107 added support for arbitrary annotations on parts of a function definition. Just like default values, annotations are evaluated at function definition time. This creates a number of issues for the type hinting use case:

> - forward references: when a type hint contains names that have not been defined yet, that definition needs to be expressed as a string literal;
> - type hints are executed at module import time, which is not computationally free.

> Postponing the evaluation of annotations solves both problems.

Labels:
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
+/- 0 on doing this now versus waiting until its fully implemented in the standard library. Does this have a noticeable impact on import performance or is it stylistic for where we are at now?
I've not tested performance, presumably the import time gets a little longer with every type annotation added.

from a style POV. we can get rid of some noqa: F401 and get better formatting with black (although that's only really the case when PRs with whole typing expressions that sometimes get merged)

for a contributor POV, there are less things to get right if don't need to ensure types are enclosed in quotes see https://github.com/pandas-dev/pandas/pull/36022#issuecomment-684952618

from a reviewer POV, it's nice to just show the type in the IDE which can't be done with string literals (VS code anyway)

> +/- 0 on doing this now versus waiting until its fully implemented in the standard library.

That'll be Python 3.10
I would be +1 on this.
thanks @simonjayhawkins