BrandonFors/pandas-github-issues huggingface dataset:
REF: share astype code in MaskedArray
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry

cc @jorisvandenbossche i dont know if this is the optimal way to do this, but it seems like there is a lot of share-ability here.

Labels:
name: Refactor
description: Internal refactoring of code
name: NA - MaskedArrays
description: Related to pd.NA and nullable extension arrays

Comments:
> Thanks! Yes, this is certainly one of the methods on the list of "to share" for masked arrays

also pls link to the issue & check the box
updated per request + green
REF: share .astype code for astype_nansafe + TDA.astype
State: closed

- [ ] closes #xxxx
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry

This changes TDI.astype and TDA.astype in the cast with no-NAs to return float64 instead of int64, matching Series behavior.

Labels:
name: Refactor
description: Internal refactoring of code
name: Datetime
description: Datetime data dtype
name: Timedelta
description: Timedelta data type

Comments:
Is this "pandas convention" documented anywhere?
> Is this "pandas convention" documented anywhere?

Not that I'm aware of.  I'm not a fan of this behavior, looking forward to making it unnecessary when we support non-nano td64
thanks @jbrockmendel yep nice refactor here.
>  I'm not a fan of this behavior, looking forward to making it unnecessary when we support non-nano td64

We already might want to start deprecating this in advance.
CI: Supress moto server logs in tests
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry

https://github.com/pandas-dev/pandas/pull/38096 originally did not do the trick. This change worked for me locally.

Labels:
name: CI
description: Continuous Integration
name: Clean

Comments:
cool ping on green (let's verify that it does the trick as well :->)
thanks @mroeschke 
@meeseeksdev backport 1.2.x
Something went wrong ... Please have  a look at my logs.
ENH: skip_blank_columns in read_csv
State: closed

- [ ] closes #34231
- [ ] adds skip_blank_columns functionality
- [ ] updates API documentation


Labels:
name: Enhancement
name: IO CSV
description: read_csv, to_csv
name: Needs Discussion
description: Requires discussion from core team before further action

Comments:
Is this any different than calling `.dropna` on the frame after parsing? We have a *ton* of keywords as is in read_csv and tend not to expand that further
Thanks for the PR, but it appears that there is not much interest this particular implementation of this feature. I think more discussion should live in the original issue, https://github.com/pandas-dev/pandas/issues/34231, before moving forward with a PR. Closing.
BENCH: Fix CategoricalIndexing benchmark
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry

This benchmark behavior changed after https://github.com/pandas-dev/pandas/pull/38372

Additionally, exploring if its feasible to always run the benchmarks to catch these changes earlier.

Labels:
name: Benchmark
description: Performance (ASV) benchmarks

Comments:
thanks @mroeschke
DOC: Add doc-string examples for pd.read_sql using custom parse_dates arg values
State: closed

follow on PR  for  #37823

- [x] closes #xxxx
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry


Labels:
name: Docs
name: IO SQL
description: to_sql, read_sql, read_sql_query

Comments:
Added doctests pass. 

CI fails due to other reasons, since I only added documentation. Will merge master later to run CI agian.
very nice @avinashpancham
BUG: require arraylike in infer_dtype_from_array
State: closed

- [ ] closes #xxxx
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Dtype Conversions
description: Unexpected or buggy dtype conversions

Comments:
hmm 32-bit error legit?
> hmm 32-bit error legit?

I think so, still troubleshooting it (and similar in #38505).  in the meantime ill shamelessly direct your attention to #38504, #34683, #37965, #37950, #38021, #38073, #38285, #36131, #35417 (the latter two being harder than the rest)
> > hmm 32-bit error legit?
> 
> I think so, still troubleshooting it (and similar in #38505). in the meantime ill shamelessly direct your attention to #38504, #34683, #37965, #37950, #38021, #38073, #38285, #36131, #35417 (the latter two being harder than the rest)

looking now
BUG: disallow scalar in Categorical constructor
State: closed

- [x] closes #38433
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry


Labels:
name: Categorical
description: Categorical Data Type
name: Deprecate
description: Functionality to remove in pandas
name: Constructors
description: Series/DataFrame/Index/pd.array Constructors

Comments:
We could also easily deprecate this first. I don't know if others would rely on it, but at least we did ourselves (the behaviour was explicitly tested)
indifferent here. am ok pushing this thru or deprecating. 
doesn't this duplicate #38614 (or vice versa)
> doesn't this duplicate #38614 (or vice versa)

no.  one is for Categorical, the other for CategoricalIndex
> > doesn't this duplicate #38614 (or vice versa)
> 
> no. one is for Categorical, the other for CategoricalIndex

ok.

should we deprecate this? i agree its very weird, but i suppose it might happen in the real world.
Given that we had tests for it, and that it is actually also allowed for Series, I would go for deprecation first (it shouldn't be hard I think?)
DOC: fixes for assert_frame_equal check_freq argument
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry

This is just a minor documentation change to fix the `versionadded` tag on the `check_freq` argument to `assert_frame_equal` and also to add a description of this backwards incompatible chage to the 1.1.0 Whats New (as recommended in https://github.com/pandas-dev/pandas/issues/35570#issuecomment-674064533).

Labels:
name: Docs

Comments:
CI failures look unrelated?
thanks @kylekeppler 
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
$ git checkout 1.2.x
$ git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
$ git cherry-pick -m1 89b3d6b201b5d429a202b5239054d5a70c8b5071
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
$ git commit -am 'Backport PR #38471: DOC: fixes for assert_frame_equal check_freq argument'
```

4. Push to a named branch :

```
git push YOURFORK 1.2.x:auto-backport-of-pr-38471-on-1.2.x
```

5. Create a PR against branch 1.2.x, I would have named this PR:

> "Backport PR #38471 on branch 1.2.x"

And apply the correct labels and milestones.

Congratulation you did some good work ! Hopefully your backport PR will be tested by the continuous integration and merged soon!

If these instruction are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
TYP : DataFrame.(merge, join)  core.reshape.merge.(merge, ...) (easy)
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
looks good. @simonjayhawkins if you can double check here.
thanks @arw2019
Update 10min.rst
State: closed

Added Links for Matplotlib Close along with a line explaining what it does.
I had faced difficulty when I read it for the first time and had to google search what close does

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Docs

Comments:
Hey! 
Can someone explain what went wrong. This is the first time I used Sphinx directly but it seems to fail and I can't understand why
@jbrockmendel 
Hey
I saw your contributions and commits on the top page 
I want some help here can you tell me what's going wrong and causing all this fail?
Near the bottom of the page there is a red x for the "pre-commit" CI check.  click on the "details" link to see the logs for the failed check.  There is a line that the code-check wants to see changed, I think it just wants some whitespace removed.
Thank You @jbrockmendel 
@jbrockmendel 
Hey I'd like to apologize in case my message was aggressive to say the least. Someone pointed it out to me and I am just starting but that is no excuse so I will try my best to keep this in mind in the future.
Thank You for all the help though
Looks like the code check passed this time around.  Good job.  Someone more familiar with the docs will come along to review this before too long.
thanks @aflah02 
@jreback You're Welcome
TYP: DataFrame.to_gbq, DataFrame.to_html (easy: copy-paste from format module)
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry

Spinning off from #38416

Labels:
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
thanks @arw2019
REF: simplify _sanitize_column
State: closed

It gets called indirectly from `iloc._setitem_with_indexer`, which we're trying to simplify

Labels:
name: Refactor
description: Internal refactoring of code
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves

Comments:
thanks, yeah we have a lot of built up code over the years. +1 on organizaing / simplifying.
STYLE: use types_or in pre-commit
State: closed

- [x] closes #38022 
- [ ] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Code Style
description: Code style, linting, code_checks

Comments:
Thanks, @MarcoGorelli for correcting my mistakes. I have fixed the PR. Can you please review it again? 
Also, please update the minimum pre-commit version in `environment.yml` and in `requirements-dev.txt:
```
  - pre-commit>=2.9.2
```
Thank you so much for correcting my mistakes again. I am in process of understanding the whole pandas library. Thank you for your help :) 
I have fixed it. Can you please review it again? 
thanks @rkc007
REF: Separate values-casting from Block.astype
State: closed

This separates Block._astype from Block.astype.  Block._astype only requires Block.values, the idea being that we can roll it into astype_nansafe before long.

Labels:
name: Dtype Conversions
description: Unexpected or buggy dtype conversions
name: Internals
description: Related to non-user accessible pandas implementation

Comments:
great, yeah this is a PITA internally. ping on green.
ping
ENH: Improve error message when names and usecols do not match for read_csv with engine=c
State: closed

- [x] closes #29042
- [x] tests added / passed This case was already covered with differing error messages. I adjusted the message accordingly
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry


Labels:
name: Error Reporting
description: Incorrect or improved errors from pandas
name: IO CSV
description: read_csv, to_csv

Comments:
Hello @phofl! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:



There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2020-12-13 16:22:49 UTC
Revert "BUG: first("1M") returning two months when first day is last day of month"
State: closed

Reverts pandas-dev/pandas#38331

Labels:
name: Bug
name: Frequency
description: DateOffsets

Comments:
known test failure

```
=========================== short test summary info ============================
FAILED pandas/tests/io/test_html.py::TestReadHtml::test_banklist_url_positional_match[bs4]
= 1 failed, 146315 passed, 3162 skipped, 1095 xfailed, 66 warnings in 2648.26s (0:44:08) =
```
BUG: first("2M") returning incorrect results
State: closed

- [x] closes #29623
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
 
This would fix #29623 for offsets with n > 1
cc @simonjayhawkins 

Labels:
name: Bug
name: Frequency
description: DateOffsets

Comments:
Thanks @phofl for the quick response.

I'm not sure where this leaves us w.r.t. backports. whether we should backport #38331 (i.e. reopen #38436) and then backport this as well.

or let these changes sit in master for a while (and move release note added in #38331 to 1.3 whatsnew)
If we decide to backport, we could cherry pick this one on top of the other Commit. But I am not quite sure either if we should backport.
we can just revert the original and then merge to master 
Oh right, that would work. Should I open a Revert PR?
#38448
Ok, thx. Will adjust this one after the other is merged
after the revert is merged, will reopen issue and change milestone. so can update the OP here to close issue again.
Done. This fixes both cases now.
thanks @phofl
BUG: read_csv usecols and names parameters inconsistent between c and python
State: closed

- [x] closes #16469
- [x] tests added / passed The tests added by @gfyoung which were marked as xfail are passing now.
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry


Labels:
name: Bug
name: IO CSV
description: read_csv, to_csv

Comments:
thanks @phofl 
Thx, opened #38453 as a follow up. Could not figure this out yesterday for the python part.
CI: add slash dispatch workflow to trigger pre-commit checks with comment
State: closed

This would be an on-demand bot to run pre-commit checks on a PR, which can be triggered by commenting

```
@github-actions pre-commit
```

on a pull request (see [here](https://github.com/nbQA-dev/nbQA/pull/518) for a demo).

Use case: if a PR is submitted and is good-to-go except for some linting error, we can just comment `/pre-commit-run` and have the bot fixup the errors.

Labels:
name: CI
description: Continuous Integration
name: Code Style
description: Code style, linting, code_checks

Comments:
Cool, thanks! 
That actually reminded me that we use something similar in the arrow repo, using comments to trigger actions: https://github.com/apache/arrow/blob/master/.github/workflows/comment_bot.yml. This seems to check the content of the comment, instead of using one workflow (with slash-command-dispatch) to trigger another, which seems a bit simpler to me?
Thanks @jorisvandenbossche ! I didn't know you could do that, that indeed does look simpler, I'll see if I can update this to do it that way
@jorisvandenbossche OK, done

If the comment-bot is now largely taken from `arrow`, my understanding is that we need to include their license at the top of the file - is this right?
> If the comment-bot is now largely taken from arrow, my understanding is that we need to include their license at the top of the file - is this right?

I don't think that that is necesaarily needed in this case. I mean, I contribute to several projects, and I am constantly applying what I tried / learned how to set up github actions in one project to other projects. Also, if you include the license, you would need to clearly state what has changed. And in the end, the main thing you copied is the line `if: startsWith(github.event.comment.body, '@github-actions autotune')` and the idea to use the pr-push action, for the rest it's a lot different compared to the original file from arrow.
What are your thoughts on https://github.com/pandas-dev/pandas/pull/38444#issuecomment-746988296 ?
Hey Joris - sorry, I missed that comment, somehow my brain only registered the review comments - I've removed the license header as I think your assessment makes perfect sense (as someone with far less experience, I was trying to tread carefully)
this looks fine to me. let's give it a try.
@MarcoGorelli can you add somewhere the various actions that are availabel (take is in the user docs, but this one plus the backport ones could go somewhere)
Thanks!
BUG: Fix Index.__repr__ when `display.max_seq_items` = 1
State: closed

- [x] closes #38415 
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry


Labels:
name: Bug
name: Output-Formatting
description: __repr__ of pandas objects, to_string

Comments:
Thanks @skvrahul for the PR. please add a test.
@jreback @simonjayhawkins 
Could you confirm that this parameter is expected to affect even the list of names of columns in the index
I.e. `repr` evaluates to:
```
MultiIndex([...
            ('qux', 'two')],
           names=['first', ...], length=6)
```
and not
```
MultiIndex([...
            ('qux', 'two')],
           names=['first', 'second'], length=6)
```
> Could you confirm that this parameter is expected to affect even the list of names of columns in the index
> I.e. `repr` evaluates to:
> 
> ```
> MultiIndex([...
>             ('qux', 'two')],
>            names=['first', ...], length=6)
> ```
> 
> and not
> 
> ```
> MultiIndex([...
>             ('qux', 'two')],
>            names=['first', 'second'], length=6)
> ```

Checked on `master` and can confirm this (parameter affects even the list of column names)
Added tests accordingly.

@jreback Added tests and moved one of the tests to its own TC. 
thanks @skvrahul very nice
BENCH/REF: parametrize CSV benchmarks on engine
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry

The diff in basic PR implementing the `pyarrow`-based CSV engine (#38370) is quite big. A part of that PR is a small refactor of the CSV I/O benchmarks such that they take `engine` as a parameter. Most of that refactor is not dependent on having the pyarrow engine so I'm spinning it off here to de-bloat #38370.

Labels:
name: IO CSV
description: read_csv, to_csv
name: Benchmark
description: Performance (ASV) benchmarks

Comments:
looks fine, checks are failing though, ping on green.

cc @gfyoung @WillAyd if comments.
Atm not sure what's going on with the benchmark failure  - the failure is in `asv_bench/benchmarks/tslibs/timedelta.py` which doesn't get touched here
https://github.com/pandas-dev/pandas/pull/38476 should fix the unrelated benchmark failure. 
> #38476 should fix the unrelated benchmark failure.

Thanks @mroeschke!!
> Is there some logic in which classes are parameterized and which not?

Not that I know of beyond (AFAIK) that the python engine supports all these whereas the c/pyarrow engines only support a proper subset. 

I'll do a pass through the entire file and parametrize where both c and python engines are supported.
CI green
thanks @arw2019
TYP: pandas/core/frame.py (easy: Axis/Level)
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry

Splitting this away from #38416

Labels:
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
looks fine. merge master and ping on green.
Rebased + green
thanks
BUG: add sorting_reamaining=True to avoid IndexError GH25831
State: closed

- [x] closes #25831
- [x] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Bug
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode
name: MultiIndex
name: Stale

Comments:
Sure!
Just added!
Hello @YuRenee! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:











* In the file [`test_issue1.py`](https://github.com/pandas-dev/pandas/blob/86928636178c14c6411d968b03324d663eedf3ee/test_issue1.py):

> [Line 5:14](https://github.com/pandas-dev/pandas/blob/86928636178c14c6411d968b03324d663eedf3ee/test_issue1.py#L5): [E231](https://duckduckgo.com/?q=pep8%20E231) missing whitespace after ','
> [Line 15:1](https://github.com/pandas-dev/pandas/blob/86928636178c14c6411d968b03324d663eedf3ee/test_issue1.py#L15): [E265](https://duckduckgo.com/?q=pep8%20E265) block comment should start with '# '
> [Line 16:1](https://github.com/pandas-dev/pandas/blob/86928636178c14c6411d968b03324d663eedf3ee/test_issue1.py#L16): [E265](https://duckduckgo.com/?q=pep8%20E265) block comment should start with '# '
> [Line 17:60](https://github.com/pandas-dev/pandas/blob/86928636178c14c6411d968b03324d663eedf3ee/test_issue1.py#L17): [W292](https://duckduckgo.com/?q=pep8%20W292) no newline at end of file



##### Comment last updated at 2020-12-14 17:38:40 UTC
can you merge master
can you merge master and update
This pull request is stale because it has been open for thirty days with no activity. Please update or respond to this comment if you're still interested in working on this.
closing as stale. if you want to continue, pls ping and can re-open.
Backport PR #38331 on branch 1.2.x (BUG: first("1M") returning two months when first day is last day of month)
State: closed

Backport PR #38331: BUG: first("1M") returning two months when first day is last day of month

Labels:
name: Bug
name: Frequency
description: DateOffsets

Comments:
closing pending discussion in https://github.com/pandas-dev/pandas/pull/38331#issuecomment-744002535
Backport PR #38247 on branch 1.2.x (BUG: Limited available colors)
State: closed

Backport PR #38247: BUG: Limited available colors

Labels:
name: Visualization
description: plotting

Comments:
This is an enhancement, we shouldn't start backporting such PRs IMO
> This is an enhancement, we shouldn't start backporting such PRs IMO

this is to 1.2 and actually was ready way before the rc. i don't view this as a big deal.

and now would require even more work to revert the original and change the note.
Yes, sure, for this specific PR it would be more work to undo it, but so I was speaking in general for other PRs
agreed

only regressions for 1.2  at this point