BrandonFors/pandas-github-issues huggingface dataset:
BUG: Index([date]).astype("category").astype(object) roundtrip
State: closed

- [ ] closes #xxxx
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry

This also makes some MultiIndex constructor behavior consistent with Index behavior, with the side-effect of fixing the thorny cases in #36131

Labels:
name: Datetime
description: Datetime data dtype
name: MultiIndex

Comments:
Fixes #28152, though doesnt yet have a test for that
looks good. i guess this is technically a bug fix in 1.3? hard to tell.
>  guess this is technically a bug fix in 1.3?

thats a fair description.  ill add a note
BUG: Support timespec argument in Timestamp.isoformat() (#26131)
State: closed

- [x] closes #26131
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry

I have just tried to support the same behaviour as `datetime.datetime()`, without adding the additional `"nanoseconds"` option, but this could be handled as well.

Labels:
name: Enhancement
name: Datetime
description: Datetime data dtype
name: Stale

Comments:
This pull request is stale because it has been open for thirty days with no activity. Please update or respond to this comment if you're still interested in working on this.
Thanks for the PR, but going to close due the lack of activity. Let us know if you'd like to continue working on this. 
I'm planning to take over this PR as the original author seems to have lost time/interest.
ENH: Map pandas integer to optimal SQLAlchemy integer type (GH35076)
State: closed

- [x] closes #35076
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry


Labels:
name: IO SQL
description: to_sql, read_sql, read_sql_query

Comments:
Note for future reference: On windows numpy 1.16 `dtype=int` gives a 32 bit int, in later versions this changed to 64 bit, which is line with unix. See also https://stackoverflow.com/a/36279549/14089081
Mind fixing the merge conflict @avinashpancham? 
Sure, updated it.
@mroeschke CI is greenish, only Travis gives an error due to other reasons.
Thanks @avinashpancham
patch wsl compatibility
State: closed

Change wsl check to something more universal and consistent

- [x] closes #38527
- [ ] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: IO Data
description: IO issues that don't fit into a more specific label
name: Windows
description: Windows OS

Comments:
> Thanks. I am not sure this can reliably be tested, but confirmed this works on your end?

I see, and yes, it's working for me.
> > Thanks. I am not sure this can reliably be tested, but confirmed this works on your end?
> 
> I see, and yes, it's working for me.

works for me too.

@jreback I've rebased the original commit and added a line in whatsnew. Let me know if you need anything else from my end.
thanks @mbkupfer
DEPR: datetimelike.astype(int)
State: closed

- [x] closes #24381
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry

Sits on top of #38535

Labels:
name: Dtype Conversions
description: Unexpected or buggy dtype conversions
name: Deprecate
description: Functionality to remove in pandas

Comments:
can you merge master
do we have sufficient `.view` tests on *Index* ?
> do we have sufficient .view tests on Index ?

looks decent, yah
I ran into this case of datetime->integer casting while exploring our casting rules (https://github.com/pandas-dev/pandas/issues/22384). @jbrockmendel do you remember the rationale for deprecating this casting? It's not really clear to me why we would in the future disallow datetimelike->integer casting.

This PR doesn't seem to give much reasoning, and the referenced issue is about the bitwidth / sign being ignored, but not sure deprecating altogether is necessarily the fix for that (the bitwidth being ignored *also* applies for integer->integer casting, which we didn't deprecate).
https://github.com/pandas-dev/pandas/issues/22384#issuecomment-909660696

>> We currently raise a warning about the dt64 -> int64 that this will raise instead in the future and to use view instead. Do you know what the rationale was for this deprecation?

> The idea is that 'astype' in general means something like 'the same information but in a different format' and that doesn't make sense with i8 <-> M8


Whoops, sorry, I totally forgot I already asked / you answered in the other thread .. :) Will continue there
testing case sort_index with pd.set_option('use_inf_as_na', True)
State: closed

- [ ] closes #29687
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry

Create test cases for sort_index works with 'pd.set_option('mode.use_inf_as_na',True)


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Stale

Comments:
Hello @weikhor! Thanks for opening this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:

* In the file [`pandas/tests/frame/methods/test_sort_index.py`](https://github.com/pandas-dev/pandas/blob/e1c5a881d12275ea55c90a46a6d2a4328de9bf0d/pandas/tests/frame/methods/test_sort_index.py):

> [Line 860:1](https://github.com/pandas-dev/pandas/blob/e1c5a881d12275ea55c90a46a6d2a4328de9bf0d/pandas/tests/frame/methods/test_sort_index.py#L860): [W293](https://duckduckgo.com/?q=pep8%20W293) blank line contains whitespace
> [Line 866:1](https://github.com/pandas-dev/pandas/blob/e1c5a881d12275ea55c90a46a6d2a4328de9bf0d/pandas/tests/frame/methods/test_sort_index.py#L866): [W293](https://duckduckgo.com/?q=pep8%20W293) blank line contains whitespace
> [Line 879:1](https://github.com/pandas-dev/pandas/blob/e1c5a881d12275ea55c90a46a6d2a4328de9bf0d/pandas/tests/frame/methods/test_sort_index.py#L879): [W391](https://duckduckgo.com/?q=pep8%20W391) blank line at end of file


@weikhor can you merge master and update to comments
This pull request is stale because it has been open for thirty days with no activity. Please update or respond to this comment if you're still interested in working on this.
closing as stale. if you want to continue, pls ping and can re-open.
Tests .loc on sparse DataFrame #34687
State: closed

- [ ✔️] closes #34687
- [ ✔️] tests added / passed

Created test case for ttps://github.com/pandas-dev/pandas/issues/34687 

Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Dtype Conversions
description: Unexpected or buggy dtype conversions
name: Sparse
description: Sparse Data Type
name: Stale

Comments:
Hello @weikhor! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:



There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2020-12-20 14:21:18 UTC
@arw2019 

I have tried my test cases on my side. It work perfectly. I not sure why automatic test case still failed. 
> @arw2019
> 
> I have tried my test cases on my side. It work perfectly. I not sure why automatic test case still failed.

If you look at the output it's because scipy is not being imported correctly. Take a look at what we do in other similar tests to get this to green (we won't merge if the checks don't pass)
you also need to format the code (look at output from pre-commit hook)
Hi @arw2019, I failed on AssertionError: ExtensionArray NA mask are different https://dev.azure.com/pandas-dev/pandas/_build/results?buildId=50437&view=logs&j=2d7fb38a-2053-50f3-a67c-09f6e91d3121&t=449937cc-3d50-56b5-5662-e489f41f1268&l=125  and pre-commit / pre-commit (pull_request). Please assist. 
> Hi @arw2019, I failed on AssertionError: ExtensionArray NA mask are different https://dev.azure.com/pandas-dev/pandas/_build/results?buildId=50437&view=logs&j=2d7fb38a-2053-50f3-a67c-09f6e91d3121&t=449937cc-3d50-56b5-5662-e489f41f1268&l=125 and pre-commit / pre-commit (pull_request). Please assist.

Well they *are* different:
``` python
In [18]: res2 = df.loc[range(2)].loc[range(1)]
    ...: exp2 = pd.DataFrame(
    ...:             [[1.0, 0.0, np.nan, np.nan, np.nan]], dtype=pd.SparseDtype("float64", 0.0)
    ...:         )

In [19]: res2
Out[19]: 
     0    1    2    3    4
0  1.0  0.0  0.0  0.0  0.0

In [20]: exp2
Out[20]: 
     0    1   2   3   4
0  1.0  0.0 NaN NaN NaN
```
so you need to change the expected output.

Re: pre-commit, the easiest thing is to run pre-commit locally and let the hooks do the modifications
@arw2019 

All checks have passed. Thank
@arw2019 @jreback 
Please review and approve the pull request. Thank
can you merge master and update
This pull request is stale because it has been open for thirty days with no activity. Please update or respond to this comment if you're still interested in working on this.
closing as stale. if you want to continue, pls ping and can re-open.
> closing as stale. if you want to continue, pls ping and can re-open.

I was thinking of taking over this pull request, since it looks like it is almost finished. What would be the best way of doing that, with this living on the fork of weikhor? Just copy the changes over to a new pr?
sure you can cherry pick off of the other branch is prob best
BUG: fix array conversion from Arrow for slided array
State: closed

Closes #38525

Labels:
name: Bug
name: NA - MaskedArrays
description: Related to pd.NA and nullable extension arrays

Comments:
lgtm, needs a whatsnew note bug fix (ok for 1.2 or 1.3)
Yeah, I didn't yet add a whatsnew because we need to decide which version to target. Normally, plain bug fixes should go in 1.3 right now, but since this is actually causing a silent data corruption, it might be worth putting in 1.2
1.2 sounds good to me
thanks @jorisvandenbossche 
@meeseeksdev backport 1.2.x
TYP: pandas/io/sql.py (easy: bool/str) 
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
thanks @arw2019
Added Documentation to specify that DataFrame.last() needs the index to be sorted to deliver the expected results
State: closed

Added Documentation mentioning that DataFrame.last() needs the index to be sorted to deliver the expected results

Haven't yet worked on raising an error will work as advised

- [ ] closes #38000
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Docs

Comments:
Hello @aflah02! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:



There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2020-12-17 08:21:54 UTC
Looks good, thanks @aflah02 (and thanks @arw2019 for reviewing!)
Thanks @MarcoGorelli
BUG&TST: HTML formatting error in Styler.render() in rowspan attribute
State: closed

- [x] closes [#38234](https://github.com/pandas-dev/pandas/issues/38234)
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry


Labels:
name: Bug
name: Styler
description: conditional formatting using DataFrame.style

Comments:
Hello @JoseNavy! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:





There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2020-12-22 18:13:36 UTC
lgtm ping on greenish
The Travis check didn't pass and after checking why, I don't see a clear error on what went wrong. All the other checks did pass, can you help me with this? @jreback  
thanks @JoseNavy the travis CI is acting up, unrelated
@meeseeksdev backport 1.2.x
BUG: Regression in logical ops raising ValueError with Categorical columns with unused categories
State: closed

- [x] closes #38367
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry

@simonjayhawkins 

This would fix the issue, but is pretty dirty. If we decide to merge this, we should remove this again with #38140 Also test would have to be adjusted (expected remove unused category)

Labels:
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: Regression
description: Functionality that used to work in a prior pandas version
name: Categorical
description: Categorical Data Type

Comments:
thanks @phofl 
@meeseeksdev backport 1.2.x
Editing Pandas Documentation homepage
State: closed

Trying to improve the user experience and interaction with the webpage.

1.Edited button hover effect.
2. Make links open in new tabs instead of redirect in the current webpage because this may be not good for some people.
3. Added data manipulation note in the introduction section.

Labels:
name: Docs
name: Stale

Comments:
Just removing the errors in the first 2 commits.
so this is just a change to hover (1) right?
cc @datapythonista 
> so this is just a change to hover (1) right?
yes

Thanks @MOHED1224. I think it's fine here, but in general we prefer to not mix PRs with unrelated changes, even if they are small.

@jorisvandenbossche is this hover effect something that should be implemented here, or in the original sphinx theme?
This pull request is stale because it has been open for thirty days with no activity. Please update or respond to this comment if you're still interested in working on this.
Sorry for the slow reply here

>  is this hover effect something that should be implemented here, or in the original sphinx theme?

At the moment the "custom-button" (and its css) is defined here in pandas, so a hover effect on it is also to be implemented in pandas. 
Although long term we should probably look into replacing it with the buttons provided by sphinx-panels: https://sphinx-panels.readthedocs.io/en/latest/#link-buttons
I'm going to close this. @MOHED1224 let's do this properly even if it's small. Can you open an issue with screenshots of the button you think should be improved, with and without your proposed hover. And tag me there, if there is agreement please reopen the PR only with those changes, and nothing else.
CI: pin xlrd<2.0
State: closed

- [ ] closes #38524
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: IO Excel
description: read_excel, to_excel
name: Dependencies
description: Required and optional dependencies

Comments:
@meeseeksdev backport 1.2.x
BUG: read_excel forward-filling MI names
State: closed

- [x] closes #34673
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry

Added test with OP's issue as a sheet in the style of existing sheets in `tests/io/data/excel/testmultiindex.*`

(the same as the "both_name" sheet, but with the cell below the MI name empty)

New sheet looks like:
<img width="393" alt="Screen Shot 2020-12-15 at 9 50 02 PM" src="https://user-images.githubusercontent.com/37011898/102298700-961dbf00-3f1f-11eb-9c98-034268ad9d68.png">

In master without this change, the empty cell was being filled with the index name `ilvl2`

Labels:
name: Bug
name: IO Excel
description: read_excel, to_excel

Comments:
Thanks @mzeitlin11 !
API: CategoricalDtype.__eq__ with categories=None stricter
State: closed

- [ ] closes #xxxx
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry

xref #37929

This will allow us to remove special-casing for Categorical in Block._astype, Index.astype, and Categorical.astype.

Labels:
name: Categorical
description: Categorical Data Type
name: API - Consistency
description: Internal Consistency of API/Behavior

Comments:
ok this looks good. though having a hard time telling what changed from a user POV, can you add a whatsnew note.
whatsnew added + greenish
can you rebase
rebased + greenish
excellent.  this blocked #38530, which in turn is one of only two remaining special cases preventing us from ripping out Block._astype
ENH: support none in header list #37625
State: closed

closes #37625
tests passed
read_excel funtion can now accept header={'Sheet1': 0, 'Sheet2': None} as dictionary with header for each sheet.


Labels:
name: IO CSV
description: read_csv, to_csv
name: Stale

Comments:
This pull request is stale because it has been open for thirty days with no activity. Please update or respond to this comment if you're still interested in working on this.
closing as stale. if you want to continue, pls ping and can re-open.
df.melt() column name 'variable'
State: closed

- [x] closes #37716
- [x] tests added / passed
- [x] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry

This is my first time contributing to the open source project. Although I didn't make a significant change, I saw a bug report and asked to just raise a warning instead of fixing it. That's what I did and returned original DataFrame instead of returning wrong frame. 


Labels:
name: Bug
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode
name: Stale

Comments:
This pull request is stale because it has been open for thirty days with no activity. Please update or respond to this comment if you're still interested in working on this.
closing as stale. if you want to continue, pls ping and can re-open.
BUG: MultiIndex.equals returning incorrectly True when Indexes contains NaN
State: closed

- [x] xref #38439
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry

Depending on the expected behavior of align, this may have hidden an additional bug in align in the referenced issue.

Labels:
name: MultiIndex

Comments:
thanks @phofl
REG: DataFrame.shift with axis=1 and CategoricalIndex columns
State: closed

- [x] closes #38434
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry


Labels:
name: Bug
name: Regression
description: Functionality that used to work in a prior pandas version
name: Algos
description: Non-arithmetic algos: value_counts, factorize, sorting, isin, clip, shift, diff
name: Categorical
description: Categorical Data Type

Comments:
Thanks for taking this.

This PR should IMO be part of 1.2, since it’s a regression since 1.1.
> This PR should IMO be part of 1.2, since it’s a regression since 1.1.

and whatsnew probably not necessary
added test case with periods=2, removed whatsnew
@meeseeksdev backport 1.2.x
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
$ git checkout 1.2.x
$ git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
$ git cherry-pick -m1 d08f12c0409e8de977ae1821dbfc583942f35bef
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
$ git commit -am 'Backport PR #38504: REG: DataFrame.shift with axis=1 and CategoricalIndex columns'
```

4. Push to a named branch :

```
git push YOURFORK 1.2.x:auto-backport-of-pr-38504-on-1.2.x
```

5. Create a PR against branch 1.2.x, I would have named this PR:

> "Backport PR #38504 on branch 1.2.x"

And apply the correct labels and milestones.

Congratulation you did some good work ! Hopefully your backport PR will be tested by the continuous integration and merged soon!

If these instruction are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
DOC: Corrected Out-of-date list of sort algorithms
State: closed

Reference Issues/PRs
Fixes: #38287

Implemented changes:
Changed the docstring of all the methods using np.sort as it now includes one additional stable algorithm


Labels:
name: Docs
name: Algos
description: Non-arithmetic algos: value_counts, factorize, sorting, isin, clip, shift, diff

Comments:
can you also update the tests in pandas/tests/series/methods/test_sort_index.py to use stable (where we use mergesort), just parameterize a test is prob good enough.
> can you also update the tests in pandas/tests/series/methods/test_sort_index.py to use stable (where we use mergesort), just parameterize a test is prob good enough.

Done :)
LGTM, mind fixing up the merge conflict @DonPablo99? 
can you merge master and update
thanks @DonPablo99
DOC: Fix incorrect doc string for infer_dtype (#38375)
State: closed

- [x] closes #38375
- ~~tests added / passed~~
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- ~~whatsnew entry~~

A minor fix to https://github.com/pandas-dev/pandas/issues/38375 correcting the output of an example in `infer_dtype` doc string


Labels:
name: Docs

Comments:
thanks @tsinggggg
TST: don't use global fixture in the base extension tests
State: closed

Follow-up on https://github.com/pandas-dev/pandas/pull/37867. In the discussion about using this fixture, I forgot that we should not use global pandas fixtures in those tests, but only the ones that are defined in the local /pandas/tests/extension/conftest.py, otherwise this doesn't work for downstream projects (eg https://github.com/geopandas/geopandas/issues/1735). For this case it's easier to simply not use a fixture, I think (but still share the common definition of the list)

Labels:
name: Testing
description: pandas testing functions or related to the test suite

Comments:
thanks @jorisvandenbossche 
@meeseeksdev backport 1.2.x
TYP: func argument to DataFrame.apply, DataFrame.applymap, core.apply.frame_apply
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry

Spinning off from #38416, xref https://github.com/pandas-dev/pandas/pull/38416#discussion_r542911035

Labels:
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
thanks @arw2019
BUG: CategoricalIndex.reindex fails when Index passed with labels all in category
State: closed

- [x] closes #28690
- [x] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry


Labels:
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: Categorical
description: Categorical Data Type

Comments:
thanks @GYHHAHA
Updated README
State: closed

 I have updated the README with the following - 
1. Added Links for Conda, Cython and AQR.
2. Added a brief description for what all the Dependencies do so people who are unfamiliar with them can atleast get a basic idea about them
3. Corrected PyPI spelling in one instance

Labels:
name: Docs

Comments:
Hey
If any more changes are needed in the README just comment them below I'll do them as well.
Thanks :)
I can't seem to understand why it failed. I think it is unrelated to my changes
@jreback Just wanted to mention I've made the changes but some tests seem to fail however I think they are independent of my changes
thanks @aflah02 failures are unrelated to your PR