BrandonFors/pandas-github-issues huggingface dataset:
WIP: BUG: fix Timestamp.normalize with nonexistent midnight on DST change
State: closed

- [ ] closes #40517
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] whatsnew entry

Attempting an approach to fix this. Let's see if it works.

Not surprisingly, the bruteforce approach breaks a bunch of stuff, because I ended up with twice the utcoffset shift instead of once, which makes perfect sense.

What we want here instead is to get the first valid nanosecond of the day by default and normalize to that. And probably expose to the user the ability to shift back for normalization if midnight does not exist.

Will try to fix this tomorrow.

Labels:
name: Stale

Comments:
This pull request is stale because it has been open for thirty days with no activity. Please update or respond to this comment if you're still interested in working on this.
closing as stale. ping when ready to continue.
CI: Combined the doctests checks
State: closed

- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them


Doing as was asked [here](https://github.com/pandas-dev/pandas/pull/40903#discussion_r612486254)


Labels:
name: CI
description: Continuous Integration

Comments:
Could you merge master again? It appears that there were networking issues in the CI
gentle ping @mroeschke 
thanks @ShaharNaveh
DOC: add example for plotting asymmetrical error bars
State: closed

- [x] closes #41034 
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] whatsnew entry


Labels:
name: Docs
name: Visualization
description: plotting

Comments:
@rhshadrach PR updated, thanks!
Thanks @mortada!
REF: remove Categorical._shallow_copy
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] whatsnew entry

@phofl parts of this may be worth doing in #40967

Labels:
name: Refactor
description: Internal refactoring of code
name: Index
description: Related to the Index class or subclasses
name: Constructors
description: Series/DataFrame/Index/pd.array Constructors

Comments:
I'll move it over later today
added test case for iloc function if it returns the same output for b…
State: closed

…oth numpy and pandas arrays to fix the issue #40933 

- [x] closes #40933 
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] whatsnew entry


Labels:
name: Stale

Comments:
We can not cast to numpy here, our ExtensionArrays behave different
This pull request is stale because it has been open for thirty days with no activity. Please update or respond to this comment if you're still interested in working on this.
closing as stale. ping when ready to continue and i'll reopen.
BUG: Handling NaN case before dtype conversion #40638
State: closed

- [ ] closes #40638
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] whatsnew entry
Handles the NA/NaN case before the dtype conversion

Labels:
name: Stale

Comments:
Hello @Sudomarko! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:



There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2021-04-19 03:59:09 UTC
This pull request is stale because it has been open for thirty days with no activity. Please update or respond to this comment if you're still interested in working on this.
@Sudomarko closing as stale. ping when ready to continue and i'll re-open.
BUG: Changing return is_named_tuble to a tuple sublclass #40682
State: closed

- [ ] closes #40682
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] whatsnew entry

Changed pandas/core/dtypes/inference.py so that the return of the is_named_tuple function to be a tuple subclass so that it is compatible with sqlalchemy.

Passed test_inference.py

Labels:
name: Stale

Comments:
This pull request is stale because it has been open for thirty days with no activity. Please update or respond to this comment if you're still interested in working on this.
@Sudomarko can you add the code sample from the issue OP as a test
@Sudomarko Thanks for the PR. closing as stale. ping if you want to continue.
ENH: check for string and convert to list in DataFrame.dropna subset argument
State: closed

- [x] closes #41021
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [x] whatsnew entry


Labels:
name: API - Consistency
description: Internal Consistency of API/Behavior

Comments:
@phofl @mzeitlin11 the test `pandas/tests/reductions/test_reductions.py::TestSeriesReductions::test_all_any_params` runs succesfully on my machine, but fails here in CI. So cannot reproduce the error to check what is wrong. 


edit: my branch is up to date with master, so that cannot be the problem.
You can ignore the ``ResourceWarning`` tests. But pre commit and ci checks is real
This pull request is stale because it has been open for thirty days with no activity. Please update or respond to this comment if you're still interested in working on this.
@erfannariman can you resolve conflicts and address @jreback comments
@erfannariman closing as stale. ping when ready to continue.
> @erfannariman closing as stale. ping when ready to continue.

Could you please reopen, will finish this one.
I get a typing error since subset is `IndexLabel`, while `np.compress` expects other type. The functionality should still work, since `com.maybe_make_list` returns a list. 
Hello @erfannariman! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:





There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2021-10-18 14:37:09 UTC
@rhshadrach @jreback think this one is good to go.
Thanks @erfannariman!
BUG: isna not returning copy for MaskedArray
State: closed

- [x] closes #40935
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [x] whatsnew entry

Think further improvements like zero-copy access fall under #34873, so #40935 can be closed.

Labels:
name: Bug
name: NA - MaskedArrays
description: Related to pd.NA and nullable extension arrays

Comments:
thanks @mzeitlin11
[ArrowStringArray] fix test_astype_int, test_astype_float
State: closed


Labels:
name: Strings
description: String extension data type and string data

Comments:
the changes to to_numpy are implicitly tested (and different behavior to StringArray). will need to add tests to explicitly tests this with the changes requested here. will open separate PRs for to_numpy/astype and close this to clear the queue.
merging, @simonjayhawkins if you can note the followups
> merging, @simonjayhawkins if you can note the followups

sure. The typing is included in https://github.com/pandas-dev/pandas/issues/35169#issuecomment-727217462 and will be done after the work by @Dr-Irv sorting out the base EA types.
Fix 33634: If aggregation function returns NaN the order of the index on the resulting df is not maintained 
State: closed

- [x] closes #33634
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [x] whatsnew entry


Labels:
name: Bug
name: Apply
description: Apply, Aggregate, Transform, Map

Comments:
cc @jreback 
Thanks @DriesSchaumont!
Revert "Skipt failing tests for numpy dev (#40877)"
State: closed

- [x] closes #40874

new numpy wheel is out, bug was fixed during the week


Labels:
name: CI
description: Continuous Integration
name: Dependencies
description: Required and optional dependencies
name: Upstream issue
description: Issue related to pandas dependency

Comments:
greenish, failure unrelated
lgtm go ahead merge and backport
@meeseeksdev backport 1.2.x
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
$ git checkout 1.2.x
$ git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
$ git cherry-pick -m1 70435eba769c6bcf57332306455eb70db9fa1111
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
$ git commit -am 'Backport PR #41015: Revert "Skipt failing tests for numpy dev (#40877)"'
```

4. Push to a named branch :

```
git push YOURFORK 1.2.x:auto-backport-of-pr-41015-on-1.2.x
```

5. Create a PR against branch 1.2.x, I would have named this PR:

> "Backport PR #41015 on branch 1.2.x"

And apply the correct labels and milestones.

Congratulation you did some good work ! Hopefully your backport PR will be tested by the continuous integration and merged soon!

If these instruction are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
                
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
$ git checkout 1.2.x
$ git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
$ git cherry-pick -m1 70435eba769c6bcf57332306455eb70db9fa1111
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
$ git commit -am 'Backport PR #41015: Revert "Skipt failing tests for numpy dev (#40877)"'
```

4. Push to a named branch :

```
git push YOURFORK 1.2.x:auto-backport-of-pr-41015-on-1.2.x
```

5. Create a PR against branch 1.2.x, I would have named this PR:

> "Backport PR #41015 on branch 1.2.x"

And apply the correct labels and milestones.

Congratulation you did some good work ! Hopefully your backport PR will be tested by the continuous integration and merged soon!

If these instruction are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
DOC: update `style.ipynb` user guide for recent enhancements
State: closed

I added small changes to the Styler User Guide to exemplify the new builtin highlighting functions.

Also removed some of the longer detail which has been moved to individual documentation pages (for example for `background_gradient`)

Labels:
name: Docs
name: Code Style
description: Code style, linting, code_checks

Comments:
Hi @attack68. I wish to get started in contributing to open source. Is there any way I can contribute to this issue, please? I see it needs a documentation enhancement but I am not quite sure well. I can always help in writing for a start
> Hi @attack68. I wish to get started in contributing to open source. Is there any way I can contribute to this issue, please? I see it needs a documentation enhancement but I am not quite sure well. I can always help in writing for a start

@dayvheed this is not an issue. It is a pull request where I have already made the necessary changes. If you would like to contribute to pandas I would suggest the place to start is reading the documentation on contributing and then tackling real issues (not pull requests) that have the "good first issue" label.
Thank you@attack68. It was the label that got me confused. 
sure ping on gree
@jreback pinging on green
thanks @attack68
DOC: Improve describe() documentation
State: closed

- [x] closes #40972 
- [ ] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] whatsnew entry


It feels more verbose, but necessary?

Labels:
name: Docs

Comments:
Test failure seems unrelated
Yeah this test is flaky
Thanks for looking at this @Bharat123rox. This patch doesn't look to be addressing the issue - the line before already says that different pandas operations might return a Series or DataFrame. 

The issue here is that the next line mentions that this is an example of a Series being returned, but a DataFrame is actually being returned from this describe example
> The issue here is that the next line mentions that this is an example of a Series being returned, but a DataFrame is actually being returned from this describe example

So should we just stick to this example and change it to `DataFrame` because the given `describe()` operation can still return both right? So what should be done then?
I would agree with just mentioning DataFrame and Series, e.g. removing the column reference. Do you agree @mzeitlin11?
> I would agree with just mentioning DataFrame and Series, e.g. removing the column reference. Do you agree @mzeitlin11?

Updated, please re-review
Thanks @Bharat123rox
BUG: groupby.rank with MaskedArray incorrect casting
State: closed

- [ ] closes #xxxx
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] whatsnew entry

In master:
```
df = pd.DataFrame({"key": ["a"] * 4, "val": [1] * 4})
df["val"] = df["val"].astype("Int64")

>>> result = df.groupby("key").rank()
TypeError: cannot safely cast non-equivalent float64 to int64
```

Moves in the direction of sharing/generalizing _ea_wrap_cython_operation.

If I did this right, should avoid overlap with #40651.

Labels:
name: Bug
name: Groupby
name: NA - MaskedArrays
description: Related to pd.NA and nullable extension arrays

Comments:
thanks @jbrockmendel
Bug in to_datetime raising ValueError with None and NaT and more than 50 elements
State: closed

- [x] closes #39882
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [x] whatsnew entry

None and NaT are different for unique while convert_listlike casts None to NaT, hence causing dups
Not sure if we could do something better.

Labels:
name: Datetime
description: Datetime data dtype
name: Dtype Conversions
description: Unexpected or buggy dtype conversions

Comments:
@jreback greenish
thanks @phofl
BUG: Fix Dataframe constructor missing sql column names (#40682)
State: closed

…ames

- [ ] closes #40682  
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] whatsnew entry


Labels:
name: Bug
name: IO SQL
description: to_sql, read_sql, read_sql_query
name: Stale

Comments:
Hello @jnchngc! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:





There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2021-04-23 21:46:40 UTC
This pull request is stale because it has been open for thirty days with no activity. Please update or respond to this comment if you're still interested in working on this.
Thanks for contributing @jnchngc, closing as stale for now, but please ping to reopen if you'd like to continue and address comments (would also be good to avoid the added dependency here as well).
fix: function _take_with_is_copy was defined final but overriden
State: closed

- [x] closes #40974 
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] whatsnew entry


Labels:
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
Thanks! Can you also add the return type to this function in series.py? Now that final is removed, that should work
> Thanks! Can you also add the return type to this function in series.py? Now that final is removed, that should work

should just be `-> Series`
Thanks @ajiteshsingh
Regression modifying obj with all false indexer
State: closed

- [x] closes #40885
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them


Regression on master so no whatsnew


Labels:
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: Regression
description: Functionality that used to work in a prior pandas version

Comments:
very nice @phofl
REGR: memory_map with non-UTF8 encoding
State: closed

- [x] closes #40986
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [x] whatsnew entry

My best guess is that `memory_map=True` always assumed UTF-8 with the python engine. Now that the c and python engine use the same IO code, the c engine assumed UTF-8 as well.


Labels:
name: Regression
description: Functionality that used to work in a prior pandas version
name: IO CSV
description: read_csv, to_csv

Comments:
Hello @twoertwein! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:







There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2021-04-25 14:56:08 UTC
@meeseeksdev backport 1.2.x
thanks @twoertwein keep em coming!
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
$ git checkout 1.2.x
$ git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
$ git cherry-pick -m1 0a0540c30bb438a443b10516d9d0826089a4b396
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
$ git commit -am 'Backport PR #40994: REGR: memory_map with non-UTF8 encoding'
```

4. Push to a named branch :

```
git push YOURFORK 1.2.x:auto-backport-of-pr-40994-on-1.2.x
```

5. Create a PR against branch 1.2.x, I would have named this PR:

> "Backport PR #40994 on branch 1.2.x"

And apply the correct labels and milestones.

Congratulation you did some good work ! Hopefully your backport PR will be tested by the continuous integration and merged soon!

If these instruction are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
Deprecate joining over a different number of levels
State: closed

- [x] closes #34862
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [x] whatsnew entry

I think deprecating this is a good idea, I stumbled across this accidentally this week and found this really odd

Labels:
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode
name: Deprecate
description: Functionality to remove in pandas

Comments:
lgtm ping when updated
thanks @phofl
Deprecate suffixes in merge producing duplicate columns
State: closed

- [x] closes #22818
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [x] whatsnew entry

I think we should not allow these column collisions at all. Hence deprecating and raising in 2.0
Alternative here would be to add an ``errors`` keyword to the merge and join functions as @simonjayhawkins mentioned in the op. 

Labels:
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode
name: Deprecate
description: Functionality to remove in pandas

Comments:
thanks @phofl
TYP: libperiod
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] whatsnew entry


Labels:
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
Thanks @jbrockmendel for the updates. some new mypy failures.
mypy complaints fixed; new CI failure is 3.10, looks unrelated
thanks @jbrockmendel
Bug in loc returning multiindex in wrong oder with duplicated indexer
State: closed

- [x] closes #40978
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [x] whatsnew entry

This fixes the ordering, but I am not sure if an indexer with duplicates in this case should return the same as an indexer which is unique with the same elements.

Labels:
name: Bug
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: MultiIndex

Comments:
@phofl can you rebase and will look again
merged master
thanks @phofl
DOC: Reindexing behaviour of dataframe column-assignment missing
State: closed

- [X] xref #39845 (.loc[] portion of issue)
- [X] tests added / passed
- [X] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] whatsnew entry


Labels:
name: Docs
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: Stale

Comments:
Do we need a whatsnew entry for a documentation-only issue? If so, where and in what general format to we put it?
Don't think we need a whatsnew, you can not use closes if you are fixing part of the issue, because this would close the issue. E.g. your number is wrong, you are referencing the pr
This pull request is stale because it has been open for thirty days with no activity. Please update or respond to this comment if you're still interested in working on this.
> This pull request is stale because it has been open for thirty days with no activity. Please update or respond to this comment if you're still interested in working on this.

I apologize for the lack of activity. I was very busy with finals and will be working towards closing this pull request in the coming days.
Hello @hewittk! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:



There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2021-06-09 21:14:12 UTC
This pull request is stale because it has been open for thirty days with no activity. Please update or respond to this comment if you're still interested in working on this.
Sorry this went un-reviewed for a while. Could you merge master again to this branch?
Thanks for the PR, but it appears to have gone stale. Let us know if you're still interested in working on this and we can reopen. Closing.