BrandonFors/pandas-github-issues huggingface dataset:
REGR: unpickling in 1.3.0 DataFrame created in 1.2.x #42345
State: closed

- [x] closes #42345
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [x] whatsnew entry


Labels:
name: IO Data
description: IO issues that don't fit into a more specific label
name: Regression
description: Functionality that used to work in a prior pandas version

Comments:
@jbrockmendel ok i think then let's merge this and then backport the full testing of the 1.2.x pickles in 1.3?
> yah
@meeseeksdev backport 1.3.x
Something went wrong ... Please have  a look at my logs.
BUG: fix regression with SerieGrouper with Timestamp index (#42390)
State: closed

closes https://github.com/pandas-dev/pandas/issues/42390

This fixes a regression introduced in c355ed1 where cache is not
initialized with correct state of islider and vslider.

On Timestamp index this trigger a "ValueError Length of values does not match length of index"

Labels:
name: Datetime
description: Datetime data dtype
name: Groupby
name: Regression
description: Functionality that used to work in a prior pandas version

Comments:
OK, I think the underlying problem here is that we shouldn't be getting to libreduction at all when we have a dt64tz DTI.  I think DatetimeIndex._has_complex_internals should be True for tzaware (possibly all of the datetimelike indexes, not totally sure)
> OK, I think the underlying problem here is that we shouldn't be getting to libreduction at all when we have a dt64tz DTI. I think DatetimeIndex._has_complex_internals should be True for tzaware (possibly all of the datetimelike indexes, not totally sure)

see https://github.com/pandas-dev/pandas/issues/42390#issuecomment-888158663


status here?
needs a release note and IIUC @jbrockmendel prefers to fix the underlying issue https://github.com/pandas-dev/pandas/pull/42391#issuecomment-886911368

@jbrockmendel 1.3.2 scheduled for end of this week, can you take a look. (There are several open issues on 1.3.2 so I think will schedule 1.3.3 for 3 weeks after 1.3.2 so no pressure)
The example in #42390 is fixed by changing DatetimeTimedeltaMixin._has_complex_internals to return True.  This will be a less-invasive alternative to ripping out SeriesBinGrouper which i expect we'll do for 1.4
closing this one as stale.
DOC fix the incorrect doc style in 1.2.1
State: closed

- [x] closes #39290
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] whatsnew entry


Labels:
name: Docs

Comments:
hmm, this looks fine, IIRC did we somehow change the whatsnew way back when?

cc @simonjayhawkins @datapythonista 
thanks @debnathshoham 
@meeseeksdev backport 1.3.x
Something went wrong ... Please have  a look at my logs.
TYP: clean for PY3.8 (Literal)
State: closed


Labels:
name: Clean
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
39 failure looks unrelated; ive seen it on at least one other PR today
#42381
@simonjayhawkins I don't think this will work with python 3.7.  Does that mean we are dropping support for 3.7 in the near future?
already dropped. #41678

thanks @simonjayhawkins
Fix Formatting Issue
State: closed

Escape the asterisks so they appear in the generated doc. Closes #42371. Untested. Asterisks formatted bold in my editor, kept it that way for consistency, someone should test.

Labels:
name: Docs

Comments:
Hello @DavidDeLord! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:

* In the file [`pandas/core/series.py`](https://github.com/pandas-dev/pandas/blob/8c04d2ad4fb17210fa2ed5c6ddef78a01278e33a/pandas/core/series.py):

> [Line 204:41](https://github.com/pandas-dev/pandas/blob/8c04d2ad4fb17210fa2ed5c6ddef78a01278e33a/pandas/core/series.py#L204): [W605](https://duckduckgo.com/?q=pep8%20W605) invalid escape sequence '\*'
> [Line 204:45](https://github.com/pandas-dev/pandas/blob/8c04d2ad4fb17210fa2ed5c6ddef78a01278e33a/pandas/core/series.py#L204): [W605](https://duckduckgo.com/?q=pep8%20W605) invalid escape sequence '\*'
> [Line 204:47](https://github.com/pandas-dev/pandas/blob/8c04d2ad4fb17210fa2ed5c6ddef78a01278e33a/pandas/core/series.py#L204): [W605](https://duckduckgo.com/?q=pep8%20W605) invalid escape sequence '\*'



##### Comment last updated at 2021-07-07 05:44:49 UTC
wow it detects this but not that the docstrings are >72 chars long
> someone should test

see https://pandas.pydata.org/pandas-docs/dev/development/contributing_documentation.html#building-the-documentation for generating the documentation and then can post a screenshot of the relevant section
closing because this is messy and ill open another where i do it properly
also its wrong
CI: Prefer conda-forge over defaults
State: closed

- [ ] closes #42370
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] whatsnew entry

Apparently its recommended to install pyarrow from conda forge instead of the default conda channel(xref https://github.com/ContinuumIO/anaconda-issues/issues/12314, https://arrow.apache.org/docs/python/install.html#using-conda), trying to see if this fixes it. I don't think this will mess up any other packages since conda-forge packages are published by package maintainers, so they shouldn't have build issues?

Labels:
name: CI
description: Continuous Integration

Comments:
Windows CI failure is a flaky ResourceWarning, feel free to restart the job(I cannot).
ci failure on numpydev also also seen on other PR #42367

```
=========================== short test summary info ============================
FAILED pandas/tests/scalar/timedelta/test_arithmetic.py::TestTimedeltaMultiplicationDivision::test_td_div_nan[nan1]
= 1 failed, 156395 passed, 15976 skipped, 857 xfailed, 6 xpassed in 1944.90s (0:32:24) =
```
Thanks @lithomas1
BUG: truncate has incorrect behavior when index has only one unique v…
State: closed

…alue (#42365)

- [x] closes #42365
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [x] whatsnew entry


Labels:
name: Bug
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves

Comments:
looks fine can you add a whatsnew note in 1.4 bug fixes, indexing section
@jreback I think this PR should be good as well since the errors above seem transient and unrelated to any of the code changes here (and I'm seeing similar errors in other PRs)
thanks @neelmraman
CLN: make some private Index methods keyword-only
State: closed

Not sure how much mileage we can get out of doing this for only private methods, but if we had it for everything then it'd be a lot easier to address #21677

Labels:
name: Clean
name: Index
description: Related to the Index class or subclasses

Comments:
> CLN: make some private Index methods positional-only

keyword only?
> keyword only?

yep, fixed.  thanks.
thanks @jbrockmendel
BUG: fixes PyPy build (do not use _PyHASH_Imag)
State: closed

- [x] closes #42355
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [x] what's new entry

`_PyHASH_Imag` is not defined in PyPy, thus we define and use our own `_PandasHASH_Imag`.


Labels:
name: Build
description: Library building on various platforms

Comments:
@realead can you add a release note in 1.3.1; ping on green.
thanks @realead 
@meeseeksdev backport 1.3.x
Something went wrong ... Please have  a look at my logs.
@realead Thanks for the quick fix!
ERR: better error message with invalid argument in sort_values
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] whatsnew entry
added a if branch to check the ascending type, as the existing one accepts value with string as well instead of boolean.

k=df.sort_values(by=['col1'],ascending="False") does not give exception with str.
added piece of code to raise exception if the type is not boolean




Labels:
name: Error Reporting
description: Incorrect or improved errors from pandas

Comments:
Hello @deej197! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:

* In the file [`pandas/core/series.py`](https://github.com/pandas-dev/pandas/blob/d064edc441dcc16033d9c002bdfefb5cc04ea3aa/pandas/core/series.py):

> [Line 3444:1](https://github.com/pandas-dev/pandas/blob/d064edc441dcc16033d9c002bdfefb5cc04ea3aa/pandas/core/series.py#L3444): [W293](https://duckduckgo.com/?q=pep8%20W293) blank line contains whitespace
> [Line 3445:9](https://github.com/pandas-dev/pandas/blob/d064edc441dcc16033d9c002bdfefb5cc04ea3aa/pandas/core/series.py#L3445): [E303](https://duckduckgo.com/?q=pep8%20E303) too many blank lines (2)
> [Line 3445:27](https://github.com/pandas-dev/pandas/blob/d064edc441dcc16033d9c002bdfefb5cc04ea3aa/pandas/core/series.py#L3445): [E225](https://duckduckgo.com/?q=pep8%20E225) missing whitespace around operator



##### Comment last updated at 2021-07-03 19:22:47 UTC
Thanks for the contribution @deej197, but unfortunately you won't be able to contribute with a proper development environment. If you could run this test locally you could see that:
- It's adding new bugs, and doesn't affect the current one
- The problem is not the one you mention in the title, the error message is fine, the problem is that it only raises on `Series`, and not in `DataFrame`.

And it's also better to start by opening an issue, finding a reproducible example, and when there is agreement with the solution, then open the PR (which would require tests and a release note, besides the code changes). I opened an issue myself, so you can see how would be better to start: #42434. Feel free to work on the issue, but as I said, you need a working development environment. Otherwise you won't be able to test what you're doing, and you'll be submitting obviously wrong PRs like this, that don't add value, and waste reviewers time.

If you follow the contribution guide, you shouldn't have problems setting up the develoment environment, if something is not clear enough, feel free to open an issue to ask for clarification in the docs.
DEPR: ignoring missing labels when indexing on MultiIndex level
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] whatsnew entry

There are a bunch of issues that boil down to this weird behavior.  I need to track down the full list (@attack68 do you have such a list handy?)

Labels:
name: MultiIndex
name: Deprecate
description: Functionality to remove in pandas

Comments:
> 
> There are a bunch of issues that boil down to this weird behavior. I need to track down the full list (@attack68 do you have such a list handy?)

I haven't got a list but I have a script/notebook that tries to iterate over all possible combinations and display the results. I will try and post it later.

I hold the view that the tests for multiindex loc is quite sparse in terms of patchy parametrised tests, but I think a more robust algorithmic approach to all combinations might be worthwhile. Perhaps a routine for this might develop from the script??
https://github.com/attack68/pandas/tree/multiindex_ipynb 
file in root: `LocIndexing.ipynb`

It gathers and analyses many different combinations of `loc` slicing.

For a MultiIndex with `['a', 'b', 'c', 'd']` in each level, (i.e. `'!'` not in any level) here is a summary of some odd `series.loc` slices, when combining with different index types (duplicated vs unique, and monotonic vs unsorted).

![Screen Shot 2021-07-13 at 16 32 18](https://user-images.githubusercontent.com/24256554/125472004-4d3b204c-c4ac-453e-91ed-507c1eb1732a.png)
BUG: Don't cache args during rolling/expanding.apply with numba engine
State: closed

- [x] closes #42287
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [x] whatsnew entry


Labels:
name: Bug
name: Window
description: rolling, ewma, expanding
name: numba
description: numba-accelerated operations

Comments:
thanks @mroeschke
ENH: Rename index when using DataFrame.reset_index
State: closed

- [x] closes #6878
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [x] whatsnew entry

Probably the hardest part of this PR is choosing a good name for the new argument. As mentioned in #6878 the `name` keyword is already used in `Series.reset_index`, although to rename the data column. It could be argued that using `name` also in this PR would not cause confusion, since it's hard to assume one can rename a column's name in `DataFrame.reset_index` (which column would it be?). 

Anyway, I've opted for `names`. I've also thought of `rename` but that sounds like asking for a boolean. `new_name` is an option too, although a bit ugly.

Labels:
name: Enhancement
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode

Comments:
> 
> 
> Now I see the problem with that `else` statement, you're absolutely right!
> 
> By the way, could you point me to where the `whatsnew` file is so that I can edit it?

Whatnew file is here: ``doc/source/whatsnew/v1.4.0.rst``
Hello @gcaria! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:









There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2021-09-09 11:12:17 UTC
This pull request is stale because it has been open for thirty days with no activity. Please update or respond to this comment if you're still interested in working on this.
Sorry for the long pause, I've finally added the `Index.get_default_index_names` and `MultiIndex.get_default_index_names` methods, and rebased to master since I was at it.
@gcaria - this looks almost there

Please can you get the code-checks passing - see the contributing guide
```
pandas/core/indexes/multi.py:1407:21: PDF005 leading space in concatenated strings
pandas/core/frame.py:5639:89: E501 line too long (94 > 88 characters)
pandas/core/frame.py:5640:89: E501 line too long (89 > 88 characters)
```

Looks like these changes are also causing some test failures - see Azure Pipelines.

`TestResetIndex.test_reset_index_rename_multiindex `

Can you also merge master and address any remaining comments above

Thanks!
this was looking good, if you'd merge master and address comments
@gcaria - do you still want to work on this? See comment above from Jeff
Unfortunately I don't have time to work on this anymore, but all the work is almost done.
Thanks for the update. If you happen to find time to work on this PR in the future, or if anyone else wants to finish up this feature, we can reopen.
DOC: fix docs after deprecation of args
State: closed

follows up an oversigh from #40628.
`False` was deprecated as an input so the example should be updated.

Labels:
name: Docs
name: Deprecate
description: Functionality to remove in pandas

Comments:
thanks @attack68 
@meeseeksdev backport 1.3.x
Something went wrong ... Please have  a look at my logs.
DOC: Unsuppress relevant dataframe in split-apply-combine guide
State: closed

Currently, the [user guide for groupby](https://pandas.pydata.org/docs/dev/user_guide/groupby.html#dataframe-column-selection-in-groupby) shows the following code, but it is unclear where the "C" and "D" columns come from.

![image](https://user-images.githubusercontent.com/7686641/124226033-48346700-dabd-11eb-90a1-be764c93f934.png)

In the rendered documentation, the preceding definition of `df` does not have "C" and "D" columns.

```python
In [50]: arrays = [
   ....:     ["bar", "bar", "baz", "baz", "foo", "foo", "qux", "qux"],
   ....:     ["one", "two", "one", "two", "one", "two", "one", "two"],
   ....: ]
   ....: 

In [51]: index = pd.MultiIndex.from_arrays(arrays, names=["first", "second"])

In [52]: df = pd.DataFrame({"A": [1, 1, 1, 1, 2, 2, 3, 3], "B": np.arange(8)}, index=index)

In [53]: df
Out[53]: 
              A  B
first second      
bar   one     1  0
      two     1  1
baz   one     1  2
      two     1  3
foo   one     2  4
      two     2  5
qux   one     3  6
      two     3  7
```

Instead, the code under the heading "DataFrame column selection in GroupBy" is referring to a completely different dataframe which is in the RST file but is suppressed in the rendered documentation.

This pull request un-suppresses that relevant dataframe so that readers can see where the "C" and "D" columns come from.

Labels:
name: Docs
name: Groupby

Comments:
Thanks @chrisyeh96!
ENH: Add `method='table'` for EWM.mean
State: closed

- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [x] whatsnew entry
close #42273 

Labels:
name: Enhancement
name: Window
description: rolling, ewma, expanding

Comments:
Since this only works for `engine='numba'` and for `mean` would it not be better to wrap this simply into `mean(engine='numba-table')` or the like?

Or, is the plan to try and expand this functionality more generally and therefore your way is better?
Yes, the hope is to eventually enable `method="table"` for other `EWM` operations (`var`, `std`) in the future with numba.

This also mirrors the API we have for `rolling` and `expanding`
thanks @mroeschke
PERF/REGR: revert #41785
State: closed

- [x] closes #42248
- [ ] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [x] whatsnew entry


Labels:
name: Performance
description: Memory or execution speed performance
name: Constructors
description: Series/DataFrame/Index/pd.array Constructors

Comments:
Thanks @jbrockmendel as this now goes in 1.3.1, will need a release note.
> Thanks @jbrockmendel as this now goes in 1.3.1, will need a release note.

#42312 now merged
@jbrockmendel lgtm can you add a release note for 1.3.1
@meeseeksdev backport 1.3.x
Something went wrong ... Please have  a look at my logs.
PERF/REGR: restore IntervalIndex._intersection_non_unique
State: closed

- [ ] xref #42240
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] whatsnew entry

Reverts the rest of #41929

Labels:
name: Performance
description: Memory or execution speed performance
name: Interval
description: Interval data type
name: Index
description: Related to the Index class or subclasses
name: setops
description: union, intersection, difference, symmetric_difference

Comments:
i've milestoned 1.3 and hopefully the backport will be straightforward
@jbrockmendel have you ran the asvs for this change?
no i havent
ok. i'm winding down for the day.

this targets `index_object.IntervalIndexMethod.time_intersection_both_duplicate(1000)`?

should the asv be run before merging to check or confident this fixes?

> should the asv be run before merging to check or confident this fixes?

im moderately confident, but my intuition on this doesn't have a great track record over the last few days
let's merge and just do the rleease tomorrow. this is the point of point releases, we can always fix things.
@meeseeksdev backport 1.3.x
Something went wrong ... Please have  a look at my logs.
> im moderately confident, but my intuition on this doesn't have a great track record over the last few days

looking good... https://simonjayhawkins.github.io/fantastic-dollop/#index_object.IntervalIndexMethod.time_intersection_both_duplicate?branch=1.3.x&branch=1.2.x&commits=5940c9cf&p-param1=1000
BENCH: update imports to allow testing against 1.2.5
State: closed


Labels:
name: Benchmark
description: Performance (ASV) benchmarks

Comments:
I've milestoned 1.4. I don't think we need to backport this. we use master for the benchmarks.

This changes allows our current benchmarks to run against 1.2.5 to help prioritize and track regression fixes from 1.2.5 to 1.3x
BUG: .loc with MultiIndex with tuple in level GH#27591
State: closed

- [x] closes #27591
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [x] whatsnew entry


Labels:
name: Bug
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: MultiIndex

Comments:
This PR does better than existing but its still ambiguous for pathological cases - hence why I'm floating the idea of deprecating tuples as sequence identifiers for multiple keys and relying only on lists (which can't be used as index keys).

**edit**: actually not even doing that would solve the cases below - because nowhere is a list needed, the below depends on how the `slice(None)` might be automatically added or not based on implied levels. 

### Effect of PR

| Before | After |
| --- | --- |
| ![Screen Shot 2021-07-01 at 15 26 34](https://user-images.githubusercontent.com/24256554/124132141-ca526a80-da80-11eb-8359-c09a6d1eea95.png) | ![Screen Shot 2021-07-01 at 15 26 11](https://user-images.githubusercontent.com/24256554/124132158-d0484b80-da80-11eb-853a-8be2dce90f93.png) | 

> hence why I'm floating the idea of deprecating tuples as sequence identifiers for multiple keys and relying only on lists (which can't be used as index keys).

I think we should do this too.  This PR is about fixing the bug in the interim.
thanks @jbrockmendel, looking forwards to cleanups on this!
add Juneteenth to USFederalHolidayCalendar
State: closed

* Add **Juneteenth National Independence Day**

*  Standardize the [official federal holiday names](https://www.opm.gov/policy-data-oversight/pay-leave/federal-holidays/#url=2021)

Labels:
name: Datetime
description: Datetime data dtype
name: Stale

Comments:
Can you write a test for this.
Hello @ChuliangXiao! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:



There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2021-07-02 15:35:29 UTC
Thanks @ChuliangXiao, besides the test, can you also please an entry in the release notes? Thanks!
This pull request is stale because it has been open for thirty days with no activity. Please update or respond to this comment if you're still interested in working on this.
Closing as stale, but if you want to finish this @ChuliangXiao just let me know.