BrandonFors/pandas-github-issues huggingface dataset:
REGR: fillna not filling NaNs after pivot without explicitly listing pivot values on 1.1.x
State: closed

- [ ] xref #36495

yet another alternative to fix directly on 1.1.x pending further discussion in #34407 on how to resolve the issues on master.

cc @jorisvandenbossche @jbrockmendel 

This won't close issue until fix on master. could copy test to master with xfail (and would need to sync release notes if we merge this direct to 1.1.x)

Labels:
name: Missing-data
description: np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate
name: Regression
description: Functionality that used to work in a prior pandas version
name: Internals
description: Related to non-user accessible pandas implementation

Comments:
travis environment setup failure unrelated
Hello @simonjayhawkins! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:





There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2020-09-26 15:05:56 UTC
Fine for me to merge this now in 1.1.x, and have another version applied to master with the test xfailed
I'll try to find the underlying problem so this is unnecessary.  In the interim, this is fine by me.
may need more discussion.

I can't block by requesting changes on my own PR, moving to 1.1.4 so release note needs to be moved. closing for now instead. Will re-open after 1.1.3 release.

I've started the final release readiness checks. merging now would require going back to square one.
Why not include this in the release? This PR is ready it seems (small fix, tests pass)
merging directly to 1.1.x can be a bit messy. If there's a better way https://github.com/pandas-dev/pandas/pull/36668#issuecomment-699524035 we can do that for 1.1.4

If this issue still persists when we are ready for 1.1.4 release we could merge this instead?
Sounds good!
REGR: fillna not filling NaNs after pivot without explicitly listing pivot values
State: closed

- [ ] closes #36495

alternative to #34407 that only restores `self._consolidate_inplace()` to fix reported regression in #36495

This WILL fail see https://github.com/pandas-dev/pandas/pull/34407#issuecomment-699488350 and https://github.com/pandas-dev/pandas/pull/34407#issuecomment-699495512

if all green apart from those two failures can then maybe revert #35839 and #35854

Labels:
name: Missing-data
description: np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate
name: Regression
description: Functionality that used to work in a prior pandas version
name: Internals
description: Related to non-user accessible pandas implementation

Comments:
cc @jbrockmendel @jorisvandenbossche 
yep, just the two tests failing with this.
Backport PR #36560 on branch 1.1.x ([BUG]: Fix regression in read_table with delim_whitespace=True)
State: closed

Backport PR #36560: [BUG]: Fix regression in read_table with delim_whitespace=True

Labels:
name: IO Data
description: IO issues that don't fit into a more specific label

Comments:
lint failure fixed in #36663
travis failure unrelated. see https://github.com/pandas-dev/pandas/pull/36396#issuecomment-695195148
DEPR: DateOffset args don't end in 's'
State: closed

Make deprecation DateOffset args don't end in 's'.

Mentioned [here](https://github.com/pandas-dev/pandas/pull/36516#discussion_r492885786)


Labels:
name: Frequency
description: DateOffsets
name: Deprecate
description: Functionality to remove in pandas
name: Stale

Comments:
Test1: 

```
pandas/tests/io/generate_legacy_storage_files.py:300: in create_pickle_data
    data = create_data()
pandas/tests/io/generate_legacy_storage_files.py:262: in create_data
    "DateOffset_h_ns": DateOffset(hour=6, nanoseconds=5824),
pandas/_libs/tslibs/offsets.pyx:976: in pandas._libs.tslibs.offsets.RelativeDeltaOffset.__init__
    off, use_rd = _determine_offset(kwds)
```

Test2:

```
test_dt64arr_add_sub_DateOffsets from pandas.tests.arithmetic.test_datetime64.TestDatetime64DateOffsetArithmetic
```

Test3:

```
pandas/tests/tseries/offsets/test_offsets.py:4196: in _test_all_offsets
    self._test_offset(offset_name=name, offset_n=n, **kwds)
pandas/tests/tseries/offsets/test_offsets.py:4199: in _test_offset
    offset = DateOffset(**{offset_name: offset_n})
pandas/_libs/tslibs/offsets.pyx:976: in pandas._libs.tslibs.offsets.RelativeDeltaOffset.__init__
    off, use_rd = _determine_offset(kwds)
```

Test4 (kwd on argument doesn't end with 's'):

```
    @pytest.mark.parametrize("kwd", sorted(liboffsets._relativedelta_kwds))
    def test_valid_relativedelta_kwargs(kwd):
         # Check that all the arguments specified in liboffsets._relativedelta_kwds
         # are in fact valid relativedelta keyword args
         DateOffset(**{kwd: 1})
```

I think these are test relating to this. But I'm not very familiar to handle them. @arw2019 
This pull request is stale because it has been open for thirty days with no activity. Please update or respond to this comment if you're still interested in working on this.
@GYAHHA sorry for not responding here. Can you rebase again and will take another look at test failures
@GYHHAHA reviving this. I think we should do it. likely need to put `assert_produces_warning(FutureWarning)` around any related tests to get this to pass. move the note to 1.3
ok, I will take this.
This pull request is stale because it has been open for thirty days with no activity. Please update or respond to this comment if you're still interested in working on this.
Thanks for the PR, but it appears to have gotten stale. Going to close for now but let us know if you can merge master and update and we'd be happy to reopen.
Backport PR #36595 on branch 1.1.x (Partial Revert "ENH: infer freq in timedelta_range (#32377)")
State: closed

Backport PR #36595: Partial Revert "ENH: infer freq in timedelta_range (#32377)"

Labels:
name: Timedelta
description: Timedelta data type

Comments:
failure unrelated. see https://github.com/pandas-dev/pandas/pull/36396#issuecomment-695195148
Backport PR #36444 on branch 1.1.x (BUG: inconsistent replace)
State: closed

Backport PR #36444: BUG: inconsistent replace

Labels:
name: replace
description: replace method

Comments:
I don't think we want to backport the changes for the black upgrade.
will merge this for now. and open a separate PR against 1.1.x using the older version of black to tidy up.
CLN: More pytest idioms in pandas/tests/window
State: closed

- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`

Removed some unused variables, added some fixtures for DataFrames used in `test_pairwise.py`, and utilize more `pytest.mark.parameterize`

Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Clean
name: Window
description: rolling, ewma, expanding

Comments:
havent looked through test_pairwise yet, but the other files all LGTM
ASV: used integer ndarray as indexer for Series.take indexing benchmark
State: closed

- [x] closes #18000
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`

Series.take was using a list of booleans as indexers. Now, It is replaced with a random list of integers.
Here are the benchmark results with `asv run --bench indexing.Take`
### Benchmarks Results:

#### Before:

```
[100.00%] ··· indexing.Take.time_take                                                                                                                                                ok
[100.00%] ··· ========== =============
                index                 
              ---------- -------------
                 int      4.13±0.08ms 
               datetime   4.13±0.01ms 
              ========== =============
```

#### After:

```
[100.00%] ··· indexing.Take.time_take                                                                                                                                                ok
[100.00%] ··· ========== =============
                index                 
              ---------- -------------
                 int      6.36±0.08ms 
               datetime   6.29±0.03ms 
              ========== =============
```

Labels:
name: Benchmark
description: Performance (ASV) benchmarks
name: ExtensionArray
description: Extending pandas with custom dtypes or arrays.

Comments:
#### Updated benchmarks after changing indexer from list to ndarray:

```
[100.00%] ··· indexing.Take.time_take                                                                                                                                                ok
[100.00%] ··· ========== =========
                index             
              ---------- ---------
                 int      336±2μs 
               datetime   348±2μs 
              ========== =========
```
thanks @hardikpnsp
BUG: DataFrame.pivot drops column level names when both rows and columns are multiindexed
State: closed

- [x] closes #36360
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry

Fixed by checking for multi-indexed columns in `DataFrameGroupBy._wrap_aggregated_output`

Labels:
name: Bug
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode
name: MultiIndex

Comments:
@jreback not sure you'll accept 390e9a9 (but atm it's needed to pass CI)

in dfc609f we're passing `Index.names` as the `name` kwarg 

This means that for a simple index we end up with `name=[['index_name']]` in the `Index` constructor so I check for that and set `name=['index_name']`
@jreback this one is ready for re-review. I now handle simple index and multiindex together with `_set_names(aggregation_axis.names)`
thanks @arw2019
CLN: dont special-case DatetimeArray indexing
State: closed

- [x] closes #36210
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry

cc @jorisvandenbossche 

Labels:
name: Datetime
description: Datetime data dtype
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves

Comments:
can you add some tests that lock this down (eg from the OP).

I think this would be ok to push for 1.2 with a whatsnew note.

cc @jorisvandenbossche 
whatsnew+test+green
any other thoughts here?  after this i can get back to de-duplicating EA code
Test amended + green
Thanks Brock!
DEPR: min_periods=None behavior for Rolling.count
State: closed

- [x] closes #31302
- [x] closes #35579 
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry

Additionally refactors `count` to take the `_apply` path to make this op consistent with the others.

Labels:
name: Deprecate
description: Functionality to remove in pandas
name: Window
description: rolling, ewma, expanding

Comments:
This will close #35579 too.

You could use
```
def test_fixed_forward_indexer_count():
    # GH: 35579
    df = DataFrame({"b": [None, None, None, 7]})
    indexer = FixedForwardWindowIndexer(window_size=2)
    result = df.rolling(window=indexer, min_periods=0).count()
    expected = DataFrame({"b": [0.0, 0.0, 1.0, 1.0]})
    tm.assert_frame_equal(result, expected)
```
as a test.
Thanks @phofl. Added that unit test to this PR 
thanks @mroeschke
TST: GH23452 test reorder_categories() on categorical index
State: closed

- [x] closes #23452
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry

Couldn't see entries for tests in whatsnew files so I assume it's not needed if only a test case is added This is my first Pandas contribution, looking forward to your feedback.

Labels:
name: Testing
description: pandas testing functions or related to the test suite

Comments:
Requested changes have been applied, is this good to merge now?
Thanks @fokoid 
Sorry to bother you guys - could you please mark this PR with `hacktoberfest-accepted` as per #36837?
ENH: match stdlib behavior for datetimelike comparisons
State: closed

- [x] closes #28507
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry


Labels:
name: Error Reporting
description: Incorrect or improved errors from pandas
name: Timezones
description: Timezone data dtype

Comments:
Thanks @jbrockmendel
TYP: selection and groups type-hinting in groupby
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Groupby
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
can you rebase and ping on green.
/azp run
<samp>
Azure Pipelines successfully started running 1 pipeline(s).<br>

</samp>
@jreback green
CLN: cleanup DataFrameInfo
State: closed

- [ ] closes #xxxx
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry

Slightly clean-up DataFrameInfo.
Make construction of header and separator cleaner.

Labels:
name: Clean

Comments:
Thanks @ivanovmg
TST: GH 32431 check print label indexing on nan
State: closed

- [x] closes #32431 
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry --> not applicable

This is my first contribution to pandas. I added a simple test that was requested in one of the open issues.

Labels:
name: Testing
description: pandas testing functions or related to the test suite

Comments:
This pull request is stale because it has been open for thirty days with no activity. Please update or respond to this comment if you're still interested in working on this.
thanks @wesleyboelrijk
Backport PR #36613 on branch 1.1.x (BUG: Fix unordered cut with Series labels)
State: closed

Backport PR #36613: BUG: Fix unordered cut with Series labels

Labels:
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode

Comments:
web and docs now passing after restart (can't start ci/checks and ci/web and docs independently) may consider splitting.
failure unrelated. see https://github.com/pandas-dev/pandas/pull/36396#issuecomment-695195148
Replace single with double backticks in RST file #36617
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry

closes https://github.com/pandas-dev/pandas/issues/36617

Labels:

Comments:
 Replaces single with double backticks


 doc/source/user_guide/10min.rst
 doc/source/user_guide/timeseries.rst
 doc/source/user_guide/visualization.rst
 doc/source/user_guide/missing_data.rst
 doc/source/user_guide/options.rst
 doc/source/user_guide/reshaping.rst
 doc/source/user_guide/scale.rst
 doc/source/user_guide/merging.rst
 doc/source/user_guide/cookbook.rst
 doc/source/user_guide/enhancingperf.rst
 doc/source/user_guide/groupby.rst
 doc/source/user_guide/io.rst
 doc/source/getting_started/overview.rst
 doc/source/reference/panel.rst
 doc/source/reference/plotting.rst
 doc/source/user_guide/10min.rst
 doc/source/user_guide/basics.rst
 doc/source/user_guide/categorical.rst
 doc/source/development/contributing.rst
 doc/source/development/contributing_docstring.rst
 doc/source/development/extending.rst
 doc/source/ecosystem.rst
 doc/source/getting_started/comparison/comparison_with_sql.rst
 doc/source/getting_started/install.rst
 doc/source/getting_started/intro_tutorials/06_calculate_statistics.rst
@dsaxton Can I ask why this test is failing?
Is this Issue solved?
Hi @dsaxton , I've removed many unnecessary backticks
now I cann't find problems that are failing the test.
can you tell me what other problems I'd made in the commits
Can you also merge master? Several files were just fixed.
@dsaxton thank you. done the changes
@dsaxton thank you for your help :smile: 
love to contribute something to pandas
CLN:replace single with double backticks
State: closed

#36617


    doc/source/development/contributing.rst
    doc/source/development/contributing_docstring.rst
    doc/source/development/extending.rst
    doc/source/ecosystem.rst
    doc/source/getting_started/install.rst
    doc/source/getting_started/comparison/comparison_with_sql.rst
    doc/source/getting_started/intro_tutorials/06_calculate_statistics.rst
    doc/source/getting_started/intro_tutorials/08_combine_dataframes.rst
    doc/source/user_guide/categorical.rst


Labels:

Comments:
@satrio-hw The doc build for this seems to be failing, note that cross-referencing syntax like :class:\`Series\` _does_ use single backticks: https://www.sphinx-doc.org/en/master/usage/restructuredtext/roles.html#xref-syntax
Thanks @satrio-hw for the PR but the issue is now fixed
TST: check whether printing an IntervalIndex works (GH32553)
State: closed

- [x] closes #32553
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry (not applicable)

I just added a test that checks whether print succeeds, would we also want to check the exact output of the `str` function on an `IntervalIndex`?


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Stale

Comments:
This pull request is stale because it has been open for thirty days with no activity. Please update or respond to this comment if you're still interested in working on this.
This pull request is stale because it has been open for thirty days with no activity. Please update or respond to this comment if you're still interested in working on this.
Closing in favor of #38242
CI: add py38 slow build #35160
State: closed

- [x] closes #35160

This issue was created before we dropped py36 support. So I think we should add equivalent CI build for py38.



Labels:
name: CI
description: Continuous Integration

Comments:
can u out this on azure instead? we have more parallelism there - and they are slightly less opaque than travis 
> can u out this on azure instead? we have more parallelism there - and they are slightly less opaque than travis

I'll look at how to set up sql on azure. It's more straightforward to do this on Travis. 
@fangchenli another alternative is to move one of the other travis builds to azure if that's easier. 
can you rebase; also is the 3.7_locale duplicated in azure & travis? (or is it different)?
> can you rebase; also is the 3.7_locale duplicated in azure & travis? (or is it different)?

The one on travis has sql related tests. Maybe we can combine those two.
ok this lgtm. can you merge master to retrigger and ping on green.
thanks @fangchenli
CLN: de-duplicate numeric type check in _libs/testing
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry

In `testing.pyx` we keep a list of numeric dtypes and use that for type checking. But we have functions that do this in `tslibs/util.pxd` so it's better to use those instead

Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Clean

Comments:
looks good, still some lint issues, ping on green (small comment)
Green
@arw2019 can you merge master
Sorry to have been quiet!

Merged again just now - all green
thanks @arw2019
CLN: simplify interpolate_2d and callers
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Missing-data
description: np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate
name: Clean
name: API - Consistency
description: Internal Consistency of API/Behavior

Comments:
> or can do it after when all non-named keyword args are banned

yah lets save it for the end
BUG: Categorical setitem, comparison with tuple category
State: closed

- [x] closes #20439
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry


Labels:
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: Categorical
description: Categorical Data Type

Comments:
> can you also check getitem?

not sure i follow.  you mean check that `cat[i]` gives a tuple?  pretty sure we already have tests for that
> > can you also check getitem?
> 
> not sure i follow. you mean check that `cat[i]` gives a tuple? pretty sure we already have tests for that

ok then.
Comment on stale PRs
State: closed

Updating the stale PR action to comment in the PR. Should help to automate the process of pinging contributors who go quiet. Also updating to run once daily instead of every six hours.

Labels:
name: CI
description: Continuous Integration

Comments:
/azp run
<samp>
Azure Pipelines successfully started running 1 pipeline(s).<br>

</samp>
lgtm . so this runs once daily but how often does it comment?
> lgtm . so this runs once daily but how often does it comment?

It should comment only when it marks something as stale, and if something is already marked it'll just skip it (so ideally it'll comment at most once on a PR, unless something is going in and out of a stale state)
Cool - let's see how this goes
CLN: Break up aggregate.transform
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry

Precursor to #36478. Just breaks up transform without any code changes.

Labels:
name: Clean
name: Apply
description: Apply, Aggregate, Transform, Map

Comments:
Thanks for the suggestion @arw2019. In this particular case, I think not. In #36478, we'll change e.g. `isinstance(func, dict)` to using `is_dict_like`. I believe that makes singledispatch not viable.
> Thanks for the suggestion @arw2019. In this particular case, I think not. In #36478, we'll change e.g. `isinstance(func, dict)` to using `is_dict_like`. I believe that makes singledispatch not viable.

Right, that makes sense.

Another question, should `transform_dict_like` and `transform_str_or_callable` be underscored? Since AFAICT they're private methods
> > Thanks for the suggestion @arw2019. In this particular case, I think not. In #36478, we'll change e.g. `isinstance(func, dict)` to using `is_dict_like`. I believe that makes singledispatch not viable.
> 
> Right, that makes sense.
> 
> Another question, should `transform_dict_like` and `transform_str_or_callable` be underscored? Since AFAICT they're private methods

this entire hierarchy is private pandas/core/* (yes it doesn't actually look like it, but its been document this way for a long time)
thanks @rhshadrach 

happy to have typing passes as well
Revert "[BUG]: Fix regression when adding timeldeta_range to timestamp"
State: closed

Reverts pandas-dev/pandas#36582

Labels:
name: Timedelta
description: Timedelta data type

Comments:
merge on green
unrelated failure xref #36467

```
=========================== short test summary info ============================
FAILED pandas/tests/io/test_html.py::TestReadHtml::test_banklist_url_positional_match[bs4]
= 1 failed, 76281 passed, 1021 skipped, 1167 xfailed, 331 warnings in 1651.71s (0:27:31) =
```
BUG: Fix unordered cut with Series labels
State: closed

- [x] closes #36603
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry


Labels:
name: Bug
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode

Comments:
@meeseeksdev backport 1.1.x