BrandonFors/pandas-github-issues huggingface dataset:
paste_windows() wrong arg for c_wchar_p
State: closed

paste_windows() now directly used handle return from safeGetClipboardData(CF_UNICODETEXT) as argument for c_wchar_p, which should used safeGlobalLock(handle) instead, while copy_windows(text) doing the right thing.

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Windows
description: Windows OS
name: Stale

Comments:
@folger Is there an issue for this? Usually the first step would be to create one if you feel the code needs to change. Also you'll want to add a test and release note to this PR.
> @folger Is there an issue for this? Usually the first step would be to create one if you feel the code needs to change. Also you'll want to add a test and release note to this PR.

@dsaxton I have not created issue for this yet. But it can only be reproduced by python embedded in another application. Not sure how to add a testing case.
this is vendored from Pyperclip; have they made this change upstream?
yeah we need this to be pushed upstream, then we can re-vendor a copy here.
This pull request is stale because it has been open for thirty days with no activity. Please update or respond to this comment if you're still interested in working on this.
closing. this needs to be pushed upstream first (and then we can update our vendored version).
CLN: resolve UserWarning in `pandas/plotting/_matplotlib/core.py` #35945
State: closed

- [x] closes #35945
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`


Labels:
name: Visualization
description: plotting
name: Clean

Comments:
> thanks for the PR, @VirosaLi
> 
> just curious where is this warning getting caught or is the warning raised on a newer matplotlib version, do we have any tests right now to catch the warning?

pandas/tests/plotting/test_frame.py: 10 warnings
pandas/tests/plotting/test_series.py: 5 warnings
  /home/fangchenli/Workspace/pandas-VirosaLi/pandas/plotting/_matplotlib/core.py:1235: UserWarning: FixedFormatter should only be used together with FixedLocator
    ax.set_xticklabels(xticklabels)

@VirosaLi so the warning is now gone? I don't think we actually need a test for this, but if you can put an assert_produces_warning(None) around something and have it pass would be great.
the first one will be resolved once https://github.com/pandas-dev/pandas/pull/36085 gets merged.
thanks @VirosaLi 
@meeseeksdev backport 1.1.x
Can't Dooooo.... It seem like this is already backported (commit is empty).I won't do anything. MrMeeseeks out.
Version Number correction in to_json table
State: closed

Added a line of code to obtain version of Pandas and display it in Schema of a table json when to_json(orient='table') is used.

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:

Comments:
Perhaps a comment explaining that this is the version of the json schema and should only be updated if the schema is changed.
> why are ou adding this?
> the point is this is the format of the schema which has not been changed in a while

I read it after the fact. I will close PR.
CLN: resolve DeprecationWarning in `pandas/_testing.py` #35942
State: closed

- [x] closes #35942
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Stale

Comments:
Thanks for the PR. Can you confirm that this is the correct fix? On master I see this

```
> /Users/taugspurger/sandbox/pandas/pandas/_testing.py(790)assert_index_equal()
-> diff = np.sum((left.values != right.values).astype(int)) * 100.0 / len(left)
(Pdb) left
Index([<NA>, NaT], dtype='object')
(Pdb) left.values
array([<NA>, numpy.datetime64('NaT')], dtype=object)
(Pdb) right.values
array([<NA>, NaT], dtype=object)
(Pdb) pp np.no
np.nonzero    np.not_equal
(Pdb) pp np.not_equal(left.values, right.values)
*** TypeError: boolean value of NA is ambiguous
```

Are we raising a TypeError but not seeing it since the test is xfailed?
> Thanks for the PR. Can you confirm that this is the correct fix? On master I see this
> 
> ```
> > /Users/taugspurger/sandbox/pandas/pandas/_testing.py(790)assert_index_equal()
> -> diff = np.sum((left.values != right.values).astype(int)) * 100.0 / len(left)
> (Pdb) left
> Index([<NA>, NaT], dtype='object')
> (Pdb) left.values
> array([<NA>, numpy.datetime64('NaT')], dtype=object)
> (Pdb) right.values
> array([<NA>, NaT], dtype=object)
> (Pdb) pp np.no
> np.nonzero    np.not_equal
> (Pdb) pp np.not_equal(left.values, right.values)
> *** TypeError: boolean value of NA is ambiguous
> ```
> 
> Are we raising a TypeError but not seeing it since the test is xfailed?

There are 4 xfail at `pandas/tests/indexes/test_index_new.py:99`. Before the change, it was an
`AttributeError: 'bool' object has no attribute 'astype'`. After the change, it became `TypeError: boolean value of NA is ambiguous`.
Right, do we want that TypeError? What's the desired behavior?
> Right, do we want that TypeError? What's the desired behavior?

Not sure about this. Might need some discussion. xref #31884
This pull request is stale because it has been open for thirty days with no activity. Please update or respond to this comment if you're still interested in working on this.
DOC: complement the documentation for pandas.DataFrame.agg #35912
State: closed

- [x] closes #35912
- [x] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry
<img width="676" alt="Screenshot 2020-08-28 at 11 59 23" src="https://user-images.githubusercontent.com/21543236/91519967-a71dd280-e926-11ea-87f5-e647fe650168.png">


Labels:
name: Docs

Comments:
Hello @onshek! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:



There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2020-08-29 07:56:56 UTC
Sorry for closing it by mistake :)
Can anyone help me to figure out why two new checks failed here (This PR passed all 15/15 checks two days ago and I didn't make new changes)? Thanks! @MarcoGorelli @charlesdong1991 @rhshadrach 
> Can anyone help me to figure out why two new checks failed here (This PR passed all 15/15 checks two days ago and I didn't make new changes)? Thanks! @MarcoGorelli @charlesdong1991 @rhshadrach

Failures are unrelated, if you merge master and push again I'd expect them to be fixed
> > Can anyone help me to figure out why two new checks failed here (This PR passed all 15/15 checks two days ago and I didn't make new changes)? Thanks! @MarcoGorelli @charlesdong1991 @rhshadrach
> 
> Failures are unrelated, if you merge master and push again I'd expect them to be fixed

Got it, thanks!
Thanks @onshek. The CI failure is unrelated
```

============================= test session starts ==============================
platform linux -- Python 3.8.5, pytest-6.0.1, py-1.9.0, pluggy-0.13.1
rootdir: /home/vsts/work/1/s, configfile: setup.cfg, testpaths: pandas
plugins: hypothesis-5.28.0, forked-1.2.0, asyncio-0.12.0, xdist-2.1.0, azurepipelines-0.8.0
gw0 I / gw1 I
gw0 [76009] / gw1 [76009]

##[error]1 test(s) failed, 76009 test(s) collected.
Skipping uploading of coverage data.

=================================== FAILURES ===================================
_____________________ TestIndex.test_tab_complete_warning ______________________

```
TYP: annotations in core.groupby
State: closed

I'm still seeing a couple of mypy complaints, suggestions @simonjayhawkins ?

Labels:
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
> I'm still seeing a couple of mypy complaints, suggestions @simonjayhawkins ?

pandas\core\groupby\categorical.py:101: error: "CategoricalIndex" has no attribute "set_categories"; maybe "categories"?  [attr-defined]
pandas\core\groupby\categorical.py:104: error: "CategoricalIndex" has no attribute "add_categories"; maybe "categories"?  [attr-defined]

These are because mypy is a static type checker and these methods are dynamically added to CategoricalIndex

in `DatetimeIndex`, tz_localize and to_period were added statically https://github.com/pandas-dev/pandas/pull/34630#discussion_r436402735

I'm OK with # type: ignores on this pass (with error code) 

error: Argument 1 to "map" has incompatible type overloaded function; expected "Callable[[object], Iterator[_T]]"  [arg-type]

not sure on this one.

mypy is happy with
```
        for name, lev, in_axis in zip(
            reversed(self.grouper.names),
            reversed(self.grouper.get_group_levels()),
            reversed([grp.in_axis for grp in self.grouper.groupings]),
        ):
```
which I think is easier to grok anyhow.
> error: Argument 1 to "map" has incompatible type overloaded function; expected "Callable[[object], Iterator[_T]]" [arg-type]
> 
> not sure on this one.

see https://github.com/python/mypy/issues/6811#issuecomment-491332516

>As for the originally reported bug, it also repros with this simpler code:

>```
>def get_common_index(*names: str) -> int:
>    x = map(reversed, names)
>    return 0
>```


Thanks @jbrockmendel
REGR: Fix comparison broadcasting over array of Intervals
State: closed

- [x] closes https://github.com/pandas-dev/pandas/issues/35931
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry


Labels:
name: Regression
description: Functionality that used to work in a prior pandas version
name: Interval
description: Interval data type

Comments:
Looks like `Interval.__richcmp__` returns `NotImplemented` in nearly all cases.  I think if we handle `util.is_array(other)` cases directly that might solve this problem without breaking addition
> Looks like `Interval.__richcmp__` returns `NotImplemented` in nearly all cases. I think if we handle `util.is_array(other)` cases directly that might solve this problem without breaking addition

Nice, I think that's working. So far I only did a minimal copy / paste from another Cython class and stuff is passing, but am I missing some cases you can think of?
LGTM
thanks @dsaxton very nice!
@meeseeksdev backport 1.1.x
BUG: BlockSlider not clearing index._cache
State: closed

- [ ] closes #xxxx
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry


Labels:
name: Internals
description: Related to non-user accessible pandas implementation

Comments:
lgtm. if you can rebase, ping on green.
ping
REGR: Fix inplace updates on column to set correct values
State: closed

Closes #35731

Labels:
name: Missing-data
description: np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate
name: Regression
description: Functionality that used to work in a prior pandas version
name: Categorical
description: Categorical Data Type

Comments:
LGTM pending green

I'd also be OK with chasing the unwrapping up the call chain to unwrap earlier
thanks @jorisvandenbossche 
@meeseeksdev backport 1.1.x
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
$ git checkout 1.1.x
$ git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
$ git cherry-pick -m1 b45327f5fd262e835a1007587c2882c3d77b3158
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
$ git commit -am 'Backport PR #35936: REGR: Fix inplace updates on column to set correct values'
```

4. Push to a named branch :

```
git push YOURFORK 1.1.x:auto-backport-of-pr-35936-on-1.1.x
```

5. Create a PR against branch 1.1.x, I would have named this PR:

> "Backport PR #35936 on branch 1.1.x"

And apply the correct labels and milestones.

Congratulation you did some good work ! Hopefully your backport PR will be tested by the continuous integration and merged soon!

If these instruction are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
Inconsistencies between python/cython groupby.agg behavior
State: closed

This is pretty ugly, but tentatively is sufficient to make #34997 pass.

The upshot is that we have two problems:

1) in libreduction setting `setattr(cached_ityp, '_index_data',  islider.buf)` silently does the wrong thing for EA-backed indexes
2) when we go through the non-libreduction path, we do things slightly differently, which requires more patches to get tests passing.

cc @WillAyd


Labels:
name: Groupby

Comments:
> you are right, this is not pretty.

it started off even worse.

> does this have a test that is added that doesn't work now?

No.  The motivation is to get #34997 working (which is needed in order to upgrade to cython3 when its eventually released)
This is also prerequisite to killing off the _index_data kludge
if u can rebase will have a look
rebased per request
> do we have tests that show the inconsistency now? and how does this fix it?

This does not fix anything that is broken _in master_.  But under #34997 (which is needed for cy3 compat), some paths that currently go through cython will instead go through python, and would _then_ fail.

> > do we have tests that show the inconsistency now? and how does this fix it?
> 
> This does not fix anything that is broken _in master_. But under #34997 (which is needed for cy3 compat), some paths that currently go through cython will instead go through python, and would _then_ fail.

maybe i am not following everything, but do we *need* to use cy3 in 3.9?
> maybe i am not following everything, but do we need to use cy3 in 3.9?

i hope not; cy3 isnt even out yet
> > maybe i am not following everything, but do we need to use cy3 in 3.9?
> 
> i hope not; cy3 isnt even out yet

so we *don't* need this rn then?
> so we don't need this rn then?

correct AFAIK
closing in favor of #34997
REF: window/test_dtypes.py with pytest idioms
State: closed

- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`


Labels:
name: Refactor
description: Internal refactoring of code
name: Testing
description: pandas testing functions or related to the test suite

Comments:
Hello @mroeschke! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:





There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2020-09-05 17:17:13 UTC
thanks @mroeschke
Make MultiIndex.get_loc raise for unhashable type
State: closed

- [x] closes #35878
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry


Labels:
name: Regression
description: Functionality that used to work in a prior pandas version
name: MultiIndex

Comments:
@meeseeksdev backport 1.1.x
CI: Attempt to unpin pytest-xdist
State: closed

- [x] closes #35756

2.1.0 was released yesterday: https://pypi.org/project/pytest-xdist/#history

Labels:
name: CI
description: Continuous Integration

Comments:
/azp run
<samp>
Azure Pipelines successfully started running 1 pipeline(s).<br>

</samp>
restarting CI to check no flakeyness. 
Thanks @alimcmaster1
Truncate columns list to match tr_frame for correct dict formatters lookup
State: closed

When using a dictionary of formatters and a truncated DataFrame is being displayed, the column list used for looking up the formatter in the dict are incorrect.  The columns attribute of the DataFrameFormatter needs to be truncated as well for the lookup to be correct.

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Output-Formatting
description: __repr__ of pandas objects, to_string

Comments:
I'm not sure that overwriting self.columns is the way to go here - is there a way to localize the fix?
I think the new pull request localizes it down to just the spot where the fix is needed.  My original fix was based on a monkey patch I was using in my library to make it work in older versions of pandas.
Hello @kesmit13! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:





There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2021-03-15 14:00:51 UTC
This pull request is stale because it has been open for thirty days with no activity. Please update or respond to this comment if you're still interested in working on this.
AFAIK, I've addressed all of the issues. I'm just waiting on final approval.
@kesmit13 yeah can you merge master and fix conflict.
Regarding solving the conflicts.
After recent changes in ``pandas/io/format/format.py`` it will be necessary to move changes in ``_get_formatter`` method into the same method in class ``DataFrameFormatter``.
And you probably won't need to type ``tr_frame``.
@kesmit13, can you please merge master and resolve conflicts in ``doc/source/whatsnew/v1.2.0.rst``?
Without it we cannot witness CI checks.
Thank you!
> I guess that your whatsnew entry should go now to v1.3.

@kesmit13 can you address @ivanovmg comment


can you merge master and will have a look (also note would need to be in 1.4)
Thanks for the PR, but it appears to have gone stale. This appears close, so if you could merge master and address the remaining comments, we'd be happy to reopen and have another look over.
TST/API: test column indexing copy/view semantics
State: closed

This is adding some tests for current behaviour related to the discussions in #33457 and #35417 (and also adds the tests from https://github.com/pandas-dev/pandas/pull/35266 which got never merged). I think it is good to have some more explicit tests of current behaviour, while we are considering changing things.

Note: some semantics I am testing are certainly debatable, but I was experimenting a bit with what is happening on current master.

Labels:
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: ExtensionArray
description: Extending pandas with custom dtypes or arrays.
name: Copy / view semantics
name: Stale

Comments:
cc @jbrockmendel @TomAugspurger @jreback 
> some semantics I am testing are certainly debatable, but I was experimenting a bit with what is happening on current master.

I'm generally positive on this PR, lean towards only testing explicitly-desired behavior
> lean towards only testing explicitly-desired behavior

And do you find the currently tested behaviours desired?
This pull request is stale because it has been open for thirty days with no activity. Please update or respond to this comment if you're still interested in working on this.
@jorisvandenbossche can you merge master and address comments
@jorisvandenbossche closing as stale. reopen when ready.
REF: use BlockManager.apply for cython_agg_blocks, apply_blockwise
State: closed

- [x] closes #34714
- [ ] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Refactor
description: Internal refactoring of code
name: Internals
description: Related to non-user accessible pandas implementation

Comments:
rebased+green
CI: docker 32-bit linux build #32709
State: closed

- [x] closes #32709



Labels:
name: CI
description: Continuous Integration
name: 32bit
description: 32-bit systems

Comments:
Thanks @VirosaLi. Are you able to integrate this into our existing setup at all? See how we handle the Python 3.9 build (e.g. https://github.com/pandas-dev/pandas/blob/master/ci/setup_env.sh#L3-L6 and elsewhere).
This pull request is stale because it has been open for thirty days with no activity. Please update or respond to this comment if you're still interested in working on this.
@fangchenli can you update this. would be great to get this to work (xfailing the tests as needed of course). W/o this or similar we are going to have a very hard time continuing to support 32-bit.
Windows py38_np18 had stalled. restarted.

the following tests are failing on the 32bit wheel builds

```

=========================== short test summary info ===========================
FAILED test_venv/lib/site-packages/pandas/tests/groupby/test_groupby.py::test_groupby_nat_exclude
FAILED test_venv/lib/site-packages/pandas/tests/io/formats/test_info.py::test_info_int_columns
FAILED test_venv/lib/site-packages/pandas/tests/window/test_grouper.py::TestGrouperGrouping::test_groupby_rolling_string_index
FAILED test_venv/lib/site-packages/pandas/tests/window/test_grouper.py::TestGrouperGrouping::test_groupby_rolling_no_sort
= 4 failed, 64345 passed, 7420 skipped, 1047 xfailed, 6 xpassed, 14 warnings in 932.75s (0:15:32) =
```

so will want to merge some of the changes here into 1.1.x.

@jreback could wait till this is approved and merged and do a partial backport or open a separate PR now directly against 1.1.x with the changes to those tests here.
thanks a lot @fangchenli this is great
@simonjayhawkins pls backport what u need against 1.1.x
@meeseeksdev backport 1.1.x
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
$ git checkout 1.1.x
$ git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
$ git cherry-pick -m1 d89331b96e919acc2a83f7f6201830d246018e36
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
$ git commit -am 'Backport PR #35898: CI: docker 32-bit linux build #32709'
```

4. Push to a named branch :

```
git push YOURFORK 1.1.x:auto-backport-of-pr-35898-on-1.1.x
```

5. Create a PR against branch 1.1.x, I would have named this PR:

> "Backport PR #35898 on branch 1.1.x"

And apply the correct labels and milestones.

Congratulation you did some good work ! Hopefully your backport PR will be tested by the continuous integration and merged soon!

If these instruction are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
CI: Mark s3 tests parallel safe
State: closed

Closes https://github.com/pandas-dev/pandas/issues/35856

I think we need to update the pytest pattern though, so this should
fail.

Labels:
name: CI
description: Continuous Integration
name: Unreliable Test
description: Unit tests that occasionally fail

Comments:
> so this should fail.

Does this mean that this PR should not be merged despite being green?
I'm going to rerun it a few times. I suspect we'll get a failure.
Updated to just make these fixtures parallel safe. At the start of each session, each worker starts up a moto server on `localhost:555{worker_id}`. Then we just need to plumb that through everywhere.

I think Martin mentioned possible memory issues on windows workers with this approach, but that may have been starting the moto server at the test level. Hopefully doing things at the session level avoids that.
Rerunning azure at https://dev.azure.com/pandas-dev/pandas/_build/results?buildId=40904&view=results. But we should be good.
https://github.com/pandas-dev/pandas/pull/35655 wasn't backported, so this shouldn't need a backport either.
I've seen a couple failures on re-runs, but nothing to do with s3 / moto. So this should be good to go.
> I think Martin mentioned possible memory issues on windows workers

Yes, this was for the case of restarting the process for every test (i.e., standard pytest scope) - I believe moto may be less than optimal at cleaning up on exit, at least on Windows.