BrandonFors/pandas-github-issues huggingface dataset:
Asset Frame with duplicate index
State: closed

- [ ] closes #26494
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Hey guys, I started working on a way to make testing.assert_frame_equal accept an argument so it can test equals a frame with duplicate index just like the issue I marked to close here. For this I had to make a couple of changes in some functions and I think I got to the point that I should ask for a better direction and if this is valid for the project.

The error pointed in the issue is definitely not the best message for this case. I got to the point to not get that error by making clear there is a duplicate index. But I got stuck in this error now:

`ValueError: Shape of passed values is (5, 1), indices imply (3, 1)`


Labels:
name: Testing
description: pandas testing functions or related to the test suite

Comments:
Hello @gabriellm1! Thanks for opening this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:

* In the file [`pandas/core/frame.py`](https://github.com/pandas-dev/pandas/blob/471c7cec3cda92fb6d23336213aa7bfe982b0c56/pandas/core/frame.py):

> [Line 3743:89](https://github.com/pandas-dev/pandas/blob/471c7cec3cda92fb6d23336213aa7bfe982b0c56/pandas/core/frame.py#L3743): [E501](https://duckduckgo.com/?q=pep8%20E501) line too long (97 > 88 characters)

* In the file [`pandas/core/generic.py`](https://github.com/pandas-dev/pandas/blob/471c7cec3cda92fb6d23336213aa7bfe982b0c56/pandas/core/generic.py):

> [Line 3832:89](https://github.com/pandas-dev/pandas/blob/471c7cec3cda92fb6d23336213aa7bfe982b0c56/pandas/core/generic.py#L3832): [E501](https://duckduckgo.com/?q=pep8%20E501) line too long (104 > 88 characters)

* In the file [`pandas/util/testing.py`](https://github.com/pandas-dev/pandas/blob/471c7cec3cda92fb6d23336213aa7bfe982b0c56/pandas/util/testing.py):

> [Line 1348:46](https://github.com/pandas-dev/pandas/blob/471c7cec3cda92fb6d23336213aa7bfe982b0c56/pandas/util/testing.py#L1348): [E231](https://duckduckgo.com/?q=pep8%20E231) missing whitespace after ','


Hmm not sure I am on board with making this change. The point of `check_like` is to ignore position but align on label. If labels are duplicated how would that alignment work?
Hmmm now I see the problem. There would be multiple rows to align and check if equals. In a big dataframe that alignment solution would probably take too long to run. So probably there isn't a solution for this right?
Yea I think the alignment here as to what should be checked is ambiguous. Appreciate the PR as always but I don't think this one is worth pursuing (if anyone disagrees feel free to reopen)
Debug 38 build
State: closed

Seeing if this helps with 38 build hanging until something more permanent comes along

@TomAugspurger @jorisvandenbossche 

Labels:

Comments:
Adding a verbose flag seemed to work here. I *think* the issue is just that there is no output coming from setup.py during install and it takes over 10 minutes. We could roll with this and see how it goes (knowing this is a temporary build solution anyway) or look into global settings, though I doubt we care to increase timeouts for other builds:

https://docs.travis-ci.com/user/common-build-problems/#build-times-out-because-no-output-was-received

makes sense, lgtm
@TomAugspurger thoughts?  might fix the travis 38 builds
Thanks @WillAyd!
ENH: Add StringArray.__arrow_array__ for conversion to Arrow
State: closed

Similar as https://github.com/pandas-dev/pandas/pull/28368 (which did this for IntegerArray)

Labels:
name: Dtype Conversions
description: Unexpected or buggy dtype conversions
name: ExtensionArray
description: Extending pandas with custom dtypes or arrays.

Comments:
The failing check is codecov, so ignoring that
> The failing check is codecov, so ignoring that

However, we should probably ensure we are testing somewhere pyarrow 0.15 / latest release (from a quick look that doesn't seem to be included on any build). Will open an issue for that.
TST: Test Series integer mod behavior on string
State: closed

Closes https://github.com/pandas-dev/pandas/issues/24396

Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Numeric Operations
description: Arithmetic, Comparison, and Logical operations
name: Series
description: Series data structure

Comments:
lgtm but cc @jbrockmendel for verification.
lgtm
could parametrize over Series/Index/DataFrame if you wanted.  not a big deal
Thanks @gfyoung
Removed raise_from_traceback
State: closed

ref #29170

This is a pretty old compat function that I think actually hides the errors we are looking for when run via load scheduling. I've converted to a mixture of `raise...from` and `raise (e).with_traceback()` depending on what was available. Not sure it matters too much either way

Let's see...

Labels:
name: Error Reporting
description: Incorrect or improved errors from pandas

Comments:
Looks like this does actually preserve the traceback:

https://travis-ci.org/pandas-dev/pandas/jobs/601566770#L2267
unused sys import
Comments addressed and all green here
LGTM
normally on a deprecation removal we should have the whatsnew note along with all of the changes in the same PR as otherwise it’s harder to track / follow



Nice to see this removed as a result of our move to `pytest` !
Fixed segfaults and incorrect results in GroupBy.quantile with NA Values in Grouping
State: closed

- [X] closes #28882
- [X] tests added / passed
- [X] passes `black pandas`
- [X] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [X] whatsnew entry


Labels:
name: Groupby
name: Segfault
description: Non-Recoverable Error
name: quantile
description: quantile method

Comments:
All green on this. FWIW I think worth a 0.25.3 release since its a regression and happy to volunteer for that if others agree
lgtm cc @jreback
> All green on this. FWIW I think worth a 0.25.3 release since its a regression and happy to volunteer for that if others agree

@jorisvandenbossche @TomAugspurger 

ok by me
Doesn't matter to me. How far are we from 1.0? ~1 month?

Just FYI Will, there are some strange issues with the pdf build in the
docker container. I had to build it outside of docker. I can probably help
out with that if needed.

On Wed, Oct 30, 2019 at 7:11 AM Jeff Reback <notifications@github.com>
wrote:

> All green on this. FWIW I think worth a 0.25.3 release since its a
> regression and happy to volunteer for that if others agree
>
> @jorisvandenbossche <https://github.com/jorisvandenbossche> @TomAugspurger
> <https://github.com/TomAugspurger>
>
> ok by me
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> <https://github.com/pandas-dev/pandas/pull/29173?email_source=notifications&email_token=AAKAOIXMOSBTY5Y5X7MA6EDQRF2WXA5CNFSM4JDZGOT2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOECT5WMA#issuecomment-547871536>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AAKAOIRBDUK6PSMJ5SRYG4TQRF2WXANCNFSM4JDZGOTQ>
> .
>

Sounds good. I'll do the back port and start on the release today / tomorrow. I'll reach out as questions come up
Remove TestData in frame tests in multiple files 
State: closed

Part of #22471 

This is removing/replacing TestData usage in the following files:
* `test_convert_to.py`
* `test_nonunique_indexes.py`
* `test_query_eval.py`
* `test_replace.py`
* `test_repr_info.py`


--------
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`


Labels:
name: Testing
description: pandas testing functions or related to the test suite

Comments:
I took care of all comments and as suggested by jreback I'll create a follow-up PR to resolve the remaining `TestData` usage in `test_query_eval.py`.

@WillAyd @jreback Can you please take another look? Thanks!
thanks @SaturnFromTitan
TST: Document and test na_filter in read_excel
State: closed

Closes https://github.com/pandas-dev/pandas/issues/25453

Picks up where https://github.com/pandas-dev/pandas/pull/26353 left off

Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Docs
name: IO Excel
description: read_excel, to_excel

Comments:
Thanks @gfyoung
CI: Avoid passing attrs=None
State: closed

Trying to fix the allowed-failure build on Travis; I haven't been able to reproduce that error locally, so this is a shot in the dark.

Labels:

Comments:
Any chance getting rid of `raise_with_traceback` helps identify? Seems like that is a Python2/3 compat thing so might be making things worse
I tend to agree.  Hard to tell until I find a way to reproduce locally
Closing in favor of #29174
Clarify referred command for flake8 caveat
State: closed

Follow-up to: https://github.com/pandas-dev/pandas/pull/27233

With all of the content in-between about pre-commit hooks, the phrase `"This command"` does not make sense any more in the documentation about this `flake8` caveat.

Labels:
name: Docs
name: Code Style
description: Code style, linting, code_checks

Comments:
Great thanks @gfyoung !
REF: avoid getattr pattern in libgroupby rank functions
State: closed

cc @WillAyd do we need to do anything special w/r/t the comment on L1220 re #19560?

Labels:
name: Refactor
description: Internal refactoring of code
name: Internals
description: Related to non-user accessible pandas implementation

Comments:
> cc @WillAyd do we need to do anything special w/r/t the comment on L1220 re #19560?

Is this changing behavior when trying to rank objects? I think on master raises an error

```python
>>> pd.DataFrame([[1]], dtype=object).groupby([1]).rank()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/williamayd/clones/pandas/pandas/core/groupby/groupby.py", line 2144, in rank
    axis=axis,
  File "/Users/williamayd/clones/pandas/pandas/core/groupby/groupby.py", line 867, in _cython_transform
    result, _ = self.grouper.transform(obj.values, how, **kwargs)
  File "/Users/williamayd/clones/pandas/pandas/core/groupby/ops.py", line 621, in transform
    return self._cython_operation("transform", values, how, axis, **kwargs)
  File "/Users/williamayd/clones/pandas/pandas/core/groupby/ops.py", line 583, in _cython_operation
    result, values, labels, func, is_numeric, is_datetimelike, **kwargs
  File "/Users/williamayd/clones/pandas/pandas/core/groupby/ops.py", line 658, in _transform
    transform_func(result, values, comp_ids, ngroups, is_datetimelike, **kwargs)
  File "/Users/williamayd/clones/pandas/pandas/core/groupby/ops.py", line 441, in wrapper
    return f(afunc, *args, **kwargs)
  File "/Users/williamayd/clones/pandas/pandas/core/groupby/ops.py", line 393, in <lambda>
    kwargs.get("na_option", "keep"),
TypeError: 'NoneType' object is not callable
```

Though not sure how well tested that is. Ideally would be explicitly disallowed but looks like that PR is still open, so might want to be careful when changing this
> Is this changing behavior when trying to rank objects? I think on master raises an error

The same example now raises `pandas.core.base.DataError: No numeric types to aggregate`
thanks @jbrockmendel
TYPING : Series.name -> Optional[Hashable]
State: closed

xref https://github.com/pandas-dev/pandas/pull/29062#discussion_r337644656

cc @WillAyd @TomAugspurger 

Labels:
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
No problem with changing but what caused this issue exactly? Something we need to account for with pickle compat?
> No problem with changing but what caused this issue exactly? Something we need to account for with pickle compat?

no. with check_untyped_defs (i.e. checks body of _unpickle_series_compat)..

`None` is assigned to `name`: `index, name = own_state[0], None`

and then assigned to `self.name`: `self.name = name`


Thanks!
Update setup.py
State: closed

jinja2 in the install_requires list


Labels:

Comments:
Sorry, I made a mistake. You can close this PR.

Tiarles da Rocha Moralles Guterres

Engenheiro de Computação
Mestrando do Programa de Pós-graduação em Engenharia Elétrica
Membro IEEE da Robotics and Automation Society
Universidade Federal de Santa Maria

________________________________
De: Jeff Reback <notifications@github.com>
Enviado: Tuesday, October 22, 2019 3:05:36 PM
Para: pandas-dev/pandas <pandas@noreply.github.com>
Cc: Tiarles Guterres <tiarlesmoralles@hotmail.com>; Author <author@noreply.github.com>
Assunto: Re: [pandas-dev/pandas] Update setup.py (#29163)


@jreback requested changes on this pull request.

what is the purpose of this PR?

and why do you think it is necessary

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub<https://github.com/pandas-dev/pandas/pull/29163?email_source=notifications&email_token=ADKEG6PPRTNQODWXIRT4KMDQP46HBA5CNFSM4JDTFE52YY3PNVWWK3TUL52HS4DFWFIHK3DMKJSXC5LFON2FEZLWNFSXPKTDN5WW2ZLOORPWSZGOCI2BLGQ#pullrequestreview-305403290>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ADKEG6L7GJAJC7PRGKLQIU3QP46HBANCNFSM4JDTFE5Q>.
Period 29073
State: closed

- [ ] closes #29073
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Docs
name: Period
description: Period data type

Comments:
One of the CI checks failed so I closed and was going to work back through it but looking at the checks for recent commits to Master this might be a Travis issue so reopening. 
Looks like the failure is trailing whitespace on line 1781
hmm thanks, fixed. Initially the check that failed was that Travis timed out while compiling. 

Is it normal for commits to master to be showing CI check fails? When i saw that, I assumed there was something going on and that I should just try again later.  
> Is it normal for commits to master to be showing CI check fails? When i saw that, I assumed there was something going on and that I should just try again later.

We recently added a Travis build for python3.8 and it is timing out annoyingly often.  Just re-started that, so hopefully it will resolve itself.
Great thanks @HughKelley
CLN: replace Dict with Mapping to annotate arguments
State: closed

xref https://docs.python.org/3/library/typing.html#typing.Dict

cc @WillAyd 


Labels:
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
> lgtm merge on green.

shows green on Travis, github shows in progress. will merge.


> Not sure if overkill but note in contributing guide may be useful as well for follow up

i'll do similar changes for List first, xref https://docs.python.org/3/library/typing.html#typing.List

probably go with Iterable, and use Sequence where needed.

see how that goes and then update guide accordingly.
> probably go with Iterable, and use Sequence where needed.

Both of these would allow strings then right? Might be less universally applicable to go from List to one of these
> > probably go with Iterable, and use Sequence where needed.
> 
> Both of these would allow strings then right? Might be less universally applicable to go from List to one of these

if these are typed as generic, then maybe so, but not if being precise. 
(we should probably have `disallow_any_generics`  in our config.)
Remove TestData from series-tests test_api.py
State: closed

Part of #22550

* Replaced TestData usage in `pandas/tests/series/test_api.py` with fixtures

- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`


Labels:
name: Testing
description: pandas testing functions or related to the test suite

Comments:
Thanks @SaturnFromTitan very nice cleanup
DOC: attempt to fix contributors for 0.24.0, 0.25.0/1/2
State: closed

This hopefully addresses https://github.com/pandas-dev/pandas/issues/29151, please review!

CC @TomAugspurger @jreback 

Labels:
name: Docs

Comments:
lgtm. can you have a look at the other whatsnew for the same issue?
Looks like there are a few

```
doc/source/whatsnew/v0.24.0.rst:1936:.. contributors:: v0.23.4..HEAD
doc/source/whatsnew/v0.25.1.rst:119:.. contributors:: v0.25.0..HEAD
doc/source/whatsnew/v0.25.0.rst:1270:.. contributors:: v0.24.x..HEAD
doc/source/whatsnew/v0.25.2.rst:49:.. contributors:: v0.25.1..HEAD
```

I think all of those can be updated.
Okay, the `.x` technique only works with branches i.e. w/o the `v` prefix, and even then the underlying git command might fail with

```
$ git shortlog -s 0.24.x..v0.25.0
fatal: ambiguous argument '0.24.x..v0.25.0': unknown revision or path not in the working tree.
```

Not sure how the repo is cloned / checked out in the environment used for building documentation. But in my clone of the repo I need to do
```
$ git shortlog -s origin/0.24.x..v0.25.0
```
to make this technique work.

Edit: so, either we use tags as  in `v0.24.2..v0.25.0` which seems to work in every build environment or we use a combination of branch names and tags as in `origin/0.24.x..v0.25.0` which seems to depend on the checkout (or at least my git knowledge is not good enough to understand the requirements about the checkout here).

Going to adjust the PR to only use tags (using the _latest_ as of today, i.e. 0.23.4 and 0.24.2 as the base).
Thanks!
Remove TestData from series-tests test_timeseries.py
State: closed

Part of #22550

* Replaced TestData usage in `pandas/tests/series/test_timeseries.py` with fixtures

- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`


Labels:
name: Testing
description: pandas testing functions or related to the test suite

Comments:
Thanks @SaturnFromTitan
Remove TestData from series-tests test_sorting.py
State: closed

Part of #22550

* Replaced TestData usage in `pandas/tests/series/test_sorting.py` with fixtures

- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`


Labels:
name: Testing
description: pandas testing functions or related to the test suite

Comments:
thanks @SaturnFromTitan
Remove TestData from series-tests test_repr.py
State: closed

Part of #22550

* Replaced TestData usage in `pandas/tests/series/test_repr.py` with fixtures

- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`


Labels:
name: Testing
description: pandas testing functions or related to the test suite

Comments:
I added a new `empty_series` fixture as it is mentioned in the issue description. I've seen more places where this can be used, so I hope this is fine with you
Done ✅ Please have another look @jreback
thanks
Remove TestData from series-tests test_replace.py
State: closed

Part of #22550

* Replaced TestData usage in `pandas/tests/series/test_replace.py` with fixtures

- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`


Labels:
name: Testing
description: pandas testing functions or related to the test suite

Comments:
thanks @SaturnFromTitan keep em coming. If these are smallish changes you can group multiple in one PR. We just don't want to have really large PRs
DOC: Edit MultiIndex.set_levels() docstring to clarify that set_levels() interprets passed values as new components of the .levels attribute (#28294)
State: closed

Added documentation on MultiIndex.set_levels to clarify that MultiIndex.set_levels() interprets passed values as new components of the .levels attribute. Continuation of #28797 due to issues with Travis CI for Python 3.8 and subsequent git diff issues resulting from resolution of Travis CI issues.

- [x] closes #28294 - or at least attempts to
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`

Labels:
name: Docs
name: MultiIndex

Comments:
can you rebase
Hello @jbrockmendel, do you mean 'git rebase upstream/master' or 'git fetch upstream && git merge upstream/master' to resolve conflicts?

Just wanted to confirm in order to avoid the git diff issues I encountered earlier in #28797 
> do you mean 'git rebase upstream/master' or 'git fetch upstream && git merge upstream/master' to resolve conflicts?

I usually do `git pull upstream master` and manually resolve any conflicts.
Not sure what happened, but I did a `git fetch upstream`, `git pull upstream/master`, manually resolved conflicts, and `git push origin` on my branch - and my tests are failing.

Edit: Did a `git fetch upstream && git merge upstream/master`, checked `black pandas` and `git diff`, and `git push origin` on my branch to get latest updates - tests passed.
lgtm @gfyoung care to take a look?
Hello @hweecat! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:



There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2020-01-02 01:51:02 UTC
can you merge master
@jbrockmendel I've done a "git fetch upstream" and "git merge upstream/master" - let's hope tests pass this time.
Okay, I have no idea what happened to my branch when I'm merging master, and tests are failing - suspect that it's caused by my previous commits made by merging master.

Anyone has any idea what to do with this?
Did you merge `upstream/master` or master from your branch? I'm guessing the latter given the amount of files changed

You can maybe reset hard back to before the merge and try again with `upstream/master`
Just checked - tests finally passed after the reset hard and merge with `upstream master`. Thanks for the help!
Thanks for the work on this @hweecat
BUG: fix non-existent variable in NDFrame.interpolate
State: closed

- [x] closes #29132
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry


Labels:
name: Bug
name: Missing-data
description: np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate

Comments:
Can you add a test for the bug this fixes
> Can you add a test for the bug this fixes

Absolutely! I added a test in the test_generic.TestNDFrame, is that reasonable?
thanks @ellequelle 
> thanks @ellequelle

👍