BrandonFors/pandas-github-issues huggingface dataset:
CI: temporary skip parquet tz test for pyarrow>=2.0.0
State: closed

See https://github.com/pandas-dev/pandas/issues/37286, this PR is not the actual fix, but at least ensures we don't have failing CI in other PRs (I prefer to skip the test instead of pinning pyarrow to <= 1.0, since all other tests are passing on 2.0, which is good to still run)

Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: IO Parquet
description: parquet, feather

Comments:
is this not a genuine error https://travis-ci.org/github/pandas-dev/pandas/jobs/737659028 the s3 test
Yes, I split a fix for the genuine failure into its own PR: https://github.com/pandas-dev/pandas/pull/37304
> note we actually do want to pin pyarrow in some older builds but i'll comment on the issues

Yes, indeed, we will want to pin some builds to pyarrow 1.0, and keep some at latest (separate issue, though, see also https://github.com/pandas-dev/pandas/pull/35259#issuecomment-712966527 for some related discussion)
Opened a dedicated issue for this (pinning pyarrow): https://github.com/pandas-dev/pandas/issues/37306
1.1.x is not yet failing with pyarrow related failures. https://dev.azure.com/pandas-dev/pandas/_build/results?buildId=45669&view=results 

changing milestone here and will backport if/when ci starts to fail.
> 1.1.x is not yet failing with pyarrow related failures. https://dev.azure.com/pandas-dev/pandas/_build/results?buildId=45669&view=results
> 
> changing milestone here and will backport if/when ci starts to fail.

test_timezone_aware_index was added in #36004 which is only on master



Ah, yes, that's a good reason it is not failing on 1.1.x ;-)
Does the 1.1.x branch have Travis? Because then https://github.com/pandas-dev/pandas/pull/37304 might still need to be backported
Hmm, it seems the Travis build on 1.1.x is failing with a conda solve issue
ah, was just mentioning that over on the other issue. It last build on https://travis-ci.org/github/pandas-dev/pandas/builds/736681105

we have some failures that we may need to resolve if we can get it to build consistently. I started to investigate #36779 but my local setup was obviously incorrect.
BUG: Allow empty chunksize in stata reader when using iterator
State: closed

- [X] closes #37280 
- [X] tests added / passed
- [X] passes `black pandas`
- [X] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry


Labels:
name: Regression
description: Functionality that used to work in a prior pandas version
name: IO Stata
description: read_stata, to_stata

Comments:
can you add a what's new note for the precision loss change (1.1.4)
also pls merge master
I rebased on master and added a what's new.

Hello @bashtage! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:





There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2020-10-23 06:22:14 UTC
small conflict and pls rebase
thanks @bashtage 
@meeseeksdev backport 1.1.x
CI: tz dtype mismatch in test_parquet following pyarrow upgrade
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: IO Parquet
description: parquet, feather

Comments:
prob need to backport this 
this caused a different failure in Travis.  cc @jorisvandenbossche thoughts on the right way to solve this?
I am looking into the actual cause (it's our testing machinery that cannot handle with different but equivalent timezones), let's use https://github.com/pandas-dev/pandas/issues/37286 to further discuss this. 
As a temporary fix to get green CI, I put up https://github.com/pandas-dev/pandas/pull/37303 (I prefer that intead of this PR, because setting `check_categorical` to False is only fixing it "by accident", I think)
> this caused a different failure in Travis

Did a separate PR for that one (not a timezone related failure): https://github.com/pandas-dev/pandas/pull/37304
DOC: fix a small typo
State: closed

an alternative fix would be "pandas provides a wide range of functions for cleaning textual data and extracting useful information from it."

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Docs

Comments:
thanks @partev
CLN: make PeriodArray signature match DTA/TDA
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Period
description: Period data type
name: API - Consistency
description: Internal Consistency of API/Behavior

Comments:
lgtm merge on green-ish
Regression in offsets caused offsets to be no longer hashable
State: closed

- [x] closes #37267
- [x] tests added / passed Not quite sure, if these test is sufficient, if yes, I would tests for other functions.
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry


Labels:
name: Datetime
description: Datetime data dtype
name: Regression
description: Functionality that used to work in a prior pandas version
name: Frequency
description: DateOffsets

Comments:
great code change looks good. if you'd report on the asv's we can merge.
Thanks, benchmark has just finished. No changes

``BENCHMARKS NOT SIGNIFICANTLY CHANGED.``
thanks @phofl nice
REF/TST: collect reindex tests
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves

Comments:
azure didn't run here for some reason as well
cool merge on green-ish
failures are due to pyarrow upgrade (currently troubleshooting locally).  greenish
REF/TST: collect astype tests
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Dtype Conversions
description: Unexpected or buggy dtype conversions

Comments:
hmm

i think this one is actually failing some CI (that IS related)

e.g. CI/web and docs is failing
can you rebase
rebased+green
ENH: Add is_sparse method to check for sparse columns in a DataFrame (GH26706)
State: closed

- [x] closes #26706 
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry


Labels:
name: Enhancement
name: Sparse
description: Sparse Data Type
name: Needs Discussion
description: Requires discussion from core team before further action
name: DataFrame
description: DataFrame data structure

Comments:
Implemented the method `df.is_sparse` and not the method `df.sparse.is_sparse`, since using the sparse accessor requires that all columns are sparse.
Thanks, @avinashpancham.

Is this not too specific? I´d like better an is_dtype method, e.g. doing df.is_dtype(“sparse”), which would be more generally useable.

@jorisvandenbossche 
Considering it again, I think:

```
>>> df = DataFrame({"A": pd.arrays.SparseArray([1, np.nan, 1]), "B": [1, 2, 3]})
>>> df.dtypes == "sparse"
Series([True, False], index=["A", "B"])
```

would achieve the same and be more general. Does the added method add anything over the above?

The current imp does only that. That's also (part of) the reason why not everyone agreed on adding such a method
Ok. IMO this method is too narrow to add to the dataframe namespace. So I`m -1 on this.
Np, I will wait for others to share their opinion and then we can decide whether to continue with or close this PR
The actual proposal in the issue was to add this not in the "top-level" DataFrame namespace, but in the `sparse` accessor, so you would do like `df.sparse.is_sparse`. Given that we already have the `.sparse` accessor for sparse-related things, I personally don't think that it is "too specific".

Now, there is already a lot of discussion about it on the issue (https://github.com/pandas-dev/pandas/issues/26706), so let's have the discussion whether we want this or not there.

> Implemented the method df.is_sparse and not the method df.sparse.is_sparse, since using the sparse accessor requires that all columns are sparse.

Yes, to do that, we need to change the sparse accessor to also work when not all columns are sparse. 

> Considering it again, I think: ... `df.dtypes == sparse` ... would achieve the same and be more general. 

@topper-123 that's potentially a nice idea, but to be clear, that doesn't work currently, as far as I can see:

```
In [2]: df = DataFrame({"A": pd.arrays.SparseArray([1, np.nan, 1]), "B": [1, 2, 3]})

In [3]: df.dtypes == "sparse"
Out[3]: 
A    False
B    False
dtype: bool
```

(but the proposal can then be to make this work?)
This pull request is stale because it has been open for thirty days with no activity. Please update or respond to this comment if you're still interested in working on this.
Thanks for the PR but it appears we need further discussion on https://github.com/pandas-dev/pandas/issues/26706 on the API before moving forward with a PR. Closing
REGR: Make comparisons consistent for PeriodDtype
State: closed

- [x] closes #37265
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry


Labels:
name: Dtype Conversions
description: Unexpected or buggy dtype conversions
name: Regression
description: Functionality that used to work in a prior pandas version
name: Period
description: Period data type

Comments:
LGTM
thanks @dsaxton 
@meeseeksdev backport 1.1.x
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
$ git checkout 1.1.x
$ git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
$ git cherry-pick -m1 ce5a1120649fd870dd258154cf7b7657841b094d
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
$ git commit -am 'Backport PR #37270: REGR: Make comparisons consistent for PeriodDtype'
```

4. Push to a named branch :

```
git push YOURFORK 1.1.x:auto-backport-of-pr-37270-on-1.1.x
```

5. Create a PR against branch 1.1.x, I would have named this PR:

> "Backport PR #37270 on branch 1.1.x"

And apply the correct labels and milestones.

Congratulation you did some good work ! Hopefully your backport PR will be tested by the continuous integration and merged soon!

If these instruction are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
                
probably conflicts on the whatsnew with #37198, but the backport #37305 can't be merged just yet. https://github.com/pandas-dev/pandas/pull/37305#pullrequestreview-513566564
DOC: troubleshoot docbuild
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:

Comments:
didn't we just merge this?
yah, pushed this before realizing another branch already did it.  closing.
BUG: Don't raise TypeError when converting NA from string to numeric
State: closed

- [x] closes #37262
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry


Labels:
name: Bug
name: Strings
description: String extension data type and string data
name: NA - MaskedArrays
description: Related to pd.NA and nullable extension arrays

Comments:
Thanks @dsaxton. It's probably fine to fix it like this in the short term, since a proper fix to actually refactor the `maybe_convert_numeric` to produce nullable results is a bigger change?
> Thanks @dsaxton. It's probably fine to fix it like this in the short term, since a proper fix to actually refactor the `maybe_convert_numeric` to produce nullable results is a bigger change?

That was my thinking, remove the error now then "enhance" to cast to nullable as a follow-up. I can open a separate issue for that if that makes sense?
thanks @dsaxton very nice
REF: nargsort incorrectly calling _values_for_argsort
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry

cc @jorisvandenbossche 

Labels:
name: Refactor
description: Internal refactoring of code

Comments:
@jreback this fixes the docbuild thats failing on master
thanks
ENH: implement matching warning message
State: closed

- [ ] closes #37261
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry

Implement matching warning message.
New kwarg ``match`` in ``assert_produces_warning``.

Labels:
name: Testing
description: pandas testing functions or related to the test suite

Comments:
thanks @ivanovmg very nice
TST: split up test_concat.py #37243
State: closed

- [ ] closes #37243 
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode

Comments:
Hello @laffite56! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:













* In the file [`pandas/tests/reshape/concat/test_empty.py`](https://github.com/pandas-dev/pandas/blob/dcb2717e2cfaa80a34e057540bc644019df42259/pandas/tests/reshape/concat/test_empty.py):

> [Line 84:89](https://github.com/pandas-dev/pandas/blob/dcb2717e2cfaa80a34e057540bc644019df42259/pandas/tests/reshape/concat/test_empty.py#L84): [E501](https://duckduckgo.com/?q=pep8%20E501) line too long (91 > 88 characters)



* In the file [`pandas/tests/reshape/concat/test_series.py`](https://github.com/pandas-dev/pandas/blob/dcb2717e2cfaa80a34e057540bc644019df42259/pandas/tests/reshape/concat/test_series.py):

> [Line 230:89](https://github.com/pandas-dev/pandas/blob/dcb2717e2cfaa80a34e057540bc644019df42259/pandas/tests/reshape/concat/test_series.py#L230): [E501](https://duckduckgo.com/?q=pep8%20E501) line too long (89 > 88 characters)





##### Comment last updated at 2020-10-20 17:03:34 UTC
I have put a few functions into classes, but i don't know if this is fine. 
> I have put a few functions into classes, but i don't know if this is fine.

I generally would go the other way actually.

But most important for this PR. please let's do just straight copy-paste, its very hard to see changes otherwise. my requests were all about just moving existing code around in blocks, not actually change that much.

so we want to merge this soon. I am also happy to do this in a separate PR by first literally moving the entire file, then moving pieces of it.

I'm a little confused. You want me to create a new PR, with one commit containing only copy-paste original "test_concat.py" to new folder, like i have right now:

from **pandas/tests/reshape/test_concat.py**  => **pandas/tests/reshape/concat/test_concat.py**

and then in next commit reshape "test_concat.py" to submodules?

Sir, @jreback, i don't really know what to do with this issue. I can easly make new PR with every commit separated, like one commit for moving one file, then next ones for every new submodule and for every significant change, so You'll be able to see every change. 
> Sir, @jreback, i don't really know what to do with this issue. I can easly make new PR with every commit separated, like one commit for moving one file, then next ones for every new submodule and for every significant change, so You'll be able to see every change.

so 1) pls merge master. I would actually just do a straight move & split and NOTHING else in a the first pr. split by classes is fine.
let's do that and will merge quickly. Then its very easy to do followups.
#37360 here is new PR.
BUG: with integer column labels, .info() throws KeyError 
State: closed

- [ ] closes #37245
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: Regression
description: Functionality that used to work in a prior pandas version

Comments:
thanks @MarcoGorelli 
@meeseeksdev backport 1.1.x
fixed spelling errors in whatsnew, userguide, and ecosystem
State: closed

Hi there, 

I fixed the spelling errors mentioned here. https://github.com/pandas-dev/pandas/issues/37252

 The 2 "wit" spelling errors were confusing, and I could not understand the sentences. 

This is my first pull request, so please excuse my ignorance if I did something incorrectly.

Thanks. 






Labels:
name: Docs

Comments:
Thanks @asharma13524 !
DOC: update PeriodArray docstring
State: closed

- [x] closes #37237 
- [ ] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Docs

Comments:
> @arw2019 thanks for the PR.
> I would also mention `pd.array(..)` as the way to create a PeriodArray from a sequence of Period scalars.

Added
Thanks for the update!
Small development related comment: can you generally try to avoid rebasing / force pushing to a branch, but rather only add new commits? (which means merging master instead of rebasing to update to latest master) Github works better that way (eg you can specifically see the new changes since your last review) making reviewing easier. 
(not of any concern of course for this specific PR of small size ;))
CI move code directives to pre-commit, remove some outdated checks
State: closed

Moving checks for incorrect code block or IPython directives to pre-commit so they're cross-platform and give faster feedback to devs.

Also removing some no-longer-necessary code checks:

- "Check that no file in the repo contains trailing whitespaces" is taken care of by the `trailing-whitespace` hook
- "Check for extra blank lines after the class definition" is enforced by `black`

Labels:
name: CI
description: Continuous Integration
name: Code Style
description: Code style, linting, code_checks

Comments:
LGTM pending green
thanks @MarcoGorelli
CI move non-standard-import checks over to pre-commit
State: closed

Like this they're cross-platform, provide faster feedback to devs, and can use pre-commit `pygrep` instead of the custom `invgrep`

Labels:
name: CI
description: Continuous Integration
name: Code Style
description: Code style, linting, code_checks

Comments:
This looks fine.  Has there been a discussion somewhere where theres consensus to move as much as possible/convenient to the pre-commit?
> This looks fine. Has there been a discussion somewhere where theres consensus to move as much as possible/convenient to the pre-commit?

There hasn't, there was just this discussion as a precedent #36386 , and Will's comment

> These in particular are managed through the configuration file, so they won't differ from being run in pre-commit which has the added bonus of being cross platform

There was one PR (#37136) where using pygrep instead of invgrep revealed a case which wasn't being found.

I'd welcome a discussion about whether this is welcome in general of course - as per this PR's description, I think it would be beneficial
can you merge master and pin gon green.
> can you merge master and pin gon green.

ping
thanks @MarcoGorelli
CLN: clean-up test on addition of series/frames
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry

Cleanup in ``pandas/tests/arithmetic/test_numeric.py``.
- Split ``test_arith_ops_df_compat`` into ``test_add_series`` and ``test_add_frames``.
- Parametrize tests.

Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Numeric Operations
description: Arithmetic, Comparison, and Logical operations

Comments:
thanks @ivanovmg
TST/CLN: Split out some to_string tests
State: closed

test_format.py is over 3000 lines long so breaking some (but not all) to_string-specific tests into a separate test_to_string.py file (we already have test_to_html.py, test_to_latex.py, and test_to_markdown.py)

Labels:
name: Testing
description: pandas testing functions or related to the test suite

Comments:
> looks fine. can you confirm we have same number of tests before & after here.

master
```
pandas/tests/io/formats/test_format.py .............................................................................................................................................................. [ 67%]
.............................................................................                                                                                                                         [100%]

============================================================================================ 235 passed in 4.39s ============================================================================================
```

branch
```
pandas/tests/io/formats/test_format.py .............................................................................................................................................................. [ 78%]
...........................................                                                                                                                                                           [100%]

============================================================================================ 201 passed in 4.21s ============================================================================================
```

```
pandas/tests/io/formats/test_to_string.py ..................................                                                                                                                          [100%]

============================================================================================ 34 passed in 0.42s =============================================================================================
```

thanks @dsaxton
TST: indexing tests for #21168, #27420, #15928, #30053
State: closed

- [x] closes #21168
- [x] closes #27420
- [x] closes #15928
- [x] closes #30053
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves

Comments:
lgtm merge when green-ish (something failing on master?)