BrandonFors/pandas-github-issues huggingface dataset:
remove trailing commas for #35925
State: closed

- [x] pandas/io/parquet.py
- [x] pandas/io/parsers.py
- [x] pandas/io/pytables.py


Labels:
name: Code Style
description: Code style, linting, code_checks
name: Clean

Comments:
xref #35925
Thanks @rajanshoo25 for the PR. generally lgtm.
@rajanshoo25 CI / Checks (pull_request) failing. need to run black again after the changes

```
black, version 19.10b0
Checking black formatting
would reformat /home/runner/work/pandas/pandas/pandas/io/parsers.py
Oh no! 💥 💔 💥
1 file would be reformatted, 1122 files would be left unchanged.
```


Thanks @rajanshoo25 
> Thanks @rajanshoo25

Thanks a lot! @simonjayhawkins for guiding me. Actually I am new in open-source contribution.
TYP: add type annotation to expr.py #36027
State: closed

- [x] closes #36027
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`



Labels:
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
> Thanks @VirosaLi for the PR.
> 
> This may be fixed in open PR. see [#36012 (review)](https://github.com/pandas-dev/pandas/pull/36012#pullrequestreview-478726562)

Cool, I'll close this one.
DOC: Update documentation for pd.Interval if string endpoints are not allowed anymore
State: closed

- [X] closes #36002 
- [X] tests added / passed
- [X] passes `black pandas`
- [X] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry

Removed outdated example in documentation for pd.Interval (string endpoints are not allowed anymore).

Labels:
name: Docs

Comments:
Thanks @souris-dev
TYP: add type annotation to `_xlwt.py` #36024
State: closed

- [x] closes #36024
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`

pandas\io\excel\_xlwt.py:52: error: Need type annotation for 'style_dict' (hint: "style_dict: Dict[<type>, <type>] = ...")  [var-annotated]


Labels:
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
Thanks @VirosaLi
TYP/CLN: cleanup `_openpyxl.py`, add type annotation #36021
State: closed

- [x] closes #36021
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`



Labels:
name: Clean
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
@VirosaLi to fix the import errors, use quotes around the type (or can use `from __future__ import annotations` if #36034 approved. )
TYP: Annotate plotting stacker
State: closed

In AreaPlot._plot we call `ax.fill_between`, which means `ax` must be an `Axes` object (and in particular, not an `Axis` object).  This chases down all the other places we can infer `Axes` from that.

Then some edits in an `__init__` to get mypy passing, and revert annotations of `_plot` in a few places because mypy complained about signature mismatch.

Labels:
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
/azp run
<samp>
Azure Pipelines successfully started running 1 pipeline(s).<br>

</samp>
yea also lgtm test failures seems unrelated so restarted. merge on green
Thanks @jbrockmendel
CI: Unpin MyPy
State: closed


Labels:
name: CI
description: Continuous Integration
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
LGTM
LGTM
lgtm ex @jreback comments; definitely need to get this updated given all of the changes between 0.73 and now
thanks @simonjayhawkins 

note may want to add this to the whatsnew for development changes (we usually list pytest version changes)
TYP: check_untyped_defs core.internals.concat
State: closed

pandas\core\internals\concat.py:106: error: Incompatible types in assignment (expression has type "Tuple[Any, ...]", variable has type "List[Any]")  [assignment]
pandas\core\internals\concat.py:131: error: Incompatible types in assignment (expression has type "Tuple[Any, ...]", variable has type "List[Any]")  [assignment]
pandas\core\internals\concat.py:377: error: Need type annotation for 'upcast_classes'  [var-annotated]
pandas\core\internals\concat.py:378: error: Need type annotation for 'null_upcast_classes'  [var-annotated]

Labels:
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
LGTM
Thanks @simonjayhawkins
TYP: misc typing cleanup in core/indexes/multi.py
State: closed

pandas\core\indexes\multi.py:496: error: Need type annotation for 'arrays'  [var-annotated]
pandas\core\indexes\multi.py:722: error: Incompatible types in assignment (expression has type "List[Any]", variable has type "FrozenList")  [assignment]
pandas\core\indexes\multi.py:893: error: Incompatible types in assignment (expression has type "List[Any]", variable has type "FrozenList")  [assignment]
pandas\core\indexes\multi.py:2438: error: List item 0 has incompatible type "slice"; expected "str"  [list-item]
pandas\core\indexes\multi.py:3095: error: Unsupported left operand type for | ("None")  [operator]

Labels:
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
thanks @simonjayhawkins
TYP: misc typing cleanup for core/computation/expressions.py
State: closed

pandas\core\computation\expressions.py:78: error: Need type annotation for 'dtypes' (hint: "dtypes: Set[<type>] = ...")  [var-annotated]
pandas\core\computation\expressions.py:258: error: Need type annotation for '_TEST_RESULT' (hint: "_TEST_RESULT: List[<type>] = ...")  [var-annotated]

Labels:
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
LGTM
thanks @simonjayhawkins
BUG: Can't restore index from parquet with offset-specified timezone #35997
State: closed

- [x] closes #35997
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry


Labels:
name: Bug
name: Timezones
description: Timezone data dtype
name: IO Parquet
description: parquet, feather

Comments:
Hello @alippai! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:







There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2020-10-07 00:09:08 UTC
Thanks @alippai, however the test doesn't seem to be passing? I think we would first need to fix the issue before committing the test.
Like with TDD, you write the expectation as a testcase and it fails, then the code is changed and the untouched test passes. Isn't this the workflow for fixing a bug? I didn't work on pandas codebase before, but I can give it a try. 
BTW Setting up the environment works like a charm, but the thing I did is in the official docs too: https://pandas.pydata.org/docs/dev/development/contributing.html#test-driven-development-code-writing
> So, before actually writing any code, you should write your tests. Often the test can be taken from the original GitHub issue.
It's a good practice to write tests first, but a PR should include both the test and the fix that makes the test pass (unless the test is already passing of course)
@dsaxton @jbrockmendel I've pushed a naive fix, what do you think? Should I skip regex or is it OK? 
@jreback Run it, there are no significant changes within 10% (beside the fact that `index_cached_properties.IndexCache.*` looks totally random)
~I'm giving this up for now, feel free to grab & fix it~
Do you need any changes? What else needed for the merge? 
> Do you need any changes? What else needed for the merge?

ok this lgtm. if you'd add a whatsnew note in 1.2 under I/O section can get this in . ping on green.
@jreback The what's new entry is added, the CI failures are not related. Thanks!
thanks for hanging in there @alippai nice work!
This was my first contribution to pandas, thank you @dsaxton, @jbrockmendel, @jorisvandenbossche and @jreback for the help. I really enjoyed working on this and I appreciate the fact that I had close to zero technical issues setting up the dev environment.
ENH: Arithmetic with Timestamp-based intervals
State: closed

- [X] closes #35908 
- [X] tests added / passed
- [X] passes `black pandas`
- [X] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [X] whatsnew entry


Labels:
name: Datetime
description: Datetime data dtype
name: Numeric Operations
description: Arithmetic, Comparison, and Logical operations
name: Interval
description: Interval data type
name: Stale

Comments:
@arw2019 - Thanks for the review!
I'll try incorporating these.
Also, the point you've raised about different timezones is interesting, I'll have a look at that.
This pull request is stale because it has been open for thirty days with no activity. Please update or respond to this comment if you're still interested in working on this.
closing as stale. if you want to continue, pls ping and can re-open.
ENH: vendor typing_extensions
State: closed

- [x] closes #34869
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry

haven't figured out how to make isort and black ignore this file

Labels:
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
Hello @jbrockmendel! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:

* In the file [`pandas/_vendored/typing_extensions.py`](https://github.com/pandas-dev/pandas/blob/1b29450597510adb80c8b29cfadecdfc1c86dce2/pandas/_vendored/typing_extensions.py):

> [Line 1985:89](https://github.com/pandas-dev/pandas/blob/1b29450597510adb80c8b29cfadecdfc1c86dce2/pandas/_vendored/typing_extensions.py#L1985): [E501](https://duckduckgo.com/?q=pep8%20E501) line too long (98 > 88 characters)



##### Comment last updated at 2020-09-01 16:25:26 UTC
cc @WillAyd @simonjayhawkins thoughts on getting this to pass checks?
FWIW I think OK to run the new module through isort / black; doesn't need to be a perfect reproduction

AFAICT code check failures are mostly around internal style rules we place; OK to update to those as well as they are mostly cosmetic
> do we need any sort of testing for this?

I wouldn't bother.  FWIW not needing to worry about that is part of why i suggested making this a dependency in #34869
thanks @jbrockmendel
BUG: None in Float64Index raising TypeError, should return False
State: closed

- [x] closes #35788
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry


Labels:
name: Bug
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves

Comments:
thanks @jbrockmendel 
@meeseeksdev backport 1.1.x
Issue35925 Remove trailing commas
State: closed

#35925 
Files edited: 
- pandas/tests/test_multilevel.py
- pandas/tests/test_nanops.py
- pandas/tests/window/moments/test_moments_consistency_rolling.py
- pandas/tests/window/moments/test_moments_ewm.py
- pandas/tests/window/moments/test_moments_rolling.py
- pandas/tests/window/test_base_indexer.py
- pandas/tests/window/test_pairwise.py
- pandas/tests/window/test_rolling.py
- pandas/tseries/frequencies.py
- pandas/util/_test_decorators.py

Labels:
name: Code Style
description: Code style, linting, code_checks
name: Clean

Comments:
Thanks @metehankutlu
TYP: misc typing fixes for pandas\core\frame.py
State: closed

pandas\core\frame.py:1091: error: namedtuple() expects a string literal as the first argument  [misc]
pandas\core\frame.py:4594: error: Need type annotation for 'names' (hint: "names: List[<type>] = ...")  [var-annotated]

Labels:
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
thanks @simonjayhawkins
TST: Verify operators with IntegerArray and list-likes (22606)
State: closed

- [x] closes #22606
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: ExtensionArray
description: Extending pandas with custom dtypes or arrays.

Comments:
Do we also want tests for other operators ( -, /, * ,etc)?
@jreback could you have another look? Thanks!
I think we just merged a change by @jbrockmendel that updated this fixture (the box_1d), so pls merge master again and ping on green.
@jreback merged master and everything passes. I did see that the fixtures are now updated to use `pd.array` instead of `tm.to_array`. Do we want to update this PR to reflect this new approach?
thanks @avinashpancham
DOC: Add Notes about difference to numpy behaviour for ddof in std() GH35985
State: closed

- [x] closes #35985
- [ ] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Docs

Comments:
> Thanks @timhunderwood
> 
> > normalizes by N
> 
> Are you sure this is correct? Does dividing by N actually normalise the quantity?
> 
> IMO it should be enough to write something like
> 
> ```
> Notes
> -----
> To have the same behaviour as `numpy.std`, use `ddof=0` (instead of the default `ddof=1`)
> ```



> Thanks @timhunderwood
> 
> > normalizes by N
> 
> Are you sure this is correct? Does dividing by N actually normalise the quantity?
> 
> IMO it should be enough to write something like
> 
> ```
> Notes
> -----
> To have the same behaviour as `numpy.std`, use `ddof=0` (instead of the default `ddof=1`)
> ```

thanks @MarcoGorelli , I agree this could be confusing. I've changed to your simpler wording, thanks.
> Looks good to me, can you fetch and merge upstream/master and then push again to restart the CI process in order to get it green?

Done - CI process has started...
Thanks @timhunderwood !
ENH: Add axis argument to Dataframe.corr
State: closed

- [x] closes #35002
- [x] tests added / passed


Labels:
name: Enhancement
name: Numeric Operations
description: Arithmetic, Comparison, and Logical operations
name: Stale

Comments:
Hello @kc611! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:





There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2020-08-31 15:28:08 UTC
Also can you add a whatsnew note for v1.2.0?
@dsaxton Thanks for the review. I made the changes accordingly.

I was also thinking about adding a callable method in the test but since this is a minor improvement (and most probably will work for every case for which Dataframe.corr worked normally,since we're only transposing the original matrix), I dropped the idea. Any changes to tests are welcome.

And also under what section of whatsnew note for v1.2.0 should I add the note?
> And also under what section of whatsnew note for v1.2.0 should I add the note?

You can add it under enhancements (a bullet within other enhancements is probably fine)
> LGTM @kc611 nice work

Thanks

@MarcoGorelli Thanks for review, I added the type annotations as requested.

But it seems that the tests are failing and I cannot figure out the exact reason. Can you assist me.
@kc611 Can you merge master and address @jreback comments above?
This pull request is stale because it has been open for thirty days with no activity. Please update or respond to this comment if you're still interested in working on this.
@kc611 Can you address comments and merge master?
BUG: Respect errors="ignore" during extension astype
State: closed

- [x] closes https://github.com/pandas-dev/pandas/issues/35471
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry


Labels:
name: Bug
name: Dtype Conversions
description: Unexpected or buggy dtype conversions
name: ExtensionArray
description: Extending pandas with custom dtypes or arrays.

Comments:
Hello @dsaxton! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:







There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2020-09-06 00:58:13 UTC
thanks @dsaxton 
@meeseeksdev backport 1.1.x
BUG: instantiation using a dict with a period scalar
State: closed

- [x] closes #35965
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry

Fixing bug discussed in [issue 35965](https://github.com/pandas-dev/pandas/issues/35965) where `pd.DataFrame({'a': pd.Period('2020-01')})` created `a` as an object column instead of a `period[m]` column.

Changing the functionality of `infer_dtype_from_scalar` isn't necessarily required here, but the fact that `infer_dtype_from_scalar` would return the `period.ordinal` value seems inconsistent with the behavior for other dtypes in this function. Additionally, that functionality was only used in a single place within the code (`interval.py`), which I fixed accordingly.

Labels:
name: Bug
name: Dtype Conversions
description: Unexpected or buggy dtype conversions
name: Period
description: Period data type
name: ExtensionArray
description: Extending pandas with custom dtypes or arrays.
name: Constructors
description: Series/DataFrame/Index/pd.array Constructors

Comments:
Hello @justinessert! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:













There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2020-09-06 22:33:19 UTC
@jreback there's one that I updated in [test_infer_dtype_from_period](https://github.com/pandas-dev/pandas/pull/35966/files#diff-fe46d698847ecf7fbacc9eccf0bd4722L80-L94), is that what you mean or are you after something else?
@jreback What are you looking to be tested outside of [test_infer_dtype_from_period](https://github.com/pandas-dev/pandas/pull/35966/files#diff-fe46d698847ecf7fbacc9eccf0bd4722L80-L94)?
> @jreback What are you looking to be tested outside of [test_infer_dtype_from_period](https://github.com/pandas-dev/pandas/pull/35966/files#diff-fe46d698847ecf7fbacc9eccf0bd4722L80-L94)?

yep saw that, your change is good
thanks @justinessert very nice
@jreback Thanks for all your helpful and timely replies, it's much appreciated!
BUG/CLN: Decouple Series/DataFrame.transform
State: closed

- [x] closes #35811
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry

First step toward #35725. Currently `transform` just calls `aggregate`, and so if we are to forbid `aggregate` from transforming, these need to be decoupled. Other than the bugfix (#35811), the only other behavioral change is in the error messages.

Assuming the bugfix #35811 is the correct behavior, docs/whatsnew also needs to be updated.

I wasn't sure if tests should be marked with #35725 or perhaps this PR #. Any guidance here?

Labels:
name: Bug
name: Clean
name: Apply
description: Apply, Aggregate, Transform, Map

Comments:
> I wasn't sure if tests should be marked with #35725 or perhaps this PR #. Any guidance here?
either is fine
@jreback Thanks for the review. Code has been reorganized and ready for another. For the comment about not adding things to the core files - I think the idea behind this that we should be distributing the implementation of methods in e.g. NDFrame into various submodules like aggregate instead of having one monolithic generic.py. Is that correct?
@jreback changes made and green. In fixing the one test I found cumcount doesn't exist for DataFrames/Series (which, after thinking a bit, makes sense). This prompted me to add in tests for (almost) all the transformation kernels.
thanks @rhshadrach
TYP: misc cleanup in core\generic.py
State: closed

pandas\core\generic.py:4707: error: Unsupported operand types for in ("Optional[str]" and "str")  [operator]
pandas\core\generic.py:6559: error: 'builtins.object' object is not iterable  [misc]


Labels:
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
LGTM
thanks @simonjayhawkins
TYP: annotate plotting based on _get_axe_freq
State: closed

In some places in plotting `ax` is an Axes object and in other its an Axis object.  Current goal is to pin these down.

in timeseries._get_ax_freq we call `ax.get_shared_x_axes()`, which is an Axes method that does not exist on Axis.  This annotates that usage and annotates all the other places where we can infer Axes from that.

Labels:
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
Thanks @jbrockmendel
TYP: misc cleanup in core\groupby\generic.py
State: closed

pandas\core\groupby\generic.py:610: error: Too many arguments  [call-arg]
pandas\core\groupby\generic.py:1212: error: "create_series_with_explicit_dtype" gets multiple values for keyword argument "dtype_if_empty"  [misc]

Labels:
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
Thanks @simonjayhawkins
TYP: misc typing cleanups for #32911
State: closed

pandas\io\excel\_odswriter.py:39:5: error: Argument 5 of "write_cells" is incompatible with supertype "ExcelWriter"; supertype defines the argument type as "Optional[Tuple[int, int]]"  [override]
pandas\io\excel\_odswriter.py:62:35: error: Argument 1 to "_validate_freeze_panes" has incompatible type "Optional[List[Any]]"; expected "Optional[Tuple[int, int]]"  [arg-type]

Labels:
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
LGTM
Thanks @simonjayhawkins
TYP: misc typing cleanups for #29116
State: closed

pandas\core\frame.py:7429:59: error: Argument 3 to "relabel_result" has incompatible type "Optional[List[str]]"; expected "Tuple[Any, ...]"  [arg-type]
pandas\core\frame.py:7429:68: error: Argument 4 to "relabel_result" has incompatible type "Optional[List[int]]"; expected "List[int]"  [arg-type]

Labels:
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
LGTM pending green
Thanks @simonjayhawkins
BUG: Fix DataFrame.groupby().apply() for NaN groups with dropna=False 
State: closed

- [X] closes #35889 
- [X] tests added / passed
- [X] passes `black pandas`
- [X] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [X] whatsnew entry in `v1.1.2.rst`


Labels:
name: Bug
name: Groupby
name: Apply
description: Apply, Aggregate, Transform, Map

Comments:
@cwkwong might want to have a more descriptive title `BUG: Fix DataFrame.groupby().apply() for NaN groups with dropna=False`
Thanks @arw2019 for approving the changes. What's the next step to hopefully make this as part of the `1.1.2` release?
thanks @cwkwong very nice