BrandonFors/pandas-github-issues huggingface dataset:
added names, fastpath parameters explanation to pandas.Series
State: closed

- [x] closes #xxxx
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry


Labels:
name: Docs

Comments:
Can you use a more informative title?

Looks like a .ipynb file got included accidentally?
Yes sure I will do that.

And could you please ignore .ipynb file. It accidently got uploaded.

Sent from my iPhone

> On Aug 17, 2019, at 12:06 PM, jbrockmendel <notifications@github.com> wrote:
> 
> Can you use a more informative title?
> 
> Looks like a .ipynb file got included accidentally?
> 
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub, or mute the thread.

Thanks.  We can ignore the extra file for now, but you'll need to remove it before this can be merged.

Looks like the lint checker is unhappy about the removed newline.  Can you run `black pandas/core/series.py`
Hello @rmunjal29! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:



There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2019-11-08 00:20:49 UTC
@rmunjal29 can you merge master and update to comments?
Hello William,

Sorry for the late reply. I am actually pretty engaged in my project these days but I will definitely find time and merge these changes.

Thanks and regards,
Ritika

Sent from my iPhone

> On Sep 12, 2019, at 8:34 PM, William Ayd <notifications@github.com> wrote:
> 
> @rmunjal29 can you merge master and update to comments?
> 
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub, or mute the thread.

@rmunjal29 can you address comments?
Already updated the changes

Regards,
Ritika

> On Oct 21, 2019, at 8:42 PM, William Ayd <notifications@github.com> wrote:
> 
> ﻿
> @rmunjal29 can you address comments?
> 
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub, or unsubscribe.

Looks like there is a notebook in the diff - can you remove that?

Sent from my iPhone

> On Oct 21, 2019, at 7:17 PM, rmunjal29 <notifications@github.com> wrote:
> 
> ﻿Already updated the changes
> 
> Regards,
> Ritika
> 
> > On Oct 21, 2019, at 8:42 PM, William Ayd <notifications@github.com> wrote:
> > 
> > ﻿
> > @rmunjal29 can you address comments?
> > 
> > —
> > You are receiving this because you were mentioned.
> > Reply to this email directly, view it on GitHub, or unsubscribe.
> —
> You are receiving this because you commented.
> Reply to this email directly, view it on GitHub, or unsubscribe.

Thanks @rmunjal29
pandas.read_parquet now has info that it accepts path_to_directory that contains multiple parquet files
State: closed

- [x] closes #27820 
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry


Labels:
name: IO Parquet
description: parquet, feather

Comments:
Can you also delete the `.clang-format` file?
Can you merge master and address comments?
@prajvalgupta can you address comment above?
Hey. @WillAyd  Can I pick this up? Also what did you mean by this 

> Can you merge master and address comments?

I am new to contributing and didn't understand what this meant.
can you merge master

```git merge upstream/master```

and for each comment above you need to either fix or comment what is the issue
I think this is stale as it hasn't been updated since first posting. @punndcoder28 if you'd like to take over you should get a copy of master locally on your own branch and merge these changes in. Can definitely take a look then

Of course @prajvalgupta if you are still actively working this please let us know
DOC: Add missing parameters to Series constructor
State: closed

Added documentation to pandas.Series for "name" and "fastpath" parameters.

Labels:
name: Docs
name: Series
description: Series data structure

Comments:
Hello @SHuangISU! Thanks for opening this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:

* In the file [`pandas/core/series.py`](https://github.com/pandas-dev/pandas/blob/fbe63af20be0fc0ccb95c88231bdc4177a739843/pandas/core/series.py):

> [Line 174:82](https://github.com/pandas-dev/pandas/blob/fbe63af20be0fc0ccb95c88231bdc4177a739843/pandas/core/series.py#L174): [W291](https://duckduckgo.com/?q=pep8%20W291) trailing whitespace
> [Line 175:56](https://github.com/pandas-dev/pandas/blob/fbe63af20be0fc0ccb95c88231bdc4177a739843/pandas/core/series.py#L175): [W291](https://duckduckgo.com/?q=pep8%20W291) trailing whitespace
> [Line 176:1](https://github.com/pandas-dev/pandas/blob/fbe63af20be0fc0ccb95c88231bdc4177a739843/pandas/core/series.py#L176): [W293](https://duckduckgo.com/?q=pep8%20W293) blank line contains whitespace
> [Line 179:1](https://github.com/pandas-dev/pandas/blob/fbe63af20be0fc0ccb95c88231bdc4177a739843/pandas/core/series.py#L179): [W293](https://duckduckgo.com/?q=pep8%20W293) blank line contains whitespace
> [Line 180:33](https://github.com/pandas-dev/pandas/blob/fbe63af20be0fc0ccb95c88231bdc4177a739843/pandas/core/series.py#L180): [W291](https://duckduckgo.com/?q=pep8%20W291) trailing whitespace


1. You should install this tool on your machine: https://github.com/tj/git-extras
2. Then make the five changes as instructed by the pep8speaks comment above
3. Use `git squash HEAD~1` to undo the last commit: https://github.com/tj/git-extras/blob/master/Commands.md#git-squash
4. Do `git commit -am "DOC: add API docstrings to pandas.Series for 'name' and 'fastpath'"` as per the guidelines: https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#committing-your-code
5. To override your first commit you'll have to do `git push -f` that should automatically updat e this PR
for what its worth, when merging a PR we squash all commits into one, so its really only step 2 that is needed
@SHuangISU can you address the review comments please? We've got another PR adding these parameters to this doc, if you're not planning to finish this PR, please let me know, so I'll keep the other one open. Thanks!
@datapythonista - I'd say you could give @SHuangISU 24hrs to respond and if he doesn't get to it you can close this and perhaps point out a similar issue that's available
Thanks for the suggestion @sbrice, but contributors to pandas are volunteers working in their scarce free time, and giving 24h to respond sounds unreasonable and rude to me.
Yeah, the suggestion was within the context of our event i wasn't sure who the other PR was coming from
Ah, that makes sense, didn't get that. Thanks!
I think the other PR referenced is #27964 which was updated more recently. I think that one is pretty close so to keep things simple let's close this one out

@SHuangISU certainly would accept any other contributions you'd like to offer
BUG: Remove null values before sorting during groupby nunique calculation
State: closed

- [x] Closes #27904
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry


Labels:
name: Datetime
description: Datetime data dtype
name: Groupby
name: Missing-data
description: np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate

Comments:
Does this completely close #27904? Or should it remain open for the case of `dropna=False`?
In the case of `dropna=False`, what should `nunique` return? As in, do the `NaN`s all count as distinct values? I wouldn't expect them to, although as `np.nan == np.nan` returns `False`, so I'm not sure
Just one NA value I think.
And to be clear, we can leave handling `dropna=False` to a followup PR. Just want to make sure that we have an open issue for it.
Actually... I'm going to push this off the 0.25.1 milestone (sorry @MarcoGorelli)

Can you check a few things:

1. How's the performance of this? I assume `np.lexsort` has to do some kinds of NA masking and filtering internally. Are we much slower than that?
2. It looks like there's a now unnecessary `if dropna` starting around line 1171 now. IIUC, `val` should now have all the na values removed, so masking again shouldn't be necessary.
Sorry for the delay - yes, I agree with your decision, Tom :) I'll pick this up again next week
Hello @MarcoGorelli! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:





There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2019-09-06 15:50:47 UTC
As noted as a comment in #27904, this works fine with numpy's own `nat` value...is it considered cheating just replace `pandas.NaT` with `np.datetime64("NaT")`? I tried that, and both the existing and the new tests (which include `dropna=False`) pass.
Haven't looked deeply yet but does this also impact Series and DataFrame methods? The location of the change would imply so hence my reason for asking
Thanks for your review!

Sure, whatsnew entry added.

If I've understood your question, then the `pandas.DataFrame.nunique` and `pandas.Series.nunique` methods didn't have this issue to begin with, and are unaffected by this change.
Ah misread the file location but makes sense so thanks for confirming. Should be able to review more deeply over the next few days
Thanks @MarcoGorelli!

Can you confirm that this fixed all of #27904, or are there outstanding tasks?
That's right - thanks for having taken the time to review my work!
CLN: docstrings, circularity in core.computation
State: closed

Things noticed while looking through this directory:

- computation.pytables is only used io.pytables, might not belong in core.computation
- UndefinedVariableError looks like it was defined as part of a `# python 3 compat kludge`, might no longer be necessary

Labels:
name: Clean

Comments:
thanks @jbrockmendel
REF: implement array_ops
State: closed

This begins the process of separating the (ndarray|EA)-specific parts of Series ops out from the Series ops themselves.  Eventually these will a) become the op implementations for PandasArray, and b) be used block-wise instead of column-wise for DataFrame ops.

Labels:
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode

Comments:
sure
BUG: set_precision format fixed (#13257)
State: closed

- [x] closes #13257
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry

I've tried to fix the set_precision format, does that fix it? 

### INPUT -> df.style.set_precision(2)
| A | B |
| --- | --- |
| 1 | 2.0090 |
| 3.2121 | 4.566 |
### OUTPUT
| A | B |
| --- | --- |
| 1 | 2.01 |
| 3.21 | 4.57 |

@vinigl
@samuelgranato




Labels:
name: Output-Formatting
description: __repr__ of pandas objects, to_string
name: IO HTML
description: read_html, to_html, Styler.apply, Styler.applymap

Comments:
Hello @AntonioAndraues! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:





There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2019-10-16 14:57:46 UTC
Can you add a test for this
> Can you add a test for this

I just added a test.
Is that it? 
> Looks good. Can you add a release note to doc/source/whatsnew/v1.0.0.rst under "bug fixes"? Probably the "Other" section.

Done
Thanks. Looks like there's a merge conflict in 1.0.0.rst, if you could fix that.
> Thanks. Looks like there's a merge conflict in 1.0.0.rst, if you could fix that.
@TomAugspurger 
I think i fix it.
CI is failing now. Do your tests need to be updated?
> CI is failing now. Do your tests need to be updated?
@TomAugspurger 
I'm sorry forgot to update them... 
I think everything will be fine now!


Hmm CI is still failing. Haven't taken a look though.
> Hmm CI is still failing. Haven't taken a look though.
I think CI is failing because some test from others contributors are expecting the older use of style...
Do i change the test?
<img width="656" alt="Captura de Tela 2019-09-19 às 09 34 27" src="https://user-images.githubusercontent.com/26484794/65244445-ef026b80-dac0-11e9-8da3-c75e783f8576.png">
@TomAugspurger 
If the old test is wrong then for sure should update
@AntonioAndraues is this still active? If so can you fix up merge conflict?
@WillAyd Yes. I Will fix it...
Sunday in the afternoon i commit it. 
Great thanks @AntonioAndraues - nice PR!
I _think_ this PR should be identified as a breaking change.

```python
import pandas as pd
pd.__version__
df = pd.DataFrame([0.1])
df
df.style.highlight_min()
```
in pandas 0.25.1
![image](https://user-images.githubusercontent.com/13159005/71480033-f5566100-27ee-11ea-89aa-016ae56e9b21.png)

on master
![image](https://user-images.githubusercontent.com/13159005/71479979-d2c44800-27ee-11ea-9f0b-868551a595c0.png)
TST: Asserts all types are caterogical in function fillna with correct error message
State: closed

- [x] closes #13628
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff
- [x] whatsnew entry


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Categorical
description: Categorical Data Type

Comments:
needs `isort pandas/tests/arrays/categorical/test_missing.py`
thanks @LiuSeeker
BUG: Ensure that fill_na in Categorical only replaces null values
State: closed

- [x] closes #26215
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry


Labels:
name: Bug
name: Missing-data
description: np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate
name: Categorical
description: Categorical Data Type

Comments:
Can you add the test from https://github.com/pandas-dev/pandas/pull/27932#discussion_r314318140?

We've had issues in the past with unordered categories where the categories / codes didn't line up.
Sure :) I've added a test where the fill value is a series with the same categories as the original series but in a different order, sorry for having missed this the first time
Looks great @MarcoGorelli. Thanks!
Fix regression in .ix fallback with IntervalIndex
State: closed

closes #27865

cc @jbrockmendel this is another one from limiting the catched exceptions in https://github.com/pandas-dev/pandas/pull/27223/files#diff-7489e290a23303c4db4f803011ecaf8eR121. But also this time it's again rather a bug in the code that raises this error.

For some reason, `DataFrame._get_value` for a DataFrame with an IntervalIndex raises an AttributeError. That's a bug in itself which should be solved (and probably is the cause of more bugs in other places where `_get_value` is used, eg in `.at`). But will open a separate issue for that. 

Labels:
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves

Comments:
Opened https://github.com/pandas-dev/pandas/issues/27927
really? we haven't patched .ix in many releases. And we are removing it. 
@jorisvandenbossche you work on windows right?  can you reproduce the azure failure?
No, sorry, no windows :) (linux)
> really? we haven't patched .ix in many releases. And we are removing it.

Then we maybe also shouldn't have tried to clean up the code .. ;)

Anyway, this is just for 0.25.1, in master afterwards we can remove it again (when removing ix). 
If the idea is that people can upgrade to 0.25, see all the warnings to fix them, to then be able to upgrade to 1.0, then the their code needs to work to be able to do that.
Do I understand correctly: there is a bug in IntervalTree and fixing that would make this fix unnecessary?
Yep. But I think that catching AttributeError here is a bit more conservative for the backport.
Yes, I opened  #27927 for that (the bug appears in other places as well, but there it was already failing before, so not a regression)
Any objections to merge this?
I think we're good. I had to restart the CI earlier for a failed build from the flaky test.
@MeeseeksDev backport to 0.25.x
BUG: Help python csv engine read binary buffers
State: closed

The file buffer given to read_csv could have been opened in
binary mode, but the python csv reader errors on binary buffers.

- [x] closes #23779
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry


Labels:
name: IO CSV
description: read_csv, to_csv

Comments:
What does `Linux py37_np_dev` know that I don't?
Made the requested changes. Do you want a rebase to squash?
Is it normal for random unrelated errors to fail builds?
Yes, we're debugging it still. Restarted azure.
> this is hard to follow, can you just use BufferedIOBase here rather than no_close?

I figured this way was more general in case there were future issues discovered beyond just BufferedIOBase, but consider it done.

Thanks @fiendish!
CLN: Remove incorrect check, comment, rename
State: closed

cc @jorisvandenbossche 

Labels:
name: Clean

Comments:
@jorisvandenbossche did i miss anything?
Given the confusion / discussion around it, I would keep *some* comment explaining why you set the name after finalize (instead of passing it to `_constructor`). At least for me it would not be directly obvious.
Hello @jbrockmendel! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:



There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2019-08-19 22:49:18 UTC
updated per request
@jorisvandenbossche gentle ping
Thanks!
BUG: Lambda function returns KeyError in DataFrameGroupBy.agg
State: closed

- [x] closes #27519 
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry


Labels:
name: Bug
name: Groupby

Comments:
emm, not sure, could take a look, but probably will change few functions as well. This looks indeed a bit complex, but I found out there are quite few issues around it: 
1. allow lambda in `groupby.agg`
2. correctly order different lambda functions and dataset which involves different lambdas on same column, mixed lambda and aggfunc.
3. potentially could remove duplication warning, right now, if `gb.agg(a=(col, 'min'), b=(col, 'min'))` would raise an error, but `gb.agg(a=(col, 'min'), b=(col, lambda x: np.min(x)))` won't. Users might actually want to do it, so that they could use `b` column for other ops while based on `np.min` result. There is also an error test in `test_aggregate.py` which could be potentially removed.
But I will take a look tomorrow to see if I can come up with a simpler version. @TomAugspurger 
Hello @charlesdong1991! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:





There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2019-08-26 18:03:11 UTC
Hi, @TomAugspurger I simplified the code quite a lot, but still in `_normalize_keyword_aggregation` because i feel it is better than changing `_maybe_mangle_lambdas`. Right now, hopefully it wont look too complex. And I also add more complex test cases to prove the lambda works in different scenarios and order of columns is correct. It looks all good locally so feel free to take a look. ^^
wanna give a follow-up review? @TomAugspurger 
Thanks!
Updated table with read_fwf()
State: closed

- [X] closes #27884 
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry

@TomAugspurger Can you merge?

Labels:
name: Docs
name: IO CSV
description: read_csv, to_csv

Comments:
@TomAugspurger Can you merge, or need any other step?
Thanks.
BUG: fix to_timestamp out_of_bounds
State: closed

- [x] closes #19643
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Period
description: Period data type
name: Compat
description: pandas objects compatability with Numpy or Python functions

Comments:
added whatsnew, found this also fixes the analogous bug in PeriodIndex, need to add test for that
updated with additional test
@jorisvandenbossche gentle ping
Thanks!
CLN: Catch more specific exceptions in groupby
State: closed

Working hypothesis is that this will help turn up the problem underlying #27902.  Regardless catching more specific exceptions is worthwhile.

Labels:
name: Groupby
name: Clean

Comments:
Thoughts here?  Conflicts are going to start building up with local branches addressing catching-too-much
can you run groupby perf benchamarks. some of the code you changes was a sore point for perf recently.
Rebased.  asvs look like noise, which makes sense.

```
asv continuous master HEAD -E virtualenv -f 1.1 -b groupby
[...]
       before           after         ratio
     [91e5b85a]       [0c8ae4a9]
     <master>         <winapply>
+         111±3μs          170±2μs     1.53  groupby.GroupByMethods.time_dtype_as_field('float', 'shift', 'transformation')
+      10.0±0.7ms       12.9±0.2ms     1.28  groupby.Categories.time_groupby_nosort
+         121±4μs          135±6μs     1.12  groupby.GroupByMethods.time_dtype_as_group('object', 'any', 'direct')
-         174±4μs          121±4μs     0.70  groupby.GroupByMethods.time_dtype_as_group('datetime', 'any', 'transformation')
```

e.g. re-running `time_groupby_nosort` comes back with NOT SIGNIFICANTLY CHANGED.

Thanks!
Revert 37 pins
State: closed

~second commit is temporary.~

Closes #27907 

cc @jjhelmus 

Labels:

Comments:
what's the trigger condition for the "temporary" commit?
> what's the trigger condition for the "temporary" commit?

Those packages being uploaded to the defaults channel by Anaconda (tonight if all goes well).
We won't merge this with the second commit. Just testing to ensure there aren't any issues with the packages.
Everything seemed to pass using those packages. Reverting the channel commit, which will cause this to fail again. I'll re-run CI once they've been uploaded to defaults.
Corrected Python 3.7.4 package are now in `pkgs/main`, CI should now pass if re-run.
Something went wrong ... Please have  a look at my logs.
CI: Skip flaky test that fails Azure to make pipeline succeed
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:

Comments:
I'd like to try to troubleshoot the failure before suppressing it.  Especially since its windows, once it is xfailed it is likely to slip through the cracks for a long time.
agree! i did not know how to test in windows environment, thus hard to debug for me... so I was trying to avoid it by directly suppressing the issue. @jbrockmendel this suppressing is definitely not long term