BrandonFors/pandas-github-issues huggingface dataset:
Added docs to fix GL08 errors in CustomBusinessHours
State: closed

- [ ] xref #27977
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry

Currently even after adding the docstrings the command `python scripts/validate_docstrings.py --errors=GL08` does not pick up the changes.

Labels:
name: Docs

Comments:
> Currently even after adding the docstrings the command python scripts/validate_docstrings.py --errors=GL08 does not pick up the changes.

not sure we are checking this yet (idea is to fix the errors then add the check). dependens on how much fails across the codebase.
> not sure we are checking this yet (idea is to fix the errors then add the check). dependens on how much fails across the codebase.

There are still lots of GL08 errors in many files, so adding this error to the check would require lots more docstrings.

Though this

> Currently even after adding the docstrings the command python scripts/validate_docstrings.py --errors=GL08 does not pick up the changes.

may require further investigation, not sure why it wouldn't pick up the changes (this happened even after running `python setup.py build_ext -i -j 4`)
thanks @mohdkashif93
DOC: SS01 docstrings errors fixed
State: closed

xref #27977
- [ ] closes part of [MarcoGorelli/PyDataGlobal2020-sprint#4](https://github.com/MarcoGorelli/PyDataGlobal2020-sprint/issues/4)
- [ ] tests added / passed
- [X] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Docs

Comments:
Does this fix all the SS01 errors? If so, do you want to add this error code to the relevant check in `scripts/code_checks.sh`?
Sure @MarcoGorelli , thanks
I think you accidentally committed path_to_file.zip ?
@MarcoGorelli Indeed... Got it removed!
Thanks @lucasrodes , and congrats on your first PR to pandas!
DOC: add examples to docstrings of assert_ functions
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Docs

Comments:
Thanks a lot @nguevara !
namespace consistency for test_dtype.py
State: closed

- [x ] references #37838
- [x] tests added / passed
- [x] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Code Style
description: Code style, linting, code_checks

Comments:
Can you drop the first commit?

So
```
git fetch --all --prune
git rebase -i upstream/master
```
then, in the interactive window, change the first `pick` to `drop`. Then (you may need to run this one twice)
```
pre-commit run --files pandas/tests/arithmetic/test_interval.py
```
Then,
```
git add pandas/tests/arithmetic/test_interval.py
git commit -m "fixup"  # or whatever message you like
git push origin namespace -f
```

Let me know if anything trips you up here
Fix cases of inconsistent namespacing in tests
State: closed

Additional fixes for MarcoGorelli/PyDataGlobal2020-sprint#1.
Continues the work of lpkirwin in #37838.

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Code Style
description: Code style, linting, code_checks

Comments:
Thanks @jan-mue !
CI: Skip test when no xlwt or openpyxl
State: closed

Ref: https://github.com/pandas-dev/pandas/pull/37818

Seeing error in CI: https://dev.azure.com/pandas-dev/pandas/_build/results?buildId=47726&view=logs&j=acc1347f-8235-55b0-95c7-0e2189e61659&t=486f34f1-eda6-516a-fc5d-d8195128dacd

Skip logic same as `test_to_excel` in this file

cc @jreback 

Labels:
name: CI
description: Continuous Integration
name: IO Excel
description: read_excel, to_excel

Comments:
thanks @alimcmaster1 merge on green.
> thanks @alimcmaster1 merge on green.

will do thanks!
Potentially flaky test below - other issues seem fixed.
```
self = <pandas.tests.plotting.test_converter.TestRegistration object at 0x000001C5058A3C48>

    def test_pandas_plots_register(self):
        plt = pytest.importorskip("matplotlib.pyplot")
        s = Series(range(12), index=date_range("2017", periods=12))
        # Set to the "warn" state, in case this isn't the first test run
        with tm.assert_produces_warning(None) as w:
            s.plot()
    
>       assert len(w) == 0
E       assert 2 == 0
E        +  where 2 = len([<warnings.WarningMessage object at 0x000001C51321A188>, <warnings.WarningMessage object at 0x000001C506E39348>])

pandas\tests\plotting\test_converter.py:82: AssertionError
```
/azp run
<samp>
Azure Pipelines successfully started running 1 pipeline(s).<br>

</samp>
its fine to merge this
TST: Add test for setting item using python list (#19406)
State: closed

- [X] closes #19406 
- [X] tests added / passed
- [X] passes `black pandas`
- [X] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [NA] whatsnew entry


Labels:
name: Testing
description: pandas testing functions or related to the test suite

Comments:
One nitpick from my side: you coud parametrize somethink like ``func`` with ``[list, np.array, Series]`` and define mask in the test through ``mask=func([True, False, True])`` Would make parametrization shorter
@phofl Ooh nice. Will do.
@phofl There. Much more concise. ✨
lgtm
Thanks @paxcodes , congrats on your first PR to pandas! Do let us know if you want pointers for other issues if you'd like to tackle them too
Thanks @MarcoGorelli and again, thanks for your time yesterday at the sprint!
DOC: section in indexing user guide to show use of np.where
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry

Add a section in the user guide to illustrate the use of `np.where` and `np.select` to enlarge a dataframe conditionally.  The example is taken from SO, as mentioned in [this issue](https://github.com/MarcoGorelli/PyDataGlobal2020-sprint/issues/5#issuecomment-727241262).

Labels:
name: Docs
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves

Comments:
thanks @suvayu
Fix cases of inconsistent namespacing in tests
State: closed

Sometimes e.g. Series and pd.Series are used
in the same test file. This fixes some of these
cases, generally by using the explicitly
imported class.

- [ ] closes #xxxx
- [ ] tests added / passed
- [x] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Code Style
description: Code style, linting, code_checks

Comments:
so ideally we would add these to the pre-commit rules that checks this. these are likley inconsistent in more places.

so this PR is ok, but would prefer that you change them across the entire codebase so that we are enforcing the rule. (and it may make sense to not change everything, e.g. do just Grouper, or just the Index ones). I am not sure how big a change this is.
For context, this follows on from @dsaxton 's work on making the namespace using more consistent. Will modify the `scripts/check_for_inconsistent_pandas_namespace.py` file accordingly once this and the rest of the fixes of these kind are in 
> I am not sure how big a change this is.

It's not too big, should get them done today along with the expanded CI check (see https://github.com/MarcoGorelli/PyDataGlobal2020-sprint/issues/1 )
> > I am not sure how big a change this is.
> 
> It's not too big, should get them done today along with the expanded CI check (see [MarcoGorelli/PyDataGlobal2020-sprint#1](https://github.com/MarcoGorelli/PyDataGlobal2020-sprint/issues/1) )

ok sounds great. feel free to merge these on green.
Thanks @lpkirwin
CLN: remove unnecessary close calls and add a few necessary ones
State: closed

- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`

Follow up to #37639. Close file handles as early as possible.

Remove `close` calls that are already within a try-except/finally block that closes the file handles.

Add `close` call for `read_json` when we read the entire content in one go.

Add close call in `read_csv` when reaching the end of the iterator.

Put `to/read_excel` in a try-finally block to make sure that file handles are closed even in case of an exception.

Labels:
name: IO Data
description: IO issues that don't fit into a more specific label

Comments:
small comment, ping on green.
thanks @twoertwein
Backport PR #37787 on branch 1.1.x (Fix regression for loc and __setitem__ when one-dimensional tuple was given for MultiIndex)
State: closed

Backport PR #37787: Fix regression for loc and __setitem__ when one-dimensional tuple was given for MultiIndex

Labels:
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: MultiIndex

Comments:
fixture ``frame_or_series`` is not found
@phofl yeah can you cherry-pick this commit on the the 1.1.x branch and fix up.
Yep, will open a new PR.
#37849
COMPAT: should an empty string match a format (or just be NaT)
State: closed

- [x] closes #13044
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry (not applicable)


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Datetime
description: Datetime data dtype

Comments:
Seems like the failing tests are not related to my changes - @alimcmaster1 could you please verify? How should I proceed with this issue? Would rebasing on current master help?
thanks @fgebhart very nice
BUG: Parse missing values using read_json with dtype=False to NaN instead of None (GH28501)
State: closed


- [x] closes #28501 
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry


Labels:
name: Missing-data
description: np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate
name: IO JSON
description: read_json, to_json, json_normalize

Comments:
Nice PR! Looks like there is a merge conflict in the whatsnew - if you can fix that up we can get this merged
Merged master, but CI fails due to unrelated reasons. Will retry with new updates from master later today
@avinashpancham can you merge master once again
@jreback merged master and updated whatsnew entry
hmm precommit checks are failing, can you merge master and ping on green
Still some issues, will try again tomorrow
thanks @avinashpancham 
I'm not sure that there was consensus in the issue discussion on the correct behaviour or whether it is a bug. Maybe worth considering moving the release note into the breaking changes section in case anyone is depending on this behaviour.
> I'm not sure that there was consensus in the issue discussion on the correct behaviour or whether it is a bug. Maybe worth considering moving the release note into the breaking changes section in case anyone is depending on this behaviour.

type dtype option is actually pretty odd anyhow (and should deprecate)


I think most voices in the issue actually argued that the current behaviour was correct, and so IMO we should not have changed it to start with. 
there was exactly 1 comment
Will opened the issue, and 2 people commented on it arguing not to change it (and a third one commented as well, but off-topic), no one else argued in favor of the change.

Anyway, I don't think counting the numbers is that important, but I agree with @simonjayhawkins that there was certainly no clear consensus


On the None vs NaN, I am personally quite ambivalent, but when disabling parsing I think we should *not* return float dtype for this case. 
you can revert if u want
not that important 

this was removing a special case though - it was very odd
Return view for xs when droplevel=False with regular Index
State: closed

- [x] xref #37776
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`

Is there a better way to test, if the object is a view?

cc @jbrockmendel 


Labels:
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves

Comments:
thanks for following up on this
thanks @phofl
BUG: MultiIndex.drop does not raise if labels are partially found
State: closed

- [x] closes #37820
- [x] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry


Labels:
name: Bug
name: MultiIndex

Comments:
Hello @GYHHAHA! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:





There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2020-11-26 00:35:30 UTC
cc @jreback 
thanks @GYHHAHA very nice, keep em coming!
CLN: File handling for PyArrow parquet
State: closed

- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`

De-duplicate file handling of PyArrows' parquet.

Labels:
name: Refactor
description: Internal refactoring of code
name: IO Parquet
description: parquet, feather

Comments:
I'm looking into this, I fear that the docs are testing a case that isn't captured in the tests itself.
@twoertwein can you merge master and apply suggestions, ping on greenish
@jreback green'ish. I think I incorporated @jorisvandenbossche comments (renamed variable and function; do not type-abuse IOHandles)
thanks @twoertwein
BUG: DataFrame reductions inconsistent with Series counterparts
State: closed

- [x] closes #36076
- [x] closes #28949
- [x] closes #21020
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry


Labels:
name: Categorical
description: Categorical Data Type
name: API - Consistency
description: Internal Consistency of API/Behavior
name: Reduction Operations
description: sum, mean, min, max, etc.
name: Nuisance Columns
description: Identifying/Dropping nuisance columns in reductions, groupby.add, DataFrame.apply

Comments:
#28948 is an incorrect ref (top of PR)
BUG: Allow custom error values in parse_dates argument of read_sql like functions (GH35185)
State: closed

- [x] closes #35185 
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry


Labels:
name: IO SQL
description: to_sql, read_sql, read_sql_query

Comments:
@jreback could you have another look?
@avinashpancham ok this is good. Can you do a followon PR (or open an issue to do so), that adds examples to the `read_sql` doc-string. e.g. using a passed dict like this (but since there are NO examples would be good to have some general examples using the various options).
Sure, will create a follow on PR to add some examples in the docstring.
Thanks @avinashpancham
CI: pin build to use pyarrow==1.0
State: closed

closes #37306

@jorisvandenbossche picked build at random. we can discuss here what the preferences are? cc @jreback 

Labels:
name: CI
description: Continuous Integration
name: IO Parquet
description: parquet, feather

Comments:
thanks @simonjayhawkins 
We should maybe try to get some of the builds that are now in practice using 1.0 (but not pinned) or pinned to 0.15 to use pyarrow 2.0, because we now only have one build with pyarrow 2.0 (the windows 3.8 one). For some reason the Travis builds are using 1.0, although they are not pinned.
Deprecate partial slicing of unordered DatetimeIndex when both keys are not present
State: closed

- [x] xref #18531
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry

cc @jreback @jorisvandenbossche 

This would deprecate #37796 before removing it. 

Labels:
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: Deprecate
description: Functionality to remove in pandas

Comments:
thanks @phofl
ENH: storage_options for to_excel
State: closed

- [ ] ref #33987 (follow up to #37639)
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry: not added, already implied in whatsnew of #37639?

Labels:
name: Enhancement
name: IO Excel
description: read_excel, to_excel

Comments:
thanks @twoertwein 
is there a way to re-use documentation? We have blocks about compression storage_options in many places. It would be cool to have them in one place and then import them.
> is there a way to re-use documentation? We have blocks about compression storage_options in many places. It would be cool to have them in one place and then import them.

yes for sure, we have the _shared_docs infrastructure, but for excel its prob enough just to use the Appenders
Think this broke CI - https://github.com/pandas-dev/pandas/pull/37818/checks?check_run_id=1399068710 - opening up a fix
oh yes! I think a pytest.importorskip should fix this. @alimcmaster1  Are you already making a PR?

edit: just saw your PR :) Thank you!
REF: define nanargminmax without values_for_argsort
State: closed


Labels:
name: Refactor
description: Internal refactoring of code
name: ExtensionArray
description: Extending pandas with custom dtypes or arrays.

Comments:
This was done explicitly this way in the original PR https://github.com/pandas-dev/pandas/pull/27801 (@jbrockmendel you actually explicitly questioned that use, which led to some discussion about it: https://github.com/pandas-dev/pandas/pull/27801#issuecomment-626183973)

This gives a big slowdown, as doing a full argsort (to then only use a single index) is more expensive as argmin or argmax.
Thanks for linking back there.

> But so, then we can simply expand this implmentation note in the docs to: you can either implement _values_for_argsort (and then argsort, argmin and argmax provide a default implementation based on those values), or implement argsort, argmin and argmax yourself. I don't think that is any problem?

I think a Technically Correct solution would look something like:

```
class ExtensionArray:
    def argsort(self, ...):
        raise NotImplementedError

    def argmax(self):
        return the_implementation_using_argsort

class VFAMixin:
    def _values_for_argsort(self):
        return the_implementation_in_master
    
    def argsort(self, ...):
        return the_implementation_in_master
    
    def argmax(self):
        return the_implementation_using_values_for_argsort
```

Getting there may be a hassle given backwards compatibility, but i think getting rid of the ambiguity as to values_for_argsort is worth it.
What's the benefit of your proposed scheme compared to the current situation? (i.e. what is quoted, assuming we properly document this)
I think its clearer.  It would also make it much harder for us mess up and use _values_for_argsort in places we're not supposed to (even if we document it correctly, many a slip)
But also in your scheme we have a `the_implementation_using_values_for_argsort` function that will use `_values_for_argsort`, which was exactly the purpose of ``nanargminmax`` (the version from before this PR). 
But it wouldn't be in the base class EA, which wouldn't have values_for_argsort at all
@jorisvandenbossche if you want to revert this i wont argue the point
prob should open a new issue so don't forget about this @jbrockmendel 
opened #38201 to revert this
Ah, I already did that yesterday -> #38177

I think Jeff was asking for an issue about a better solution (eg your sketch above at https://github.com/pandas-dev/pandas/pull/37815#issuecomment-727257259). Now, I think we can use the existing https://github.com/pandas-dev/pandas/issues/27218 for that.
CI: The `set-env` and `add-path` commands are deprecated and will be disabled on November 16th.
State: closed

https://github.com/pandas-dev/pandas/actions/runs/361523610

Labels:
name: CI
description: Continuous Integration

Comments:
sure, marked for 1.1.5 as likley it will break there too
@meeseeksdev backport 1.1.x
DOC add note about how to skip tests during CI
State: closed

Tomorrow there will be a PyDataGlobal sprint, which'll probably involve many, many documentation-related pull requests, so this might help with not using up azure pipelines resources unnecessarily

EDIT
----
doesn't seem to be working :flushed:

Labels:

Comments:
:thinking: would've expected this to work, from https://docs.microsoft.com/en-us/azure/devops/pipelines/repos/azure-repos-git?view=azure-devops&tabs=yaml#skipping-ci-for-individual-commits
DOC: capitalize Python as proper noun
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Docs

Comments:
Thanks - can you also fix this in other doc pages, such as doc/source/ecosystem.rst and doc/source/user_guide/cookbook.rst? There's a few other places where it appears uncapitalised
> Thanks - can you also fix this in other doc pages, such as doc/source/ecosystem.rst and doc/source/user_guide/cookbook.rst? There's a few other places where it appears uncapitalised

@MarcoGorelli sure thing! I ran:

```
grep python -r *
grep jupyter -r *
```

in the `doc/source/` directory to find instances and changed them. I also found some other random capitalizations along the way.

I didn't touch anything in the `whatsnew/` directory.
@MarcoGorelli hopefully got them now! Thanks for the suggestions. (Side note, just realized I can't spell my branches right and caused me some confusion while making edits 😅)
thanks @erictleung
ENH: Add end and end_day options for origin from resample
State: closed

- [x] closes #37804
- [x] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Enhancement
name: Resample
description: resample method

Comments:
Hello @GYHHAHA! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:











There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2020-11-28 10:31:24 UTC
i think this is an edge case, but no big deal to add.
cc @mroeschke 
I will rewrite this [part](https://pandas.pydata.org/docs/user_guide/timeseries.html#use-origin-or-offset-to-adjust-the-start-of-the-bins) of user guide. Need detail enhancement description in whatsnew? 

Thanks for doc-string, tests, codes review.