BrandonFors/pandas-github-issues huggingface dataset:
DOC: Add description for tz parameter in DatetimeIndex
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Docs

Comments:
@nsiregar Should we have `time zone` or `timezone`? 
@za it's `time zone` in dictionary.
@nsiregar I think CI error is unrelated - can you merge master and repush to be sure?
@nsiregar can you please merge master into your branch, and address the remaining comment? Thanks!
Closing as stale.
CLN 29574 Replace old string formating
State: closed

Its my first contribution, modified the str formating to use fstrings
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ]  pandas/tests/frame/test_to_csv.py


Labels:
name: Code Style
description: Code style, linting, code_checks

Comments:
Thanks @panjacek
DOC: Fix pandas.index.copy summary documentation
State: closed

- [ ] closes https://github.com/pandanistas/pandanistas_sprint_jakarta2020/issues/8
- [ ] tests added / passed
- [ ] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry
Output of `python scripts/validate_docstrings.py pandas.Index.copy`

################################################################################
######################## Docstring (pandas.Index.copy)  ########################
################################################################################

Make a copy of this object.

Name and dtype sets those attributes on the new object.

Parameters
----------
name : Label, optional
    Set name for new object.
deep : bool, default False
dtype : numpy dtype or pandas type, optional
    Set dtype for new object.
names : list-like, optional
    Kept for compatibility with MultiIndex. Should not be used.

Returns
-------
Index
    Index refer to new object which is a copy of this object.

Notes
-----
In most cases, there should be no functional difference from using
``deep``, but if ``deep`` is passed it will attempt to deepcopy.

################################################################################
################################## Validation ##################################
################################################################################

3 Errors found:
        Parameter "deep" has no description
        See Also section not found
        No examples section found

Labels:
name: Docs

Comments:
Thanks @asepwhite. If you can please address the review comments in a follow up PR that would be great.
doc: Add period to parameter description
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry

```

there are a few errors left:
################################################################################
################################## Validation ##################################
################################################################################

11 Errors found:
        Parameters {'**fields', 'name', 'ordinal'} not documented
        Unknown parameters {'second', 'minute', 'year', 'month', 'quarter', 'hour', 'day'}
        Parameter "year" has no description
        Parameter "month" has no description
        Parameter "quarter" has no description
        Parameter "day" has no description
        Parameter "hour" has no description
        Parameter "minute" has no description
        Parameter "second" has no description
        Parameter "dtype" has no description
        Examples do not pass tests
```

Labels:
name: Docs

Comments:
Thanks @DavaIlhamHaeruzaman
DOC: Update pandas.DataFrame.droplevel docstring
State: closed

- [ ] closes https://github.com/pandanistas/pandanistas_sprint_jakarta2020/issues/23
- [ ] tests added / passed
- [X] passes `black pandas`
- [X] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry

Output of `python scripts/validate_docstrings.py pandas.DataFrame.droplevel`:
```
################################################################################
#################### Docstring (pandas.DataFrame.droplevel) ####################
################################################################################

Return DataFrame with requested index / column level(s) removed.

.. versionadded:: 0.24.0

Parameters
----------
level : int, str, or list-like
    If a string is given, must be the name of a level
    If list-like, elements must be names or positional indexes
    of levels.

axis : {0 or 'index', 1 or 'columns'}, default 0
    Axis along which the level(s) is removed:

    * 0 or 'index': remove level(s) in column.
    * 1 or 'columns': remove level(s) in row.

Returns
-------
DataFrame
    DataFrame with requested index / column level(s) removed.

Examples
--------
>>> df = pd.DataFrame([
...     [1, 2, 3, 4],
...     [5, 6, 7, 8],
...     [9, 10, 11, 12]
... ]).set_index([0, 1]).rename_axis(['a', 'b'])

>>> df.columns = pd.MultiIndex.from_tuples([
...     ('c', 'e'), ('d', 'f')
... ], names=['level_1', 'level_2'])

>>> df
level_1   c   d
level_2   e   f
a b
1 2      3   4
5 6      7   8
9 10    11  12

>>> df.droplevel('a')
level_1   c   d
level_2   e   f
b
2        3   4
6        7   8
10      11  12

>>> df.droplevel('level_2', axis=1)
level_1   c   d
a b
1 2      3   4
5 6      7   8
9 10    11  12

################################################################################
################################## Validation ##################################
################################################################################

1 Errors found:
	See Also section not found
```

Labels:
name: Docs

Comments:
@asyarif93 can you merge master and repush again to get CI green?
Thanks @asyarif93
DOC: Fix SA04 errors from DataFrame.melt
State: closed

- [ ] closes https://github.com/pandanistas/pandanistas_sprint_jakarta2020/issues/10
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Docs

Comments:
@giovanism Thank you for the PR!

Can you run "black" on ```pandas/core/frame.py```?
Closing as stale.
@giovanism in case you want to further work on this (based on your gitter comment), you might need to open a new PR. 
(I can't reopen this PR, github indicates this is because there has been a force push to this branch after the PR was closed)
DOC PR09 Add missing . on freq parameter on groupby.py
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Docs

Comments:
Thanks @za 
@simonjayhawkins Happy to help!
DOC: Add See Also section on series.py
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Docs

Comments:
This fixed the error, but I don't think we want to add a WIP note into the official documentation of pandas. We can discuss what to add in the see also if you want. 
@datapythonista Yes please. Let's get this PR done. What relevant function that we can add at `See Also` section? Any hint? `s.to_markdown()`?
After had a discussion with @datapythonista should add `See Also` section in the parent file somewhere in `pandas/core/generic.py`.
Closing this, as this needs to be started from the beginning. Better open a new PR with the correct changes. Thanks!
BUG: Fix wrong reading sparse matrix
State: closed

- [x] closes #29814 
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry


Labels:
name: Sparse
description: Sparse Data Type

Comments:
Previously the comment in #29814 had suggested using `data.indices` instead of `data.nonzero()`. However, this PR instead strips any explicit zeros in the sparse input.

One of my use cases involves taking the difference of two sparse matrices, and then passing the result to scipy.sparse and pandas (ideally filling missing with NaN, instead of 0). There's a meaningful difference between "this element has no information from either matrix" (NaN) and "this element had the exact same value in both matrices" (0). It looks to me like this PR would obscure that distinction.

This would also make it hard to round-trip from `scipy.sparse`, which seems to handle explicit zeros.

Is there a solution that doesn't involve losing the information about explicit zeros?

(This wouldn't be a show-stopper for us; right now export to pandas is basically a convenience for users who want it. We'd just have to add a warning on that function.)
@jorisvandenbossche or @TomAugspurger any thoughts here?
What’s the performance cost of your PR be using indicies? It seems like stripping explicit zeros will require scanning the full data?
@TomAugspurger 
Eliminating explicit zeros will require scanning the full data in my PR.
I didn't come up with other way(not scanning the full data). Is there another good way?
@m7142yosuke as I also mentioned on the issue, there has been a PR that refactored the `from_spmatrix` dataframe accessor method: https://github.com/pandas-dev/pandas/pull/32825

This seems to have fixed the DataFrame bug (it has its own logic now, and does not use SparseArray.from_spmatrix under the hood), but so the SparseArray is still buggy (what you are fixing here). 
But so, it's probably worth to take a look at how that PR changed the implementation, and do something similar here (or try to see if some of the code can be shared).

Also adding a test case for the original case from https://github.com/pandas-dev/pandas/issues/29814 would be welcome
@jorisvandenbossche @jreback 
Please review this PR

Done.
- Refactoring code referencing #32825
- Add a test case for the original case from #29814 
thanks @m7142yosuke
CLN: 29547 replace old string formatting 7
State: closed

I splitted PR #31844 in batches, this is the seventh
For this PR I ran the command `grep -l -R -e '%s' -e '%d' -e '\.format(' --include=*.{py,pyx} pandas/` and checked all the files that were returned for `.format(` and changed the old string format for the corresponding `fstrings` to attempt a full clean of, [#29547](https://github.com/pandas-dev/pandas/issues/29547). I may have missed something so is a good idea to double check just in case

- [ x  ] tests added / passed
- [ x ] passes `black pandas`
- [ x ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`

Labels:
name: Code Style
description: Code style, linting, code_checks

Comments:
Looks good. Can you merge master?
> Looks good. Can you merge master?

no point atm. failing ci.
Thanks @3vts
REF: dont use numexpr in places where it doesnt help.
State: closed

DataFrame ops will still use it when dispatching to series/block ops.

Perf neutral:

```
In [3]: arr = np.arange(10**4).reshape(100, 100)                                                                                                                                    
In [4]: df = pd.DataFrame(arr)                                                                                                                                                      
In [5]: df[4] = df[4].astype(float)                                                                                                                                                 
In [6]: ser = df[0]                     
                                                                                                                                            
In [8]: %timeit res = df+ser                                                                                                                                                        
19.8 ms ± 317 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)  # <-- PR
19.8 ms ± 267 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)  # <-- master

In [10]: %timeit res2 = df.add(ser, axis=0)                                                                                                                                         
19.2 ms ± 276 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)  # <-- PR
19.2 ms ± 301 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)  # <-- master

In [13]: %timeit res3 = df + df                                                                                                                                                     
23.6 ms ± 382 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)  # <-- PR
24 ms ± 551 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)  # <-- master
```

Labels:
name: Performance
description: Memory or execution speed performance
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode

Comments:
closing in favor of #32047.
CLN 29547 Replace old string formatting syntax with f-strings
State: closed

Addresses #29547 for pandas/io/sas/sas_xport.py. I searched through for instances of the `%` operator and `.format`.

- [X] passes `black pandas`
- [X] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`

I noticed that `_read_sas_doc` doesn't seem to be used anymore. I could delete that as well, or we could save that for a separate PR.


Labels:
name: Code Style
description: Code style, linting, code_checks

Comments:
thanks
CLN: 29547 replace old string formatting 6
State: closed

I splitted PR #31844 in batches, this is the sixth
For this PR I ran the command `grep -l -R -e '%s' -e '%d' -e '\.format(' --include=*.{py,pyx} pandas/` and checked all the files that were returned for `.format(` and changed the old string format for the corresponding `fstrings` to attempt a full clean of, [#29547](https://github.com/pandas-dev/pandas/issues/29547). I may have missed something so is a good idea to double check just in case

- [ x  ] tests added / passed
- [ x ] passes `black pandas`
- [ x ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`

Labels:
name: Code Style
description: Code style, linting, code_checks

Comments:
Thanks @3vts
Avoid importing from pandas at _libs files
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry

Something that brought at to me at first [here](https://github.com/pandas-dev/pandas/pull/30395#discussion_r363101229)

---

cc @jbrockmendel 

Labels:

Comments:
What is the benefit of doing this? I think I prefer it as is so we have only one way of testing for 32 bit
@WillAyd TBH I don't really know, all I do know is that one time I imported ```pandas.compat``` at the top, and I was requested to change it.
@MomIsBestFriend is right that in general we try to avoid non-_libs imports in _libs, but that rule is not hard-and-fast, e.g. _libs.parsers is all over the place.  tslibs is the only place where im really adamant about this.
Unless there is a performance improvement I'd be -1 on this change
what actually need is a *cython* version of compat that we can use; this is not what we are going for.
CI: Removed pattern check for specific modules
State: closed

- [ ] closes #xxxx
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: CI
description: Continuous Integration

Comments:
Looks good. Do you have a larger plan with doctest or just this?
@WillAyd The end goal is to have the line
```
pytest -q --doctest-module pandas/
```

But ATM it's not possible.

---

I am just making sure that we keep some modules with correct examples.
My next PR will cover ```pd.Series```
Thanks @MomIsBestFriend 
Could be convinced otherwise but I think worth backporting to keep CI consistent - @MomIsBestFriend can you do manually?
@WillAyd Sure! I have opened a PR here #31993
STY: Fixed wrong placement of whitespace
State: closed

- [x] ref #30755
- [ ] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Code Style
description: Code style, linting, code_checks

Comments:
Thanks @MomIsBestFriend
DOC: Use use recommended library over deprecated library
State: closed

Importing `pandas.io.json.json_normalize` in pandas 1.0.1 gives the following warning

```
FutureWarning: pandas.io.json.json_normalize is deprecated, use pandas.json_normalize instead
```

This PR updates the documentation here https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.json_normalize.html to use the recommended approach.

Labels:
name: Docs

Comments:
Thanks @martinbjeldbak !
CLN: 29547 replace old string formatting 5
State: closed

I splitted PR #31844 in batches, this is the fifth
For this PR I ran the command `grep -l -R -e '%s' -e '%d' -e '\.format(' --include=*.{py,pyx} pandas/` and checked all the files that were returned for `.format(` and changed the old string format for the corresponding `fstrings` to attempt a full clean of, [#29547](https://github.com/pandas-dev/pandas/issues/29547). I may have missed something so is a good idea to double check just in case

- [ x  ] tests added / passed
- [ x ] passes `black pandas`
- [ x ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`

Labels:
name: Code Style
description: Code style, linting, code_checks

Comments:
Great thanks @3vts
CLN: 29547 replace old string formatting 4
State: closed

I splitted PR #31844 in batches, this is the fourth
For this PR I ran the command `grep -l -R -e '%s' -e '%d' -e '\.format(' --include=*.{py,pyx} pandas/` and checked all the files that were returned for `.format(` and changed the old string format for the corresponding `fstrings` to attempt a full clean of, [#29547](https://github.com/pandas-dev/pandas/issues/29547). I may have missed something so is a good idea to double check just in case

- [ x  ] tests added / passed
- [ x ] passes `black pandas`
- [ x ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`

Labels:
name: Code Style
description: Code style, linting, code_checks
name: Clean

Comments:
Thanks @3vts
REF: remove _convert_scalar_indexer
State: closed

This sits on top of #31867, so is partially a demonstration of how much complication is caused by our inconsistent error-raising.

Labels:
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: Error Reporting
description: Incorrect or improved errors from pandas

Comments:
@jbrockmendel can you merge master
xref #25996
Definitely simplifies things so +1 there. Does this need to get bundled with the other PR to comprehensively raise a `KeyError`? Again on board, just want to be careful of messaging and backwards compat for users that previously caught TypeError (if even an issue - maybe they always caught both?)
> Does this need to get bundled with the other PR to comprehensively raise a KeyError?

Yes.
closing to clear the queue, will re-open following #31867.
re-opened following #31867
love it. if you can rebase (not sure if it was before). merge on green.
Thanks @jbrockmendel
CLN: D414: Section has no content
State: closed


Labels:
name: Clean

Comments:
cool thanks! just out of curiosity how many more checks do you plan on running?
Some others that could be useful may conflict with validate_docstrings or dynamic docstring generation.

pydocstyle is a static checker whereas validate_docstrings imports the docstrings and checks.

pydocstyle can check all static docstrings. validate_docstrings only checks the public docstrings.

I think the new doc decorator could be an advantage here to allow static checking of more of the dynamically generated docstrings.

I think that the 'quick hit' checks done so far would add sufficient value if included in CI and enforced.

I see two main benefits. Long term to improve the quality of non-public docstrings to help contributers, but in the short term to support the review process by formalising some more style nits.
add eval examples
State: closed

xref #31952 

Screenshot for `pandas.eval`:
![image](https://user-images.githubusercontent.com/33491632/74459941-04b76980-4e84-11ea-9f1b-2b6ad9cb5494.png)

Screenshot for `pandas.DataFrame.eval`:
![image](https://user-images.githubusercontent.com/33491632/74460007-23b5fb80-4e84-11ea-931b-0f73eebb6725.png)


Labels:
name: Docs
name: expressions
description: pd.eval, query

Comments:
Great thanks @MarcoGorelli
CLN: index related attributes on Series/DataFrame
State: closed

Followup to #31126. IMO the current approach to adding the index related class attributes is too indirect and therefore unnecessary difficult to follow. Just adding the class attributes directly on the relevant class makes reading the code easier, IMO.

Notice that the types are already defined in pandas/core/generic.py:304-316.

Labels:
name: Clean

Comments:
Ok, I've made them strings/ints.
Yea definitely find this easier to follow as well
Thanks @topper-123
API: replace() should raise an exception if invalid argument is given
State: closed

- [x] closes #18634
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry

I used `is_scalar()` instead of `is_numeric()` to allow data structures like pd.Timestamp to be used in `replace()`. See tests in pandas/tests/frame/methods/test_replace.py, pandas/tests/series/methods/test_replace.py for examples.

Labels:
name: Error Reporting
description: Incorrect or improved errors from pandas

Comments:
thanks @a-y-khan 
Glad I could make a contribution!
CLN: 29547 replace old string formatting 3
State: closed

I splitted PR #31844 in batches, this is the third
For this PR I ran the command `grep -l -R -e '%s' -e '%d' -e '\.format(' --include=*.{py,pyx} pandas/` and checked all the files that were returned for `.format(` and changed the old string format for the corresponding `fstrings` to attempt a full clean of, [#29547](https://github.com/pandas-dev/pandas/issues/29547). I may have missed something so is a good idea to double check just in case

- [ x  ] tests added / passed
- [ x ] passes `black pandas`
- [ x ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`

Labels:
name: Code Style
description: Code style, linting, code_checks
name: Clean

Comments:
> @simonjayhawkins @WillAyd do we have a poilicy on this? e.g. i don't think its a big deal to have extra f'' on lines that don't need them.

yeah, no big deal since we don't have code check to enforce.

my preference would be to not include but would not request changes on a PR that does.

however, would accept clean-up PRs that remove unused.

> > @simonjayhawkins @WillAyd do we have a poilicy on this? e.g. i don't think its a big deal to have extra f'' on lines that don't need them.
> 
> yeah, no big deal since we don't have code check to enforce.
> 
> my preference would be to not include but would not request changes on a PR that does.
> 
> however, would accept clean-up PRs that remove unused.

Ok will clean it since there is no gain and this is a cleanup anyways
> Ok will clean it since there is no gain and this is a cleanup anyways

yeah, that's probably the objection here, the intentional adding of the unused string literal prefixes.
Hello @3vts! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:











There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2020-02-13 19:36:56 UTC
Thanks @3vts