BrandonFors/pandas-github-issues huggingface dataset:
DOC: Fix capitalization among headings in documentation files (#32550)
State: closed

Following #32978 and #32944.

Headers updated for the following files:
```
- [ ] doc/source/getting_started/comparison/comparison_with_stata.rst
- [ ] doc/source/getting_started/intro_tutorials/01_table_oriented.rst
- [ ] doc/source/getting_started/tutorials.rst
- [ ] doc/source/reference/arrays.rst
- [ ] doc/source/reference/window.rst
- [ ] doc/source/user_guide/boolean.rst
```

Labels:
name: Docs

Comments:
Can you please merge master?
HI @MomIsBestFriend , sorry I am quite new to git. Could you please expand on what I should do?
> HI @MomIsBestFriend , sorry I am quite new to git. Could you please expand on what I should do?

@themien Sure!

If your working environment is set up with the instructions found in the [contributing guide](https://dev.pandas.io/docs/development/contributing.html), you can merge "master" by running:

```
git checkout bug_32550_part_3
git fetch upstream
git merge upstream/master
```

And then, push your code again:

```
git push origin bug_32550_part_3
```

@MomIsBestFriend great! Thank you
looks ok. @datapythonista @jorisvandenbossche 
Thanks @themien, nice clean up
BUG: Multiple unstack using row index level labels and multi level columns DataFrame
State: closed

- [x] closes #24729
- [x] closes #28306
- [x] closes #32624
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry

This should fix the issue with a long list of indices and unstack. The issue appeared with 6 indices for the first time.


Labels:
name: Bug
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode
name: MultiIndex

Comments:
Thanks @phofl This also fixes #24729 and maybe others. can you add a test for those and also add a what's new.

> This also fixes #24729 and maybe others.

also fixes #28306

I'll add the tests in the evening
Added the tests for both issues and the whats new entry.

Reverted the typo fix. I'll add a pull request separately for this shortly
Thanks @phofl!
DOC: Modify validate_rst_title_capitalization.py script
State: closed

Regarding issue #32550.

@datapythonista

Modify script in order to not take into account words with 2 or more capital letters, which should all be proper words.

Labels:
name: Docs

Comments:
Sorry, not sure if we understood each other. We're adding all the exceptions to the list, which I think is good enough. I think this will allow cases like all capital letter, which we don't want, unless explicitly in the list. My preference is not to have this, and just have all the exceptions in the list, even if it's long.
Ok no problem!
PERF: Enable %z in parsing datetime
State: closed

- [ ] closes #32792
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry

Labels:
name: Performance
description: Memory or execution speed performance
name: Timezones
description: Timezone data dtype

Comments:
Is this really a performance issue? It feels like an enh to free up a use case that previously have to use workaround. I feel weird putting it under ```Performance improvements``` in whatsnew.
Sure you can put in in the enhancement section
very nice @quangngd keep em coming!
This is amazing. Thank you!

In which release is this going to be available? 1.1 on August?
likely 1.1 will be soonish - next month or 2
Here's a test case notebook showing why pandas should support this:

https://www.kaggle.com/eparizzi/pandas-to-datetime-is-slow
@eparizzi this is already fixed and will be released soon
TST: Move _get_cython_table_params into pandas/_testing.py
State: closed

Part of #30914
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`



Labels:
name: Testing
description: pandas testing functions or related to the test suite

Comments:
LGTM
thanks @SaturnFromTitan
Add error message for tz_localize
State: closed

- [x] closes #32967
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry


Labels:
name: Error Reporting
description: Incorrect or improved errors from pandas

Comments:
Thanks @sumanau7 can you merge master.
@simonjayhawkins Don't see the option to merge, request you to do the same.
the merge is to get the changes in #32929 shown in the diff for before review. see https://pandas.pydata.org/docs/development/contributing.html#updating-your-pull-request 

sorry the request should have been to merge upstream master
@simonjayhawkins is the failing test related ?
`test_s3_roundtrip`
> @simonjayhawkins is the failing test related ?
> `test_s3_roundtrip`

known issue, see #33077
Green
@sumanau7 can you merge master and update for the comment above
sorry accid closed
also pls merge master (conflict)
@sumanau7  Could you merge master and address the review comments?
closing as stale, if you'd like to continue pls ping.
DOC: Fix capitalization among headings in documentation files (#32550)
State: closed

Files updated:

```
- [ ] doc/source/development/policies.rst
- [ ] doc/source/development/roadmap.rst
- [ ] doc/source/ecosystem.rst
- [ ] doc/source/getting_started/comparison/comparison_with_sas.rst
- [ ] doc/source/getting_started/comparison/comparison_with_sql.rst
```

more files updated in different PRs: #32944, #32991

Labels:
name: Docs

Comments:
@datapythonista, a pleasure. I am actually practising how to use git.
Thanks @themien
DOC: Mention default behaviour of index_col in readcsv
State: closed

- [x] closes #32664 
- [ ] tests added / passed (Not needed)
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry (Not needed)


Labels:
name: Docs

Comments:
Thanks @Bharat123rox
DOC: Change doc template to fix SA04 errors in docstrings #28792
State: closed

- [x] xref #28792

As requested in #32823 creating separate PR.
Also all descriptions for methods starts with upper case and it looks a little out of place. Should it be casted to lowercase perhaps?



Labels:
name: Docs

Comments:
cc @datapythonista 
Thanks @dilex42. 

My preference would be, instead of just listing the reverse of each method (`add` <-> `radd`...), adding all the related methods in all docstrings:
```
See Also
--------
Series.add : ...
Series.radd : ...
Series.mul : ...
...
```
It'll be a bit long, but I don't think too much. I think all them are relevant for the users. And it'll reduce the complexity of that function, instead of increasing it.

I guess it makes sense to have two, one for comparisons, and one for algebra operations.

What do you think?

I don't have experience to comment from designing perspective to what approach better.

I guess having all functions listed is reasonable. But if I understand correctly there will be 2 static templates, additional `type` param for each method and then some templates concatenation based on type so I don't know if it's gonna be less complex.
Can't really tell without spending a decent amount of time where those docstrings are being used. I think for dataframe docstrings there are different templates, I thought it was the same for series. In that case my preferred option would be to have arithmetic and comparison methods all together in the See Also.

@WillAyd do you have an opinion here?
The way I see it, we need to answer can we make it work with single flex template? If yes then sure let's do that. But if not then let's find the best solution with two templates.
Is this description and formatting good?
/azp run
<samp>
Azure Pipelines successfully started running 1 pipeline(s).<br>

</samp>
Thanks @dilex42
CLN: Clean NDFrame.__finalize__ signature
State: closed

`NDFrame.__finalize__` doesn't do anything with `method` or `kwargs` so removing them from the signature and all calls. Had to change a few tests which overrode this method with another that also included those arguments.

Labels:

Comments:
I think @TomAugspurger had some plans for this?
https://github.com/pandas-dev/pandas/pull/28394 is still a WIP, though I split out the finalize bits.

But we'll want `method` for us, and subclasses may be using the other arguments.
Sounds like this may not make sense so I'll go ahead and close
TST: Use indices fixture in tests/indexes/test_setops.py
State: closed

part of #30914
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`


Labels:
name: Testing
description: pandas testing functions or related to the test suite

Comments:
LGTM
thanks @SaturnFromTitan
TST: Use indices fixture in tests/indexes/test_base.py
State: closed

part of #30914
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`


Labels:
name: Testing
description: pandas testing functions or related to the test suite

Comments:
Something I've been trying to do that you might be interested in: these tests are a mix between using `indices` fixture and `self.create_index`.  If we take all of the create_index methods and put those indexes into the `indices` fixture, then we can switch over completely to using the fixture.

Good idea @jbrockmendel, I can look into that. Is there a ticket for replacing `self.create_index`?
+1 for renaming the fixture to `index` @simonjayhawkins 

I wonder about a good transition strategy though. I could create a fixture `index = indices` in `pandas/conftest.py` and then follow-up with a few PRs that replace `indices` by `index`. Wdyt?
> +1 for renaming the fixture to `index` @simonjayhawkins
> 
> I wonder about a good transition strategy though. I could create a fixture `index = indices` in `pandas/conftest.py` and then follow-up with a few PRs that replace `indices` by `index`. Wdyt?

sounds good. that'll keep the diff here smaller here while allowing others to give their opinions on naming fixtures in general before making changes to other modules.
A much more digestible diff now @simonjayhawkins :)
I find `@pytest.mark.parametrize("index", ["string"], indirect=True)` much less clear than `index = tm.makeStringIndex(100)`
@jbrockmendel I kinda agree. Instead of `tm.makeStringIndex(100)` we should use a `string_index` fixture though
@jreback @jbrockmendel @simonjayhawkins 
As there seems to be no consent for `index` as an alternative name for `indices`, I refrained from introducing an alternative name in this PR altogether. This PR is exclusively using the `indices` fixture now. The renaming should come in follow-up PRs.
@jreback CI is finally green
sorry a small comment, also merge master and ping on green.
@jreback Using indirect instead of a separate fixture now + CI is green
thanks!

I think it’s worth mentioning in our contributing docs on how - when to use indirect fixtures 
xref: #33128
TYP: ensure Block.putmask, Block.where get arrays, not Series/DataFrame
State: closed

Avoids the `getattr(obj, "values", obj)` pattern (there's an issue for that)

Labels:
name: Internals
description: Related to non-user accessible pandas implementation
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
travis yellow is wrong, really is green
thanks
CLN/STY: Nitpicks
State: closed

Things I find myself doing in multiple branches

Labels:
name: Code Style
description: Code style, linting, code_checks

Comments:
Looks fine.

Brainstorming ideas for expanding the range of what you work on: when editing a file, consider searching for TODO/FIXME comments and seeing if any of them are things you can address. (best case is right at the edge of what you're comfortable with).  E.g. there is one TODO in index.pyx that looks doable.
I forgot to mention the changes I've made which are not related to styling.

@jbrockmendel @WillAyd Can you please re-review?
Personally, I would stop doing such styling changes to cython (to resemble what black does) until there is black support for cython at which point we can do it in one go and consistently.
> Personally, I would stop doing such styling changes to cython (to resemble what black does) until there is black support for cython at which point we can do it in one go and consistently.

Noted.
thanks @MomIsBestFriend 

yeah let's hold off on cython changes for style until we have an auto formatter, however typing / fixing things are all ok.
BUG: df.iloc[:, :1] with EA column
State: closed

- [x] closes #32957
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry


Labels:
name: Bug
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves

Comments:
im now thinking the test doesnt belong in extension tests; if it were to fail (e.g. in the status quo), that wouldnt indicate that the EA author needed to fix somthing
i think comments have been addressed @jorisvandenbossche, can you confirm?
@jorisvandenbossche have your concerns/comments been addressed here?
gentle ping @jorisvandenbossche 
needs a rebase as well
thanks @jbrockmendel
CLN: Remove encoding specifier
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:

Comments:
LGTM
What is the point of doing this?
> What is the point of doing this?

I'm pretty sure this is a py2 holdover
> > What is the point of doing this?
> 
> I'm pretty sure this is a py2 holdover

Correct, there is no need to specify the encoding (if it's utf8), in python3
thanks @MomIsBestFriend
BUG: passing a non-dict mapping to pd.concat raises a TypeError
State: closed

closes #32954 
closes #32863 
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode

Comments:
thanks @pbourguignon
REF: move mixed-dtype frame_apply check outside of _reduce try/except
State: closed

cc @jorisvandenbossche this is the same branch I linked the other day.  Some variant of #32867 may make this irrelevant, but I don't think getting this improvement in the interim will make that any more difficult.

Labels:
name: Numeric Operations
description: Arithmetic, Comparison, and Logical operations
name: Clean

Comments:
I can't reproduce this test failure locally
Maybe add a test specifically for the median of a dataframe with categorical columns, so it might be clearer what is wrong with that (as I suppose that is the reason that the `fillna(df.median()` is giving something else)
btw I think we simply need to create pandas.core.reduce, similiar to how we have pandas.core.apply to handle all of this logic. much easier to reason about in a separate module (and can easily break down / dispatch cases).


thanks, pls note potential followon above...
CLN: remove unnecessary Series._convert calls
State: closed

cc @WillAyd my confidence on the correctness here is only "pretty sure", so could use an extra set of eyes.

Labels:
name: Dtype Conversions
description: Unexpected or buggy dtype conversions
name: Clean

Comments:
@jorisvandenbossche 
Brock, can you respond to my comments?
can you merge master, this looks ok. I guess @jorisvandenbossche has some comments
this is fine
DOC: Fix capitalization among headings in documentation files (#32550) 
State: closed

Files updated:

```
doc/source/development/contributing_docstring.rst
doc/source/development/developer.rst
doc/source/development/extending.rst
doc/source/development/maintaining.rst
doc/source/development/meeting.rst
```


Labels:
name: Docs

Comments:
@themien can you merge master

cc @datapythonista 
Thanks @themien
CLN: xarray tests
State: closed

- [ ] closes #xxxx
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Clean

Comments:
LGTM
Thanks @MomIsBestFriend
pandas/core/ops/: replace .format() with f-strings
State: closed

- https://github.com/pandas-dev/pandas/issues/29547
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`

Labels:

Comments:
Hello @SvoONs! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:



There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2020-03-23 17:31:46 UTC
Thanks for the PR. I'm not sure we should be replacing templates. lets see what others think.
Agreed with @simonjayhawkins - this is a use case where format is better than an f-string. 

Going to close since this isn't actionable, but thanks for the PR and of course would welcome contributions on other modules or issues
Fixing scatter plot size (#32904)
State: closed

This fixes the marker size in scatter plots (see https://github.com/pandas-dev/pandas/issues/32904).

- [x] closes #32904
- [ ] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Visualization
description: plotting

Comments:
Can you add a test for this? In `pandas/tests/plotting/test_frame.py` Your original example should be fine. That should have a comment referring back to your issue.

And we'll need a release note in whatsnew/v1.1.0.rst.
I made some errors with merging the commits and ended up deleting the original fork. Not sure if it's possible to adjust this pull request, so will create a new one instead.
REF: "bare_pytest_raises" to use the ast module
State: closed

- [ ] closes #xxxx
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry

---

There is also a performance boost:

```
In [1]: import os                                                                                             

In [2]: from scripts.validate_unwanted_patterns import bare_pytest_raises, main                               

In [3]: SOURCE_PATH = "pandas/tests/"                                                                         

In [4]: %timeit main(function=bare_pytest_raises, source_path=SOURCE_PATH, output_format="{source_path}:{line_number}:{msg}.") 

12.6 s ± 76.4 ms per loop (mean ± std. dev. of 7 runs, 1 loop each) # Master
4.33 s ± 27 ms per loop (mean ± std. dev. of 7 runs, 1 loop each) # PR
```

Labels:
name: CI
description: Continuous Integration

Comments:
@MomIsBestFriend tangential to this bc I was thinking about an ast-based check: one anti-pattern we have is using private functions across modules (e.g. core.indexing._maybe_numeric_slice used in io.formats.style).  How hard would it be to track those down?
> @MomIsBestFriend tangential to this bc I was thinking about an ast-based check: one anti-pattern we have is using private functions across modules (e.g. core.indexing._maybe_numeric_slice used in io.formats.style). How hard would it be to track those down?

@jbrockmendel If you meant tracking every use of private function that is not defined is the exact same file, I spouse is doable. One problem I think is going to be with ```.pyx``` files, but I can start small and add that feature later on.
Dont go out of your way on my account; focus on what you're most interested in.  We appreciate the effort your putting in.
> Dont go out of your way on my account; focus on what you're most interested in. We appreciate the effort your putting in.

I want to make progress, I am not really familiar with the code base, I really like when you give me issues you'd like to see me work on, because this is how I make progress.
I will never make progress if I don't do things I am not 100% sure on them.

---



> > @MomIsBestFriend tangential to this bc I was thinking about an ast-based check: one anti-pattern we have is using private functions across modules (e.g. core.indexing._maybe_numeric_slice used in io.formats.style). How hard would it be to track those down?
> 
> @jbrockmendel If you meant tracking every use of private function that is not defined is the exact same file, I spouse is doable. One problem I think is going to be with `.pyx` files, but I can start small and add that feature later on.

@jbrockmendel Can you confirm I got it correctly?
thanks @MomIsBestFriend
TST: bare pytest raises in tests/scalar
State: closed

- [ ] ref #30999
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry

https://github.com/pandas-dev/pandas/issues/30999



Labels:
name: Testing
description: pandas testing functions or related to the test suite

Comments:
Thanks @quangngd
Added file paths to ignore in linter scripts/validate_unwanted_patterns.py
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry

---

While running ```./ci/code_checks.sh lint``` on my local machine I got the output of:

```
Check for use of not concatenated strings
./asv_bench/env/40d026129f63c107b1ac48bf9ad6964c/lib/python3.6/_sitebuiltins.py:99:String unnecessarily split in two by black. Please merge them manually..
./asv_bench/env/40d026129f63c107b1ac48bf9ad6964c/lib/python3.6/inspect.py:2768:String unnecessarily split in two by black. Please merge them manually..
./asv_bench/env/40d026129f63c107b1ac48bf9ad6964c/lib/python3.6/inspect.py:2890:String unnecessarily split in two by black. Please merge them manually..

....
```

This fixes it.

Labels:
name: Performance
description: Memory or execution speed performance
name: CI
description: Continuous Integration

Comments:
sure