BrandonFors/pandas-github-issues huggingface dataset:
BUG: Throw a ParserError when header rows have unequal column counts …
State: closed

(GH43102)

- closes #43102 
- Fixed uncaught IndexError that is raised when the iterator `i` in `extract(r)` inside base_parser.py exceeds the length of a header row (when `field_count > len(r)`).
- A Parser error is raised when the header row had unequal columns. Outputs the first row where a mismatch is found.
- Added a test to match the error being raised.

Labels:
name: Error Reporting
description: Incorrect or improved errors from pandas
name: IO CSV
description: read_csv, to_csv

Comments:
Hello @quantumalaviya! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:





There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2021-09-05 08:27:17 UTC
@jreback I added a test inside pandas/tests/io/parser/test_header.py. Is this fine?
I had added "A Parser error is raised when the header row had unequal columns. Outputs the first row where a mismatch is found." as a substitute for whatsnew. Do you want it to be more detailed or am I missing something?
Please add something about the uncaught error before to avoid the impression that a new error is raised
@jreback @phofl Does this look good?
Please be patient, we are all volunteers and will look when we have time.

Whatsnew is still missing
Thanks for the clarification, I was confused as to what the next step is. 

EDIT: Changed whatsnew.
Yes the whatsnews are our release notes. Small comment, otherwise lgtm
@lithomas1 merge away
thanks @quantumalaviya 
My pleasure. Thanks for guiding me through it!
Typo fixes
State: closed

- [x] closes #39158 
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [x] Fixed typo in pandas/io/parser/c_parser_wrapper.py


Labels:
name: Docs
name: Error Reporting
description: Incorrect or improved errors from pandas

Comments:
Closing as this was already being worked on before you opened the PR https://github.com/pandas-dev/pandas/issues/39158#issuecomment-901325351 . Thanks anyway for your contribution
EHN: partially upgrade sql module for SQLAlchemy 2.0 compat
State: closed

xref #40686


Labels:
name: IO SQL
description: to_sql, read_sql, read_sql_query
name: Compat
description: pandas objects compatability with Numpy or Python functions

Comments:
Hello @fangchenli! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:





There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2021-08-20 04:05:22 UTC
very nice @fangchenli
REF: names no longer needed in _form_blocks, create_block_manager_from_arrays
State: closed

1) copy/paste create_block_manager_from_arrays and _form_blocks to the pseudo-public internals.api
2) rename create_block_manager_from_arrays -> create_block_manager_from_column_arrays
3) prune no-longer-needed args from create_block_manager_from_arrays, _form_blocks

Two follow-ups after this:
4) remove no-longer-needed checks in no-longer-pseudo-public create_block_manager_from_column_arrays
5) deprecate pseudo-public create_block_manager_from_arrays

Labels:
name: Refactor
description: Internal refactoring of code
name: Internals
description: Related to non-user accessible pandas implementation

Comments:
> 5) deprecate pseudo-public create_block_manager_from_arrays

yeah we think pyarrow (and maybe fastparquet) uses this internal api?
> yeah we think pyarrow (and maybe fastparquet) uses this internal api?

yah will have to see when the time comes
failure is codecov

looking on github for external uses of create_block_manager_from_arrays and mostly just seeing it in pandas forks
kk great. lookign forward to code deletions.
I don't think that pyarrow or fastparquet use this (in any case pyarrow doesn't, xref https://github.com/pandas-dev/pandas/issues/40226). 
great, let's rip it out and see if anyone complains
cc @TomAugspurger looks like we kept create_block_manager_from_arrays in the internals namespace for dask.  Is it still needed?
Looks like that was just the other function create_block_manager_from_blocks
Ah, yes, dask/partd uses `create_block_manager_from_blocks` (https://github.com/dask/partd/blob/main/partd/pandas.py)
TYP: Replace namedtuple with dataclass #40531
State: closed

This PR replaces namedtuple with dataclass in pandas/core/groupby/base.py and closes #40531.

The parameters `order` and `frozen` is set equal to True to make sure that that behaviour of OutputKey does not change. 



Labels:
name: Clean

Comments:
@github-actions pre-commit

EDIT
----
looks like not all tests get run from the automated commit, so closing and reopening. but then that stopped the azure checks, so I added an empty commit - now all the CI checks  seem to be running fine
DOC: `styler.format` does not, in fact, raise a KeyError
State: closed

This was actually never true from the start #40134. It might have been true at one point during the PR and then was overwritten without docs being corrected before final merge.

I.e. these docs have always been wrong since introduction.

Should be back portable.

```python
df = DataFrame([[1.0, 2.0]], columns=["A", "B"])
df.style.format({"A": "{:.1f}", "B": "{:.2f}"}, subset = ["A"])  
# does not raise a KeyError for "B", just ignores.
```

Labels:
name: Docs
name: Styler
description: conditional formatting using DataFrame.style

Comments:
@meeseeksdev backport 1.3.x
Something went wrong ... Please have  a look at my logs.
#latest features update
State: closed

- [x] closes #xxxx
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [x] whatsnew entry


Labels:

Comments:
see https://github.com/pandas-dev/pandas/pull/43089#issuecomment-900792857

please stop
rom
State: closed

- [ ] closes #xxxx
-
[answer.pdf](https://github.com/pandas-dev/pandas/files/7004115/answer.pdf)
 [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] whatsnew entry


Labels:

Comments:
Thanks for the PR. Please fill in the PR template with what issue you are addressing so it's clear to the maintainers. Closing but feel free to reach out if you need help
Here is the solution to issue #43085 Basically, you have to make sure that the df2 after reindexing is stored again in df2
State: closed

- [ ] closes #43085 




Labels:

Comments:
Hello @pragatimehra21! Thanks for opening this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:

* In the file [`pandas/reindex.py`](https://github.com/pandas-dev/pandas/blob/4f687079dce519d6642bf2260c1086ce770c7572/pandas/reindex.py):

> [Line 6:4](https://github.com/pandas-dev/pandas/blob/4f687079dce519d6642bf2260c1086ce770c7572/pandas/reindex.py#L6): [E225](https://duckduckgo.com/?q=pep8%20E225) missing whitespace around operator
> [Line 7:1](https://github.com/pandas-dev/pandas/blob/4f687079dce519d6642bf2260c1086ce770c7572/pandas/reindex.py#L7): [W391](https://duckduckgo.com/?q=pep8%20W391) blank line at end of file


Hi @pragatimehra21. Thanks for the contribution, but this should not be added as a PR. This is just a code snippet which you have inserted into a file. It is better for you to post the code snippet in the underlying Issue.

We would of course welcome a PR that corrected the part of the code where the problem lies, with your solution according to proper code standards.
Added some more concepts and 3rd page #Update #CheatSheet #Pandas
State: closed

- [x] closes #xxxx
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [x] whatsnew entry


Labels:

Comments:
Thanks for the PR, but closing for a similar reason as https://github.com/pandas-dev/pandas/pull/43086#issuecomment-900570201
#fixed syntax and grammatical errors, Added quick notes for simple go - through/ follow-ups
State: closed

… through. (Open to further updates)

#fixed syntax and grammatical errors, Added quick notes for simple go through. (Open to further updates)

- [x] closes #xxxx
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [x] whatsnew entry


Labels:

Comments:
Thanks for contributing @harshkr1999 but I notice most of these changes are not grammatical changes or syntax fixes but your own preference for the way the sentence is written or the example is structured.

There is nothing fundamentally wrong with the way they have currently been written by the original author, and therefore I don't approve this PR.

There are many issues labelled "good first issue" or "contributions welcome" if you would like to contribute further. 
Yeah, agree with the above, see here https://pandas.pydata.org/pandas-docs/stable/development/contributing.html for how to contribute
Backport PR #43041 on branch 1.3.x (Fix len one list dataframe constructor bug for datetime)
State: closed

Backport PR #43041: Fix len one list dataframe constructor bug for datetime

Labels:
name: Regression
description: Functionality that used to work in a prior pandas version
name: DataFrame
description: DataFrame data structure
name: Constructors
description: Series/DataFrame/Index/pd.array Constructors

Comments:
merging
Backport PR #43041 on branch 1.3.x (Fix len one list dataframe constructor bug for datetime)
State: closed


Labels:
name: Regression
description: Functionality that used to work in a prior pandas version
name: DataFrame
description: DataFrame data structure
name: Constructors
description: Series/DataFrame/Index/pd.array Constructors

Comments:
Thanks @phofl the autobackport was failing due to conflicts because another PR was not yet backported. There are merge conflicts again here now.

closing this one. opened #43077
Thx
BENCH: Add pyarrow to env
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] whatsnew entry

This came up while I was benchmarking the pyarrow csv reader that I am planning to PR soon, which was erroring with an ImportError for pyarrow. Anyone know if this is the right patch? (It works correctly for me afterwards, however I am curious as to how ArrowStringArray was getting benchmarked)

pytest also should be removable I think. (there really shouldn't be any reason why we are benchmarking that)
cc @simonjayhawkins @TomAugspurger 

Labels:
name: Benchmark
description: Performance (ASV) benchmarks

Comments:
lgtm thanks @lithomas1 

yeah this should also help with the string functions, we *should* have the parameterizations already. cc @simonjayhawkins
TST: df.combine_first duplicates rows for nan index values (#39881)
State: closed

- [x] closes #39881
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] whatsnew entry (n/a - added test case only)


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: Missing-data
description: np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate
name: MultiIndex

Comments:
thanks @ssche
BUG: Allow named aggregation with Resampler.agg
State: closed

- [x] closes #32803 
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [x] whatsnew entry


Labels:
name: Bug
name: Groupby
name: Apply
description: Apply, Aggregate, Transform, Map

Comments:
Thanks @dicristina for the PR. can you add a release note doc/source/whatsnew/v1.4.0.rst
thanks @dicristina very nice!
Update frame.py
State: closed

Built the example inside the docstring and removed the unnecessary inplace=true

- [x] closes #xxxx
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [x] whatsnew entry


Labels:

Comments:
Hello @trevorkask! Thanks for opening this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:

* In the file [`pandas/core/frame.py`](https://github.com/pandas-dev/pandas/blob/d1c8182e7430252247a7da6ff8b8c44725c55516/pandas/core/frame.py):

> [Line 6809:1](https://github.com/pandas-dev/pandas/blob/d1c8182e7430252247a7da6ff8b8c44725c55516/pandas/core/frame.py#L6809): [W293](https://duckduckgo.com/?q=pep8%20W293) blank line contains whitespace


can u please just push to the open PE

creating new PRs just makes more work
DOC: Updating grammar for clarity and consistency
State: closed

### Location of the documentation
[10 minutes to pandas](https://pandas.pydata.org/pandas-docs/stable/user_guide/10min.html)

### Documentation problem
Throughout the page, there are code examples with an accompanying explanation of what is happening. The explanation will sometimes end in a colon, and other times end with a period.  Here is a screenshot of the inconsistency:

![image](https://user-images.githubusercontent.com/23269446/129495989-5b172a42-7bcc-49f4-8a4f-30d3087e872c.png)

### Suggested fix for documentation
I think that having a colon is the correct usage considering that the sentences are explaining the code block underneath it. I also think it would be better for the page to be consistent in punctuation. In my pull request, I've changed all the instances where I think this is applicable, as well as changing a couple of wordings to be easier to understand.


Labels:
name: Docs

Comments:
thanks @andrewwoood
BUG: nlargest/nsmallest can now consider nan values like sort_values(ascending=True).head(n)
State: closed

- [x] closes #28984 
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [x] whatsnew entry

nsmallest/nlargest are now consistent with x.sort_values(ascending=False).head(n)/x.sort_values(ascending=True).head(n)
Edit: I've changed test_nlargest_misc(self) in \tests\serie\methods\test_nlargest.py to reflect the fact that Nans are included


Labels:
name: Numeric Operations
description: Arithmetic, Comparison, and Logical operations

Comments:
Hello @usersblock! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:









There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2021-09-07 07:41:45 UTC
@usersblock i've closed #42997 as a duplicate of #28984, can you update the references to the issue also include the code sample from #28984 as a test.
Updated reference and added new test to test_nlargest.py in frames\methods
ok can you add a replica of the test in the OP for `df.groupby('grp').apply(lambda x: x.sort_values('z', ascending=False).head(1))`
thanks @usersblock very nice!
PERF: read_stata
State: closed

```
from asv_bench.benchmarks.io.stata import *

self = StataMissing()
self.setup("tc")

%timeit self.time_read_stata("tc")
150 ms ± 3.3 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)  # <- master
114 ms ± 2.81 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)  # <- PR
```

cc @bashtage

Labels:
name: Performance
description: Memory or execution speed performance
name: IO Stata
description: read_stata, to_stata

Comments:
Look correct to me 
lgtm can you add a whatsnew note, ping on green.
ping
fixed transform docs
State: closed

- [ ] closes #42907 
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] whatsnew entry


Labels:
name: Docs
name: Apply
description: Apply, Aggregate, Transform, Map
name: Stale

Comments:
Hello @willie3838! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:



There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2021-08-17 16:34:43 UTC
This pull request is stale because it has been open for thirty days with no activity. Please update or respond to this comment if you're still interested in working on this.
Thanks for the PR, but appears that it has gone stale. Closing for now, but if interested in continuing let us know and we can reopen.
Doc test errors
State: closed

- [ ] closes #42907 
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] whatsnew entry


Labels:

Comments:
Hello @willie3838! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:

* In the file [`pandas/core/groupby/groupby.py`](https://github.com/pandas-dev/pandas/blob/227fc0db6a48d0c6c5c5361b491631d14452b6e3/pandas/core/groupby/groupby.py):

> [Line 321:89](https://github.com/pandas-dev/pandas/blob/227fc0db6a48d0c6c5c5361b491631d14452b6e3/pandas/core/groupby/groupby.py#L321): [E501](https://duckduckgo.com/?q=pep8%20E501) line too long (90 > 88 characters)
> [Line 321:91](https://github.com/pandas-dev/pandas/blob/227fc0db6a48d0c6c5c5361b491631d14452b6e3/pandas/core/groupby/groupby.py#L321): [W291](https://duckduckgo.com/?q=pep8%20W291) trailing whitespace





##### Comment last updated at 2021-08-15 18:52:02 UTC
ENH:included anti join functionality
State: closed

- [x] closes #42916
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] whatsnew entry

Extremely naive implementation. Happy to take suggestions.


Labels:
name: Enhancement
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode
name: Stale

Comments:
I haven't reviewed fully but my instinct is telling me that you have implemented this in a very complicated way, and I think a search/consideration for a much simpler approach is worthwhile. (see my comment in the underlying issue).

Additionally, your solution uses `numpy` and numpy array functions. Does that mean this functionality might fail with `dtypes` that are native to `pandas`?  
Hi @attack68 , this https://github.com/pandas-dev/pandas/issues/42916#issuecomment-894430300 was really helpful. I have made some changes, let me know how this looks. I believe this should be able to handle pandas dtypes.
This is rendering little odd. I had to split it into two lines due to pep8 length limitation, but not sure how to ensure the render is in a single line.

```
how : {'left', 'right', 'outer', 'inner', 'cross',
  'anti_left', 'anti_right', 'anti_full'}, default 'inner'
```

![Screenshot 2021-08-19 at 7 44 18 PM](https://user-images.githubusercontent.com/6472290/130084275-5d830f9b-e3ce-4705-8731-66842bc60f98.png)

Hello @debnathshoham! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:







There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2022-01-14 18:16:37 UTC
hi guys, just checking if anyone got a chance to look into this..
thanks @attack68 for the detailed review.
I have added a couple of more comments, would be happy to add more, if you can point somewhere you think is still unclear).
I have added a couple of comments in the tests too. In the rest, I think the name of the tests are self-descriptive.
This pull request is stale because it has been open for thirty days with no activity. Please update or respond to this comment if you're still interested in working on this.
conceptually ok with adding this. but the implementation needs tight integration with the existing. you are bolting it on in an unmaintainable manner. if you want to continue, merge master and update.
Thanks for the PR, but it appears to have gone stale. Feel free to reopen if you have time to continue with addressing the review (splitting the validate and merge operations).
REGR: SeriesGrouper using Timestamp index
State: closed

- [x] closes https://github.com/pandas-dev/pandas/issues/42390
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [x] whatsnew entry

Not sure what the defining characteristic of the failing cases is so there's a blank in the whatsnew note.

Labels:
name: Datetime
description: Datetime data dtype
name: Groupby
name: Regression
description: Functionality that used to work in a prior pandas version

Comments:
@meeseeksdev backport 1.3.x
Something went wrong ... Please have  a look at my logs.
REGR: window ewm slowdown
State: closed

- [x] closes #42333
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [x] whatsnew entry (will move once 1.3.3 notes merged)

```
-     3.22±0.09ms       1.29±0.3ms     0.40  rolling.EWMMethods.time_ewm('DataFrame', 10, 'int', 'mean')
-      2.92±0.1ms       1.08±0.1ms     0.37  rolling.EWMMethods.time_ewm('DataFrame', 1000, 'int', 'mean')
-      2.68±0.1ms        959±100μs     0.36  rolling.EWMMethods.time_ewm('DataFrame', 1000, 'float', 'mean')
-      3.00±0.2ms      1.02±0.03ms     0.34  rolling.EWMMethods.time_ewm('DataFrame', 10, 'float', 'mean')
-      2.78±0.1ms        850±100μs     0.31  rolling.EWMMethods.time_ewm('Series', 10, 'int', 'mean')
-      2.34±0.2ms         683±30μs     0.29  rolling.EWMMethods.time_ewm('Series', 10, 'float', 'mean')
-     2.74±0.06ms         739±60μs     0.27  rolling.EWMMethods.time_ewm('Series', 1000, 'float', 'mean')
-      2.55±0.3ms         681±40μs     0.27  rolling.EWMMethods.time_ewm('Series', 1000, 'int', 'mean')
```

Labels:
name: Performance
description: Memory or execution speed performance
name: Regression
description: Functionality that used to work in a prior pandas version
name: Window
description: rolling, ewma, expanding

Comments:
nice speedup, cc @mroeschke 
thanks @mzeitlin11 
@meeseeksdev backport 1.3.x
Something went wrong ... Please have  a look at my logs.