BrandonFors/pandas-github-issues huggingface dataset:
wip: depr partial row-wise multi-column-wise enlargement
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

just curious to see what fails, and I'm questioning whether it's worth adding complexity to support this

Is anyone particularly attached to this? If not, I'd suggest deprecating, because:
- the workaround on the users' side is very simple
- `__setitem__` is already incredibly complex and spaghettified
- there's only 3 tests which use this pattern
- there's little time left until 3.0, and I don't have time to work on this further before 2.2

NOTE: I'm not actually sure that disallowing this is even the way to go, will sleep on it

Labels:
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: Deprecate
description: Functionality to remove in pandas

Comments:
actually addressed this in the end, so closing
BUG fix for datetime null (closes #56853)
State: closed

Closes #56853

This is my first time contributing to this repo and I'm not super experienced with open source, sorry if the style isn't right or if there's some better way of doing it.

From what I can tell, the issue was caused by recurring calls to `get_loc` functions caused by the fact that when you pass in a datetime, it automatically converts subsequent insertions in the same level to datetime. If you don't supply one, it converts it to `NaT`, which causes an indexing error due to having a null value as an index, which gets you caught in endless recursion from `try`/`except` blocks. If you switch the datetime out for a float (or any other common type), the problem is avoided because the unfilled indexes are filled in will empty strings rather that null values. As far as I can tell, this problem only exists with datetime types.

It's kind of a hacky fix, but I figured it would just be easiest to check for null values in the indexes at the end of an insertion and replace them with empty strings. I tried fiddling around with the datetime stuff itself, but it just ended up messing other things up. Let me know if there's some other solution. I also added this as a test case, just to see whether it breaks.

- [x] xref #56853
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions (no new arguments/methods/functions).
- [x] Added an entry in the latest doc/source/whatsnew/vX.X.X.rst file if fixing a bug or adding a new feature.

Labels:
name: Bug
name: Missing-data
description: np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate
name: datetime.date
description: stdlib datetime.date support

Comments:
pre-commit.ci autofix
pre-commit.ci autofix
Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen.
DOC: fix EX03 errors in docstrings - pandas.core.resample.Resampler.interpolate, pandas.pivot, pandas.merge_asof, pandas.wide_to_long
State: closed

All EX03 Errors resolved in the following cases:

1. scripts/validate_docstrings.py --format=actions --errors=EX03 pandas.core.resample.Resampler.interpolate 
2. scripts/validate_docstrings.py --format=actions --errors=EX03 pandas.pivot  
3. scripts/validate_docstrings.py --format=actions --errors=EX03 pandas.merge_asof 
4. scripts/validate_docstrings.py --format=actions --errors=EX03 pandas.wide_to_long

Output:

1. scripts/validate_docstrings.py --format=actions --errors=EX03 pandas.core.resample.Resampler.interpolate 
```
################################################################################
################################## Validation ##################################
################################################################################

2 Errors found for `pandas.core.resample.Resampler.interpolate`:
	Parameters {'**kwargs'} not documented
	Unknown parameters {'``**kwargs``'}
```

2. scripts/validate_docstrings.py --format=actions --errors=EX03 pandas.pivot  
```
################################################################################
################################## Validation ##################################
################################################################################

1 Errors found for `pandas.pivot`:
	Parameter "data" has no description
```

3. scripts/validate_docstrings.py --format=actions --errors=EX03 pandas.merge_asof 
```
################################################################################
################################## Validation ##################################
################################################################################

3 Errors found for `pandas.merge_asof`:
	Parameter "left" has no description
	Parameter "right" has no description
	Return value has no description
```

4. scripts/validate_docstrings.py --format=actions --errors=EX03 pandas.wide_to_long
```
################################################################################
################################## Validation ##################################
################################################################################

Docstring for "pandas.wide_to_long" correct. :)
```

- [ ] xref https://github.com/pandas-dev/pandas/issues/56804
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Docs

Comments:
Thanks @jordan-d-murphy, nice clean up
DOC: fix EX03 in `pandas.errors`
State: closed

Fix flake8 errors of some exceptions in `pandas.errors`.

- [ ] xref #56804 (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Docs

Comments:
Thank you for fixing those @yuanx749
DOC: fix EX03 Errors for pandas.DataFrame.to_sql
State: closed

- [X] Ref: https://github.com/pandas-dev/pandas/issues/56804

Fix these EX03 Errors for `pandas.DataFrame.to_sql`:

```
9 Errors found for `pandas.DataFrame.to_sql`:
        flake8 error: line 8, col 4: E111 indentation is not a multiple of 4
        flake8 error: line 15, col 4: E111 indentation is not a multiple of 4
        flake8 error: line 19, col 4: E111 indentation is not a multiple of 4
        flake8 error: line 27, col 1: F821 undefined name 'df_conflict'
        flake8 error: line 27, col 89: E501 line too long (124 > 88 characters)
        flake8 error: line 28, col 1: F811 redefinition of unused 'insert' from line 22
        flake8 error: line 39, col 1: F821 undefined name 'df_conflict'
        flake8 error: line 39, col 89: E501 line too long (123 > 88 characters)
        flake8 error: line 46, col 3: E111 indentation is not a multiple of 4
```

Labels:
name: Docs

Comments:
@erichxchen, thank you for working on this. I think, there are some errors left that need to be fixed:
```
5 Errors found for `pandas.DataFrame.to_sql`:
        EX03    flake8 error: line 27, col 1: F821 undefined name 'df_conflict'
        EX03    flake8 error: line 27, col 89: E501 line too long (124 > 88 characters)
        EX03    flake8 error: line 28, col 1: F811 redefinition of unused 'insert' from line 22
        EX03    flake8 error: line 39, col 1: F821 undefined name 'df_conflict'
        EX03    flake8 error: line 39, col 89: E501 line too long (123 > 88 characters)
```
For example, the line 
https://github.com/pandas-dev/pandas/blob/02011f2b81743f4e9d100aea6c791fa5e8de9933/pandas/core/generic.py#L3041
is too long. Could you please rewrite it as multi-line statement? It will fix one of the errors.

Closed by https://github.com/pandas-dev/pandas/pull/57025
DOC: fix EX03 Errors for pandas.read_json
State: closed

- [X] Ref: https://github.com/pandas-dev/pandas/issues/56804

Fix these EX03 Errors for `pandas.read_json`
```
flake8 error: line 6, col 23: F821 undefined name '_'
flake8 error: line 8, col 23: F821 undefined name '_'
flake8 error: line 10, col 23: F821 undefined name '_'
```

Labels:
name: Docs

Comments:
The CI wasn't working to validate those docstrings. It's now been fixed, and there are some errors here and in your other PR, can you have a look please?
> The CI wasn't working to validate those docstrings. It's now been fixed, and there are some errors here and in your other PR, can you have a look please?

I am actually confused about why it still errored out when commented with `  # doctest: +SKIP` and `  # noqa: E501` (also in another PR). I will dig into when I have more spare time.
Closed by https://github.com/pandas-dev/pandas/pull/57025
DOC: fix RT03 error for pandas.DataFrame.hist
State: closed

- [ ] ~~closes https://github.com/pandas-dev/pandas/issues/56804~~ (Not relevant, it is a RT03 error )
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).

~~Detected EX03 error for `pandas.DataFrame.hist`:~~
```
python scripts/validate_docstrings.py --format=actions --errors=EX03 pandas.DataFrame.hist
```
Output:
```
################################################################################
################################## Validation ##################################
################################################################################

1 Errors found for `pandas.DataFrame.hist`:
        Return value has no description
```

The output after this changes:
```
################################################################################
################################## Validation ##################################
################################################################################

Docstring for "pandas.DataFrame.hist" correct. :)
```

Labels:
name: Docs
name: Stale

Comments:
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
This seems stale and conflicting, closing. Thanks for the work on this @erichxchen, if you have the time to finish this, or to work on other contributions, you're very welcome to do so.
DOC: ex03 - no fix required
State: closed

- xref #56804

Used this script to validate:

```py
import subprocess as sp

def validate_func(func_name):
    proc = sp.Popen(['./scripts/validate_docstrings.py', '--errors=EX03', func_name], stdout=sp.PIPE, stderr=sp.PIPE)
    proc.wait()
    out = proc.stderr.read().decode().strip()
    return out.endswith(f'Docstring for "{func_name}" correct. :)')

funcs = ['pandas.DataFrame.var',
 'pandas.DatetimeIndex.day_name',
 'pandas.core.groupby.DataFrameGroupBy.apply',
 'pandas.DatetimeIndex.month_name',
 'pandas.core.groupby.DataFrameGroupBy.hist',
 'pandas.core.groupby.SeriesGroupBy.apply',
 'pandas.core.groupby.SeriesGroupBy.transform',
 'pandas.DataFrame.hist',
 'pandas.DataFrame.tz_localize',
 'pandas.CategoricalIndex.set_categories',
 'pandas.core.groupby.DataFrameGroupBy.boxplot',
 'pandas.core.groupby.SeriesGroupBy.pipe',
 'pandas.DataFrame.plot.bar',
 'pandas.DataFrame.tz_convert',
 'pandas.core.groupby.DataFrameGroupBy.pipe',
 'pandas.DataFrame.skew',
 'pandas.core.window.rolling.Rolling.corr']

for func in funcs:
    print(func, validate_func(func))
```

Let me know if it's too many at once.

Labels:
name: Docs

Comments:
Thanks @asishm
TST: Move tests out of test_groupby
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Moving tests for specific methods out of `test_groupby.py`

Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Groupby
name: Clean

Comments:
Thanks @rhshadrach
add ignore_index parameter to query
State: closed

This PR adds `ignore_index` parameter to `DataFrame.query` method. The `ignore_index()` parameter allows other DataFrame methods that filter out rows (e.g. `dropna`, `drop_duplicates` etc.) to reset to the default RangeIndex in the same method call. By adding it to `query`, we improve consistency. So with this parameter, instead of
```python
df.query(condition).reset_index(drop=True)
```
which requires two method calls, we could do
```python
df.query(condition, ignore_index=True)
```
This also allows for in-place index reset as well.
```python
df.query(condition, inplace=True)
df.reset_index(drop=True, inplace=True)                       # <--- two lines and a bit cumbersome

df.query(condition, inplace=True, ignore_index=True)  # <--- cleaner
```


Labels:

Comments:
Thanks for the PR, but for enhancement requests we usually require an issue to be opened up first before opening a PR.

Of note, I don't think this functionality would meet the bar of inclusion into pandas since the project is trying to limit multiple ways of doing the same task.

If still interested in pursuing please open an issue first so closing
DOC: fix EX03 errors in docstrings
State: closed

- [ ] xref #56804 
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Docs

Comments:
hi @mroeschke, I rebased and resolved the conflict but then noticed from https://github.com/pandas-dev/pandas/pull/56863 that `ruff` can autofix all these errors. Let me know if I should close this MR in favor of https://github.com/pandas-dev/pandas/pull/56863
It's OK to still work on this PR. While the ruff PR may also reformat this docstring in the future, it is still appreciated to have your contribution in this PR.
@mroeschke this PR is ready for review. I was about to expand it a bit but then noticed that `pd.MultiIndex` is already removed in [this PR](https://github.com/pandas-dev/pandas/pull/56880/files#diff-ddd4b0b4b985cabe7be60218e66a87d5b92e977a009d6c22bfa770572a2046b8L93) while the docstring is not fixed yet (The pipeline indeed raised an error [here](https://github.com/pandas-dev/pandas/actions/runs/7525838093) and also [here](https://github.com/pandas-dev/pandas/actions/runs/7532519414/job/20503312629) on `main` but the pipeline is still green!). 

I think we can merge this PR first to heal `main`?
It's not the first error there too.

![image](https://github.com/pandas-dev/pandas/assets/25203655/2a845878-6e60-4b9f-b4f3-1deade290c5e)

I managed to reproduce the error [here](https://github.com/pandas-dev/pandas/actions/runs/7534583802/job/20509194445?pr=56897). Will go ahead and fix all the current bugs on `main`
DOC: Fix EX03
State: closed

Error EX03 has cleaned.

pandas.errors.CategoricalConversionWarning

<img width="470" alt="image" src="https://github.com/pandas-dev/pandas/assets/77875500/d007f149-24f0-478c-b2fb-c32be2039bcc">

pandas.errors.ChainedAssignmentError

<img width="395" alt="image" src="https://github.com/pandas-dev/pandas/assets/77875500/3d7b7c4c-4900-42a6-846f-977bcd621c31">

pandas.errors.ClosedFileError

<img width="356" alt="image" src="https://github.com/pandas-dev/pandas/assets/77875500/c0e4dd3d-79ad-4d27-89cf-25042decfc0f">

pandas.errors.NumExprClobberingError

<img width="391" alt="image" src="https://github.com/pandas-dev/pandas/assets/77875500/acbc09fa-35cf-4790-8d30-f02b47a089a5">

- [ ] xref #56804 (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Labels:
name: Docs

Comments:
Thanks @luke396
REGR: freq "m" (as alias of deprecated "M") raises an error
State: closed

- [ ] closes #56847 (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

No whatsnew note as this hasn't hit users yet (thanks @jorisvandenbossche for reporting it in time!)

Labels:
name: Regression
description: Functionality that used to work in a prior pandas version
name: Frequency
description: DateOffsets

Comments:
Thanks @MarcoGorelli 
Thanks!
PERF: use the libjoin fast paths in a few more cases
State: closed

- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/v2.3.0.rst` file if fixing a bug or adding a new feature.

Additional cases where the fast path can be used:
* left join with many:1 (right unique)
* right join with 1:many (left unique)

These are covered by existing tests. 

```
import pandas as pd
import numpy as np

left = pd.Index(np.arange(100_000)).repeat(10)
right = pd.Index(np.arange(100_000))

%timeit left.join(right, how="left")

# 22.7 ms ± 7.84 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)
# 3.8 ms ± 60.4 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
```

```
import pandas as pd
import numpy as np

data = [f"i_{i:06}" for i in range(100_000)]
left = pd.Index(data, "string[pyarrow_numpy]")
right = pd.Index(data, "string[pyarrow_numpy]").repeat(10)

%timeit left.join(right, how="right")

# 393 ms ± 4.64 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
# 150 ms ± 3.98 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
```


Labels:
name: Performance
description: Memory or execution speed performance
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode

Comments:
Nice finds keep them coming!
CLN: Drop Numpy 1.22 per NEP29
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Clean
name: Dependencies
description: Required and optional dependencies

Comments:
Thanks @lithomas1
DOC: fixed EX03 errors in docstrings for `pandas.Series.dt.day_name` and `pandas.Series.dt.day_name`
State: closed

Modified the docstrings to resolve EX03/flake8 errors for,
- `pandas.Series.dt.day_name`
```
################################################################################
################################## Validation ##################################
################################################################################

4 Errors found for `pandas.Series.dt.day_name`:
        No extended summary found
        Parameters {'*args', '**kwargs'} not documented
        Unknown parameters {'locale'}
        See Also section not found
```
- `pandas.Series.str.len`
```
################################################################################
################################## Validation ##################################
################################################################################

Docstring for "pandas.Series.str.len" correct. :)
```

- [x] #56804
- [ ] [Tests passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Docs

Comments:
Thanks @roadrollerdafjorst