BrandonFors/pandas-github-issues huggingface dataset:
BUG: MultiIndex dtypes incorrect if level names not unique
State: closed

- [x] closes #45174
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [x] whatsnew entry


Labels:
name: MultiIndex

Comments:
Hello @johnzangwill! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:





There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2022-01-04 12:35:34 UTC
> also pls add a test that exercises this

https://github.com/johnzangwill/pandas/blob/7777a8dae5c3485d588e19ccb098af57e0e79031/pandas/tests/indexes/multi/test_get_set.py#L70
thanks @johnzangwill
allow multiple series argument calling in ufunc
State: closed

- [ ] closes #39853
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [ ] whatsnew entry


Labels:
name: Bug
name: Stale
name: ufuncs
description: __array_ufunc__ and __array_function__

Comments:
Hello @miettal! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:



There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2022-01-03 08:46:43 UTC
can you add a test for the bug(s) this fixes
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
Thanks for the pull request, but appears this PR is stale and broke some tests. If you'd like to continue please merge master, include a unit test, and verify the fix locally with the entire test suite, and we can reopen.
PERF: expanding/rolling.min/max with engine='numba'
State: closed

- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [x] whatsnew entry


Labels:
name: Performance
description: Memory or execution speed performance
name: Window
description: rolling, ewma, expanding

Comments:
these failures look relevant, e.g. https://dev.azure.com/pandas-dev/pandas/_build/results?buildId=71716&view=logs&j=7ae4cfe6-c794-57c3-450a-5b4ec49e7feb&t=d0040de8-63ab-5e4d-b4e2-d2ac0e07e0ad
failures are unrelated likely from https://github.com/pandas-dev/pandas/issues/45182, but if you can rebase anyhow
ASV: Add merge on categorical benchmarks
State: closed

- [x] closes #30513
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them


Labels:
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode
name: Benchmark
description: Performance (ASV) benchmarks

Comments:
thanks @lukemanley
API: Allow other na values in StringArray Constructor
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [ ] whatsnew entry

around 20-30% perf regression compared to master on added benchmarks. Still faster than 1.3.x so the regression is not user visible if we merge this for 1.4.0.
really just a quick and dirty solution to unblock @jbrockmendel.
The best solution would be to go through ensure_string_array(which is what _from_sequence does), but that would take a lot of effort to avoid perf regressions(it seems like ensure_string_array does not type its ``arr`` arg and also takes in EA inputs? we should probably restrict to numpy arrays as suggested by a code comment in the stub file).

I'll open an issue as a follow-up.

Labels:
name: Missing-data
description: np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate
name: Strings
description: String extension data type and string data
name: ExtensionArray
description: Extending pandas with custom dtypes or arrays.

Comments:
> it seems like ensure_string_array does not type its arr arg and also takes in EA inputs

yah thats a pattern we should avoid.  IIRC there is a TODO in the code about that.
gentle ping @jreback @jbrockmendel.
@github-actions pre-commit.
Hello @lithomas1! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:









There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2022-01-15 15:58:47 UTC
@github-actions pre-commit.
thanks @lithomas1
REF: separate numpy-specific part of can_hold_element
State: closed

By returning the value instead of just a bool, we open the option to return the _casted_ value, which is what we'll do in the follow-up, which will let us de-duplicate NumericIndex._validate_fill_value.  By separating out the np-specific parts, we'll be able to improve perf in a couple of places.

Labels:
name: Refactor
description: Internal refactoring of code
name: Dtype Conversions
description: Unexpected or buggy dtype conversions

Comments:
> would it be better to use None as the return value rathe than using exceptions as control flow ? (e.g. performance)

That becomes ambiguous with if the actual element is None.

There _are_ some performance concerns i have with this function, but try/except overhead is not among them.
> There are some performance concerns i have with this function, but try/except overhead is not among them.

oh ok.
Update 10min.rst
State: closed

Adding article an before the word object to solve grammatical mistakes on documentation


Labels:

Comments:
Hi @CrazyCat-Cyber 

Thanks for your PR, but it's also correct as it is

If you're looking to contribute, that's great! Please look [here](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#where-to-start) for how/where to start, feel free to reach out if you want/need help

closing for now then
BUG: clip raising with na series bounds for datetimes or ea int
State: closed

- [x] closes #44785
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [x] whatsnew entry


Labels:
name: Numeric Operations
description: Arithmetic, Comparison, and Logical operations
name: Stale

Comments:
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
@phofl can you rebase
Sorry, forgot. Will do in the coming days
Thanks for the pull request, but it appears to have gone stale. Feel free to update and reopen when you have the time, but closing to clear out the queue
BUG: fix GH45032 iloc(axis=1).__setitem__ axis argument
State: closed

- [x] closes #45032 
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [x] whatsnew entry


Labels:
name: Bug
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves

Comments:
thanks @ellequelle very nice!
DEPR: special-casing PeriodDtype in Series.where/Series/mask fallback
State: closed

- [ ] closes #xxxx
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [x] whatsnew entry


Labels:
name: Deprecate
description: Functionality to remove in pandas
name: API - Consistency
description: Internal Consistency of API/Behavior

Comments:
doesn't this overlap with #45193 ? IOW we should either merge both or neither for 1.4
> doesn't this overlap with #45193 ? IOW we should either merge both or neither for 1.4

It does overlap, yes.  My expectation is that #45193 wont make it in time for 1.4.

> IOW we should either merge both or neither for 1.4

No.  This should be done especially if we dont do 45193.
> > doesn't this overlap with #45193 ? IOW we should either merge both or neither for 1.4
> 
> It does overlap, yes. My expectation is that #45193 wont make it in time for 1.4.
> 
> > IOW we should either merge both or neither for 1.4
> 
> No. This should be done especially if we dont do 45193.

got it
we could plausibly call this a bugfix on the grounds that `Series.mask(..., inplace=True)` coerces while `Series.mask(..., inplace=False)` raises is inconsistent
closing in favor of #45546
ENH: Allow callable for on_bad_lines in read_csv when engine="python"
State: closed

- [x] closes #5686
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [x] whatsnew entry


Labels:
name: Enhancement
name: IO CSV
description: read_csv, to_csv

Comments:
@phofl if you can review
@phofl if you can review
> Could we make this work for the header columns? I think the bad_lines arguments are currently ignored there

Looks like that would require a refactor IIUC because the Python parser first "determines" what the correct header should be first and then `on_bad_lines` takes affect if there are any rows where `len(row) > len(predetermined header)`
Ok thought so. Then let's ignore this for now and keep in mind for the future. I think ignoring bad lines is not useful, if they are in the header, but modifying them with a function might be worthwile
thanks @mroeschke very nice (and @phofl for review!)
@meeseeksdev backport 1.4.x
Something went wrong ... Please have a look at my logs.
BUG: Fix for xlabel/ylabel in barh plot
State: closed

- [X] closes #45144
- [X] tests added / passed
- [X] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [X] whatsnew entry


Labels:
name: Bug
name: Visualization
description: plotting

Comments:
Hello @stefmolin! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:





There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2022-01-11 01:36:25 UTC
thanks @stefmolin very nice!
BUG: DataFrame.pivot(index=None) with MultiIndex
State: closed

- [x] closes #23955
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [x] whatsnew entry


Labels:
name: Bug
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode

Comments:
thanks @mroeschke
TYP: Add type definition for generators in  excel.py
State: closed

xref #37715

defined ````gen```` and ````gen2```` as ````Iterable[ExcelCell]````, with the latter also being an empty tuple.




Labels:
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
@twoertwein checking in, is this good to go?
> Could you merge master one more time? Otherwise LGTM

done
Thanks @kashifkhan
ENH: add `cline` to `Styler.to_latex`
State: closed

- [x] closes #21673

This allows `Styler.to_latex` to also render `\cline` to separate sparse MultiIndex rows according to user input. Tests are also included to demonstrate functionality with hidden rows and hidden index levels.

Note that DataFrame.to_latex also uses `\cline` but it is forced in certain cases without options.

```
midx = pd.MultiIndex.from_product([["A", "-", "B"], [0], ["X", "Y"]], names=["lvl 0", "lvl 1", "last lvl"])
df = pd.DataFrame([[1, "pig"], [2, "coin"], [99, ""], [99, ""], [3, "sky"], [4, "pie"]], 
                  index=midx, columns=["int", "str"])
styler = df.style
styler.hide([("-", 0, "X"), ("-", 0, "Y")])
styler.hide(level=[1,3])
print(styler.to_latex(clines="all;index", hrules=True, multirow_align="t"))
```

| | `all` | `skip-last` |
--- | --- | --- |
**`index`** | <img width="236" alt="Screenshot 2021-12-31 at 14 49 53" src="https://user-images.githubusercontent.com/24256554/147826613-bedb3728-738b-4c06-ac52-93caf4e3e652.png"> | <img width="227" alt="Screenshot 2021-12-31 at 14 55 22" src="https://user-images.githubusercontent.com/24256554/147826866-c58219a5-fef2-4e26-a769-4b0ea8277b56.png">
**`data`** | <img width="233" alt="Screenshot 2021-12-31 at 14 51 30" src="https://user-images.githubusercontent.com/24256554/147826673-d91875ab-512e-46ef-971f-621f595bf0bb.png"> | <img width="237" alt="Screenshot 2021-12-31 at 14 52 59" src="https://user-images.githubusercontent.com/24256554/147826752-664ad7de-092f-4b48-bfbb-7e1b654b6b34.png"> |

Default render without ``clines``:


<img width="218" alt="Screenshot 2021-12-31 at 15 03 14" src="https://user-images.githubusercontent.com/24256554/147827219-f604fdea-35e9-4394-8fd7-9b75b3fba351.png">







Labels:
name: IO LaTeX
description: to_latex
name: Styler
description: conditional formatting using DataFrame.style

Comments:
@meeseeksdev backport 1.4.x
thanks @attack68 
Something went wrong ... Please have a look at my logs.
ENH: to_sql returns rowcount
State: closed

- [x] closes #23998
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [x] whatsnew entry


Labels:
name: Enhancement
name: IO SQL
description: to_sql, read_sql, read_sql_query

Comments:
wow looks pretty good
DEPR: Series([np.nan], dtype='i8') silently ignoring dtype
State: closed

- [ ] closes #xxxx
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [x] whatsnew entry

xref #40110

Labels:
name: Deprecate
description: Functionality to remove in pandas
name: Constructors
description: Series/DataFrame/Index/pd.array Constructors

Comments:
hmm are failures on master?
No, caused by a typo, now fixed + green
BUG: allow use of both default+input resolvers in df.eval, GH34966
State: closed

- [x] closes #34966
- [x] tests added / passed (all tests passed except those which were auto-cancelled for taking too long)
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [x] whatsnew entry

Allows default resolvers to be used with input resolvers in `pandas.dataframe.eval`. Previously cases such as this would break:
```python
import pandas as pd

df = pd.DataFrame({'a':[0,1,2], 'b':[7,8,9]}) 
d = {'c': 5}
df.eval('a+b*c', resolvers=[d])
```
with `pandas.core.computation.ops.UndefinedVariableError: name 'a' is not defined` as explained in #34966. Following this update, the above works as expected.
Added a test similar to the above: `tests.frame.test_query_eval.TestDataFrameEval.test_eval_resolvers_combined`

Labels:
name: Bug
name: expressions
description: pd.eval, query

Comments:
Hello @bubblingoak! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:





There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2022-01-04 00:39:55 UTC
> umm can you add a test which fails w/o this change

Added the test `tests.frame.test_query_eval.TestDataFrameEval.test_eval_resolvers_combined`. The updated code passes, but current master fails on [line 174](https://github.com/bubblingoak/pandas/blob/fix-df-eval-resolvers/pandas/tests/frame/test_query_eval.py#L174): 
```python
result = df.eval("a + b * c", resolvers=[dict1])
```
with `pandas.core.computation.ops.UndefinedVariableError: name 'a' is not defined`
thanks @bubblingoak
TYP: type read_xml and deprecate passing positional arguments
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [ ] whatsnew entry


Labels:
name: Typing
description: type annotations, mypy/pyright type checking
name: IO XML
description: read_xml, to_xml

Comments:
cc @ParfaitG if any comments
LGTM! Thanks, @phofl!
cc @jreback
can you rebase (as merged other one). or is this orthogonal?
This should be orthogonal, but better to rebase
thanks @phofl
REF: Deduplicate to_xml code
State: closed

- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them

This also fixes some typing bugs detected after removing the _init_ implementation

Labels:
name: Refactor
description: Internal refactoring of code
name: IO XML
description: read_xml, to_xml

Comments:
LGTM! Thanks, @phofl. Be sure to rebase.
thanks @phofl and @ParfaitG for reviewing
TST: parameterize more tests
State: closed

- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them



Labels:
name: Testing
description: pandas testing functions or related to the test suite

Comments:
@jbrockmendel if you can have a quick glance here
comments on a few, the rest look good
Updated environment.yml 
State: closed

- [X] closes #45123
- [X] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [X] whatsnew entry
Changed line 123 in environment.yml from 
`git+https://github.com/pydata/pydata-sphinx-theme.git@master` to `git+https://github.com/pydata/pydata-sphinx-theme.git@41764f5`
By pre commit hook `requirements-dev.txt` line 86 changed to `git+https://github.com/pydata/pydata-sphinx-theme.git@41764f5`


Labels:
name: Dependencies
description: Required and optional dependencies

Comments:
Update: 2 checks are failing with the latest released version `0.7.1`. I'm trying an older release version, if that works. It may be need to build from source for pydata-sphinx-theme that is causing the failure. 

If the older version fails, I'll change it to an older commit. 
Hi @jreback, 

Using the latest release version causes failure in Doc Build and Upload / Doc Build and Upload (pull_request) pipeline and posix pipeline. I believe you have to build from source for pydata-sphinx-theme that's why installing from the repo? 

The current failure is because of timeout. Let me know what you think.
thanks @prithvijitguha
BUG: Series.__setitem__ failing to cast numeric values
State: closed

- [ ] closes #xxxx
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [x] whatsnew entry

Two cases this fixes (both tested in test_coersion)

```
ser = pd.Series([1, 2, 3, 4])
ser[1] = 1.1
>>> ser[1]
1.1  # <- PR
1    # <- master; note that setting with loc or iloc *does* cast


ser2 = pd.Series([1, 2, 3, 4], dtype=np.int8)
ser2[1] = np.int16(512)

>>> ser2[1]
512 # <- PR; casts to int16
0   # <- master; still np.int8; note that setting with with loc or iloc *does* cast
```

Downside: can_hold_element is heavier-weight than validate_numeric_casting.

Labels:
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: Dtype Conversions
description: Unexpected or buggy dtype conversions

Comments:
this is a break with numpy (good) but needs a good note here
This adresses parts of #20643 I think. The Series.__setitem__ case was not working there
Also:
- #37477
- #15413

@jbrockmendel addressing @phofl comments in followup or here?
Follwups. Definitely plural.

On Fri, Dec 31, 2021 at 6:58 AM Jeff Reback ***@***.***>
wrote:

> @jbrockmendel <https://github.com/jbrockmendel> addressing @phofl
> <https://github.com/phofl> comments in followup or here?
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/pandas-dev/pandas/pull/45121#issuecomment-1003394993>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AB5UM6GFJPCM6JOUTFCZRRDUTXAIVANCNFSM5K65KHHQ>
> .
> Triage notifications on the go with GitHub Mobile for iOS
> <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675>
> or Android
> <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.
>
> You are receiving this because you were mentioned.Message ID:
> ***@***.***>
>

k great