BrandonFors/pandas-github-issues huggingface dataset:
pytest: Use colored output
State: closed

This gives nice colored output locally and on GitHub Actions.

- [ ] closes #xxxx (Replace xxxx with the Github issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: CI
description: Continuous Integration

Comments:
Not very interesting here but will be more useful in case of failures

<img width="507" alt="Screenshot 2022-09-02 at 09 43 26" src="https://user-images.githubusercontent.com/175722/188085276-ecbb5faa-34c3-4329-8b88-7798f60b8583.png">

Thanks @jonashaag
TST: Address/catch more test warnings
State: closed

- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).

Labels:
name: Testing
description: pandas testing functions or related to the test suite

Comments:
thx @mroeschke
DOC: add pd.Grouper type to groupby by argument types
State: closed

- [ ] closes #48230 
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Docs
name: Groupby

Comments:
Thanks @paradox-lab
CLN: consistent use of rval within left_join_indexer_unique (minor cleanup)
State: closed

- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).

Very minor cleanup. Replacing `right[j]` with `rval` to be consistent with code above and below the line in question. (`rval = right[j]`) 

Labels:

Comments:
thx @lukemanley
CI: Bump timeout to 180 minutes
State: closed

Noticing a few timeouts on builds lately.

Investigating whether we need to bump the timeout limit or some tests are stalling.


Labels:
name: CI
description: Continuous Integration

Comments:
The test suite has definitely gotten slower in the last month or two.  Used to be ~15 minutes locally, now closer to twice that.
FYI just ran `pytest pandas/tests/extension/test_arrow.py --skip-slow --skip-db` after upgrading to newest pyarrow and a) took 10:15 and b) 4441 xfails
Failures are related to https://github.com/pandas-dev/pandas/issues/48432. Merging as this will help alleviate the timeouts in the meantime
@mroeschke I think we should backport, do you agree?
Sure, this should be a low risk change to backport.
@meeseeksdev backport 1.5.x
Yep agreed. Want to avoid dealing with the timeouts for the next 5-6 months
Added test for `assert_index_equal` to check that function raises an exception if two different types
State: closed

- [x] closes #31884
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Missing-data
description: np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate

Comments:
Thanks @MrShevan 
@mroeschke no problems! :)
ENH: __pandas_priority__
State: closed

Will allow dask/modin to address #38946 and https://github.com/modin-project/modin/issues/4646, respectively.

Also can be used to make non-masked EAs defer to masked EAs.

Labels:
name: Enhancement

Comments:
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
@jbrockmendel I see there is no feedback from others here. But I guess we'll need documentation if we want to merge this. It should also help the discussion, I checked the referred issues, but it's not immediately clear how the new attribute works.
@jbrockmendel Would there be a release note needed for this?
Added a whatsnew, not sure how to make a reference to _extending.pandas_priority
@jreback suggestions on how to get the whatsnew to point to the appropriate place in the docs?
#@jbrockmendel no idea sorry
> Added a whatsnew, not sure how to make a reference to _extending.pandas_priority

``` :ref:`See the developer guide <extending.pandas_priority>` ```
green
Thanks @jbrockmendel
REF: de-duplicate take_nd
State: closed

- [ ] closes #xxxx (Replace xxxx with the Github issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Refactor
description: Internal refactoring of code
name: Internals
description: Related to non-user accessible pandas implementation

Comments:
Thanks @jbrockmendel
BUG: MultiIndex.unique losing ea dtype
State: closed

- [ ] closes #xxxx (Replace xxxx with the Github issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

the MultiIndex.unique code path is casting the values to tuples and runs through unique1d. This is really slow.

```
       before           after         ratio
     [e7414aa1]       [d17eb141]
     <48184^2>        <perf_mi_unique>
-         422±2ms       30.3±0.5ms     0.07  multiindex_object.Unique.time_unique_dups(('Int64', <NA>))
-        496±70ms         28.4±1ms     0.06  multiindex_object.Unique.time_unique_dups(('int64', 0))
-         811±4ms       32.9±0.2ms     0.04  multiindex_object.Unique.time_unique(('int64', 0))
-        812±20ms       32.8±0.9ms     0.04  multiindex_object.Unique.time_unique(('Int64', <NA>))
```

Labels:
name: MultiIndex

Comments:
cc @mroeschke for a second set of eyes
BUG: read_html(extract_links=all) with no header
State: closed

- [x] closes #48316 (Replace xxxx with the Github issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).

This feature was added in 1.5.


Labels:
name: IO HTML
description: read_html, to_html, Styler.apply, Styler.applymap

Comments:
> lgtm, shouldn't we add a release note?

I had assumed that bug fixes during the rc period didn't necessarily need whatsnew notes since this feature will be new for 1.5
BUG: make Series handle dtype='int64' for string array
State: closed

- [x] closes #44923
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Dtype Conversions
description: Unexpected or buggy dtype conversions
name: Strings
description: String extension data type and string data

Comments:
Please move whatsnew to 1.6.0
Thanks @5j9
to_sql: Adding overwrite or ignore behavior for row conflicts
State: closed

- [x] closes https://github.com/pandas-dev/pandas/issues/14553, continuation of https://github.com/pandas-dev/pandas/pull/46232 and https://github.com/pandas-dev/pandas/pull/29636
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).  
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

I do not know how to continue on the PR of LSturtew (https://github.com/pandas-dev/pandas/pull/46232), but I would like to continue on this feature, as only tests or docstrings are missing I guess...
Is there some way to credit LSturtew and cvonsteg for most of the work? It would be unfair if only I get credit for just fixing tests or docstrings


Labels:
name: IO SQL
description: to_sql, read_sql, read_sql_query
name: Stale

Comments:
Hello @JustinGuese! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:







There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2022-09-12 13:00:16 UTC
so as I guessed I think the main reason is that `SqlAlchemy` has some problems with arm and mac chips (not available)... Any proposals on how to solve it? Can we make a simple exception, if the chipset is arm or mac, just throw a "Not Implemented" error? Probably not the best solution, right?

Alternatively, we could probably swap out SqlAlchemy with the native libraries (MySQL, Postgresql, ...), but that sounds horrible. What are your thoughts?

```
import sqlalchemy
E   ModuleNotFoundError: No module named 'sqlalchemy'
```
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
Looking forward to seeing this feature in action! keep up pushing
Hi @JustinGuese, any update on this? Thanks for your hard work
yeah we need someone who's good with python tests. at least my last status was that it works, but the tests are failing. any volunteers?
You are calling functions with invalid keyword arguments. You can step through this locally.
Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen.
BUG: test for empty input construct
State: closed

- [ ] closes #47959
- [ ] [Tests added ]


Labels:

Comments:
Hello @Th3nn3ss! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:



There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2022-08-31 14:40:27 UTC
Great idea. 

Closing this as all tests for empty inputs are subsequently added to #48327
WEB: Update to `jupyterlite==0.1.0b12`
State: closed

Update to the latest `jupyterlite` release to bring some more bug fixes and improvements.

This should help fix an issue with the current repl failing to load

- [x] ~closes #xxxx (Replace xxxx with the Github issue number)~
- [x] ~[Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature~
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] ~Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.~
- [x] ~Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.~


Labels:
name: Web
description: pandas website

Comments:
Thanks @jorisvandenbossche for reporting the issue today.

I tried the built archive locally downloaded from https://github.com/pandas-dev/pandas/actions/runs/2964426541, and it seems to be working well with the update:

![image](https://user-images.githubusercontent.com/591645/187778716-697fde27-007f-4d70-a347-3059c35b2887.png)

Thanks @jtpio. Agreed, a loading indicator would be a nicer user experience (to set expectation of the activity in the background)
Thanks all.

> a loading indicator

Yes that's a good point and it is currently tracked this in https://github.com/jupyterlite/jupyterlite/issues/619, which could be applied to all applications (lab, notebook, repl).
BUG: dataframe returned for alternative agg method call np.size and "size" are inconsistent
State: closed

- [ ] closes #47959
- [ ] [Tests added ]


Labels:

Comments:
```python
import pandas as pd
import numpy as np

df = pd.DataFrame(columns=['a', 'b'])

# These return equal objects
df.agg({'a':['mean','std']})
df.agg({'a':[np.mean,np.std]})

# This does not
df.agg({'a': ['size']})
df.agg({'a': [np.size]})
```
This isn't peculiar to empty inputs. And is a duplicate of #42203. I will be closing this draft PR.
BUG: inconsistent behaviour for empty DataFrames
State: closed

- [ ] Closes #47959
- [x] Tests added




Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: DataFrame
description: DataFrame data structure
name: Stale

Comments:
Hello @Th3nn3ss! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:







There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2022-09-08 07:55:53 UTC
@jorisvandenbossche Yes, The idea is to show some inconsistencies with empty inputs with some failing tests.

I will love to fix it.

This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
Still working on this @Th3nn3ss or has this been superseded with your other work?
Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen.
BUG: Add note in whatsnew for DataFrame.at behavior change
State: closed

- [x] closes #48296 (Replace xxxx with the Github issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

The issue mentioned above is just for tracking the change in behavior in 1.5. I've opened #48323 to track the remaining behavior issue mentioned there.

Labels:
name: Bug
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves

Comments:
Do you know if we have a test for the above behavior change?
Couldn't find one at first glance
Ah, should have thought of that; thanks @mroeschke - one on the way.
thx @rhshadrach
dataframe.astype() exception message to include column name
State: closed

- [X] closes #47571(Replace xxxx with the Github issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [X] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

**Description:**
Added column related info as part of `astype` exceptions

| Code  | Old Error | New Error |
| ------------- | ------------- | ----------|
| pd.DataFrame({'a':[1,2,"text"], 'b':[1,2,3]}).astype({'a': int}) | ValueError: invalid literal for int() with base 10: 'text' | ValueError: invalid literal for int() with base 10: 'text': Error while type casting for column 'a' |


Labels:
name: Dtype Conversions
description: Unexpected or buggy dtype conversions
name: Error Reporting
description: Incorrect or improved errors from pandas

Comments:
thanks for the review @phofl @mroeschke 
sure @phofl, will update the PR with changes
Hi @mroeschke , had missed following up on this PR.
feel free to let me know if any changes are required here (or) if we should hold this PR for the time being.

Thanks in advance :)
Thanks @dannyi96
TST: Filter false positive matplotlib warning
State: closed

- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).

xref https://github.com/matplotlib/matplotlib/issues/23614.

This false positive warning showed up in our test suite and can be filtered out.


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Visualization
description: plotting
name: Warnings
description: Warnings that appear or should be added to pandas

Comments:
thx @mroeschke
CI: Clean circleci setup env file
State: closed

- [x] closes #48307 (Replace xxxx with the Github issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature

#48307 is tracked by #47305. Cleaning the `setup_ev.sh` file since it's only used for this build.

Labels:
name: CI
description: Continuous Integration

Comments:
Thx for this. Wasn’t aware that the issue was still open.

should we switch to mambaforge as a follow up?
> should we switch to mambaforge as a follow up?

Yes, that would be a useful followup PR.
Thx @mroeschke
REGR: AttributeError: 'bool' object has no attribute 'to_numpy' in "mask_missing"
State: closed

- [ ] closes #47101(Replace xxxx with the Github issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Missing-data
description: np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate
name: Regression
description: Functionality that used to work in a prior pandas version

Comments:
closing so can start release tomorrow
Backport PR #48299 on branch 1.5.x (Bump s3fs to 2021.08.00)
State: closed

Backport PR #48299: Bump s3fs to 2021.08.00

Labels:
name: CI
description: Continuous Integration
name: Dependencies
description: Required and optional dependencies

Comments:
Cc @simonjayhawkins in Case we Need something similar for 1.4.x
DEPR: Deprecate positional arguments in pivot
State: closed

- [x] xref #30228 (Replace xxxx with the Github issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

cc @mroeschke 

Labels:
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode
name: Deprecate
description: Functionality to remove in pandas

Comments:
Done
Green now, merging
TYPE: Move any all to _typing
State: closed

- [ ] closes #xxxx (Replace xxxx with the Github issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

cc @mroeschke @twoertwein 

Labels:
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
Thanks @phofl