BrandonFors/pandas-github-issues huggingface dataset:
CI: Change flaky to_excel test to use xlsxwriter
State: closed

`test_write_fspath_all` has been flaky and failing when comparing the binary xml for `to_excel` with openpyxl e.g.

```
_ TestCommonIOCapabilities.test_write_fspath_all[to_excel-writer_kwargs1-openpyxl] _
[gw2] darwin -- Python 3.9.13 /Users/runner/micromamba/envs/test/bin/python3.9

self = <pandas.tests.io.test_common.TestCommonIOCapabilities object at 0x157198af0>
writer_name = 'to_excel', writer_kwargs = {'engine': 'openpyxl'}
module = 'openpyxl'

    @pytest.mark.filterwarnings("ignore:In future versions `DataFrame.to_latex`")
    @pytest.mark.parametrize(
        "writer_name, writer_kwargs, module",
        [
            ("to_csv", {}, "os"),
            ("to_excel", {"engine": "openpyxl"}, "openpyxl"),
            ("to_feather", {}, "pyarrow"),
            ("to_html", {}, "os"),
            ("to_json", {}, "os"),
            ("to_latex", {}, "os"),
            ("to_pickle", {}, "os"),
            ("to_stata", {"time_stamp": pd.to_datetime("2019-01-01 00:00")}, "os"),
        ],
    )
    def test_write_fspath_all(self, writer_name, writer_kwargs, module):
        p1 = tm.ensure_clean("string")
        p2 = tm.ensure_clean("fspath")
        df = pd.DataFrame({"A": [1, 2]})
    
        with p1 as string, p2 as fspath:
            pytest.importorskip(module)
            mypath = CustomFSPath(fspath)
            writer = getattr(df, writer_name)
    
            writer(string, **writer_kwargs)
            with open(string, "rb") as f:
                expected = f.read()
    
            writer(mypath, **writer_kwargs)
            with open(fspath, "rb") as f:
                result = f.read()
    
>           assert result == expected
E           assert b'PK\x03\x04\...0\x00\x00\x00' == b'PK\x03\x04\...0\x00\x00\x00'
E             At index 10 diff: b'\x13' != b'\x12'
E             Full diff:
E               (
E             -  b'PK\x03\x04\x14\x00\x00\x00\x08\x00\x12xbU\x07AMb\x81\x00\x00\x00\xb1\x00'
E             ?                                         ^
E             +  b'PK\x03\x04\x14\x00\x00\x00\x08\x00\x13xbU\x07AMb\x81\x00\x00\x00\xb1\x00'
E             ?           
```

This was changed from `xlwt` to `openpyxl` in https://github.com/pandas-dev/pandas/pull/49296, seeing if `xlsxwriter` is more stable.

Labels:
name: IO Excel
description: read_excel, to_excel
name: Unreliable Test
description: Unit tests that occasionally fail

Comments:
Guessing this should now pass with openpyxl as well as xlsxwriter now that we aren't comparing all file contents. Would it make sense to still test openpyxl, or is that redundant?
I think testing both openpyxl and xlsxwriter would be redundant since it seem the main point of the test is to test a `CustomFSPath`

I changed it back to testing openpyxl.
Thanks @mroeschke
DEPR: enforce non-positional, keyword deprecations
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Deprecate
description: Functionality to remove in pandas

Comments:
Thanks @jbrockmendel
BUG: Allow tz-aware Datetime SQL columns to be passed to parse_dates kwarg.
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [X] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

This PR is related to existing tests which fail for me on the main branch, but they pass after making these changes.
```
FAILED pandas/tests/io/test_sql.py::TestPostgreSQLAlchemy::test_datetime_with_timezone - AssertionError: assert False
FAILED pandas/tests/io/test_sql.py::TestPostgreSQLAlchemyConn::test_datetime_with_timezone - AssertionError: assert False
```

Both tests fail here:
```
        # this is parsed on Travis (linux), but not on macosx for some reason
        # even with the same versions of psycopg2 & sqlalchemy, possibly a
        # Postgresql server version difference
        col = df.DateColWithTz
>       assert is_datetime64tz_dtype(col.dtype)
E       AssertionError: assert False
E        +  where False = is_datetime64tz_dtype(dtype('O'))
E        +    where dtype('O') = 0    2000-01-01 00:00:00-08:00\n1    2000-06-01 00:00:00-07:00\nName: DateColWithTz, dtype: object.dtype

pandas\tests\io\test_sql.py:1794: AssertionError
```

Labels:
name: IO SQL
description: to_sql, read_sql, read_sql_query

Comments:
I will set the Postresql timezone to UTC so that these tests can pass.
Bug: Don't intercept TypeError in DataFrame.agg
State: closed

- [ ] closes #49399  (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Clean
name: Apply
description: Apply, Aggregate, Transform, Map
name: Stale

Comments:
Thanks for the PR! Could you update the title by clicking the edit button here:

![image](https://user-images.githubusercontent.com/45562402/199829857-13f0e90f-1373-41c2-9664-1769d51a87ee.png)

to be a bit more descriptive? It can even just be the title from the corresponding issue.

I also suspect there will be a number of test failures; we check the error message and that message will now need to be updated.
@rhshadrach,
Thanks for your msg. Sorry about the title. I believe it's better now. Also regarding the test failures, I believe they will have to be fixed at your end? Or would I have to change something in the test folder?
Title looks great - thanks!

> Or would I have to change something in the test folder?

You're changing tested behavior of pandas, and that is going to cause the tests to "fail". The tests that have failed need to be updated with the correct error types and messages.
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
Closing as stale. @sarvaSanjay - if you'd like to continue comment here and we can reopen.
CI: fix flake8 warning
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: CI
description: Continuous Integration

Comments:
#49457 and #49485 both removed warnings from that file and were merged in close succession (without being updated), hence the CI failure crept through - no biggie
thanks - merging then as this affects the builds of open PRs, and is also a trivial change
Thanks for catching my oversights @MarcoGorelli 
It's OK, always updating all PRs to be up-to-date with upstream/main isn't really feasible given the volume of PRs, if there's the occasional lint issue that creeps through that's OK, it's not a common occurrence
DEPR: Index.__and__, __or__, __xor__
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Deprecate
description: Functionality to remove in pandas
name: Index
description: Related to the Index class or subclasses

Comments:
LGTM. 

xref #37374
I'd be ok merging this, but am a bit new at reviewing PRs. Maybe if no one comments in the newx few hours, I'll merge it?
Thanks @jbrockmendel (and happy for you to merge these straightforward PRs as well in the future @topper-123)
DOC: Clarify deprecation of freq for Timestamp
State: closed

- [X] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [X] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [X] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Docs
name: Stale
name: Timestamp
description: pd.Timestamp and associated methods

Comments:
The failure is wrong.  Very strange, but both of these are clearly used.
This seems benign, but will be moot once #49365 is merged
Should it go in for 1.5.x?
> Should it go in for 1.5.x?

sounds good
Needs a rebase and ci is failing
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen.
Backport PR #49386 on branch 1.5.x (DOC: add name parameter to the IntervalIndex for #48911)
State: closed

…#48911



Labels:

Comments:
cc @Dr-Irv Is this a new argument? In general, we try to avoid backporting documentation fixes
> cc @Dr-Irv Is this a new argument? In general, we try to avoid backporting documentation fixes

No.  In the current docs, the argument is listed in the signature of these methods, but it is not described.

Yeah typically we only backport doc fixes if it was pertinent to something added in the last minor version or other exceptional cases to minimize backport conflicts
From the conversation above, seems like we don't want to backport this, closing.
DEPR: move NumericIndexes into base Index, part 1
State: closed

A part of the changes in pandas 2.0 will be that the current numeric indexes (Int64Index, Uint64Index & Float64Index) will be removed and their functionality added to the base Index. In addition, the base index should be able to handle all numpy numeric dtypes (int8, int16 tc.), not just the current int64, uint64 & float64.

This PR makes progress towards that. In this PR I:
* remove Int64Index, Uint64Index & Float64Index from the top namespace
* Make instantiation of `Index` with numpy numeric dtypes return a `NumericIndex` with the given dtype rather thanconverting to the old 64 bit only numeric index types, making all numeric dtypes available for indexes.

This is just the first part of the complete changes planned. In follow-ups PRs I will:
* remove Int64Index, Uint64Index & Float64Index from the code base entirely (also internally)
* Move the functionality of `NumericIndex`into the base `Index` and remove `NumericIndex`
* Update docs to reflect the above changes

## Notable changes from this PR:
One notable change from the PR is that because we now have all the numeric dtypes available for indexes, indexes created from DateTimeIndex (day, year etc.) will now be in int32, where previously they were forced to int64 (because that was the only index integer dtype available).

Using int32 is the more correct way, because the underlying `DateTimeArray` returns 32bit arrays. An example of this:

```ipython
>>> import pandas as pd
>>> 
>>> x = pd.date_range(start='1/1/2018', end='1/08/2018')
>>> x.array.day
array([1, 2, 3, 4, 5, 6, 7, 8], dtype=int32)  # 32bit, current and will not be changed in this series of PRs
>>> x.day
Int64Index([1, 2, 3, 4, 5, 6, 7, 8], dtype='int64')  # before this PR
NumericIndex([1, 2, 3, 4, 5, 6, 7, 8], dtype='int32')  # after this PR
Index([1, 2, 3, 4, 5, 6, 7, 8], dtype='int32')  # after follow-up PRs
```

xref #42717, #41272

Labels:
name: Refactor
description: Internal refactoring of code
name: Deprecate
description: Functionality to remove in pandas

Comments:
looks like a couple of sparse tests are failing
Yeah, test  suite passed locally, trying to figure out what's the difference. Probably I need to install some package locally, so tests are skipped locally.

UPDATE: everything passes locally still, so I'm a bit stomped why this fails in Github. I'll look again tomorrow.
Closing in favor of #49560.
DEPR: DatetimeIndex indexing with mismatched tzawareness
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Deprecate
description: Functionality to remove in pandas

Comments:
Thanks @jbrockmendel
Bump matplotlib min_ver to 3.6.1 (from 3.6.0)
State: closed

- [ ] closes #49498 (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

I think this will fix the minimum-versions build.

Labels:
name: CI
description: Continuous Integration
name: Dependencies
description: Required and optional dependencies

Comments:
Should `environment.yml` also be updated? Or can the documentation be updated to show up to update dependencies in a dev environment? It wasn't as easy as simply rerunning the commands [here](https://pandas.pydata.org/pandas-docs/stable/development/contributing_environment.html#creating-a-python-environment).
You can use `mamba env update` to update your environment
DEP: Enforce respecting group_keys in groupby.apply
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Groupby
name: Deprecate
description: Functionality to remove in pandas
name: Apply
description: Apply, Aggregate, Transform, Map

Comments:
Merge conflict otherwise LGTM
Thanks!
CLN: assorted follow-ups
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Clean

Comments:
Looks like the asv failure is real, will look into it
Never mind, the asv failure is present on main too.  Greenish here.
Thanks @jbrockmendel
DEPR: maybe_promote with pydate
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Deprecate
description: Functionality to remove in pandas

Comments:
```
2022-11-02T23:51:48.8199776Z /home/runner/work/pandas/pandas/doc/source/whatsnew/v2.0.0.rst:319: WARNING: more than one target found for cross-reference 'reindex': pandas.DataFrame.reindex, pandas.Series.reindex, pandas.Index.reindex
```

So in the whatsnew `DataFrame.reindex`, `Series.reindex`, and `Index.reindex`, need to be specified manually I think 
Thanks @jbrockmendel
DEPR: Index.reindex with non-unique Index
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Deprecate
description: Functionality to remove in pandas

Comments:
thx @jbrockmendel (Could you update deprecations issue?)
DEPR: remove Index.is_monotonic and Series.is_monotonic
State: closed

- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/v2.0.0.rst` file if fixing a bug or adding a new feature.

introduced in #45422

Labels:
name: Deprecate
description: Functionality to remove in pandas

Comments:
#49457
> #49457

Yep, closing
API: Series.astype(dt64_unsupported) raise
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

dt64 analogue to #49290

Labels:

Comments:
superseded by #49290
DEPR: lookup, append
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Deprecate
description: Functionality to remove in pandas

Comments:
Looks like there are append ASVs that can be removed

```
2022-11-04T16:12:14.9511166Z [ 29.19%] ··· join_merge.Append.time_append_homogenous                    failed
2022-11-04T16:12:15.0050434Z [ 29.24%] ··· join_merge.Append.time_append_mixed                         failed
```
ci fails unrelated
Thanks @jbrockmendel
enable pylint: consider-using-sys-exit
State: closed

Issue #48855. This PR enables pylint type "R" warning: `consider-using-sys-exit`.
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).

Labels:
name: Code Style
description: Code style, linting, code_checks

Comments:
Thanks @natmokval (failures were unrelated)
DEPR Series[td64].fillna(incompatible)
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Deprecate
description: Functionality to remove in pandas

Comments:
Thanks @jbrockmendel