BrandonFors/pandas-github-issues huggingface dataset:
DOC: Update link from VirtusLab to pandas-dev repo
State: closed

- [ ] closes #xxxx (Replace xxxx with the Github issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

According to https://github.com/VirtusLab/pandas-stubs:

> As of July 2022 pandas_stubs package is no longer sourced from here but instead from a repository owned and maintained by the core pandas team: https://github.com/pandas-dev/pandas-stubs

This PR updates the links in the documentation to reflect this change.

Labels:
name: Docs

Comments:
Thanks @james-d-f
ENH: Allow read_csv(engine=pyarrow) to return ArrowExtensionArray-backed columns
State: closed

- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

The `engine="pyarrow"` was added in 1.4 and is considered "experimental", so I would like opinions whether it's okay to 

1. Just change the return type to ArrowExtensionArray-backed data in 1.5
2. Raise a warning in 1.5 that the return type will change to ArrowExtensionArray-backed in 2.0. I can't immediately think of a nice way to "opt into" this behavior in 1.5 besides calling `astype()` after the `read_csv` call.


Labels:
name: IO CSV
description: read_csv, to_csv
name: Arrow
description: pyarrow functionality

Comments:
I would be -1 on anything but opt-in. IMO, the behavior should be consistent across all three engines. I know the read_csv is rather bloated with its many keywords, but I still think it would be worthwhile to add a new one, or co-opt ``use_nullable_dtypes``(the proposed new keyword for the nullable EA types), especially given that Arrow EAs are still kinda new/experimental.

In general, I think there is still some discussion to be resolved around when the CSV parser should return extension array types vs numpy types(see #42973 for example).
That's a good point and wasn't aware of expanding `use_nullable_dtypes`. Yeah agreed I think it would be better to leverage that keyword in a fashion like `use_nullable_dtypes="pandas"|"pyarrow"|None`. Closing this then.
DOC: Adds Hamilton as a development tool
State: closed

Hamilton is a paradigm that helps one manage a pandas code base in an opinionated manner.

We have had great success with it at Stitch Fix, and it's got a growing open source
following. I think it helps one to write pandas in a way that gives
confidence if you want to run Pandas in production jobs.

- [x] N/A: closes #xxxx (Replace xxxx with the Github issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Docs

Comments:
Thanks @skrawcz
Added improvements in to_datetime Error reporting message - incorrect field today/now shown in error
State: closed

- closes #47495

Splitting changes from #47597, This PR aims to close the bug #47495

- Changes description
    -  For issue #47495-  made change in `pandas/_libs/tslib.pyx` to accept "now" and "today" values. These values are accepted in other areas of parsing (` _parse_today_now` in `pandas/_libs/tslib.pyx` )

- Example of changed error messages


    | Code       | Old Error ( on 1.4.3 ) | New Error |
    | ------------- | ------------- | ------------- |
    | pd.to_datetime(["today", "yesterday"]) | dateutil.parser._parser.ParserError: Unknown string format: today  | dateutil.parser._parser.ParserError: Unknown string format: yesterday present at position 1 |



Labels:
name: Datetime
description: Datetime data dtype
name: Error Reporting
description: Incorrect or improved errors from pandas

Comments:
hi @mroeschke, could you please review this change whenever possible ?
Above change does the trick, but not sure if anything else needs to be considered here ( in terms of other existing usecases, etc )
Thanks in advance
Thanks @dannyi96
DOC: correct links of Plotly
State: closed

https://poltly.com/ -> https://plotly.com/

- [x] closes #47857 
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Docs

Comments:
Thanks @CCXXXI
DOC/ENH: Add documentation and whatsnew entry for ArrowDtype and ArrowExtensionArray
State: closed

- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added an entry in the latest `doc/source/whatsnew/v1.5.0.rst` file if fixing a bug or adding a new feature.

* Depends on https://github.com/pandas-dev/pandas/pull/47818
* Should I add a dedicated user-guide page for this functionality?

Labels:
name: Docs
name: Arrow
description: pyarrow functionality

Comments:
Will merge tomorrow if no further comments.
DEPR: args and kwargs in rolling, expanding, and ewm ops
State: closed

- [x] closes #47836 (Replace xxxx with the Github issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

We can't call `maybe_warn_args_and_kwargs` any further in the stack (e.g. in `_apply`) because args aren't passed along.

Labels:
name: Numeric Operations
description: Arithmetic, Comparison, and Logical operations
name: Deprecate
description: Functionality to remove in pandas
name: Window
description: rolling, ewma, expanding

Comments:
The numeric_only argument in Expanding.sem was missing. The tests for numeric_only passed because we were still passing through as a kwarg. So no additional tests for this argument are needed.
thanks @rhshadrach very nice!
TYP/CLN: small cleanups from flake-pyi
State: closed

Most of these changes are:

> Y041 Use "float" instead of "int | float" (see "The numeric tower" in PEP 484)

flake8-pyi would probably find similar issues in the in-line annotations but flake8-pyi does at the moment only support pyi files.

Labels:
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
Hmm I understand from a typing cleanup and numeric inheritance perspective this makes sense, but I find this a little less explicit (especially if the typing is exposed in documentation(?)). 

For example, one might think they need to unnecessarily cast an int to float or it's less clear from the typing if a function only accepts a float.

I could be convinced otherwise. Just open to thoughts on those points.
Maybe two reasons why it might be okay:

- the type annotations are not included in the doc-string (only myy&pyright see them)
- all of the type-stubs in typeshed do the same (flake8-pyi is developed by the "typeshe people")
FWIW i just stumbled on this bc I saw "int" was missing from the annotation for validate_periods
Added improvements in to_datetime Error reporting message - Outofbounds error message
State: closed

Splitting changes from #47597, This PR aims to close the bug #46509

- Changes description
    -  For issue #46509 -  added more details in OutOfBoundsDatetime exception about field and its position causing the issue in `pandas/_libs/tslib.pyx` in `array_to_datetime`
    - Updated relevant testcases

- Example of changed error messages


    | Code       | Old Error ( on 1.4.3 ) | New Error |
    | ------------- | ------------- | ------------- |
    | pd.to_datetime(pd.Series(["11:00", "12:00", "11:0"])) | pandas._libs.tslibs.np_datetime.OutOfBoundsDatetime: Out of bounds nanosecond timestamp: 1-01-01 11:00:00  | pandas._libs.tslibs.np_datetime.OutOfBoundsDatetime: Out of bounds nanosecond timestamp: 1-01-01 11:00:00 present at position 2 |
    | pd.to_datetime(['11:0']) | pandas._libs.tslibs.np_datetime.OutOfBoundsDatetime: Out of bounds nanosecond timestamp: 1-01-01 11:00:00  | pandas._libs.tslibs.np_datetime.OutOfBoundsDatetime: Out of bounds nanosecond timestamp: 1-01-01 11:00:00 present at position 0 |
    | pd.to_datetime('11:0') | pandas._libs.tslibs.np_datetime.OutOfBoundsDatetime: Out of bounds nanosecond timestamp: 1-01-01 11:00:00  | pandas._libs.tslibs.np_datetime.OutOfBoundsDatetime: Out of bounds nanosecond timestamp: 1-01-01 11:00:00 present at position 0 |



Labels:
name: Error Reporting
description: Incorrect or improved errors from pandas
name: Timestamp
description: pd.Timestamp and associated methods

Comments:
Hello @dannyi96! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:











There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2022-08-01 16:04:48 UTC
Thanks @dannyi96
Fix styling of `DataFrame` for columns with boolean label
State: closed

- [x] closes #47838
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions. (Not applicable since no new public functions added or signatures modified.)
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Summary of changes
* Make the `subset` argument in styling by default a boolean mask indicating where the numerical columns of the data frame are.
* Add new test cases for checking that the expected columns are styled in `.ctx`.

Labels:
name: Styler
description: conditional formatting using DataFrame.style

Comments:
Thanks @bherwerth
DOC: Add spaces before `:` to cancel bold
State: closed

- [x] closes #47735
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Docs
name: expressions
description: pd.eval, query

Comments:
after add spaces rendering

![image](https://user-images.githubusercontent.com/25895405/180792393-ecd4575a-f3f1-4678-9c24-1050d951ade2.png)

Thanks @Zeroto521
Faster method of normalising dictionary, allows user to select columns to ignore
State: closed

- [ ] Adds new feature and functionality to _normalize() function; ignores user specifed columns and returns the ignored columns in multiple forms (dictionary, tuple, list)
- [ ] closes https://github.com/pandas-dev/pandas/issues/47852
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.
- [ ] CODE:


Labels:

Comments:
Hello @PriestlyAdejo! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:



There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2022-07-25 22:05:08 UTC
Why does pre-commit keep on failing? I made sure to revise all the errors I made before...
> is this in reference to. specific issue?

No not really... I made it as an addition to pandas_normalize as I was having major problems with the original function when I had json files with uneven columns (due to creating extra data while scraping).
I've added an issue in issues. This was an ENH:. Sorry for the errors, it's my first time making a contribution to a community package! 

Issue: #47843 
Thanks for creating the issue (https://github.com/pandas-dev/pandas/issues/47852).

I see that this function has differences than `json_normalize`, but given that the new function is similar to `json_normalize` I would be -1 on including an entirely separate method.

 
> Thanks for creating the issue (#47852).
> 
> I see that this function has differences than `json_normalize`, but given that the new function is similar to `json_normalize` I would be -1 on including an entirely separate method.

I agree with you on that I don't think this should replace json_normalize, but I added it as an extra function WITHIN json normalize. This function could be used for whenever a user wants to ignore columns, although they can still choose to use the original json_normalize if they want to.
Hello?

Personally, I still don't believe this functionality would be suitable to include in pandas. It's a lot of additional code to maintain and `json_normalize` is the primary functionality to normalize a record/dict-like structure _to_ a pandas object
Thanks for the effort here so far, but I think it's best if discussion occur on the issue you made first to gauge interest before moving with a pull request, so closing.
Adding documentation of main function to docs file
State: closed

- [ ] closes #xxxx (Replace xxxx with the Github issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:

Comments:
I think a link to the relevant documentation page will suffice in the whatsnew document. 

It is not advisable to included documentation of all the added/changed methods in the whats new.
It appears this note is related to https://github.com/pandas-dev/pandas/pull/47843, so if https://github.com/pandas-dev/pandas/pull/47843 is accepted, this note should belong to that PR so closing in favor of including in that PR.
BUG: fix bug where appending unordered ``CategoricalIndex`` variables overrides index
State: closed

- [x] closes #24845
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Categorical
description: Categorical Data Type

Comments:
Hi @mroeschke or @jreback, I would love to have some feedback on this PR. Thanks a lot!
Hi @mroeschke, I have made the edits. I thought something was wrong with my commit, then I realized that a PR was merged with the same three errors. Could you explain, if possible, why this is happening? Thanks!
> Hi @mroeschke, I have made the edits. I thought something was wrong with my commit, then I realized that a PR was merged with the same three errors. Could you explain, if possible, why this is happening? Thanks!

We had some unrelated CI issues lately. Could you merge in main once more?
Hi @mroeschke! I merged and I am receiving errors that I'm not receiving when I run the tests on my local machine. 
> Hi @mroeschke! I merged and I am receiving errors that I'm not receiving when I run the tests on my local machine.

We're experience a similar error on main that is affecting all builds. Merging one more time should fix it.
It passed all test cases, @mroeschke. Thanks a lot for your guidance!
Thanks for sticking with all the reviews and being responsive @GivyBoy!
DEPR: inplace keyword for Categorical.set_ordered, setting .categories directly
State: closed

- [ ] closes #xxxx (Replace xxxx with the Github issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


xref #37643 (not sure if this closes it since we may want to add a copy=True keyword)

Labels:
name: Categorical
description: Categorical Data Type
name: Deprecate
description: Functionality to remove in pandas

Comments:
Thanks @jbrockmendel
DOC: added type annotations and tests to check for multiple warning match
State: closed

- [x] closes #47829 
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.



Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Docs
name: Warnings
description: Warnings that appear or should be added to pandas

Comments:
Hello @shourya5! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:





There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2022-08-03 05:49:49 UTC
@github-actions pre-commit
> @github-actions pre-commit

this was removed not long ago (the stable docs will only update after the 1.5 release, but the dev docs already don't show it) - can you run it locally?

Just
```
pre-commit run --files <file_1> <file_2> ... <file_n>
```
where you put any files you've modified after `--files`

I am failing one test besides the pre-commit test
You need to update the type annotation in _assert_caught_no_extra_warnings    too
Does this look correct?
the pre-commit action is changing 'Tuple' to tuple. Should I change it or leave it as is?
Yeah that's fine, just git add, git commit, git push
I think it should pass all tests now
WIP warn if parsing with du_parse
State: closed

- [ ] closes #12585 (Replace xxxx with the Github issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Datetime
description: Datetime data dtype
name: Warnings
description: Warnings that appear or should be added to pandas

Comments:
The issue is that in
```
        df = DataFrame(
            [[1, 2], [3, 4]],
            columns=date_range("1/1/2013", "1/2/2013"),
            index=["A", "B"],
        )
        df.index.name = name

        with tm.assert_produces_warning(warn):
            result = df.reset_index()
```
, resetting the index will try to insert an element into a datetime index. hence, it'll try to parse the new element as a date, and this'll trigger the new warning

There's currently no way of catching multiple warnings, so perhaps that should be addressed first

EDIt: passing multiple warnings as a tuple works fine
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
> This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.

still working on this
closing, as from https://github.com/pandas-dev/pandas/issues/12585#issuecomment-1249343334 I don't think this is the right solution



[log.txt](https://github.com/pandas-dev/pandas/files/9594114/log.txt)