BrandonFors/pandas-github-issues huggingface dataset:
CLN: TODOs/FIXMEs
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [ ] whatsnew entry


Labels:
name: Clean

Comments:
@mroeschke LMK if pinging you with these stops being helpful:

> AssertionError: Caused unexpected warning(s): [('ResourceWarning', ResourceWarning("unclosed <ssl.SSLSocket fd=8, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('10.1.0.137', 51532), raddr=('52.216.164.101', 443)>"), '<frozen importlib._bootstrap>', 1043)]
CLN: contextmanagers should yield in a try-finally block
State: closed

`@contextmanagers` should yield within a try-finally block to release/reset even when an error occurs.

Labels:
name: Clean

Comments:
what motivated this change? e.g. did you see an error on something?
> what motivated this change? e.g. did you see an error on something?

No error, was just grepping for `@contextmanager` to see whether they all use try-finally. I think at least the change in `file_leak_context` should be very useful to verify that IO even closes file handles in case of an error (not sure how important the other two changes are).
thanks @twoertwein
BUG: df.iloc[ndarray, ndarray] = 2dvals raising with EA column
State: closed

- [x] closes #44703
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [x] whatsnew entry


Labels:
name: Bug
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: ExtensionArray
description: Extending pandas with custom dtypes or arrays.

Comments:
lgtm, can you rebase
rebased + green
Deprecate prefix argument in read_csv and read_table
State: closed

- [x] closes #43396
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [x] whatsnew entry

cc @lithomas1 I am also on board with deprecating this. I don't think the keyword adds real value


Labels:
name: IO CSV
description: read_csv, to_csv
name: Deprecate
description: Functionality to remove in pandas

Comments:
Using ``parser.read_csv_check_warnings`` should fix the stacklevel issues. 
https://github.com/pandas-dev/pandas/blob/bdff2148ec5c81b2724ac77b3138d229994cffc4/pandas/tests/io/parser/conftest.py#L31-L39

It would be also nice to add an example to the user guide of how to add a prefix to the names.
thanks @phofl
Deprecate positional arguments for read_fwf
State: closed

- [x] xref #41485
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [x] whatsnew entry

Similar to read_csv this will simplify typing the return values

Labels:
name: Deprecate
description: Functionality to remove in pandas
name: IO Fixed Width
description: read_fwf

Comments:
Added a test.
DataFrame arithmetic operators __finalize__ call
State: closed

- [x] xref #28283 
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [x] whatsnew entry


Labels:
name: Compat
description: pandas objects compatability with Numpy or Python functions
name: Stale

Comments:
You can use xref instead of closes, which shows your pr in the issue history
@phofl Thank you for letting me know!
This pull request is stale because it has been open for thirty days with no activity. Please update or respond to this comment if you're still interested in working on this.
closing as stale if you want continue, pls ping
DOC: Removed link to Engarde project. It doesn't develop anymore. #44543
State: closed

issues #44543 

Removed link to [Engarde](https://engarde.readthedocs.io/en/latest/) project. It doesn't develop anymore [GitHab](https://github.com/engarde-dev/engarde#readme).

Added a link to the actively developing project  [Pandera](https://pandera.readthedocs.io/en/stable/) [GitHab](https://github.com/pandera-dev/pandera).





Labels:
name: Docs

Comments:
thanks @Mikhaylov-yv
DOC: fix examples in Timestamp functions
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [ ] whatsnew entry

I tried to create a new issue with the issue template 2 times but I got this error each time

![image](https://user-images.githubusercontent.com/23453691/144195459-541b63df-e84f-4a9f-9a90-855b8e68dbce.png)

Anyway, these changes are very small so perhaps there is no need to create an issue. I didn't run any tests since I made the changes through GitHub and did not clone the project locally.

Labels:
name: Docs
name: Datetime
description: Datetime data dtype

Comments:
Thanks @valentinoli
TST: Use IPython instance fixture to avoid file leak
State: closed

- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them

xref https://github.com/pandas-dev/pandas/pull/44634, https://github.com/pandas-dev/pandas/pull/35836

Probably should see if anything can be done on the iPython side.


Labels:
name: Testing
description: pandas testing functions or related to the test suite

Comments:
> Do you think it's likely this is the/a source of the warnings?

I want to say so. It aligns with an unclosed `BufferedRandom` since it was opened in `r+` mode.

Thanks for also raising an issue in iPython https://github.com/ipython/ipython/issues/12495
BUG: Series.where with incompatible NA value
State: closed

- [ ] closes #xxxx
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [x] whatsnew entry

Sits on top of #44514 (i.e. merge that first)

With this, can follow up to share where methods between Block subclasses.

Labels:
name: Bug
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: Missing-data
description: np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate

Comments:
merged the underlying if you can rebase
rebased + green
TST: read_csv raisin error with string dtype and parse dates
State: closed

- [x] closes #34066
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them



Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: IO CSV
description: read_csv, to_csv

Comments:
thanks @phofl
API: value_counts consistently return int64 dtype
State: closed

- [x] closes #44679
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [ ] whatsnew entry


Labels:
name: Dtype Conversions
description: Unexpected or buggy dtype conversions
name: ExtensionArray
description: Extending pandas with custom dtypes or arrays.

Comments:
when was the change added to make these EA types in the first place, was it for 1.4?
This is already the behaviour since at least 1.0 (checked it for Int64 for that version)
> This is already the behaviour since at least 1.0 (checked it for Int64 for that version)

ahh then this would be hard to change actually (as its the return type)
> ahh then this would be hard to change actually (as its the return type)

its all still marked as experimental
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
no apparent interest here, closing.
REF: simplify Block.where (and subtle alignment bug)
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [ ] whatsnew entry

The alignment is not user-facing AFAICT, but comes up when trying to address #44597.

Labels:
name: Refactor
description: Internal refactoring of code
name: Internals
description: Related to non-user accessible pandas implementation

Comments:
anything user facing? (since no new tests guessing no)
Not user facing.

On Tue, Nov 30, 2021 at 5:04 PM Jeff Reback ***@***.***>
wrote:

> anything user facing? (since no new tests guessing no)
>
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub
> <https://github.com/pandas-dev/pandas/pull/44691#issuecomment-983182784>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AB5UM6BFBTGMI2HSVKOBZ3DUOVYBTANCNFSM5JCUQW5A>
> .
> Triage notifications on the go with GitHub Mobile for iOS
> <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675>
> or Android
> <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.
>
>
ENH: Add indexing syntax to GroupBy.nth()
State: closed

- [x] closes #44589
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [x] whatsnew entry

Add index notation to nth for lists of integers and slices

Labels:
name: Groupby
name: API - Consistency
description: Internal Consistency of API/Behavior

Comments:
Hello @johnzangwill! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:







There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2021-12-04 16:01:30 UTC
thanks @johnzangwill very nice
Hi. Is this already implemented into pandas? I have a use case with groupby slice which is similar to this. Thank you!

> Hi. Is this already implemented into pandas? I have a use case with groupby slice which is similar to this. Thank you!

@emilmirzayev This has been merged to master for release in 1.4, which was scheduled for December 31 2021.
test dtype
State: closed

-  #43038 closes #xxxx
- 2 tests added / passed
- [ Yes] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [ ] whatsnew entry


Labels:

Comments:
@phofl Can you open this pr beacuse i was assigned to https://github.com/pandas-dev/pandas/issues/43038
and i made the change requested here https://github.com/pandas-dev/pandas/pull/44618
and my last commit passed all the tests
You can add the changes to #44618, no need to open a new pr every time you change something
TST: fix Nullable xfails
State: closed

- [ ] closes #xxxx
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [x] whatsnew entry


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Dtype Conversions
description: Unexpected or buggy dtype conversions

Comments:
user visible?
Whatsnew added + greenish
hmm a codecheck issue?
> hmm a codecheck issue?

#44590
CI: Try to not auto-cancel on master
State: closed

- [ ] closes #42076
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [ ] whatsnew entry

Seems to work. Let's try this again.

Labels:
name: CI
description: Continuous Integration

Comments:
@lithomas1 works? or we should just merge and see?
Yup, it seems to not cancel on a fake 1.3.x branch that I created locally and seems to cancel correctly on the pull request here properly. I'll still monitor to make sure it's working properly, though.
thanks @lithomas1 
I just got 14 emails about a single CI run being cancelled.  This is the most recent CI-centric PR merged AFAICT.  Any chance there's a connection?
I also just noticed this. Github actions is complaining about a line that was added in this PR (see eg https://github.com/pandas-dev/pandas/actions/runs/1535966470/workflow), but the strange thing is that this only started failing more recently on master (so after this PR was merged, there have been several other commits where github actions still ran)
REF: hold PeriodArray in NDArrayBackedExtensionBlock
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [ ] whatsnew entry

After this, Categorical will be the only NDArrayBackedExtensionArray not held in NDarrayBackedExtensionBlock.

(this also fixes a Series.where bug for PeriodDtype, the general fix (and tests) are blocked by #44514)

Labels:
name: Period
description: Period data type

Comments:
@jbrockmendel what's the direct advantage of using NDArrayBackedExtensionBlock instead of ExtensionBlock for PeriodArray?

Because this broke the pyarrow integration for period dtype up to pyarrow 3.0 (released just less than a year ago)
> what's the direct advantage of using NDArrayBackedExtensionBlock instead of ExtensionBlock for PeriodArray?

2D support!

And can you do that without breaking pyarrow <= 3.0? 
> And can you do that without breaking pyarrow <= 3.0?

I dont know what private APIs it uses.  Might be able to put in a shim.
They are already using the `pd.core.internals.make_block` function, but so in older versions pyarrow is also passing the block class (https://github.com/apache/arrow/blob/d613aa68789288d3503dfbd8376a41f2d28b6c9d/python/pyarrow/pandas_compat.py#L752-L753):

Letting it infer it works fine (what happens on the latest pyarrow releases):
```
In [10]: pd.core.internals.make_block(arr, placement=slice(0, 1, 1))
Out[10]: NDArrayBackedExtensionBlock: slice(0, 1, 1), 1 x 3, dtype: period[D]
```

but explicitly asking for an ExtensionBlock gives a mangled block with a 2D array under the hood (and for that reason it thinks it is shape (1, 1), because the underlying array is (1, 3)):

```
In [11]: pd.core.internals.make_block(
    ...:     arr, placement=slice(0, 1, 1), klass=pd.core.internals.ExtensionBlock
    ...: )
Out[11]: ExtensionBlock: slice(0, 1, 1), 1 x 1, dtype: period[D]

In [12]: _.shape
Out[12]: (1, 1)
```
> but explicitly asking for an ExtensionBlock gives a mangled block

yah it shouldn't do that.  we could put a shim in `ExtensionBlock.__init__` to check for that and raise, maybe telling users to upgrade pyarrow.
And it already doesn't do that anymore. 

> we could put a shim in ExtensionBlock.__init__ to check for that and raise, maybe telling users to upgrade pyarrow.

Can't we handle that inside `make_block`? If you pass a period array, and specify ExtensionBlock, we can return the proper block instead?
> Can't we handle that inside make_block? If you pass a period array, and specify ExtensionBlock, we can return the proper block instead?

makes sense
44414 data frame attributes not propagating with astype
State: closed

- [x] closes #44414 
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [x] whatsnew entry


Labels:
name: metadata
description: _metadata, .attrs

Comments:
Couple of comments
thanks @skwirskj 
No problem @jreback thank you for your help!
Type read_fwf
State: closed

- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them


Labels:
name: Typing
description: type annotations, mypy/pyright type checking
name: IO Fixed Width
description: read_fwf

Comments:
Thanks for the rewview
Typ parts of c parser
State: closed

- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them



Labels:
name: IO CSV
description: read_csv, to_csv
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
gentle ping @simonjayhawkins 
thanks @phofl
CLN: helper attribute to check for ArrayManager
State: closed

xref https://github.com/pandas-dev/pandas/pull/41878#discussion_r758514585

Labels:
name: Stale

Comments:
grepping for `hasattr.*blocks` shows a few more places this can be used
https://github.com/pandas-dev/pandas/runs/4364239489?check_suite_focus=true some typing :-<
Any suggestion how to deal with this? Because if mypy doesn't understand the helper function, that basically means that you need to add a `cast` (or ignore comment) every time this is used ..
> Any suggestion how to deal with this? Because if mypy doesn't understand the helper function, that basically means that you need to add a cast (or ignore comment) every time this is used ..

1) `type: ignore`
2) add a `blocks` property to AM that raises NotImplementedError
3) ping @simonjayhawkins
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
I assume would need to make DataFrame Generic and add the `_mgr` class as a type parameter. Since `_mgr` is private, not sure it this would be best practice from a typing perspective as the DataFrame typing is public.

https://mypy.readthedocs.io/en/stable/generics.html#generics
Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen.
BUG: Series.__rfloordiv__ with list/tuple
State: closed

- [ ] closes #xxxx
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [x] whatsnew entry


Labels:
name: Bug
name: Algos
description: Non-arithmetic algos: value_counts, factorize, sorting, isin, clip, shift, diff

Comments:
user visibl?
> user visibl?

whatsnew added + green
Test for dtype comparison
State: closed

- [43038 ] closes #xxxx
- [2] tests added / passed
- [X] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [ ] whatsnew entry


Labels:

Comments:
Please finish this in your othet pr
Deprecate mapper argument in rename
State: closed

- [x] supposed fix #44627
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [ ] whatsnew entry


Labels:
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode

Comments:
> CI checks are failing, can you address them please?
> 
> ```
> pandas/core/generic.py:4421: error: unused "type: ignore" comment
> pandas/core/generic.py:4485: error: unused "type: ignore" comment
> ```

Sorry, I don't know how to address failing checks, I don't know anything about CI/CD. 
you can just remove those two `# type: ignore` comments

I'd suggest you read through the contributing guide in full too https://pandas.pydata.org/docs/development/contributing.html
> you can just remove those two `# type: ignore` comments
> 
> I'd suggest you read through the contributing guide in full too https://pandas.pydata.org/docs/development/contributing.html

Yeah sure, Thanks for your effort.
Hello @Varun270! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:









There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2021-12-02 14:57:56 UTC
> you can just remove those two `# type: ignore` comments
> 
> I'd suggest you read through the contributing guide in full too https://pandas.pydata.org/docs/development/contributing.html

You can review it again I have done the changes that you suggested.
> Thanks @Varun270
> 
> ![](https://camo.githubusercontent.com/202c6f2c3bb538bc8a95cd16031bf924a90b699c6b275b9d11afad929b75be1d/68747470733a2f2f692e666c756666792e63632f32374d48517047564866387a734271783936677854355042356d53316a746a442e676966)

Thanks for helping me, I learned a lot!!
thanks @Varun270
BUG: Fix setitem with 1d matrix
State: closed

- [x] closes #42376
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [x] whatsnew entry

Fixes the case `df["mean"] = sparse.random(100, 10).mean(axis=1)` assigning a broken column.

Wasn't sure where the test should go, I feel like there must be a test for `x = np.ones((10, 1)); df["x"] = x` somewhere.

Labels:
name: Constructors
description: Series/DataFrame/Index/pd.array Constructors

Comments:
Tests fail with the new array manager, but don't with the block manager. Not sure what I should do about that, as it's unclear if this is intentional.
What should be done about the tests failing with the new data manager?

I, personally, would expect the two dimensional array that only has multiple indices along one dimension to keep working for column assignment.
> pls add a whatsnew note

Added

> i think you can just skip for array manager for now. or fix it.

I'm a bit short on time for the moment, so I'd opt to skip it.

I've opened an issue for this: #44788

Is there documentation somewhere about what the new array manager is exactly? My assumption is it's around [the BlockManager rewrite](https://pandas.pydata.org/docs/development/roadmap.html?#block-manager-rewrite)?
use `@td.skip_array_manager_not_yet_implemented` (and put the reference to the issue you are skipping).

or can do like this

```pandas/tests/reshape/concat/test_concat.py:    def test_concat_copy(self, using_array_manager):``` (see that test)
Added the skip, I think the remaining CI failures are unrelated.
> Is there documentation somewhere about what the new array manager is exactly? 

Not really.  I think there has been reticence to put it in the public docs because it is still experimental and may be removed.
> I think there has been reticence to put it in the public docs because it is still experimental and may be removed.

Might be useful to mention in the contributing docs. It was a little confusing why tests were failing on CI at first.

----------------

Was there anything else I should do here?
> Was there anything else I should do here?

just merge master pls
thanks @ivirshup very nice. sorry for the delay, quite a number of ci issues came up of late.
No worries, thanks for the merge!