BrandonFors/pandas-github-issues huggingface dataset:
DOC: add documentation to DataFrameGroupBy.skew and SeriesGroupBy.skew
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Contributes to https://github.com/noatamir/pyladies-berlin-sprints/issues/13

I copied the documentation from [here](https://github.com/pandas-dev/pandas/blob/main/pandas/core/generic.py#L11496) and [here](https://github.com/pandas-dev/pandas/blob/main/pandas/core/generic.py#L11825). Let me know if I should delete the lines in SeriesGroupBy.skew that apply to DataFrames and vice versa. 

ping @phofl @noatamir 

#PyLadies Berlin sprint


Labels:
name: Docs
name: Groupby

Comments:
I made the requested changes. I will work on the CI failures tomorrow. 
Thanks @marenwestermann - great work!
Thx @marenwestermann and thx @rhshadrach for reviewing
Thank you and thanks for reviewing!
DOC: improve the wording
State: closed

there are only two .rolling(), .expanding() and .ewm() functions, so using "etc." is not appropriate.

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Docs
name: Window
description: rolling, ewma, expanding

Comments:
Thanks @partev
DOC: Ignore functions on validate docstrings - error EX01
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Towards #37875

See https://github.com/pandas-dev/pandas/pull/50896#issuecomment-1398768593

Labels:
name: Code Style
description: Code style, linting, code_checks

Comments:
Thanks @DeaMariaLeon 
Thank you for the fast review & merge @mroeschke :-)
Updated value_counts documentation and implementation and added single label subset test
State: closed

- [ ] closes #50829
- [x] Tests added and passed.
- [x] All code checks passed.



Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Docs
name: Algos
description: Non-arithmetic algos: value_counts, factorize, sorting, isin, clip, shift, diff

Comments:
@mroeschke  this was a first attempt. Should the definition of subset be worded like [by in groupby](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.groupby.html) or linked to it? And I went with the most straight forward implementation and test if either needs to be more thorough please lmk
The doc change looks good. Appears the test you added is failing.
Not sure why the following in Code Checks / Docstring validation, typing, and other manual pre-commit hooks (pull_request) is failing:

```
 Error: /home/runner/work/pandas/pandas/pandas/core/indexes/base.py:285:EX02:pandas.Index:Examples do not pass tests:
**********************************************************************
Line 49, in pandas.Index
Failed example:
    pd.Index([1, 2, 3], dtype="uint8")
Expected:
    NumericIndex([1, 2, 3], dtype='uint8')
Got:
    Index([1, 2, 3], dtype='uint8')

Partially validate docstrings (EX02) DONE
Error: Process completed with exit code 1.
```

Is this a conflict with a change that has been merged into the main branch since I have opened the pull request?
Looks like this may have been fixed, can you merge main & resolve conflicts to check.
@tpackard1 - failures look network related; likely a hiccup. Can you try merging main again.
Thanks @tpackard1 - I'm now seeing the failed test:

> pandas/tests/frame/methods/test_value_counts.py::test_data_frame_value_counts_subset
Hopefully everything should be good now. @rhshadrach should I go ahead and update the docs (and possibly the whatsnew?) that @mroeschke mentioned about the return difference or would it be best to open a separate PR for that?
@tpackard1 - yes, updating the docs here would be good. I think a line in the whatsnew about not returning a MultiIndex for a single label would be good.
Thanks @tpackard1!
ENH: Add arrow tests for get_dummies
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

cc @mroeschke 
 
did not want to clutter the other pr with this. Is there an easier way of defining the variables?

Labels:
name: Arrow
description: pyarrow functionality

Comments:
> Is there an easier way of defining the variables?

Could do `[str(pd.ArrowDtype(typ) for typ in tm.ALL_...]`?
Thx, makes sense. Updated
Thanks @phofl
DEPR: parsing to tzlocal
State: closed

- [x] closes #50791 (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Deprecate
description: Functionality to remove in pandas

Comments:
i think issues have been addressed.
Nice thanks @jbrockmendel 
I think this needed a rebase? Seeing this on main

```
     def test_parsing_tzlocal_deprecated():
        # GH#50791
        msg = "Pass the 'tz' keyword or call tz_localize after construction instead"
        dtstr = "Jan 15 2004 03:00 EST"
    
        with tm.set_timezone("US/Eastern"):
            with tm.assert_produces_warning(FutureWarning, match=msg):
                res, _ = parse_datetime_string_with_reso(dtstr)
    
            assert isinstance(res.tzinfo, tzlocal)
    
            with tm.assert_produces_warning(FutureWarning, match=msg):
>               res = parsing.parse_datetime_string(dtstr)
E               AttributeError: module 'pandas._libs.tslibs.parsing' has no attribute 'parse_datetime_string'
```
Yah, we'll need to change that to check py_parse_datetime_string
have opened a pr
Handle CoW in BlockManager.apply
State: closed

@phofl seeing that we need the Block methods be aware of CoW (does it make a copy or not? do we need to keep track of a ref or not?) in several cases, I was thinking of this alternative of how to handle this generally. I don't know if it will turn out to be cleaner if has to handle all different cases, but for now I tested it with astype and infer_objects (https://github.com/pandas-dev/pandas/pull/50802, https://github.com/pandas-dev/pandas/pull/50428). 

The idea is that the Block method just tells the manager whether the block that is returned is a view or a copy, and then all the weakref management is centralized in the `BlockManager.apply()` method. 
It avoids the "hack" of attaching the `_ref` to the block. Now I am writing this, of course also the in case of using the `_ref` method, the extracting of those refs could be centralized in `apply()` instead of done in both `astype` and `convert`, as it is the case right now in the linked PRs. 
I _think_ I find the logic a bit simpler here in this PR, especially for `convert` with its block-splitting logic, but it's also not a big difference (and I am of course biased since I wrote this one, and the logic in your PRs I only read ;))

Note, I just copied some minimal pieces from your PRs to test this, and didn't bother with typing and cleaning it up etc. _If_ we think this might be a suitable path forward, I would also remove those parts here again, and only add the general infrastructure here, and keep astype/convert for its separate PRs.

Labels:
name: Copy / view semantics

Comments:
I agree that we have to centralise some things (which is something that is on my TODO list as well). I thought about this solution as well, but decided to go with the specialised things on the block level to be a bit more flexible for now. I'd prefer having a go at centralizing this in apply after implementing CoW for most methods that need this on the block level. Right now, it's still not totally clear to me if this is a good idea, replace for example is pretty convoluted and could be a hassle when passing this back. On the other side you are totally correct that the split block thing is annoying as hell here, what we could do though is handle this within the decorator?

But we should definitely add a central method on the manager level to unpack _ref and a central method to populate it on the block level. 
With #51144, this is now much cleaner and this can be closed.
DEPR: dt64 any/all GH#34479
State: closed

- [x] closes #34479 (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Deprecate
description: Functionality to remove in pandas
name: Reduction Operations
description: sum, mean, min, max, etc.

Comments:
Merge conflict otherwise looks good
rebased + greenish
Thanks @jbrockmendel
CI set LD_PRELOAD in circleci job
State: closed

circle CI job is failing, let's see if this fixes anything

```python-traceback
=================================== FAILURES ===================================
______________________________ test_scikit_learn _______________________________
[gw2] linux -- Python 3.8.15 /home/circleci/miniconda3/envs/pandas-dev/bin/python3.8

    def test_scikit_learn():
    
>       sklearn = import_module("sklearn")  # noqa:F841

pandas/tests/test_downstream.py:158: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
pandas/tests/test_downstream.py:27: in import_module
    return importlib.import_module(name)
../miniconda3/envs/pandas-dev/lib/python3.8/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
<frozen importlib._bootstrap>:1014: in _gcd_import
    ???
<frozen importlib._bootstrap>:991: in _find_and_load
    ???
<frozen importlib._bootstrap>:975: in _find_and_load_unlocked
    ???
<frozen importlib._bootstrap>:671: in _load_unlocked
    ???
<frozen importlib._bootstrap_external>:843: in exec_module
    ???
<frozen importlib._bootstrap>:219: in _call_with_frames_removed
    ???
../miniconda3/envs/pandas-dev/lib/python3.8/site-packages/sklearn/__init__.py:83: in <module>
    from .utils._show_versions import show_versions
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

    """
    Utility methods to print system info for debugging
    
    adapted from :func:`pandas.show_versions`
    """
    # License: BSD 3 clause
    
    import platform
    import sys
    from ..utils.fixes import threadpool_info
    from .. import __version__
    
    
>   from ._openmp_helpers import _openmp_parallelism_enabled
E   ImportError: /home/circleci/miniconda3/envs/pandas-dev/lib/python3.8/site-packages/sklearn/utils/../../../../libgomp.so.1: cannot allocate memory in static TLS block

../miniconda3/envs/pandas-dev/lib/python3.8/site-packages/sklearn/utils/_show_versions.py:14: ImportError
```

Labels:
name: CI
description: Continuous Integration
name: ARM
description: aarch64 architecture

Comments:
seems to have fixed it

fix was from https://stackoverflow.com/questions/67735216/after-using-pip-i-get-the-error-scikit-learn-has-not-been-built-correctly
Thanks @MarcoGorelli
DEPR: move NumericIndex._engine_type and .inferred_type to Index
State: closed

Moves`_engine_type` & `inferred_type` from `NumericIndex`to `Index` in preparation to remove `NumericIndex` and include numpy int/uint/float64 in the base `Index`.

xref #42717.

Labels:
name: Deprecate
description: Functionality to remove in pandas

Comments:
needs rebase
Ping.
Updated.
Thanks @topper-123
Resolved duplicate issue by updating obj object
State: closed

- [x] closes #50845 
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: DataFrame
description: DataFrame data structure
name: Stale

Comments:
@mroeschke Can you please tell me in which file I have to add the test case.
pandas/tests/frame/methods/test_align.py
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen.
DOC: Fix dark mode styles
State: closed

closes #48527

Labels:
name: Docs

Comments:
Is this in reference to an existing Github issue?
> Is this in reference to an existing Github issue?

yes, it's in reference to https://github.com/pandas-dev/pandas/issues/42744 
just realising I should've mentioned that, sorry for the misunderstanding
It is already closed 
> It is already closed

But there was a portion of the site still not corrected, ans that's what I modified. The issue is still open because of that.
Preview docs at: https://pandas.pydata.org/preview/50936/
Thanks @anushkabishnoi for working on this.

Looks like the collapsed blocks are fixed by this, but when you expand it still has white background: https://pandas.pydata.org/preview/50936/docs/getting_started/index.html

Do you want to have a look and see if you can also fix that?
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
@anushkabishnoi do you have time to address the comments above?
hey I'm sorry for not responding earlier, I tried to work out the other thing you pointed out, and I can't seem to figure it out. 
Is it fine if we just merge the original pr, or do i try the other thing again?

Thanks in advance.
What's the exact problem? Seems like it should just be repeating what you did to the style of the other component.
> To remove the white background while expanding replace line 242 with this
> 
> ```
> .tutorial-card .card-body {
>   background-color: #F0F0F0;
>   background-color: transparent;
> }
> ```

"What kind of data does pandas handle?" This Expansion Box contains an image with black Font [here](https://pandas.pydata.org/preview/50936/docs/_images/01_table_dataframe.svg) I am not sure if this is a style problem or the image. If there are no other problems this issue should be solved.
I've pushed a commit, the only issue still prevailing - the text on the svg needs to change with dark and light modes. I don't have eperience working with SVGs, so if you guys can help me out, that would be great.

(this is what I mentioned not being able to figure out)
Deleted the line myself. So far doesn't seem to be working, let's see if with this new change: https://pandas.pydata.org/preview/50936/docs/getting_started/index.html

Would be good to test things locally, since the CI is not very fast.
Thanks @anushkabishnoi
CI: Adjust test to fix wheel builders
State: closed

- [ ] closes #50898 (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Build
description: Library building on various platforms
name: CI
description: Continuous Integration

Comments:
Looks like GHA had some issues when they bumped conda. Let's wait a couple days to see if they can fix this(otherwise we'll need to apply the workaround of downgrading conda). This PR does fix the failing tests, though.

https://github.com/actions/runner-images/issues/6910.
Thanks @lithomas1
DEPR: move NumericIndex._convert_tolerance to Index
State: closed

Moves`_convert_tolerance` from `NumericIndex`to `Index` in preparation to remove `NumericIndex` and include numpy int/uint/float64 in the base `Index`.

xref #42717.

Labels:
name: Deprecate
description: Functionality to remove in pandas

Comments:
Ping.
This has also been updated :-)
thanks @topper-123
DEPR: move NumericIndex._format_native_types to Index
State: closed

This starts moving functionality from `NumericIndex`to `Index` in preparation to remove `NumericIndex` and include numpy int/uint/float64 in the base `Index`.

Also removes the `**kwargs`, as that wasn't used in the method.

I'm doing one method per PR, figuring that is easier to review. I could add ore per PR if you want.

xref #42717.

Labels:

Comments:
thanks @topper-123
BUG/PERF: Series(index=MultiIndex).rename losing EA dtypes
State: closed

- [x] closes #21055
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/v2.0.0.rst` file if fixing a bug or adding a new feature.


`Series(index=MultiIndex).rename` loses EA dtypes: 

```
import pandas as pd
import numpy as np

lev0 = pd.Index(np.arange(1000), dtype="Int64").astype("category")
lev1 = pd.Index(np.arange(1000), dtype="Int64")

mi = pd.MultiIndex.from_product([lev0, lev1], names=["A", "B"])
ser1 = pd.Series(1, index=mi)
ser2 = ser1.rename({10: 11}, level=1)

print(ser2.index.dtypes)
```

main:
```
A       int64
B       int64
dtype: object
```

PR:
```
A    category
B       Int64
dtype: object
```

Perf improves as well:

```
%timeit ser1.rename({10: 11}, level=1)

1.25 s ± 52.8 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)  -> main
350 ms ± 8.29 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)  -> PR
```

Labels:
name: Bug
name: Performance
description: Memory or execution speed performance
name: MultiIndex

Comments:
Thanks @lukemanley
ENH: support min/max/sum for pyarrow duration dtypes
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Arrow
description: pyarrow functionality

Comments:
Thanks @jbrockmendel
API: Harmonize dtype for index levels for Series.sparse.from_coo
State: closed

Currently the levels of the index the sparse series has different dtype (int32 or int64), depending on if the `dense_index` is true or False. This PR makes the dtype to be always int32, the same as the row/col attributes of the scipy `coo_matrix`.

See also discussion here: https://github.com/pandas-dev/pandas/pull/49560#discussion_r1067619162.


Labels:
name: Sparse
description: Sparse Data Type

Comments:
The failure looks unrelated.
Thanks @topper-123