BrandonFors/pandas-github-issues huggingface dataset:
DOC: Simplify pandas theme footer
State: closed

This pr is related to issue #51536.  The next changes were made:

1. Changed a version of `pydata-sphinx-theme` in environment.yml from less than 0.11 to 0.13.0
2. Updated my environment by running mamba env update -n pandas-dev --file environment.yml
3. In conf.py I changed `html_theme_options`. Now  "footer_start" is used instead of "footer_items”
4. In doc/source/_static/css/pandas.css where we override some aspects of `pydata-sphinx-theme` I removed `card-footer`
5. After cleaning and rebuilding documentation there is no footer in our html.



Labels:
name: Docs
name: Web
description: pandas website
name: Stale

Comments:
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
@natmokval do you still want to work on this?
Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen.
PERF: Series slicing
State: closed

```
import pandas as pd
ser = pd.Series(range(300_000))

%timeit ser[:30]
29.2 µs ± 749 ns per loop (mean ± std. dev. of 7 runs, 10,000 loops each)  # <- main
15.7 µs ± 459 ns per loop (mean ± std. dev. of 7 runs, 100,000 loops each)  # <- PR
```

BTW in this branch, `__finalize__` takes 13.6% of the runtime in that slicing.  

Labels:
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: Performance
description: Memory or execution speed performance

Comments:
Thanks @jbrockmendel
Update README.md
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:

Comments:
Thanks for the PR, but if you're interested in contributing please read our contributing guide first: https://pandas.pydata.org/docs/development/contributing.html
DOC/CI: [PoC] surface doc preview in PRs
State: closed

Proof of concept for 1./2. of https://github.com/pandas-dev/pandas/pull/52038#issuecomment-1481075756.

Labels:
name: Docs
name: CI
description: Continuous Integration
name: Web
description: pandas website

Comments:
Here is what it looks like when [inspecting the job directly](https://github.com/pandas-dev/pandas/actions/runs/4500786499/jobs/7920364046?pr=52139#step:15:23):

![Screenshot from 2023-03-24 10-05-47](https://user-images.githubusercontent.com/6849766/227474845-7d394718-d1b4-4b21-84a4-c3a7d8522313.png)

And here is the same information as part of the [workflow summary](https://github.com/pandas-dev/pandas/actions/runs/4500786499?pr=52139):

![Screenshot from 2023-03-24 10-06-36](https://user-images.githubusercontent.com/6849766/227474873-446b70f7-50b7-4ce9-b66a-96e5af3df15f.png)


Thanks @pmeier, this is interesting.  #52038 is not so much about how the link is shown to the users, but to publish the docs in a different server. It also has the approach of showing the link as a CI step in the main PR page. It's cool what you did here, but I think the link is still a bit hidden. But will leave this open while we still check if #52038 works for us, as having the link as the CI summary is cool even if we have other methods to show it, and if we stay with the old approach it can make sense to merge this.
> about how the link is shown to the users, but to publish the docs in a different server

Ok, in that case this PR is not the right way. Feel free to close.

> It also has the approach of showing the link as a CI step in the main PR page.

What is the "main page" here? If you mean that it is displayed as an individual step in the job, this can be trivially achieved here as well. Taking this a step further, it is also not hard to get a bot to comment on the PR with a link to the preview as soon as it is uploaded for maximum visibility.
The main PR page would be for example: https://github.com/pandas-dev/pandas/pull/52139

What you implemented requires to go to the actions page for the PR. I'll close this one for now, but feel free to reopen, suggest changes or implement a new PR once the outcome of #52038 is clear. Thanks!
Sorry, I still don't understand what you mean. How do you want to put anything in the PR thread except for having a bot or the like add a comment? Maybe I'm missing the big picture in #52038, but so far I fail to see how it would solve this as well.
We already have an action to write a comment with the preview link when a comment with the content `/preview` is created. The new option we're exploring is to create the link in the list of CI builds (where we see the result of each individual build).
Ok, got it now. Thanks for the explanation!
DOC update contributors
State: closed

In the Slack channel, a contributor has been requested to be renamed due to having changed their legal name

I think it's a reasonable request, we can just add a mapping within `announce.py` to map the old name to the new one (I've just encoded them to avoid hard-coding someone's name in a file)

Labels:
name: Docs

Comments:
even though this is easy i don't think it should be done

history is history and we should be changing it
In general I agree, but for context, this is someone who changed their name after coming out - I think we should allow it

Github already displays the new name

No need to change the commits themselves, but just displaying a new name in the release notes sounds fine
Once you change from one GitHub username to another, is your previous username available for the taking by others? If that's the case, then I'm -1 here.

Otherwise I'm +0 here. I don't think we have an obligation to update prior whatsnew (even if they are being built in the future), and this would be a bit of an annoyance if it were widely used (I don't expect it will be). At the same time, I can see the desire from a contributor's standpoint and the implementation is simple enough.
> Once you change from one GitHub username to another, is your previous username available for the taking by others? If that's the case, then I'm -1 here.

AFAIK this is not actually related to GitHub usernames. It is the name you use in your git commit, and that's just a local git config setting at the time of making the commit, that then is embodied in the commit author details. 
I can perfectly set up my git config to use any other name that could clash with other people's name or github username.
Ah, that's right, thanks. While that makes me a bit wary here, I'm +0.
looks like this is somewhat controversial...removing from the 2.0 milestone for now, let's not rush it
merging as per today's discussion
DOC: ddof in cov when nans present
State: closed

- [x] closes #47042
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

This PR adds a document about the "``nan``s must not be present in the dataframe" restriction on ddof in pandas.DataFrame.cov.


Labels:
name: Docs

Comments:
This is my first time OSS contributing.
`Code Checks / pre-commit (pull_request)` is failing, what should I fix?
Please tell me!!
Hey @whooolemilk Thanks for picking this up!

In general, how to work with errors (there will be many) is to look into the test logs and find the relevant log line. In this case try to look into the `Doc Build and Upload / Doc Build and Upload (pull_request)` one and you'll see an error message `rst inline code next to normal text`,i..e, its the "`nan`s" part that is the problem. 
To avoid the rst check issue you could just reword the sentence, for example `This argument is applicable only when no ``nan`` is in the dataframe.`
Looks good. Thanks @whooolemilk. 
@topper-123 
Thanks for letting me know
BUG: quantile only working with categorical NAs
State: closed

# WIP
closes #52099 
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: quantile
description: quantile method
name: Stale

Comments:
I guess tests are failing due to dtypes (probably due to the changes I made)
its getting float vs int
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen.
BUG: change lxml remove to drop_tree (#51629)
State: closed

According to [this SO question](https://stackoverflow.com/questions/42932828/how-delete-tag-from-node-in-lxml-without-tail) `remove` method removes tail text. In order to preserve it the method was changed to `drop_tree`. Appropriate test was added.

- [x] closes #51629
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: IO XML
description: read_xml, to_xml

Comments:
Looks okay. Could you add a whatsnew note in `v2.1.0.rst`?
> Looks okay. Could you add a whatsnew note in `v2.1.0.rst`?

Done
Thanks @m-ganko
DEPR: Categorical.to_list, NDFrame.align broadcast_axis
State: closed

- [x] closes #51254 (Replace xxxx with the GitHub issue number)
- [x] closes #51856
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Deprecate
description: Functionality to remove in pandas

Comments:
Thanks @jbrockmendel
BUG, DEPR, PERF: GroupBy aggregations
State: closed

The BUG is failing to raise for dt64/period 'var'
The DEPR is for Groupby.any/all, which we already deprecated for Series/DataFrame with these dtypes
The PERF is for Categorical dtype raising cases, raising directly instead of going through the (copy-making) agg_py_fallback path

Labels:
name: Datetime
description: Datetime data dtype
name: Groupby
name: Performance
description: Memory or execution speed performance
name: Period
description: Period data type
name: Categorical
description: Categorical Data Type

Comments:
Thanks @jbrockmendel
PERF: Groupby aggregations with Categorical
State: closed

```
import numpy as np
np.random.seed(94356)
import pandas as pd

arr = np.arange(5).repeat(10**5)
grps = np.random.randint(0, 3, size=arr.size)

cat = pd.Categorical(arr, ordered=True)
ser = pd.Series(cat)
gb = ser.groupby(grps)

%timeit gb.min()
5.4 ms ± 212 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)  # <- main
1.61 ms ± 14.3 µs per loop (mean ± std. dev. of 7 runs, 1,000 loops each)  # <- PR

%timeit gb.first()
5.59 ms ± 136 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)  # <- main
1.64 ms ± 14.5 µs per loop (mean ± std. dev. of 7 runs, 1,000 loops each)  # <- PR
```

Labels:
name: Groupby
name: Performance
description: Memory or execution speed performance
name: Categorical
description: Categorical Data Type

Comments:
Thanks @jbrockmendel
Add fillna_method 'pad' extension array tests
State: closed

Should we also be adding `'backfill'`?

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:

Comments:
Since pad is an alias of ffill, adding this is almost redundant with what's already tested. 

Additionally we deprecated & removed similarly named methods (https://github.com/pandas-dev/pandas/pull/49081) as we're trying to eliminate aliases, so I would opt to even deprecate "pad" 
Oops. I didn't realize it's just an alias.
PERF: Use String HashTable in value_counts
State: closed

- [ ] xref #14860 (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Roughly around 20-40% speedup in the benchmarkI added for largish 1000/10000 arrays. Benchmarks are pretty flaky, though.

duplicated/ismember didn't show improvement (actually regressed). I guess we aren't using the hashtable enough there to make up for the cost of doing ``is_string_array``.

Labels:
name: Performance
description: Memory or execution speed performance
name: Algos
description: Non-arithmetic algos: value_counts, factorize, sorting, isin, clip, shift, diff
name: Mothballed
description: Temporarily-closed PR the author plans to return to
name: Stale

Comments:
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
Closing to clear the queue, but feel free to reopen when you have time to revisit
BUG: Fix getsizeof when using Series(obj) and taking into account GC corrections
State: closed

Calling __sizeof__ to compute the size of an object produces an error, and in general, this method does not take Garbage Collector corrections into account.

- [x] closes #51858
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/v2.1.0.rst` file if fixing a bug or adding a new feature.


Labels:
name: Bug

Comments:
Could you put these tests in `pandas/tests/base/test_misc.py`
Thanks @balexandermunoz
PERF: DatetimeIndex comparison with Timestamp mismatched resos
State: closed

- [x] closes #52080 (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

```
import pandas as pd

rg = pd.date_range("2020-01-01", periods=100_000, freq="s")

ts_ns = pd.Timestamp("1996-01-01 00:00:00.00000000000")
ts_s = pd.Timestamp("1996-01-01")

%timeit rg < ts_s
3.68 ms ± 138 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)   # <- main
139 µs ± 3.96 µs per loop (mean ± std. dev. of 7 runs, 10,000 loops each)  # <- PR
```

@MarcoGorelli can you confirm this closes #52080

Labels:
name: Performance
description: Memory or execution speed performance
name: Non-Nano
description: datetime64/timedelta64 with non-nanosecond resolution

Comments:
let's ship it
thanks @jbrockmendel
PERF: get_block_type
State: closed

- [x] closes #48212
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

cc @jbrockmendel - this may partly close #48212, however, I suspect the OP was referring to non-EA's given the old version of pandas. 

Performance improvement is mostly for EA's where the `.kind` call can be a bottleneck.

```
import pyarrow as pa
import pandas as pd
from pandas.core.internals.blocks import get_block_type

%timeit get_block_type(pd.ArrowDtype(pa.float64()))
# 3.51 µs ± 440 ns per loop (mean ± std. dev. of 7 runs, 100,000 loops each)    <- main
# 740 ns ± 5.19 ns per loop (mean ± std. dev. of 7 runs, 1,000,000 loops each)  <- PR

%timeit get_block_type(pd.Float64Dtype())
# 1.3 µs ± 23.2 ns per loop (mean ± std. dev. of 7 runs, 1,000,000 loops each)  <- main
# 289 ns ± 2.3 ns per loop (mean ± std. dev. of 7 runs, 1,000,000 loops each)   <- PR
```

Labels:
name: Performance
description: Memory or execution speed performance
name: Internals
description: Related to non-user accessible pandas implementation

Comments:
ping on green
> ping on green

green - thanks
thanks @lukemanley
PERF: use non-copying path for Groupby.skew
State: closed

It wouldn't surprise me if there are contiguity-related lookup-optimizations that could be done in group_skew.

```
import pandas as pd
import numpy as np

nrows = 10**5
ngroups = 5
ncols = 4
nan_frac = 0.05

arr = np.random.randn(nrows, ncols)
arr[np.random.random(nrows) < nan_frac] = np.nan

df = pd.DataFrame(arr)
grps = np.random.randint(0, ngroups, size=nrows)
gb = df.groupby(grps)

%timeit result = gb.skew()
7.75 ms ± 25.7 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)  # <-main
1.78 ms ± 115 µs per loop (mean ± std. dev. of 7 runs, 1,000 loops each)  # <- PR

[same with ngroups=50]
%timeit result = gb.skew()
23.9 ms ± 101 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)  # <- main
1.81 ms ± 51 µs per loop (mean ± std. dev. of 7 runs, 1,000 loops each)  # <- PR
```



Labels:
name: Groupby
name: Performance
description: Memory or execution speed performance
name: Reduction Operations
description: sum, mean, min, max, etc.

Comments:
@willayd any guesses why npdev wouldn't like 

```
out[i, j] = <float64_t>(
                        (ct * (ct - 1) ** 0.5 / (ct - 2)) * (M3[i, j] / M2[i, j]**1.5)
                    )
```

The error in the build log

```
pandas/_libs/groupby.c:37389:15: error: aggregate value used where a floating-point was expected
37389 |               *((__pyx_t_5numpy_float64_t *) ( /* dim=1 */ ((char *) (((__pyx_t_5numpy_float64_t *) ( /* dim=0 */ (__pyx_v_out.data + __pyx_t_18 * __pyx_v_out.strides[0]) )) + __pyx_t_19)) )) = ((__pyx_t_5numpy_float64_t)__Pyx_c_prod_npy_float64(__Pyx_c_quot_npy_float64(__Pyx_c_prod_npy_float64(__pyx_t_npy_float64_complex_from_parts(__pyx_v_ct, 0), __pyx_t_npy_float64_complex_from_parts(__Pyx_CREAL(__pyx_t_20), __Pyx_CIMAG(__pyx_t_20))), __pyx_t_npy_float64_complex_from_parts((__pyx_v_ct - 2.0), 0)), __Pyx_c_quot_npy_float64(__pyx_t_npy_float64_complex_from_parts((*((__pyx_t_5numpy_float64_t *) ( /* dim=1 */ ((char *) (((__pyx_t_5numpy_float64_t *) ( /* dim=0 */ (__pyx_v_M3.data + __pyx_t_13 * __pyx_v_M3.strides[0]) )) + __pyx_t_17)) ))), 0), __pyx_t_npy_float64_complex_from_parts(__Pyx_CREAL(__pyx_t_21), __Pyx_CIMAG(__pyx_t_21)))));
```

I'd guess the ``**1.5`` is returning something with a complex type and the complaint is about casting it back to float, but no idea why that would only be an issue on npdev
It looks like Cython 3.0 has a new cpow directive that I think is responsible for this behavior. Might be ok to set that to True
IIUC the cpow thing has to wait until we use cython3. anything else here?
Thanks @jbrockmendel