BrandonFors/pandas-github-issues huggingface dataset:
ENH: drop_duplicates_index function to base series object. 
State: closed

Added drop_duplicates_index function to base series object. very useful when working with time series.


- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] whatsnew entry


Labels:
name: Enhancement
name: Needs Discussion
description: Requires discussion from core team before further action

Comments:
closing.
Update series.py
State: closed

Added drop_duplicates_index function to base series object. very useful when working with time series.

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] whatsnew entry


Labels:

Comments:
Hello @Omerdan03! Thanks for opening this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:

* In the file [`pandas/core/series.py`](https://github.com/pandas-dev/pandas/blob/a39287bbde5bd897093189b697a5f939bd952db0/pandas/core/series.py):

> [Line 2144:1](https://github.com/pandas-dev/pandas/blob/a39287bbde5bd897093189b697a5f939bd952db0/pandas/core/series.py#L2144): [W293](https://duckduckgo.com/?q=pep8%20W293) blank line contains whitespace
> [Line 2147:1](https://github.com/pandas-dev/pandas/blob/a39287bbde5bd897093189b697a5f939bd952db0/pandas/core/series.py#L2147): [W293](https://duckduckgo.com/?q=pep8%20W293) blank line contains whitespace
ENH: add `environment`, e.g. "longtable", to `Styler.to_latex`
State: closed

checks an item off #41649 

closes #37443 

adds the longtable template, similar to DataFrame.to_latex.

also allows for a custom environment if \begin{table} should be replaced by \begin{figure} etc.


Labels:
name: Enhancement
name: IO LaTeX
description: to_latex
name: Styler
description: conditional formatting using DataFrame.style

Comments:
can you rebase this
@jreback rebased and updated.
@jreback can this be merged as is? I've addressed almost all comments, with the one subjective suggestion ignored due to preference.

@ivanovmg thanks for review - I think after the iterations the tests are better and clearer.
Clean up ujson initialization
State: closed

The way we do this now is a little wonky as we are declaring the C API in the objToJSON file for re-use by JSONToObj; this happens to work because we call the objToJSON initialization function in the main extension module, but it's a bit funky and prone to errors on refactor

More background info:

https://numpy.org/doc/stable/reference/c-api/array.html#importing-the-api

https://stackoverflow.com/a/35362918/621736

Labels:
name: IO JSON
description: read_json, to_json, json_normalize
name: Clean

Comments:
thanks @WillAyd
BUG: IntervalIndex is_monotonic, get_loc, get_indexer_for, contains with np.nan
State: closed

- [x] closes #41831
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [x] whatsnew entry


Labels:
name: Bug
name: Interval
description: Interval data type

Comments:
actually can you rebase (as merged #41870 should be fine)
rebased, whatsnew added, and greenish
API: make construct_array_type non-optional
State: closed

- [x] closes #24860
- [ ] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [x] whatsnew entry


Labels:
name: API Design
name: ExtensionArray
description: Extending pandas with custom dtypes or arrays.

Comments:
release note?
whatsnew added

any idea whats going on with the 2 builds that keep failing?  usually these resolve themselves after a day or two
> whatsnew added
> 
> any idea whats going on with the 2 builds that keep failing? usually these resolve themselves after a day or two

nopes
TST: add ASV check for groupby with uint64 col
State: closed

- [x] closes #28122 
- [x] tests added/passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them

The performance regression reported in the original issue was fixed somewhere between 0.25.1 and 1.2.4. This PR just adds an ASV test to monitor for the same regression in the future. 


Labels:
name: Groupby
name: Benchmark
description: Performance (ASV) benchmarks

Comments:
Thanks for the PR - can you post the result to be sure? Also do you think we could just parametrize `GroupByMethods` to include this instead? That may be slightly more comprehensive than this one-off class, though noted that you used a similar class for Int as inspiration
> Thanks for the PR - can you post the result to be sure? Also do you think we could just parametrize `GroupByMethods` to include this instead? That may be slightly more comprehensive than this one-off class, though noted that you used a similar class for Int as inspiration

@WillAyd I prefer your suggestion to run `uint` through `GroupByMethods`. I've implemented this way and removed the one-off class. 

Results attached. 

[asv_groupby.GroupByMethods.txt](https://github.com/pandas-dev/pandas/files/6617856/asv_groupby.GroupByMethods.txt)

thanks @smithto1
REF: split out grouped masked cummin/max algo
State: closed

No user-facing change here, but I think makes logic clearer and future changes easier to understand

Benchmarks look unaffected:
<details>

```
before           after         ratio
[3f67dc33]       [51f8f9c8]
<master>         <masked_cummin_ref>
17.0±2ms         16.3±2ms     0.96  groupby.CumminMax.time_frame_transform('Float64', 'cummax')
17.7±2ms         18.4±2ms     1.04  groupby.CumminMax.time_frame_transform('Float64', 'cummin')
20.8±3ms       16.9±0.3ms    ~0.81  groupby.CumminMax.time_frame_transform('Int64', 'cummax')
20.7±3ms         18.2±3ms    ~0.88  groupby.CumminMax.time_frame_transform('Int64', 'cummin')
15.7±2ms       13.0±0.3ms    ~0.83  groupby.CumminMax.time_frame_transform('float64', 'cummax')
15.5±0.7ms       13.6±0.7ms    ~0.88  groupby.CumminMax.time_frame_transform('float64', 'cummin')
29.8±0.7ms       28.0±0.4ms     0.94  groupby.CumminMax.time_frame_transform('int64', 'cummax')
29.9±1ms         30.0±2ms     1.00  groupby.CumminMax.time_frame_transform('int64', 'cummin')
19.2±2ms         16.9±3ms    ~0.88  groupby.CumminMax.time_frame_transform_many_nulls('Float64', 'cummax')
22.6±2ms         15.6±2ms    ~0.69  groupby.CumminMax.time_frame_transform_many_nulls('Float64', 'cummin')
15.7±3ms         19.3±3ms    ~1.23  groupby.CumminMax.time_frame_transform_many_nulls('Int64', 'cummax')
19.5±2ms         14.7±1ms    ~0.75  groupby.CumminMax.time_frame_transform_many_nulls('Int64', 'cummin')
14.5±2ms       11.4±0.8ms    ~0.79  groupby.CumminMax.time_frame_transform_many_nulls('float64', 'cummax')
13.2±1ms       12.4±0.2ms     0.94  groupby.CumminMax.time_frame_transform_many_nulls('float64', 'cummin')
14.2±1ms       13.1±0.1ms     0.92  groupby.CumminMax.time_frame_transform_many_nulls('int64', 'cummax')
14.1±0.5ms       13.4±0.4ms     0.95  groupby.CumminMax.time_frame_transform_many_nulls('int64', 'cummin')
```

</details>


Labels:
name: Refactor
description: Internal refactoring of code
name: Groupby
name: Algos
description: Non-arithmetic algos: value_counts, factorize, sorting, isin, clip, shift, diff

Comments:
seems fine. IIRC we have these 2 kernels because the no mask one is >> faster?
> seems fine. IIRC we have these 2 kernels because the no mask one is >> faster?

In an original implementation, I tried forcing a mask (so only use the masked kernel), but there was a slight slowdown for float data because of the `isna` needed to actually get the mask to pass into the algo.

But all other types where `isna` is fast (eg nullables, or normal ints) forcing a mask was just as fast.  If a small perf hit (I want to say <20%) on float data is acceptable, I can dig up that code and see what the perf hit actually looks like.

Regardless of that choice, I think the split here into 2 clear kernels is helpful because it will make it much simpler if we ever do get rid of one.
thanks @mzeitlin11
⬆️ UPGRADE: Autoupdate pre-commit config
State: closed

<!-- START pr-commits -->
<!-- END pr-commits -->

## Base PullRequest

default branch (https://github.com/pandas-dev/pandas/tree/master)

## Command results
<details>
<summary>Details: </summary>



</details>

## Changed files
<details>
<summary>Changed file: </summary>



</details>

<hr>

[:octocat: Repo](https://github.com/technote-space/create-pr-action) | [:memo: Issues](https://github.com/technote-space/create-pr-action/issues) | [:department_store: Marketplace](https://github.com/marketplace/actions/create-pr-action)

Labels:

Comments:
there's a separate PR open to update `black` already
BUG: DataFrame.at with CategoricalIndex
State: closed

- [ ] closes #xxxx
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [x] whatsnew entry


Labels:
name: Bug
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: Categorical
description: Categorical Data Type

Comments:
release note?
whatsnew added + green
This is for 1.3?
don't add things to 1.3
when we merge can add if appropriate 
This has a release note in 1.3.0 whatsnew. if not 1.3 we need to request it moved. any suggestions how to track this.
i undestand and it prob will be merged but let's look first
eg don't preemptively label (i may have done that here)
its a bugfix so i wrote the note for 1.3, wont be the end of the world if it has to wait for the next one
should i move release note to 1.4 file?
> should i move release note to 1.4 file?

not yet. I think Jeff just didn't want stuff added to the 1.3 milestone for now.
if the RC is put off until tomorrow, can this go in?  it is a blocker for #41956
Doesn't look like there's a fix for the wheel issue. So may release what we
have.

On Sat, 12 Jun 2021, 22:52 jbrockmendel, ***@***.***> wrote:

> if the RC is put off until tomorrow, can this go in? it is a blocker for
> #41956 <https://github.com/pandas-dev/pandas/pull/41956>
>
> —
> You are receiving this because you commented.
> Reply to this email directly, view it on GitHub
> <https://github.com/pandas-dev/pandas/pull/41846#issuecomment-860118003>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/ADEMUXI6G5LUOT2QSZAPY5LTSPJKVANCNFSM46G2R5QA>
> .
>

> Doesn't look like there's a fix for the wheel issue. So may release what we have.

tbc. I should have said we are not necessarily putting off the rc, and that if the wheel issue is not fixed quickly, we could release rc0 and then follow-up with a rc1. So the same backport rules apply. We have a rc0, but may not advertise the fact. So the rc is not put-off, it has effectively happened.
@meeseeksdev backport 1.3.x
Something went wrong ... Please have  a look at my logs.
docstrings: typos, clarity
State: closed

Minor changes only in docstrings: capitalisation, hyphenation, "indexes" -> "indices", typos.

- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] whatsnew entry

Apologies for not testing, as the changes are minor and only affect docstrings.
No `whatsnew` entry either because minor.

Labels:
name: Docs

Comments:
> both spellings are acceptable https://grammarist.com/usage/indexes-indices/

Not sure if distinction is necessary, but from a `pandas` perspective I like the idea of 

- Indexes: Multiple `Index` objects
- Indices: Values used to index into something
Thank you both for your comments. I amended the commit to not touch indices vs indexes.
DOC: reverse rolling window (#38627)
State: closed

This adds the suggested changes from closed pull request (#38627):
https://github.com/pandas-dev/pandas/pull/39091/files/80819f65402e46caa29ff7192b34d43f91c1a0ee
After updated commit.
- remove noqa
- change line back to single quotes
- rename `df1` to `reversed_df`
- add reference to `FixedForwardWindowIndexer`
- remove repeated `df` initialization in code example
- rename `df2` to `reversed_df`
- move code example

- [x] closes #38627
- [ ] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] whatsnew entry


Labels:
name: Docs
name: Window
description: rolling, ewma, expanding

Comments:
> Could you just add the new section and save cleanups for another PR (so just add content)?
> 
> It still appears that things changed that were not supposed to in the PR.

Yeah, I can do that.
I realized I accidentally deleted part of a code example.

```
For some problems...
.. ipython:: python

   from pandas.api.indexers import FixedForwardWindowIndexer
   indexer = FixedForwardWindowIndexer(window_size=2)
   df.rolling(indexer, min_periods=1).sum()
```
Updated, does this work now?
Thanks @mdhsieh. CI failure unrelated
REF: de-duplicate symmetric_difference, _union
State: closed

Also fixes buglet in CategoricalIndex.equals(MultiIndex)

Labels:
name: Refactor
description: Internal refactoring of code
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: Index
description: Related to the Index class or subclasses

Comments:
can you rebase
once more rebase
yeah seeing failures on master
rebased + green
wow nice
This seems to have caused several performance regressions, see https://pandas.pydata.org/speed/pandas/#regressions?sort=1&dir=desc (the ones for commit "bf72b70b")
Opened https://github.com/pandas-dev/pandas/issues/42239 to track this regression
REF: share wrapping in MultiIndex setops
State: closed

If we can push _wrap_mi_setop_result further down into wrap_setop_result (not at all trivial bc different setops have different desired behavior on empty results) then we can share most of the rest of the MultiIndex setops code.

This PR changes the levels in the result of an empty .difference:

```
mi = pd.MultiIndex.from_product([range(3), ["a", "b"]])
mi2 = mi.repeat(2)

>>> mi.difference(mi2)
MultiIndex([], )

>>> mi.difference(mi2).levels
FrozenList([[], []])   # <-- master
FrozenList([[0, 1, 2], ['a', 'b']])   # <-- PR
```

I think this makes more sense, as a .difference should always be dtype-preserving, which for MultiIndex corresponds to levels-preserving.

In both the PR and in master, .intersection is levels-preserving when the intersection is empty, but it is less obvious to me that this is desirable.  In particular, it is mutually exclusive with .intersection being commutative.  cc @phofl?


Labels:
name: Refactor
description: Internal refactoring of code
name: MultiIndex

Comments:
Not following here7

> In both the PR and in master, .intersection is levels-preserving when the intersection is empty, but it is less obvious to me that    this is desirable. In particular, it is mutually exclusive with .intersection being commutative

Why is this mutually exclusive?

Have you checked performance? IIRC I have used from_arrays instead of 

```
MultiIndex.from_tuples(result, sortorder=0, names=result_names)
```

because it was faster. Not 100% sure that this was the reason though
>Not following here7

    In both the PR and in master, .intersection is levels-preserving when the intersection is empty, but it is less obvious to me that this is desirable. In particular, it is mutually exclusive with .intersection being commutative

Sorry, got it now. You are probably right, the levels should be empty, if the result is empty.

> Have you checked performance? IIRC I have used from_arrays instead of [...] because it was faster. Not 100% sure that this was the reason though

If this is the case then shouldn't we want from_tuples to always dispatch to from_arrays like we currently do here?  Or is something special in this case?
DOC: Whatsnew1.3 grammar
State: closed

- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them

This contains the non-controversial edits (in my opinion) from #41747, but is almost all grammar edits.

Labels:
name: Docs

Comments:
thanks @rhshadrach
BUG: wrong exception when slicing on TimedeltaIndex
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [x] whatsnew entry

inching towards sharing _maybe_cast_slice_bound between DTI/PI/TDI

Labels:
name: Bug
name: Timedelta
description: Timedelta data type

Comments:
release note?
@meeseeksdev backport 1.3.x
Something went wrong ... Please have  a look at my logs.
ENH/BUG: DataFrame(object_array_of_periods_or_intervals)
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [x] whatsnew entry

Get the special-casing out of sanitize_array up the chain into maybe_infer_to_datetimelike.  im hopeful we can get it all crammed into maybe_infer_objects

Labels:
name: Bug
name: Interval
description: Interval data type

Comments:
lgtm needs a whtasnew? hard to see what the user focus changed is
whatsnew added + green
Backport PR #41789 on branch 1.2.x (Bug in xs raising KeyError for MultiIndex columns with droplevel False and list indexe)
State: closed

Backport PR #41789: Bug in xs raising KeyError for MultiIndex columns with droplevel False and list indexe

Labels:
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: Regression
description: Functionality that used to work in a prior pandas version

Comments:
https://github.com/pandas-dev/pandas/pull/41789#discussion_r645481111
REF: do remove DTI._convert_arr_indexer
State: closed

Motivation: get rid of `_convert_listlike_indexer`, `_convert_arr_indexer`, `_convert_list_indexer`.  These are just convoluted bits of casting before a (inefficient-because-repeated) call to `get_indexer`.

This moves the relevant casting for the DTI/TDI/PI case directly into get_indexer, mirroring how get_loc works for these subclasses.

Upcoming PR does analogous thing for IntervalIndex and CategoricalIndex.  MultiIndex is the last one, haven't landed on a solution there.

cc @phofl

Labels:
name: Refactor
description: Internal refactoring of code
name: Index
description: Related to the Index class or subclasses

Comments:
can you rebase
rebased + green
hmm how does this improve things?
> hmm how does this improve things?

The goal is to get rid of convert_listlike_indexer entirely.  After this, its only a MultiIndex thing
Whatsnew1.3.0 links
State: closed

- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them

Part of #41747 

Labels:
name: Docs

Comments:
> some other broken links:

another pass will be fine, don't need to add to this PR. if you can merge master to resolve conflicts
> some other broken links:
> ...
> and many others. not sure is all can be fixed. maybe rewording is required in some cases.

Thanks @simonjayhawkins - I've gone through your list and fixed the `Styler` and `Styler.to_html`.

For classes where we'd like to link to but don't have docs (e.g. attributes of a MultiIndex), we could certainly reword. For example:

> - Add support for dict-like names in :meth:`MultiIndex.set_names` and :meth:`MultiIndex.rename` (:issue:`20421`)

This could be reworded as

> - Add support for dict-like names in :meth:`Index.set_names` and :meth:`Index.rename` in the case of a :class:`Multiindex` (:issue:`20421`)

My opinion is that it'd be better to have docs for `MultiIndex.set_names` rather than change the phrasing of the whatsnew (but perhaps that poses its own challenges?). Even if consensus is to change the wording instead, I'd still like to consider such modifications out of scope for this PR.

For dunder methods, when there is a clear fix I plan to make them in a subsequent PR, however many cases it isn't clear to me how the docs should be changed.
Thanks @rhshadrach for the updates.


merging as is, so that can continue with #41747 and since merge conflicts are likely to quickly arise with this .

comments here can be addressed in another pass.
REF: simplify ensure_index
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] whatsnew entry


Labels:
name: Refactor
description: Internal refactoring of code
name: Index
description: Related to the Index class or subclasses

Comments:
nice, luv deleting code!
> nice, luv deleting code!

yep, more on the way!
BLD: Adjust Numpy Minimum Versions for aarch64/arm64 compatibility
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] whatsnew entry


Labels:
name: Build
description: Library building on various platforms
name: ARM
description: aarch64 architecture

Comments:
We need to backport this. Currently, `pip install pandas --pre` doesn't work on M1 mac.
ok let me look 

we should merge the sdist test first
thanks @lithomas1 
@meeseeksdev backport 1.3.x
Something went wrong ... Please have  a look at my logs.
Hi,
It looks like this PR incorrectly excluded all python 3.9 numpy version requirements on aarch64 and since the backports, I can't build any pandas 1.3.X versions on aarch64 with python 3.9.

Opened a PR to fix it: #43722 just fyi

Thanks