BrandonFors/pandas-github-issues huggingface dataset:
REF: helpers to de-duplicate datetimelike arithmetic
State: closed

1-2 more PRs after this using these helpers

Labels:
name: Refactor
description: Internal refactoring of code
name: Datetime
description: Datetime data dtype
name: Numeric Operations
description: Arithmetic, Comparison, and Logical operations
name: Timedelta
description: Timedelta data type

Comments:
> Typing failures: https://github.com/pandas-dev/pandas/actions/runs/3148299272/jobs/5131019981

merged master.  hopefully that'll do it since the other one is OK
green
Thanks @jbrockmendel
DEPR: Enforce ewm(times=str) deprecation
State: closed

Introduced in https://github.com/pandas-dev/pandas/pull/43265


Labels:
name: Deprecate
description: Functionality to remove in pandas
name: Window
description: rolling, ewma, expanding

Comments:
Test that it raises? Whatsnew?
> Test that it raises? Whatsnew?

Added test here https://github.com/pandas-dev/pandas/pull/48842/files#diff-3f0452bb29c5fda1dfc3fb1ec8346ac3a1fffe504c857d232c1df7a043b45ad8R239
DEPR: Enforce Rolling.validate/is_datetimelike/win_type deprecation
State: closed

Introduced in https://github.com/pandas-dev/pandas/pull/43665, https://github.com/pandas-dev/pandas/pull/38963


Labels:
name: Deprecate
description: Functionality to remove in pandas
name: Window
description: rolling, ewma, expanding

Comments:
needs whatsnew (and rebase), otherwise looks good
BUG: Timestamp.__add__(np_dt64_obj) result resolution
State: closed

- [ ] closes #xxxx (Replace xxxx with the Github issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Timedelta
description: Timedelta data type
name: Non-Nano
description: datetime64/timedelta64 with non-nanosecond resolution
name: Timestamp
description: pd.Timestamp and associated methods

Comments:
there's a subtle problem here, closing for now
CLN: small cleanups
State: closed

- [ ] closes #xxxx (Replace xxxx with the Github issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Clean
name: Non-Nano
description: datetime64/timedelta64 with non-nanosecond resolution

Comments:
Thanks @jbrockmendel
BUG: EWM(online) not raising NotImplementedError for unsupported
State: closed

- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added an entry in the latest `doc/source/whatsnew/v1.6.0.rst` file if fixing a bug or adding a new feature.


Labels:
name: Error Reporting
description: Incorrect or improved errors from pandas
name: Window
description: rolling, ewma, expanding

Comments:
small comment, otherwise LGTM
thx @mroeschke
BUG: ArrowExtensionArray compared to invalid object not raising
State: closed

- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added an entry in the latest `doc/source/whatsnew/v1.5.1.rst` file if fixing a bug or adding a new feature.


Labels:
name: Arrow
description: pyarrow functionality

Comments:
would it make sense to have a code check for returning an exception rather than raising one?  (I think i wrote some methods a while back that intentionally return exceptions to be re-raised by callers which we'd have to change)
> would it make sense to have a code check for returning an exception rather than raising one?

Did you mean raising exceptions rather than returning them? If so yeah it would help avoid cases like this and https://github.com/pandas-dev/pandas/pull/48834. `try/except` should be more reliable way to re-raise exceptions (specifically) rather than using `return`?


Thx @mroeschke
Adding new variables for docstring
State: closed

- [ ] closes #48632 
- [ ] ~~[Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature~~
- [ ] ~~All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).~~
- [ ] ~~Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.~~
- [ ] ~~Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.~~

I am creating this PR to get comments on whether this solution is correct or not.

**Solution -** The Timestamp class is inheriting from datetime. Values of `month`, `day`, `hour`, `minute`, `second` and `microsecond` variables are used of that of datetime class. To add the docstring, I am proposing to create one new variable for each. I have added a variable for `hour` as an example.

I don't know whether it will affect performance and memory usage. Do we have some existing mechanism to measure it and then compare with old results?

Labels:

Comments:
@phofl Thanks for the feedback. I am closing this PR, to search for some other way to add docstring.
minimal environment
State: closed

- [ ] closes #48828

minimal_environment.yml created
minimal-requirements.txt generated from yml/environment

@MarcoGorelli 

Labels:
name: Dependencies
description: Required and optional dependencies
name: Stale

Comments:
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
yeah not really this is the direction we want to go in - closing for now then but thanks @YvanCywan for your PR
REGR: groupby fails with nullable dtypes and dropna=False
State: closed

- [x] closes #48794 (Replace xxxx with the Github issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

This reworked test is a bit ghastly, any suggestions on how to simplify or clean up are most welcome.

Labels:
name: Bug
name: Groupby
name: Regression
description: Functionality that used to work in a prior pandas version
name: NA - MaskedArrays
description: Related to pd.NA and nullable extension arrays

Comments:
Thanks @mroeschke - I was able to simplify this a bit (namely - remove an xfail, less parameters for determining sequence, less repetition of uniques). Also added back in `test_series` and sparse (thanks for catching that). What do you think - would you still like to see it broken up?
Looks better. There's still a bit to grok to understand `range(3**4)` -> `xyz` -> unique values per dtype, but I don't think making it simpler should block the fix.
Thanks @rhshadrach
DEPR: Enforce DTI.to_series(keep_tz) removal
State: closed

- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Introduced in https://github.com/pandas-dev/pandas/pull/29731


Labels:
name: Deprecate
description: Functionality to remove in pandas

Comments:
thx @mroeschke (please update the deprecation issue)
DEPR: Enforce is_extension_type removal
State: closed

- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Introduced in https://github.com/pandas-dev/pandas/pull/29457

Labels:
name: Deprecate
description: Functionality to remove in pandas

Comments:
thanks @mroeschke.  pls be sure to update #30228
DEPR: Enforce deprecation of Categorical.min/max(numeric_only)
State: closed

- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Added in https://github.com/pandas-dev/pandas/pull/27929


Labels:
name: Deprecate
description: Functionality to remove in pandas

Comments:
needs whatsnew, otherwise lgtm
thx @mroeschke (Please update the deprecations issue)
BUG: to_datetime(format='...%f') parses nanoseconds
State: closed

- [x] closes #48767 (Replace xxxx with the Github issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added an entry in the latest `doc/source/whatsnew/v1.5.1.rst` file if fixing a bug or adding a new feature.


Labels:
name: Datetime
description: Datetime data dtype

Comments:
thx @mroeschke 
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
git checkout 1.5.x
git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
git cherry-pick -x -m1 953921f2782062a67aa3886837fac563892d7ec6
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
git commit -am "Backport PR #48820: BUG: to_datetime(format='...%f') parses nanoseconds"
```

4. Push to a named branch:

```
git push YOURFORK 1.5.x:auto-backport-of-pr-48820-on-1.5.x
```

5. Create a PR against branch 1.5.x, I would have named this PR:

> "Backport PR #48820 on branch 1.5.x (BUG: to_datetime(format='...%f') parses nanoseconds)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the `Still Needs Manual Backport` label once the PR gets merged.

If these instructions are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
API: make Timestamp/Timedelta _as_unit public as_unit
State: closed

- [ ] closes #xxxx (Replace xxxx with the Github issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Timedelta
description: Timedelta data type
name: Non-Nano
description: datetime64/timedelta64 with non-nanosecond resolution
name: Timestamp
description: pd.Timestamp and associated methods

Comments:
@jreback mentioned this earlier this week
Following today's call, I'm leaning towards changing "reso" to "creso" for the cdef methods and "unit" to "reso" for the python methods.
After spending more time on this, im again really reticent to use "reso" and "as_reso" for the public keyword/attribute/method referring to the unit abbreviation (s, ms, us, ns).  "unit" is just used _everywhere_ else.  DatetimeTZDtype refers to it as .unit. numpy docs call it unit (though it isnt a keyword in the dt64/td64 constructors).  IIUC pyarrow dtypes have a .unit attribute for this.

I think the public-facing keyword/attribute/method should be unit/unit/as_unit.  The argument against has been that in the constructors this means "unit" plays two roles: 1) how to interpret int inputs, 2) how to represent the data internally.  Increasingly I think that if a user passes ints and unit="s", it makes more sense to keep this as second-resolution than cast to nanosecond (and should be more performant).  Moreover, doing this would avoid some overflow cases.

That leaves cases where a user passes a unit that is supported for input but not for storage, i.e. Y, M, D, h, m.  For these we'd treat the inputs the same and use the closest-supported (i.e. second) for storage.

Thoughts?
> I think the public-facing keyword/attribute/method should be unit/unit/as_unit

Yeah I would be okay if the "problematic" situation of `to_datetime/to_timedelta/Timestamp/TImedelta(ints, unit="s")` to interpret `[ints]` as epoch seconds _and_ having the resulting unit be seconds, so having the public naming be `unit` would be okay to me.
all good here?  we have the option of changing unit->reso later, but for now id like to get started on some follow-ups to this.
Thanks @jbrockmendel
BUG: DTI-datetime_scalar preserve freq
State: closed

- [ ] closes #xxxx (Replace xxxx with the Github issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Numeric Operations
description: Arithmetic, Comparison, and Logical operations
name: Frequency
description: DateOffsets

Comments:
Thanks @jbrockmendel
DOC: Fix whatsnew
State: closed

- [ ] closes #xxxx (Replace xxxx with the Github issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Docs

Comments:
thanks @phofl
Added padding and fixed columns for sponsor logos in mobile view
State: closed

Added padding and fixed columns for sponsor logos in mobile view

- [ ] closes #48519  (Replace xxxx with the Github issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Web
description: pandas website

Comments:
Thanks for the work on this @amay-patel. Do you mind uploading screenshots on how these changes look in both mobile and desktop please? This way reviewers can provide feedback not only in the code changes, but in the final result. Thanks!
Hi! I put a screenshot of what the website looks like on my localhost:8000 with a regular computer view and a mobile view

<img width="1440" alt="Screen Shot 2022-09-28 at 4 46 10 PM" src="https://user-images.githubusercontent.com/92037532/192885330-a7ba957c-b391-4693-8181-8898425c3811.png">
<img width="1440" alt="Screen Shot 2022-09-28 at 4 47 00 PM" src="https://user-images.githubusercontent.com/92037532/192885518-9ac2e5e8-bccc-40d8-9672-544d4ee78ae3.png">

Awesome! Thanks @amay-patel
PERF: Delay creation of indexer in RangeIndex.sort_values
State: closed

- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added an entry in the latest `doc/source/whatsnew/v1.6.0.rst` file if fixing a bug or adding a new feature.

```
In [2]: ri = pd.RangeIndex(1_000_000)

In [3]: %memit ri.sort_values() # PR
peak memory: 120.22 MiB, increment: 0.10 MiB

In [4]: %memit ri.sort_values() # main
peak memory: 119.88 MiB, increment: 0.17 MiB
```

Labels:
name: Performance
description: Memory or execution speed performance
name: Index
description: Related to the Index class or subclasses

Comments:
thx @mroeschke
REGR: fix df.apply with keyword non-zero axis
State: closed

Fixes #48656. Builds on top of #48693.

An alternative fix could be to use `inspect.signature`, since `inspect.signature` defaults to following wrapped functions. But this seems like a more minimal change.

- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).

Labels:
name: Apply
description: Apply, Aggregate, Transform, Map

Comments:
cc @MarcoGorelli 
Thanks, made the changes
Addressed all of @MarcoGorelli's comments.

As I said in the PR body, I chose not to use inspect.signature because it's often good to try and keep regression fixes minimal. But I'm happy to change it, so let me know and I will :-)
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
git checkout 1.5.x
git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
git cherry-pick -x -m1 1dee9344a4e9a3a00935d3b91587254b5fb0b4f0
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
git commit -am 'Backport PR #48797: REGR: fix df.apply with keyword non-zero axis'
```

4. Push to a named branch:

```
git push YOURFORK 1.5.x:auto-backport-of-pr-48797-on-1.5.x
```

5. Create a PR against branch 1.5.x, I would have named this PR:

> "Backport PR #48797 on branch 1.5.x (REGR: fix df.apply with keyword non-zero axis)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the `Still Needs Manual Backport` label once the PR gets merged.

If these instructions are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
                
thx @hauntsaninja
Created icons for dark theme
State: closed

- [x] closes #48762

For the `svg` images mentioned in the referred issue, I created a version for the dark theme as suggested by @datapythonista.
The images are retrieved by the pydata sphinx theme as explained in [PyData Theme documentation](https://pydata-sphinx-theme.readthedocs.io/en/stable/user_guide/light-dark.html#use-theme-dependent-content).

Labels:
name: Web
description: pandas website

Comments:
Thanks @lorenzovngl, looks good. I think we can probably keep just the new version of the icons, I think the new blue looks good for both themes, what do you think?  And the SQL one, if instead of white you make it gray, we can probably do the same. For SAS makes sense how it is now. What do you think? I think having just one version when possible will make things easier to maintain. And I think the new blue icons may look even better in the bright theme.
Hi! I absolutely agree with you. In my opinion, the new light blue looks better than the older one even for the light theme.
For the SQL logo, I chose the gray `#888888` because it seems to look good in both themes.
Since these latest changes are a kind of rollback, I created a new branch [lorenzovngl:dark-theme-icons-2](https://github.com/lorenzovngl/pandas/tree/dark-theme-icons-2). If everything is ok I can make a new pull request from that branch.
I think it'd be ok to also commit the changes here even if it's a rollback. Since the changes would vome in a separate commit, we could see differences, restore the old version... Even if it's in the same branch. But also fine to open a new PR and close this one if you think it's better.
Awesome, thanks @lorenzovngl! 
@lorenzovngl looks like the code for the SAS icon didn't work as expected. The sphinx code is leaked into the website.

Can you have a look please? https://pandas.pydata.org/docs/dev/getting_started/index.html#coming-from
I noticed there is a wrong syntax and it caused the code leak, but I can't figured out how to solve it...
> I noticed there is a wrong syntax and it caused the code leak, but I can't figured out how to solve it...

Do you mind reverting the changes in the rst file for now if you can't find the fix please? You can leave the two images and all the other changes, but at least it'll work well with the light theme while we find the fix. Also, if you can create an issue to use both images once it's reverted, that would be great. Thanks!
I opened #48935
DOC: Add example for `Series.get()`
State: closed

Add an example of `Series.get()` to `NDFrame`.

Previously, [the docs for `Series.get()`](https://pandas.pydata.org/docs/reference/api/pandas.Series.get.html) only showed examples for `DataFrame.get()`, confusingly.

Labels:
name: Docs

Comments:
Great thanks @wjandrea
ENH: Raise ParserError instead of IndexError when specifying an incorrect number of columns with index_col for the read_csv C parser.
State: closed


- [x] closes #16393 (Replace xxxx with the Github issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Enhancement
name: Error Reporting
description: Incorrect or improved errors from pandas

Comments:
Thanks @DriesSchaumont
CLN: Remove overloads made redundant by deprecation
State: closed

- [X] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [X] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.


Not 100% sure what the policy is on type annotation in pandas when there is a deprecation that makes overloads redundant, assuming the deprecated path isn't taken. 

xref pandas-dev/pandas-stubs#317


Labels:

Comments:
Don't think this is feasible since pandas uses `inplace` internally for now.