BrandonFors/pandas-github-issues huggingface dataset:
DOC fix link in bug report
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Admin
description: Administrative tasks related to the pandas project

Comments:
thanks - merging then as this is super-minor and affects the bug-reporting experience
PERF: datetime parsing with Q
State: closed

```
In [3]: %timeit pd.Timestamp("2014Q1")
3.04 µs ± 63.6 ns per loop (mean ± std. dev. of 7 runs, 100,000 loops each)  # <- main
2.8 µs ± 19.2 ns per loop (mean ± std. dev. of 7 runs, 100,000 loops each)  # <- PR
```

Labels:
name: Datetime
description: Datetime data dtype
name: Performance
description: Memory or execution speed performance

Comments:
Thanks @jbrockmendel
update dev meeting frequency
State: closed

Updated the dev meeting description to include the 2nd monthly meeting (4th Wednesday)

Labels:
name: Docs
name: Community
description: Community topics (meetings, etc.)

Comments:
Doc build looks good:
![Screenshot from the doc build artifact of the section with the proposed change](https://user-images.githubusercontent.com/6564007/213249353-872db1c1-0fab-465f-919d-9854f38a0ed4.png)
DOC: Fix formatting issue with DataFrame.info docs
State: closed

Removed a backslash that was causing the description of the **max_cols** argument from the [DataFrme.info](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.info.html#pandas.DataFrame.info) method to be merged with the description of the **buf** argument.
![image](https://user-images.githubusercontent.com/32521936/213172824-a61ba4d6-6449-4ce2-a6f2-6d16ce62b381.png)




Labels:
name: Docs

Comments:
Sure. I downloaded the CI artifact and checked the HTML. Everything seems fine with DataFrame.info:
![image](https://user-images.githubusercontent.com/32521936/213248300-d5494299-6738-4ca8-9548-55cacf3d1439.png)


Series.info also looks ok (I get the same HTML as the [live site](https://pandas.pydata.org/docs/dev/reference/api/pandas.Series.info.html))




Excellent, thanks @CarlosGDCJ. I'm re-running the CI job that failed. I think the failure is unrelated, but just to be sure.
Thanks @CarlosGDCJ
CI/WEB: Fix github quota errors by using website as cache
State: closed

xref #50485

Looks like the previous approach didn't fix the github quota errors. @mroeschke I saw you also added a github token, but that didn't seem to work either.

What I'm doing here is to save the data we fetch from github in the website. For example https://pandas.pydata.org/maintainers.json (also for releases and pdeps). The, we request data from github normally, but if the quota is exceeded, if fallbacks to that json file with the data of the previous build. In the worst case scenario we'll have some delay updating the info in the website (we use the quota on PRs, and it can happen that for commits to main that update the cache files, we don't have quota for a while). But we shouldn't see any other CI failure caused by the github quota.

Since building the website shouldn't fail anymore, in the CI or locally, I remove the `--ignore-io-errors` parameter of the script, which doesn't make sense anymore. Also the previous cache, and the token, which I think they didn't help much.

I also found a typo when fetching the PDEPs under discussion, that was preventing to render then in the roadmap page. It's also being fixed here.

Labels:
name: CI
description: Continuous Integration
name: Web
description: pandas website

Comments:
@phofl @MarcoGorelli do you want to have a look at this before I merge? CI error is because a pytest worker didn't return, but the tests passed there.

Would be good to get this merged before the release, so the quota error doesn't happen when publishing the prod docs, and we can forget about the error later too.
Thanks for having a look. I'll merge, and if there is any problem I'll address in a follow up. But I tested it quite well, should work.
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
git checkout 1.5.x
git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
git cherry-pick -x -m1 afdf0a3e409d02cdfff6c813f512717f1503a4d8
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
git commit -am 'Backport PR #50811: CI/WEB: Fix github quota errors by using website as cache'
```

4. Push to a named branch:

```
git push YOURFORK 1.5.x:auto-backport-of-pr-50811-on-1.5.x
```

5. Create a PR against branch 1.5.x, I would have named this PR:

> "Backport PR #50811 on branch 1.5.x (CI/WEB: Fix github quota errors by using website as cache)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the `Still Needs Manual Backport` label once the PR gets merged.

If these instructions are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
DOC: Updating whatsnew to release 1.5.3
State: closed

xref #49857

Labels:
name: Docs
name: Release

Comments:
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
git checkout 1.5.x
git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
git cherry-pick -x -m1 96fc51f5ec678394373e2c779ccff37ddb966e75
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
git commit -am 'Backport PR #50809: DOC: Updating whatsnew to release 1.5.3'
```

4. Push to a named branch:

```
git push YOURFORK 1.5.x:auto-backport-of-pr-50809-on-1.5.x
```

5. Create a PR against branch 1.5.x, I would have named this PR:

> "Backport PR #50809 on branch 1.5.x (DOC: Updating whatsnew to release 1.5.3)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the `Still Needs Manual Backport` label once the PR gets merged.

If these instructions are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
ENH: Add dtype to read sql to be consistent with read_sql_query
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Came across this, seemed odd to me that one supports it but not the other

Labels:
name: IO SQL
description: to_sql, read_sql, read_sql_query
name: API - Consistency
description: Internal Consistency of API/Behavior

Comments:
Thanks @phofl
BUG: to_dict not converting NA to None
State: closed

- [x] closes #50795 (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: IO Data
description: IO issues that don't fit into a more specific label
name: Missing-data
description: np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate

Comments:
Should this change be made more broadly? (e.g. tolist, iteration) - see discussion in #29738
I would think so, yes
BUG: Fix memory leak in to_json for time objects
State: closed

- [ ] closes #40443  (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

```
       before           after         ratio
     [579e0709]       [a0c41198]
     <main>           <fix-json-memleak>
-           73.7M            69.9M     0.95  io.json.ToJSONMem.peakmem_time

SOME BENCHMARKS HAVE CHANGED SIGNIFICANTLY.
PERFORMANCE INCREASED.
```


Labels:
name: Performance
description: Memory or execution speed performance
name: IO JSON
description: read_json, to_json, json_normalize

Comments:
cc @WillAyd 
Lgtm when green
Thanks @lithomas1
REF: consolidate datetime parsing paths and exception handling
State: closed

One last check I'd like to move down into dateutil_parse is the `if dt.tzinfo is not None:` in parse_datetime_string, but it turns doing that breaks a bunch of Period parsing bc of dateutil parsing "2001-11-19/2001-11-25" as 2001-11-19 20:01:00 with a tzoffset of -1500 hours.

I'm indifferent between "Unknown string format" vs "Unknown datetime string format", so happy to change to the other one.

Labels:
name: Refactor
description: Internal refactoring of code

Comments:
@mroeschke @MarcoGorelli not time-sensitive but blocker-ish for ongoing parsing simplification
sure, taking a look today
DOC: Fix some RT02 issues in docstrings
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Towards #49968

Labels:
name: Docs
name: Clean

Comments:
thanks for working on this!

looks like there's still some errors:
```
 Error: /home/runner/work/pandas/pandas/pandas/core/frame.py:4923:RT02:pandas.DataFrame.align:The first line of the Returns section should contain only the type, unless multiple values are being returned
Error: /home/runner/work/pandas/pandas/pandas/core/series.py:4643:RT02:pandas.Series.align:The first line of the Returns section should contain only the type, unless multiple values are being returned
Error: /home/runner/work/pandas/pandas/pandas/core/accessor.py:95:RT02:pandas.Series.dt.total_seconds:The first line of the Returns section should contain only the type, unless multiple values are being returned
```
Thanks @DeaMariaLeon 
Thank you!
TST: Add test_factorize_mixed_values
State: closed

- [x] closes #19721

It has mentioned some different values for `factorize`. This is the list:
```python
pd.factorize(pd.Categorical(['a', 'a', 'c']))
pd.factorize(pd.Index(pd.Categorical(['a', 'a', 'b'])))
pd.factorize(pd.Series(pd.DatetimeIndex(['2017', '2017'], tz='US/Eastern')))
pd.factorize(pd.Series(pd.Categorical(['a', 'a', 'b'])))
```

For `pd.factorize(pd.Categorical(['a', 'a', 'c']))`, it has been test via [REF/BUG/API: factorizing categorical data](https://github.com/pandas-dev/pandas/pull/19938/files#top) in [pandas/tests/categorical/test_algos.py](https://github.com/pandas-dev/pandas/pull/19938/files#diff-64e79f37f13774c4c44e99497ba9fbdf7b5e459cc4931d95288c1b2eccff4475).

The rest are added here.

Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Algos
description: Non-arithmetic algos: value_counts, factorize, sorting, isin, clip, shift, diff
name: Categorical
description: Categorical Data Type

Comments:
Thanks @luke396! Happy to see these unit testing issues getting addressed
DEPR: remove Int/Uint/Float64Index from pandas/tests/indexing & pandas/tests/series
State: closed

Extraction of part of #50479. Progress towards #42717.

Labels:
name: Deprecate
description: Functionality to remove in pandas

Comments:
Thanks @topper-123 
Great, It's getting close now to removing Int64Index & friends. Only outstanding big issue is arithemetic with indexes with bit-size lower than 64-bit. I hope I can get that done tomorrow or saturday.
CI: Remove `conda list` from sdist build causing failures
State: closed

I think the purpose of this build is to see if the sdist is pip install-able in which case I'm not sure why we are setting up conda just to do a `conda list`. 

Labels:
name: Build
description: Library building on various platforms
name: CI
description: Continuous Integration

Comments:
cc @lithomas1 I think the wheel test failures are expected (same thing is happening in your pyproject.toml PR)
> cc @lithomas1 I think the wheel test failures are expected (same thing is happening in your pyproject.toml PR)

I'll take a look.
BTW, long term, I'm thinking about replacing this workflow with the "Build sdist" workflow in the wheel builders, once we're fully ready there (just missing arm64, ideally we switch to the cibuildwheel wheel builders for the RC).

There's still some nice stuff in the current sdist workflow that's missing from it, though (testing oldest supported numpy IIRC).
> I think the purpose of this build is to see if the sdist is pip install-able in which case I'm not sure why we are setting up conda just to do a `conda list`.

IIUC, the use of the conda env is to ensure that the sdist is installable in a clean env without anything else.

(When we built the sdist, we installed stuff like numpy and Cython.)
Looks like we still have issues with numpy 1.24.x.

On main, we don't have failing CI yet since some other package is pinning numpy < 1.24.
There is a failure in main related to this: https://github.com/pandas-dev/pandas/actions/runs/3947077782

The error doesn't make sense to me, `always_yes` is only defined once. If we decide to remove conda as proposed in this PR, that would fix the error. Or maybe the problem comes from installing conda after installing python using the github action. Maybe better to setup conda directly, and just create a new environment at the end to install the sdist on it.

It'd be nice to have this green before the release, but this seems pretty specific to the CI configuration, not a problem with the sdist, so if there are no objections, I think I'll move forward with the release, regardless of the failure.

CC: @pandas-dev/pandas-core #49857
It's a problem with the provision-with-micromamba Github Action. It should be fine to cut a release without this.

I remember this from when I was messing around with cirun.
So it appears the `conda list` specifically was causing the "`always_yes`" failure here.

The environment isolation is a fair point to keep the conda environment here for now.

> BTW, long term, I'm thinking about replacing this workflow with the "Build sdist" workflow in the wheel builders, once we're fully ready there (just missing arm64, ideally we switch to the cibuildwheel wheel builders for the RC).

With this in mind, I suppose if this temporarily fixes the failures in the sdist build we can just remove this `conda list` for now
Going to push this in as it allows the sdist jobs to pass on main and all this does is remove a `conda list` output, especially since this may be replace with a step in the wheel builder workflow
PERF: Use generator expression for Blocks.replace_list
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

```
       before           after         ratio
     [852518eb]       [3902ad1a]
     <maybe-convert-less-mem^2>       <replace-use-less-mem>
-      2.68±0.02s       2.35±0.03s     0.88  series_methods.Replace.time_replace_dict(1000)
-       2.73±0.1s       2.38±0.04s     0.87  series_methods.Replace.time_replace_list(1000)
-            203M             105M     0.52  series_methods.Replace.peakmem_replace_dict(100)
-            203M             105M     0.52  series_methods.Replace.peakmem_replace_list(100)
-           1.11G             106M     0.10  series_methods.Replace.peakmem_replace_list(1000)
-           1.11G             106M     0.10  series_methods.Replace.peakmem_replace_dict(1000)

SOME BENCHMARKS HAVE CHANGED SIGNIFICANTLY.
PERFORMANCE INCREASED.
```
inplace=True is not covered by benchmarks but should be unchanged.

Labels:
name: Performance
description: Memory or execution speed performance

Comments:
Might address #6697?
> Might address #6697?

Only partially. This optimization only works for inplace=False. We'd need to write our own replace function, to avoid generating the masks.
@meeseeksdev backport 2.x
@meeseeksdev backport 2.0.x
API / CoW: DataFrame(<dict of Series>, copy=False) constructor now gives lazy copy
State: closed

xref https://github.com/pandas-dev/pandas/issues/48998 (bullet point about constructors)

When constructing a DataFrame from Series objects, and when specifying `copy=False` with CoW enabled, we need to follow the lazy copy with CoW approach to ensure proper copy/view behaviour with the original Series objects.

We _could_ in theory change the default of `copy=True` (when data passed to the constructor is a dict) to `copy=False` when CoW is enabled, and this would retain the behaviour as-if the resulting DataFrame is a copy. But, I assume that the major reason for this default is to ensure the resulting DataFrame is consolidated and not consisting of all 1-column blocks. So for that reason I left the default as is for now.

- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Constructors
description: Series/DataFrame/Index/pd.array Constructors
name: Copy / view semantics

Comments:
@phofl this should be ready now

(the current CI failures are unrelated; the arm one also fails on main)
This is a bit annoying, simply checking for new_values is old_values is not sufficient (same problem I am having in astype).

```
val = Series([1, 2, 3])
df = pd.DataFrame({"a": val}, dtype="Int64", index=[0, 1, 2])

x = get_array(df, "a")
df.iloc[0, 0] = 100
```

The is comparison is False in this case but the data aren't actually copied, hence val and df is updated with the last statement. I guess we have to create a helper function for astype doing a more sophisticated check...

Edit: Can you add this as a test case as well?
Added an xfailed test case for constructing with dtype="Int64", assuming that https://github.com/pandas-dev/pandas/pull/50802 will fix this.
BUG: resample don't work with non-nanosecond reso
State: closed

- [ ] closes #50656  (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Ideally https://github.com/pandas-dev/pandas/pull/50773 would be merged first so I can rebase

Parametrising all these tests over `unit` helped unearth both https://github.com/pandas-dev/pandas/issues/50727 and #50716 so it's probably worthwhile


Labels:
name: Non-Nano
description: datetime64/timedelta64 with non-nanosecond resolution

Comments:
thanks - good to merge or should we get a third opinion?
merge on green
BUG: DatetimeIndex with non-nano values and freq='D' throws ValueError
State: closed

- [ ] closes #50727 (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Non-Nano
description: datetime64/timedelta64 with non-nanosecond resolution

Comments:
thanks @MarcoGorelli
DOC: fixed register_extension_dtype import statement
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: ExtensionArray
description: Extending pandas with custom dtypes or arrays.

Comments:
thx @lpizzinidev