BrandonFors/pandas-github-issues huggingface dataset:
WIP/ENH: Pass tzinfos to dateutil parser
State: closed

Putting this up early because the `set_option` behavior deserves discussion.  The alternative is to make/let users pass `tzinfos` in Timestamp, to_datetime, DatetimeIndex, and probably others.

- [x] closes #22234, #18702
- [x] tests added / passed
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry

Still need to:
- [ ] document usage, whether via set_option or passing tzinfos
- [ ] tests for array_to_datetime, to_datetime, Timestamp, DatetimeIndex
- [ ] test for the fix of #22234


Labels:
name: Datetime
description: Datetime data dtype
name: Timezones
description: Timezone data dtype

Comments:
Hello @jbrockmendel! Thanks for submitting the PR.

 - There are no PEP8 issues in the file [`pandas/core/config_init.py`](https://github.com/jbrockmendel/pandas/blob/c9416786ce7226fa1f4baa230509958a6b809e38/pandas/core/config_init.py) !

 - There are no PEP8 issues in the file [`pandas/tests/tslibs/test_parsing.py`](https://github.com/jbrockmendel/pandas/blob/c9416786ce7226fa1f4baa230509958a6b809e38/pandas/tests/tslibs/test_parsing.py) !


huh? what is this supposed to accomplish from a user POV ?
> huh? what is this supposed to accomplish from a user POV ?

At the moment we aren't correctly parsing _any_ timezones aside from UTC offsets.

```
>>> ts = pd.Timestamp("2018-11-04 3:45 PM CST")
/usr/lib/python2.7/site-packages/dateutil/parser/_parser.py:1204: UnknownTimezoneWarning: tzname CST identified but not understood.  Pass `tzinfos` argument in order to correctly return a timezone-aware datetime.  In a future version, this will raise an exception.
  category=UnknownTimezoneWarning)
>>> 
>>> ts
Timestamp('2018-11-04 15:45:00')
```

(yes, yes, I know "CST" isn't a timezone. But PST, CST, EST, ... are not uncommon in the wild)
that’s not my point

how is this actually useful?
> how is this actually useful?

Because it allows users to parse date-like strings correctly.  Because at the moment we are silently returning incorrect results (well, dateutil is issuing a warning).  Because before long dateutil will start raising in these cases and we should provide _some_ way of letting users pass a `tzinfos` dict.

this is a really tiny edge case and adds lots of complexity 
I agree that this adds complexity, but not that it is a tiny corner case.  Let's put this on the back-burner until after 0.24.0.
ok, prob best to close this. and simply create a project for those issues.
Closing until after 0.24.0
CLN: remove unused import of warnings module
State: closed

I forgot to remove a import line in #24080 as the warning module is no longur used in ``frame_methods.py`` and we now get a linting error.

This PR fixes that.


Labels:

Comments:
Hello @topper-123! Thanks for submitting the PR.

 - There are no PEP8 issues in the file [`asv_bench/benchmarks/frame_methods.py`](https://github.com/topper-123/pandas/blob/659a2bf19f43618f2a7a865026a7cbb22ec31d2f/asv_bench/benchmarks/frame_methods.py) !


didn't see #24092 is already on this. Closing this PR as a duplicate.
remove calls to DataFrame.consolidate in benchmarks
State: closed

The ``DataFrame.consolidate`` method has been removed for 0.24 (in #23377), but there are still some calls to it in the asv benchmarks.  I had a failure in #23752 because of this, so hereby a fix.

This PR uses the internal  ``_consolidate`` method instead.

Labels:
name: Performance
description: Memory or execution speed performance
name: Deprecate
description: Functionality to remove in pandas

Comments:
Hello @topper-123! Thanks for submitting the PR.

 - There are no PEP8 issues in the file [`asv_bench/benchmarks/frame_methods.py`](https://github.com/topper-123/pandas/blob/78650a9ffc9cd43d11d123db19c575e6ccbcf4ba/asv_bench/benchmarks/frame_methods.py) !

 - There are no PEP8 issues in the file [`asv_bench/benchmarks/join_merge.py`](https://github.com/topper-123/pandas/blob/78650a9ffc9cd43d11d123db19c575e6ccbcf4ba/asv_bench/benchmarks/join_merge.py) !


thanks @topper-123
PERF: Use Categorical.equals in CategoricalIndex.equals
State: closed

Avoids a materialization of the full array.

```python
import pandas as pd
import numpy as np
import string


a = pd.CategoricalIndex(np.random.choice(list(string.ascii_letters[:10]), 100_000))
b = a.copy()
c = pd.CategoricalIndex(np.random.choice(list(string.ascii_letters[:10]), 100_000))

print("eq-index")
%timeit a.equals(b)

print("ne-index")
%timeit a.equals(c)
```


timings

case      | master | pr
--------- | ------- | --
eq-index  | 3.35 ms | 420 µs
ne-index  | 1.29 ms | 451 µs


Labels:
name: Performance
description: Memory or execution speed performance
name: Categorical
description: Categorical Data Type

Comments:
Hello @TomAugspurger! Thanks for submitting the PR.

 - There are no PEP8 issues in the file [`pandas/core/indexes/category.py`](https://github.com/TomAugspurger/pandas/blob/09e07d2ae26008d203bc84e4721fa83ff484e662/pandas/core/indexes/category.py) !


# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/24023?src=pr&el=h1) Report
> Merging [#24023](https://codecov.io/gh/pandas-dev/pandas/pull/24023?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/dc8d35aa53d496b651f5e1ab4cb2604e9f7236c7?src=pr&el=desc) will **increase** coverage by `<.01%`.
> The diff coverage is `100%`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/24023/graphs/tree.svg?width=650&token=eZ4WkYLtcO&height=150&src=pr)](https://codecov.io/gh/pandas-dev/pandas/pull/24023?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #24023      +/-   ##
==========================================
+ Coverage   92.31%   92.31%   +<.01%     
==========================================
  Files         161      161              
  Lines       51549    51551       +2     
==========================================
+ Hits        47586    47588       +2     
  Misses       3963     3963
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `90.71% <100%> (ø)` | :arrow_up: |
| #single | `42.45% <100%> (ø)` | :arrow_up: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/24023?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/core/indexes/category.py](https://codecov.io/gh/pandas-dev/pandas/pull/24023/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW5kZXhlcy9jYXRlZ29yeS5weQ==) | `97.88% <100%> (+0.01%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/24023?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/24023?src=pr&el=footer). Last update [dc8d35a...f35e8dd](https://codecov.io/gh/pandas-dev/pandas/pull/24023?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

thanks!
REF: Convert test_hashing to pytest idiom
State: closed


Labels:
name: Refactor
description: Internal refactoring of code
name: Testing
description: pandas testing functions or related to the test suite

Comments:
Hello @gfyoung! Thanks for updating the PR.

 - There are no PEP8 issues in the file [`pandas/tests/util/test_hashing.py`](https://github.com/gfyoung/pandas/blob/77f7ac87b9b71f132d9a36a6ecd789698f446db7/pandas/tests/util/test_hashing.py) !



##### Comment last updated on November 30, 2018 at 06:21 Hours UTC
# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/24009?src=pr&el=h1) Report
> Merging [#24009](https://codecov.io/gh/pandas-dev/pandas/pull/24009?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/dc8d35aa53d496b651f5e1ab4cb2604e9f7236c7?src=pr&el=desc) will **not change** coverage.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/24009/graphs/tree.svg?width=650&token=eZ4WkYLtcO&height=150&src=pr)](https://codecov.io/gh/pandas-dev/pandas/pull/24009?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master   #24009   +/-   ##
=======================================
  Coverage   92.31%   92.31%           
=======================================
  Files         161      161           
  Lines       51549    51549           
=======================================
  Hits        47586    47586           
  Misses       3963     3963
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `90.7% <ø> (ø)` | :arrow_up: |
| #single | `42.44% <ø> (ø)` | :arrow_up: |


------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/24009?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/24009?src=pr&el=footer). Last update [dc8d35a...dfa24a8](https://codecov.io/gh/pandas-dev/pandas/pull/24009?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

@jreback @TomAugspurger : Addressed the comment above, and all is still green.  PTAL.
LGTM. Instead of the index fixture you could also just call each of `index=False` and `index=True` in `_check_equal`, but either way works for me.
> Instead of the index fixture you could also just call each of index=False and index=True in _check_equal, but either way works for me.

Thought about it.  Just wanted something a little more modular.  Thanks for noticing the bug though!
API: fix str-accessor on CategoricalIndex
State: closed

closes #23555
closes #23556

split off from #23167
- [x] tests added / passed
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry



Labels:
name: Bug
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: Categorical
description: Categorical Data Type

Comments:
Hello @h-vetinari! Thanks for submitting the PR.

 - There are no PEP8 issues in the file [`pandas/core/strings.py`](https://github.com/h-vetinari/pandas/blob/98ff497164c431abc753c1384f076b1aeb817c45/pandas/core/strings.py) !

 - There are no PEP8 issues in the file [`pandas/tests/test_strings.py`](https://github.com/h-vetinari/pandas/blob/98ff497164c431abc753c1384f076b1aeb817c45/pandas/tests/test_strings.py) !


# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/24005?src=pr&el=h1) Report
> Merging [#24005](https://codecov.io/gh/pandas-dev/pandas/pull/24005?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/92d25f0da6c3b1175047cba8c900e04da68920b8?src=pr&el=desc) will **not change** coverage.
> The diff coverage is `50%`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/24005/graphs/tree.svg?width=650&token=eZ4WkYLtcO&height=150&src=pr)](https://codecov.io/gh/pandas-dev/pandas/pull/24005?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master   #24005   +/-   ##
=======================================
  Coverage   42.45%   42.45%           
=======================================
  Files         161      161           
  Lines       51561    51561           
=======================================
  Hits        21888    21888           
  Misses      29673    29673
```

| Flag | Coverage Δ | |
|---|---|---|
| #single | `42.45% <50%> (ø)` | :arrow_up: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/24005?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/core/strings.py](https://codecov.io/gh/pandas-dev/pandas/pull/24005/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvc3RyaW5ncy5weQ==) | `33% <50%> (ø)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/24005?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/24005?src=pr&el=footer). Last update [92d25f0...ecd08b9](https://codecov.io/gh/pandas-dev/pandas/pull/24005?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

@jreback 
Ping on green
thanks @h-vetinari
CLEAN: Removing outdated and unused CI files
State: closed

Removing outdated and unused files.
- `ci/print_versions.py` doesn't work, as the used path has changed. It's not used anywhere in the code, and when needed, `python -c "import pandas; pandas.show_versions();"` is used instead.
- `ci/README.txt` it's outdated and incomplete. After updating the CI and making it more compact, I'll try to add a section to the contributing page in the docs, about what we have. This should be more useful than this file.

Labels:
name: CI
description: Continuous Integration
name: Clean

Comments:
# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/24004?src=pr&el=h1) Report
> Merging [#24004](https://codecov.io/gh/pandas-dev/pandas/pull/24004?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/dc8d35aa53d496b651f5e1ab4cb2604e9f7236c7?src=pr&el=desc) will **not change** coverage.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/24004/graphs/tree.svg?width=650&token=eZ4WkYLtcO&height=150&src=pr)](https://codecov.io/gh/pandas-dev/pandas/pull/24004?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master   #24004   +/-   ##
=======================================
  Coverage   92.31%   92.31%           
=======================================
  Files         161      161           
  Lines       51549    51549           
=======================================
  Hits        47586    47586           
  Misses       3963     3963
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `90.7% <ø> (ø)` | :arrow_up: |
| #single | `42.44% <ø> (ø)` | :arrow_up: |


------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/24004?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/24004?src=pr&el=footer). Last update [dc8d35a...a8dcda9](https://codecov.io/gh/pandas-dev/pandas/pull/24004?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

thanks!
REF: prelim for fixing array_to_datetime
State: closed

`tslib.array_to_datetime` has an unused `format` kwarg.  This gets rid of it.

`to_datetime` (or more specifically, `_convert_listlike_datetimes`) has a bunch of boxing logic inside a try/except that doesn't need to be, obscures what Exception we're trying to catch.  This moves that code outside the try/except and de-indents it.  This overlaps with #23675.

xref #23722 (the conversation there less than the OP)  cc @mroeschke 

Labels:
name: Datetime
description: Datetime data dtype
name: Clean

Comments:
Hello @jbrockmendel! Thanks for submitting the PR.

 - There are no PEP8 issues in the file [`pandas/core/tools/datetimes.py`](https://github.com/jbrockmendel/pandas/blob/03e9ea8c57683bfeedcaf757982dc00c9bcf7d77/pandas/core/tools/datetimes.py) !


# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/24000?src=pr&el=h1) Report
> Merging [#24000](https://codecov.io/gh/pandas-dev/pandas/pull/24000?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/7653a6b43a8a302bc1d88903c8c99730a00bcc80?src=pr&el=desc) will **increase** coverage by `<.01%`.
> The diff coverage is `100%`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/24000/graphs/tree.svg?width=650&token=eZ4WkYLtcO&height=150&src=pr)](https://codecov.io/gh/pandas-dev/pandas/pull/24000?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #24000      +/-   ##
==========================================
+ Coverage    92.3%    92.3%   +<.01%     
==========================================
  Files         161      161              
  Lines       51555    51557       +2     
==========================================
+ Hits        47589    47591       +2     
  Misses       3966     3966
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `90.7% <100%> (ø)` | :arrow_up: |
| #single | `42.44% <50%> (ø)` | :arrow_up: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/24000?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/core/tools/datetimes.py](https://codecov.io/gh/pandas-dev/pandas/pull/24000/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvdG9vbHMvZGF0ZXRpbWVzLnB5) | `85.53% <100%> (+0.08%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/24000?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/24000?src=pr&el=footer). Last update [7653a6b...03e9ea8](https://codecov.io/gh/pandas-dev/pandas/pull/24000?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

thanks
DOC: work around IPython directive bug
State: closed

Closes https://github.com/pandas-dev/pandas/issues/23954

Labels:
name: Docs

Comments:
# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/23994?src=pr&el=h1) Report
> Merging [#23994](https://codecov.io/gh/pandas-dev/pandas/pull/23994?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/8ed347a25c34676399a46a4274c3dcf366f37ecf?src=pr&el=desc) will **not change** coverage.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/23994/graphs/tree.svg?width=650&token=eZ4WkYLtcO&height=150&src=pr)](https://codecov.io/gh/pandas-dev/pandas/pull/23994?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master   #23994   +/-   ##
=======================================
  Coverage    92.3%    92.3%           
=======================================
  Files         161      161           
  Lines       51538    51538           
=======================================
  Hits        47572    47572           
  Misses       3966     3966
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `90.7% <ø> (ø)` | :arrow_up: |
| #single | `42.38% <ø> (ø)` | :arrow_up: |


------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/23994?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/23994?src=pr&el=footer). Last update [8ed347a...0cabc16](https://codecov.io/gh/pandas-dev/pandas/pull/23994?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

Fixed that issue in the doc build.
CLN: remove unused import causing flake8 error on master
State: closed

Cleanup in #23971 removed the last use of the `copy` module, so `flake8` is now failing master due to an unused import. This fixes master by removing the import.

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Code Style
description: Code style, linting, code_checks

Comments:
Hello @qwhelan! Thanks for submitting the PR.

 - There are no PEP8 issues in the file [`pandas/core/groupby/ops.py`](https://github.com/qwhelan/pandas/blob/403e1cd1a2ed223ebfa26aa50654265d439d08ab/pandas/core/groupby/ops.py) !


thanks
CLN: Add missing types to tslibs/conversion.pyx
State: closed

Again just variables that were not typed, and removing some unused variables.

Labels:
name: Clean

Comments:
# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/23984?src=pr&el=h1) Report
> Merging [#23984](https://codecov.io/gh/pandas-dev/pandas/pull/23984?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/982c169a286d436481364d2a6c27b355a83472b5?src=pr&el=desc) will **not change** coverage.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/23984/graphs/tree.svg?width=650&token=eZ4WkYLtcO&height=150&src=pr)](https://codecov.io/gh/pandas-dev/pandas/pull/23984?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master   #23984   +/-   ##
=======================================
  Coverage   92.31%   92.31%           
=======================================
  Files         161      161           
  Lines       51513    51513           
=======================================
  Hits        47554    47554           
  Misses       3959     3959
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `90.71% <ø> (ø)` | :arrow_up: |
| #single | `42.42% <ø> (ø)` | :arrow_up: |


------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/23984?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/23984?src=pr&el=footer). Last update [982c169...d97cecb](https://codecov.io/gh/pandas-dev/pandas/pull/23984?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

thanks
i think this broke 32 bit

https://travis-ci.org/MacPython/pandas-wheels/jobs/461611536
cc @jbrockmendel 
cc @jschendel 
I am guessing it was one of these declarations, but it's not obvious to me which one it was since they are all derived from each other:

https://github.com/pandas-dev/pandas/pull/23984/files#diff-66d289312fd0b02e7721bf45fb281797R891