BrandonFors/pandas-github-issues huggingface dataset:
BUG: GH36928 Allow dict_keys to be used as column names by read_csv
State: closed

- [x] closes #36928 
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry

Labels:
name: Regression
description: Functionality that used to work in a prior pandas version
name: IO CSV
description: read_csv, to_csv

Comments:
Also can you add a v1.1.4 release note?
> Also can you add a v1.1.4 release note?

Where do I do that?
> > Also can you add a v1.1.4 release note?
> 
> Where do I do that?

/doc/source/whatsnew/v1.1.4 in the regressions section
Done. Thank you for your guidance throughout!
Hello @abmyii! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:





There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2020-10-08 21:11:41 UTC
@abmyii can you merge master again? I think CI failures are possibly due to upstream changes in isort that appear to be fixed now.
> @abmyii can you merge master again? I think CI failures are possibly due to upstream changes in isort that appear to be fixed now.

How would I do that? `git merge master` on the `dict_keys_as_csv_names` branch locally and then push? I don't want to mess anything more up!
> How would I do that? `git merge master` on the `dict_keys_as_csv_names` branch locally and then push? I don't want to mess anything more up!

Locally you can do
```
git checkout dict_keys_as_csv_names
git fetch upstream
git merge upstream/master
```
It should merge cleanly.
Done, thanks.
thanks @abmyii 
@meeseeksdev backport 1.1.x
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
$ git checkout 1.1.x
$ git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
$ git cherry-pick -m1 982dae89225531afabbcbe0c52203cfb2408d507
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
$ git commit -am 'Backport PR #36937: BUG: GH36928 Allow dict_keys to be used as column names by read_csv'
```

4. Push to a named branch :

```
git push YOURFORK 1.1.x:auto-backport-of-pr-36937-on-1.1.x
```

5. Create a PR against branch 1.1.x, I would have named this PR:

> "Backport PR #36937 on branch 1.1.x"

And apply the correct labels and milestones.

Congratulation you did some good work ! Hopefully your backport PR will be tested by the continuous integration and merged soon!

If these instruction are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
TYP: clean unreachable code and duplicate test #27396
State: closed

xref #27396
```
pandas/tests/frame/test_to_csv.py:363: error: Right operand of 'or' is never evaluated  [unreachable]
pandas/tests/frame/test_to_csv.py:389: error: Right operand of 'or' is never evaluated  [unreachable]
pandas/tests/frame/test_to_csv.py:417: error: Right operand of 'or' is never evaluated  [unreachable]
pandas/tests/frame/test_to_csv.py:443: error: Right operand of 'or' is never evaluated  [unreachable]
```

Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: IO CSV
description: read_csv, to_csv

Comments:
thanks @fangchenli
REF/TYP: arrays.datetimelike
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry

@simonjayhawkins this is still giving me two mypy complaints locally:

```
pandas/core/arrays/datetimelike.py:549: error: Too many arguments for "object"  [call-arg]
pandas/core/arrays/datetimelike.py:556: error: Too many arguments for "object"  [call-arg]
```

These lines are both calls to `self._scalar_type(foo)`

_scalar_type is annotated as `Type[DatetimeLikeScalar]`, which is a TypeVar for Period/Timestamp/Timedelta.  Is there a non-ignoring solution here?

Labels:
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
updated to revert the setup.cfg edit for now to get CI passing
one more rebase to make sure didn't break anything from a recent merge (as I saw some PRs using DatetimeArraylikeMixin) but could be mistaken.
are both travis and azure down?
thanks. ideally if we can see what has changed about the failing builds. maybe a package was released which bumped things?
TYP: define RangeIndex methods non-dynamically
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
(This is the last branch in this sequence for now)
TYP/REF: define comparison methods non-dynamically
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry

xref #31160, cc @simonjayhawkins @WillAyd

This implements OpsMixin which is more mypy-friendly than the status quo, and uses it in Series, Index, and DatetimeLikeArrayMixin.  There are a handful more places where it can be used in an upcoming pass.

Will do the same for arithmetic/boolean ops if there is consensus that this is worth pursuing.

Labels:
name: Refactor
description: Internal refactoring of code
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
LGTM
yep lgtm. thanks @jbrockmendel
TYP: define Index.any, Index.all non-dynamically
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry

xref #31160, cc @simonjayhawkins 

This unearths 2.5 issues:
1) Index.any/all says that we pass *args/**kwargs to np.any/all, but we don't
1b) the RangeIndex methods don't have args/kwargs at all
2) the subclasses that disable any/all don't totally make sense.  In particular it seems like Float64Index should support it and CategoricalIndex should support it iff `self.categories` supports it

Labels:
name: Numeric Operations
description: Arithmetic, Comparison, and Logical operations

Comments:
can you rebase
> This unearths 2.5 issues:
> 
> 1. Index.any/all says that we pass *args/**kwargs to np.any/all, but we don't
>    1b) the RangeIndex methods don't have args/kwargs at all
> 2. the subclasses that disable any/all don't totally make sense.  In particular it seems like Float64Index should support it and CategoricalIndex should support it iff `self.categories` supports it

@jbrockmendel maybe open issues for these and @jreback comment https://github.com/pandas-dev/pandas/pull/36929#discussion_r501099510
Thanks @jbrockmendel
DOC: add newline to fix code block format in `pandas.DataFrame.to_sql`
State: closed

- [x] closes #36922 
- [x] tests added / passed n/a (didn't add any new tests/change code)
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry

not sure if i need to add a whatsnew entry to 1.1.4 or 1.2.0...

would love to get a hacktoberfest tag if your org supports this. #36837 

Labels:
name: Docs
name: IO SQL
description: to_sql, read_sql, read_sql_query

Comments:
thanks @hkennyv
fixed capitalization on ecosystem.rst
State: closed

Fixed capitalization on doc/source/ecosystem.rst


Labels:

Comments:
Thanks but `pandas` is intentionally lowercase and was recently changed in https://github.com/pandas-dev/pandas/pull/36845

If further interested in contributing, please check out issues labeled with `good first issue`
Where do you think i got it from?
 #32550
./scripts/validate_rst_title_capitalization.py

doc/source/ecosystem.rst:83:Heading capitalization formatted incorrectly.
Looks like the `./scripts/validate_rst_title_capitalization.py` has not been modified to ignore capitalization of `pandas`. Would you like to submit a PR to modify that script?
Just to make sure,
both Pandas and pandas is fine?
It should always be referenced as lowercase `pandas`
BUG: RollingGroupby not respecting sort=False
State: closed

- [x] closes #36889
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry


Labels:
name: Bug
name: Window
description: rolling, ewma, expanding

Comments:
lgtm merge on green.
@meeseeksdev backport 1.1.x
BUG: IntervalArray.__eq__ not deferring to Series
State: closed

- [ ] closes #xxxx
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry


Labels:
name: Numeric Operations
description: Arithmetic, Comparison, and Logical operations
name: Interval
description: Interval data type

Comments:
This changes the result type of `interval_array == series`? We'll need a whatsnew for that (1.2).

Looks good otherwise.
conflict resolved, merge on green-ish
CLN: remove inplace kwarg from NDFrame._consolidate
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Refactor
description: Internal refactoring of code
name: Internals
description: Related to non-user accessible pandas implementation

Comments:
Thanks!
TYP: io.json._json, util._decorators
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
@jbrockmendel I'll review this shortly.

FYI I've started to create a branch that just tracks the outstanding mypy errors for untyped defs. Might help in resolving some more of these (and finding the easy to fix modules). https://github.com/pandas-dev/pandas/compare/master...simonjayhawkins:untyped-defs 
> FYI I've started to create a branch that just tracks the outstanding mypy errors for untyped defs

Neat.  I'm currently working on a branch that defines the relevant dunder methods on Index/Series/DataFrame directly instead of pinning them, which should address a bunch of these.
> > FYI I've started to create a branch that just tracks the outstanding mypy errors for untyped defs
> 
> Neat. I'm currently working on a branch that defines the relevant dunder methods on Index/Series/DataFrame directly instead of pinning them, which should address a bunch of these.

cool. There is quite a lot of the errors due to mypy 0.74 typing self. For now, to get the modules off the exclude list can use setattr since mypy doesn't complain about that.

The other issue with self being typed is the mixins.
Thanks @jbrockmendel
DEPR: Index.ravel returning an ndarray
State: closed

- [x] closes #19956
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry

Make Index.ravel() behavior match every other ravel() method in existence.

Labels:
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode
name: Deprecate
description: Functionality to remove in pandas
name: Index
description: Related to the Index class or subclasses

Comments:
nice!
@jbrockmendel I don't think there was consensus on changing `ravel` to return an Index. See the original discussion at https://github.com/pandas-dev/pandas/issues/19956, and you actually did a similar PR before (but to return the EA, https://github.com/pandas-dev/pandas/pull/32641) where I said the same.

- The Index method will now be inconsistent with the Series method
- What's the use case of actually needing an `Index.ravel` method?
> The Index method will now be inconsistent with the Series method

I would be OK changing the deprecation so to say the future behavior is to match self._values.ravel, matching Series behavior.

> What's the use case of actually needing an Index.ravel method?

The same as the use case for Series.ravel: you want to write code that is ndarray/index/series-agnostic
> I would be OK changing the deprecation so to say the future behavior is to match self._values.ravel, matching Series behavior.

IMO that sounds as a better option. Having consistency between Series and Index here would be good I think (although Index is more array-like than Series ..)

I am still skeptical about the need for this method at all, though (you can also do a ndim check if you eg want to ensure the array-like is 1D or converted to 1D, see eg Tom's comment at https://github.com/pandas-dev/pandas/issues/19956#issuecomment-433427374)
> I am still skeptical about the need for this method at all

as there is a top level numpy function for this https://numpy.org/doc/stable/reference/generated/numpy.ravel.html.

In the future, with \_\_array_function__ implemented, could allow much simplification of the pandas API.

so I _think_ OK to deprecate the method, in favour of np.ravel() support.

```
>>> pd.__version__
'1.2.0.dev0+765.g401d7a142c'
>>>
>>> idx = pd.Index([1, 2, 3])
>>>
>>> np.ravel(idx)
array([1, 2, 3], dtype=int64)
>>>
>>> type(_)
<class 'numpy.ndarray'>
>>>
```

so we could use np.ravel to get the current behaviour, of implement \_\_array_function__ if we wanted to return a Index instead.
Also for `np.ravel` support through `__array_function__`, we would need to decide on the return value, though (so I think a large part of the discussion applies to that as well). 

Because `np.ravel(index)` doesn't necessarily (technically) need to return an ndarray, it could also return an Index (as the warning in this PR proposes) or an EA.
BUG: Add trailing trailing newline in to_json
State: closed

- [x] closes #36888
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry


Labels:
name: Bug
name: IO JSON
description: read_json, to_json, json_normalize

Comments:
can you add a test for this
> can you add a test for this

@jbrockmendel Yes. Will do that
@jbrockmendel added a test for the bug. And also fixed the breaking tests.
cc @WillAyd
Can anyone of you complete the PR. @jbrockmendel @WillAyd . There will be again a lot of conflicts if delayed 👍 
thanks @Rohith295
regression fix for merging DF with datetime index with empty DF
State: closed

- [X ] closes https://github.com/pandas-dev/pandas/issues/36895
- [X ] tests added / passed
- [X ] passes `black pandas`
- [ X] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ X] whatsnew entry


Labels:
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode
name: Regression
description: Functionality that used to work in a prior pandas version

Comments:
Can you also add a test and a v1.1.4 release note?
needs test(s)
👍  
^^ These test failures seem unrelated and windows-dependent on a timezone feature
This looks great! This fix will help my team a great deal. 
^ These test failures look like unrelated timeouts 
Any updates on the availability of this bug-fix? 🙂 
well this hasn't even been reviewed yet 
@jreback Ready for your re-review. Thank you!
@jreback green!
thanks @PCerles very nice!
You're welcome, thanks for your help maintaining Pandas!
Single quotes
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:

Comments:
can you merge master
Is this identical to https://github.com/pandas-dev/pandas/pull/36878?
Thanks @skorani - the task here is to apply `blacken-docs`, not to find and replace single quotes with double quotes, let us know if you need help
@skorani - are you still interested in working on this? Otherwise we can close. Thanks
> @skorani - are you still interested in working on this? Otherwise we can close. Thanks

The relevant issue here has been closed, so I think safe to close this as well. Thanks anyways @skorani for the PR, there are other good first issues if you wanted to take a look at those.
ENH: allow non-consolidation in constructors
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry

cc @TomAugspurger I think something like this is what is needed for #34872.

If this were a "real" PR, I'd probably not put `consolidate` as a `DataFrame.__init__` kwarg but instead set it to `True` and only change it to `False` based on the `copy` kwarg.

The usage in core.ops is not necessary, but is an example of where we can get a perf bump.

Labels:

Comments:
closing to clear the queue, mostly wanted to park it here for reference in #34872
ti.csv
State: closed

pandas dataset for practice and understanding

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:

Comments:
thanks, but no thanks, see https://pandas.pydata.org/pandas-docs/stable/development/contributing.html for how to contribute
upgrade flake8 to 3.8.4 #36879
State: closed

closes #36879

Here are the logs after the upgrade to 3.8.4:

```console
(base) root@33d4c5d14a5b:/home/pandas-tazminia# pre-commit run flake8 --all
[INFO] Initializing environment for https://github.com/python/black.
[INFO] Initializing environment for https://gitlab.com/pycqa/flake8.
[INFO] Initializing environment for https://gitlab.com/pycqa/flake8:flake8-comprehensions>=3.1.0.
[INFO] Initializing environment for https://github.com/PyCQA/isort.
[INFO] Initializing environment for https://github.com/asottile/pyupgrade.
[INFO] Initializing environment for https://github.com/pre-commit/pygrep-hooks.
[INFO] Initializing environment for https://github.com/asottile/yesqa.
[INFO] Installing environment for https://gitlab.com/pycqa/flake8.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
[INFO] Installing environment for https://gitlab.com/pycqa/flake8.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
flake8...................................................................Passed
flake8-pyx...............................................................Passed
flake8-pxd...............................................................Passed
```

Labels:
name: Code Style
description: Code style, linting, code_checks

Comments:
This resolves #36879
Thanks @Tazminia ! pre-commit is the only CI check affected by this change, and that passes, so am merging

`isort` can probably also be upgraded if you fancy doing that
@MarcoGorelli, yeah I could do so, should I upgrade it to the latest available version ? 
Also, do you have any idea why this PR did not count on my hacktoberfest profile? all other PRs I have done so far counted.
we've not yet decided whether to take part in hacktoberfest

Yes - actually, there's a built-in pre-commit command you can run, ` pre-commit autoupdate`
re hacktoberfest - in the end we're marking individual PRs as accepted, so have done so here
New branch
State: closed

- [ ] ref #36777
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Docs

Comments:
Can you please change the name to a more reasonable one? I guess it is about using double quotes instead of single quotes in documentation.
@skorani Thanks for the PR. Can you merge master and remove the file you added? I suspect most of these things are already fixed.
Thanks @skorani , closing it looks like you've taken this forward in #36896
BUG: in DataFrame.reset_index() only call maybe_upcast_putmask with ndarrays
State: closed

- [x] closes #24206
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry


Labels:
name: Bug
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode
name: MultiIndex
name: Categorical
description: Categorical Data Type

Comments:
can you merge master and ping on green-ish
@jreback Green-ish (the failures are 6 tests from `pandas/tests/tools/test_to_datetime.py`)
@jreback all green now
thanks @arw2019 
Thanks @jreback for reviewing!
DOC: Fix GL01 docstring errors in some functions
State: closed

- [x] fixes #36874
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry

There are some GL01 pandas docstring errors in `shared_docs.py`.
```
GL01
Docstring text (summary) should start in the line immediately after the opening quotes (not in the same line, or leaving a blank line in between.
```

Labels:
name: Docs

Comments:
LGTM
thanks @Iqrar99
TST: xfailed arithmetic tests
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Timedelta
description: Timedelta data type

Comments:
can you add a whatsnew on this patch; this effectively enables some arithmetic on TDs that we couldn't before? or is this just going thru the correct path now?
whatsnew added.  this fixes `IntegerArray * timedelta`
DOC: Distinguish between different types of boolean indexing #10492
State: closed

- [x] closes #10492
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`


Labels:
name: Docs
name: Index
description: Related to the Index class or subclasses

Comments:
Thanks @arw2019 .
This was a very helpful review.  I made the changes accordingly.
would love to get a hacktoberfest tag if possible. #36837


thanks @junjunjunk