BrandonFors/pandas-github-issues huggingface dataset:
CI: Checks job aborted if a step fails GH30298
State: closed

- [ ] closes #30298
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: CI
description: Continuous Integration

Comments:
Hello @suzutomato! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:

* In the file [`pandas/io/common.py`](https://github.com/pandas-dev/pandas/blob/4a3854358b31f8f1321bb9510ab81e0b11d12dea/pandas/io/common.py):

> [Line 84:6](https://github.com/pandas-dev/pandas/blob/4a3854358b31f8f1321bb9510ab81e0b11d12dea/pandas/io/common.py#L84): [E111](https://duckduckgo.com/?q=pep8%20E111) indentation is not a multiple of four



##### Comment last updated at 2019-12-25 07:43:48 UTC
lgtm @datapythonista 
The intentional errors have been removed.
As per the last comment from @datapythonista, I won't take action for the problem with the benchmarks artifact at this moment.
Let me know if anything I can do to solve the problem.
I guess the CI errors are unrelated, can you merge master into your branch to see if the errors get fixed please?

```
git fetch ustream
git merge upsream/master
git push
```
@suzutomato mind finishing this whenever you have a second - definitely looking forward to this feature
So sorry, I overlooked @atapythonista's last message.
Working on it now.
LGTM - sure improvement

Side note maybe we really want ``if: success() || failure()``. As the changes in this PR will continue to run the workflow even if it has been cancelled as per:
https://help.github.com/en/actions/automating-your-workflow-with-github-actions/contexts-and-expression-syntax-for-github-actions.

Assume we are currently using the free plan? https://help.github.com/en/actions/automating-your-workflow-with-github-actions/about-github-actions#usage-limits

@datapythonista 
I'll give a try to `success() || failure()`, I think it's what we actually want
ENH: meth 'DataFrame.to_pickle' and func 'read_pickle' to accept URL GH#30163
State: closed

- [ ] closes #30163
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry

Labels:
name: IO Pickle
description: read_pickle, to_pickle

Comments:
@suzutomato i see multiple PRs that you are opening and then closing
don’t open new ones if u make a change, rather push to the same one

this needs tests
It feels like getting pickles over networks is antithetical to their recommended use due to the security implications. 
Sorry for the mess I've made, as well as the delay on this PR.
Tests were added.

2 Azure pipelines were failed/canceled due to timeout. I'll push agin later if that's the best way to rerun them.
Thank you,
can you merge master and ping on green.
@jreback 
Thank you for the comment, updated and merged. 
Now all is green. ptal.
Hello @suzutomato! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:







There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2020-01-09 12:17:29 UTC
@jreback Thank you for review. Updated accordingly, please take a look. Let me know if I misunderstood anything.
> It feels like getting pickles over networks is antithetical to their recommended use due to the security implications.

That was my initial reaction too. Do we want to encourage this?
> > It feels like getting pickles over networks is antithetical to their recommended use due to the security implications.
> 
> That was my initial reaction too. Do we want to encourage this?

this is a pretty common pattern actually;
security is out of scope as this is up to the user

Fair enough. We do have a warning in the read_pickle docs. May want to re-emphasize that in the `filepath_or_buffer` section.
Fixed the merge conflict. Good to merge when CI passes.
thanks @suzutomato very nice!
Thank you @jreback and @TomAugspurger for guidance and your patience!
It was fun!
JSON Support for parsing NaN, Infinity and -Infinity
State: closed

- [X] closes #12213
- [X] tests added / passed
- [X] passes `black pandas`
- [X] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [X] whatsnew entry

Not sure if we totally want this but figured I'd give it a shot any way. `simplejson` and the stdlib by comparison would decode these values


Labels:
name: IO JSON
description: read_json, to_json, json_normalize

Comments:
Ran benchmarks just to be safe and no changes appeared
Great thanks Jeff
CLN: .format to f-string formatting in various files
State: closed

- [ ] ref https://github.com/pandas-dev/pandas/issues/29547
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Code Style
description: Code style, linting, code_checks

Comments:
@kpmccahill : Thanks for opening this PR!  And welcome to pandas! 🚀 

The changes look pretty good overall.  I have one question about them that's directed towards fellow maintainers as you can see above.
@gfyoung 
> I have one question about them that's directed towards fellow maintainers as you can see above.

Thanks! And I'm gonna be honest I wasn't too sure about that one myself for the same reason you mentioned. But I thought I'd give it a go anyways just to see.

Thanks @kpmccahill !
test CI
State: closed

Debugging an issue noted in https://github.com/pandas-dev/pandas/pull/30139 and maybe in https://github.com/pandas-dev/pandas/pull/30245

Labels:

Comments:
Hopefully fixed by https://github.com/pandas-dev/pandas/pull/30293/commits/c31ee5adaf4034b0549e2e26e93f0702ded75678. Will leave the lint failure in just to check.
Good. Failed with

```
3.7.9 (flake8-comprehensions: 3.1.4, mccabe: 0.6.1, pycodestyle: 2.5.0, pyflakes: 2.1.1) CPython 3.7.3 on Linux
Linting .py code
##[error]./pandas/core/arrays/sparse/array.py:4:1:F401:'unittest' imported but unused
Linting .py code DONE
```
CLN: changed .format to f-string in pandas/core/dtypes
State: closed

- [ ] ref #29547
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Code Style
description: Code style, linting, code_checks

Comments:
@DorAmram Thanks for the PR. There are lint failures. Can you run `black pandas`, see https://dev.pandas.io/docs/development/contributing.html#python-pep8-black
Still needs `black pandas/core/dtypes/common.py` to get the linter to be happy
Thanks @DorAmram!
ENH: Add StataWriter 118 for unicode support
State: closed

Add StataWriter with unicode support

- [X] closes #23573
- [X] tests added / passed
- [X] passes `black pandas`
- [X] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry


Labels:
name: Enhancement
name: IO Stata
description: read_stata, to_stata

Comments:
@kshedden can you double-check this?  It looks fine to me, but I think you're better-informed on this than I am.
Sorry, I'm not familiar at all with the writers.
Hello @bashtage! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:







There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2019-12-31 16:38:17 UTC
scikit-learn related failure
LGTM.  @TomAugspurger are we letting the sklearn thing in travis block merging?
Nope, it's OK to ignore.
CI failure looked unrelated.

```
________________ TestRegistration.test_registering_no_warning _________________
[gw1] win32 -- Python 3.7.5 C:\Miniconda\envs\pandas-dev\python.exe

self = <pandas.tests.plotting.test_converter.TestRegistration object at 0x000002411CB02788>

    def test_registering_no_warning(self):
        plt = pytest.importorskip("matplotlib.pyplot")
        s = Series(range(12), index=date_range("2017", periods=12))
        _, ax = plt.subplots()
    
        # Set to the "warn" state, in case this isn't the first test run
        register_matplotlib_converters()
        with tm.assert_produces_warning(None) as w:
>           ax.plot(s.index, s.values)
```

Merged master to see if its fixed.
CI failure again looks unrelated (and azure won't let me re-run).  merged master to force re-run.
@jreback @jbrockmendel green. 
thanks @bashtage
BUG: DataFrame constructor raised ValueError with list-like data and …
State: closed

…dtype specified

- [ ] closes #30280
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: ExtensionArray
description: Extending pandas with custom dtypes or arrays.
name: Constructors
description: Series/DataFrame/Index/pd.array Constructors

Comments:
That was fast, thanks!
thanks @simonjayhawkins
Replaced void pointers with Types in JSON Datetime Conversions
State: closed

This also incorporates the clean up from #30271 

Benchmarks show the following benefit, though most likely just noise:

```sh
.       before           after         ratio
     [04fce81f]       [f8ed75d0]
     <stash~6^2>       <json-dates>
-         336±9ms          302±4ms     0.90  io.json.ToJSON.time_to_json('columns', 'df_int_floats')

SOME BENCHMARKS HAVE CHANGED SIGNIFICANTLY.
PERFORMANCE INCREASED.
```

One follow up to this can drastically speed up datetime index handling - right now these are done in the Python space but can reuse these same functions

Labels:
name: IO JSON
description: read_json, to_json, json_normalize

Comments:
With usual caveat about my C-fu being weak, LGTM.  cc @jreback
@jreback gentle ping, just realized this is a blocker for #28595.
thanks @WillAyd looks more clear
Unhide plt, beginners don't know how to import it.
State: closed

Alter https://github.com/pandas-dev/pandas/pull/30274.

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Docs

Comments:
pls don’t make multiple PRs for the same issue
instead just update the original
STY: Replace .format() to fstrings
State: closed

- [ ] closes #xxxx
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry

#29547

Labels:
name: Code Style
description: Code style, linting, code_checks

Comments:
@EydenVillanueva Thanks for the PR. can you merge master to resolve the merge conflicts.
> @EydenVillanueva Thanks for the PR. can you merge master to resolve the merge conflicts.

Done 👍 
> @EydenVillanueva IIUC there should only be 3 or 4 files changed?

Yes, you're right. It should be fine now. 
I'll keep checking in case you find another problem
Thanks @EydenVillanueva
[bug] don't remove timezone-awareness when using the  method from Dat…
State: closed

…aFrame

- [x] closes #30238
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry


Labels:
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode
name: Dtype Conversions
description: Unexpected or buggy dtype conversions

Comments:
> All of the column stuff is not necessary (I could be wrong but give this a try)

@jreback Tried this, but it seems it's necessary for `pandas/tests/reshape/test_concat.py::TestAppend::test_append_same_columns_type`
@MarcoGorelli this change solved the original issue. Maybe give this a shot:

```
(pandas-dev) matthewroeschke:pandas-mroeschke matthewroeschke$ git diff
diff --git a/pandas/core/frame.py b/pandas/core/frame.py
index 766437dba..62391b7cc 100644
--- a/pandas/core/frame.py
+++ b/pandas/core/frame.py
@@ -6771,7 +6771,7 @@ class DataFrame(NDFrame):
                 combined_columns = self.columns.astype(object).append(idx_diff)
             other = other.reindex(combined_columns, copy=False)
             other = DataFrame(
-                other.values.reshape((1, len(other))),
+                other._values,
                 index=index,
                 columns=combined_columns,
             )
```
@mroeschke I would still need to reshape though, wouldn't I? Else I don't see how
```
pandas/tests/frame/test_combine_concat.py::TestDataFrameConcatCommon::test_append_series_dict
```
would pass.

But if I do reshape, then `pandas/tests/frame/test_combine_concat.py::TestDataFrameConcatCommon::test_append_timestamps_aware_or_naive` fails with
```
AttributeError: 'DatetimeArray' object has no attribute 'reshape'
```

And I can't just pass the `dtype` directly to the constructor, else I'm blocked by #30428 
Extension arrays (DatetimeArray) are still essentially 1D only right now, so reshaping is not possible. As you can see in the `test_append_series_dict` it's failing the `verify_integrety=True` test which I think are not really well integrated with extension arrays.
@mroeschke Thanks for your explanation. What would you like me to do then, use `._value` and change `test_append_series_dict`, or keep as is?
@jreback thanks for your review, I've made those changes.

I've opened up another issue for the other dtypes, will work on that next week if no one else has taken it up by then
thanks!
CLN: changed .format to f-string in pandas/core/indexes
State: closed

towards #29547

- [ ] closes  
- [x] tests passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Code Style
description: Code style, linting, code_checks

Comments:
Thanks, @makeajourney . Welcome to take more of these.
Simplified objToJSON signatures
State: closed

Pre-cursor to a change that splits datetime functions that return `char *` off from datetime functions that return int.  Right now these are intermixed in functions that return void pointers, which makes type checking and grokking more difficult than they need to be

Labels:
name: IO JSON
description: read_json, to_json, json_normalize

Comments:
Hmm I don't think there's a way to make this intermediate step work so closing
Nothing more eyeballs could do to help?  This looked like a nice cleanup.
I think its just an architectural limitation trying to return a 64 bit integer from a `void *` which I think is 32 bits on a 32 bit platform, but if you see something different for sure
ENH: pass through schema keyword in to_parquet for pyarrow
State: closed

- [X] closes #30189 
- [x] tests added / passed
- [X] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry


Labels:
name: IO Parquet
description: parquet, feather

Comments:
Added! I'm a first time contributor to pandas - let me know if that whatsnew entry was appropriate.
@jreback I merged master but it doesn't look like the build failures are a result of changes from this PR. 
Fixing that in another PR. You can ignore it for now.
Awesome, let me know if there's anything else we need in here.
Thanks a lot @ieaves !
Made date time sources usage more explicit in setup.py
State: closed

Not sure this fully solves #30236 but doesn't hurt and makes things more explicit

Labels:
name: CI
description: Continuous Integration

Comments:
Hello @WillAyd! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:



There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2019-12-13 22:11:36 UTC
hmm i am always leary of changing things like this, last time we did things didn't recompile properly locally. meaning if you then *just* change something that was formerly a source (but now only a depends) it didn't recompile unless you clean.

@jbrockmendel can you test out.


I agree with both @WillAyd that this looks correct and is more explicit and @jreback that in the past this has been a fragile minefield.  On the margin I'm inclined to trust Will on this as it is easy to revert if problems pop up.
Cool thanks both for the feedback. Just to clarify on this one I'm just removing sources that aren't used by the module at all, not necessarily converting from `sources` -> `depends`

The problem I'm trying to address is that with parallel builds everything in sources gets recompiled when specified as such, and I think race conditions could make things fail

This doesn't fully solve the recompilation problem but just removes the compilation step from modules that don't reference those C extensions at all, whether directly or through import of another module
Any objection to rolling with this for now and reverting if issues pop up? Have it as a mini goal to get parallelized builds which I think this is a pre-cursor of
+1
Cool I'll merge for now and obviously can revert if anything pops up

cc @pandas-dev/pandas-core for visibility
WIP/ERR: raise consistent errors for non-existent files
State: closed

- [x] closes #29125
- [x] closes #13872
- [ ] closes #17918  <-- needs tests and double-checking
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry

xref #29102 the json case is xfailed here.


Labels:
name: Error Reporting
description: Incorrect or improved errors from pandas

Comments:
> reinventing the wheel

If there's an existing wheel we can use, I'm all for that.

Going through the issues, I'm going to try to round up all of the "should we accept strings too?" etc issues so we can at least discuss them all at once.

Also happy to break parts of this off if anything can be identified as "easy"
Closing, will re-implement in smaller pieces.
DOC: .get_slice_bound in MultiIndex needs documentation.
State: closed

- [x] closes #29967
- [ ] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry

I'm not good at English. So if there are wrong sentences or awkward sentences, please let me know.


Labels:
name: Docs
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
Hello @proost! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:



There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2019-12-23 09:43:50 UTC
thanks @proost
Make resample documentation better
State: closed

I removed the checkboxes because they were not relevant. This is a docstring update. I found the answer in https://stackoverflow.com/questions/17001389/pandas-resample-documentation but it should have been present in the docs

Labels:
name: Docs

Comments:
Hello @anshbansal! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:



There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2019-12-13 04:57:03 UTC
@anshbansal still interested on finishing this? If so, please remove the line that doesn't add value and create a link to the mentioned page/section. Thanks
@jreback  @datapythonista Can you tell me what link is being referred to in the previous comment?
Closing as stale, @anshbansal let us know if you'd like to finish this.
DEPR: change DataFrame.append default sort kwarg
State: closed

xref #20613

AFAICT this doesn't break any tests, was this supposed to have been changed along with the default in `concat` #29786 ?

Labels:
name: Deprecate
description: Functionality to remove in pandas

Comments:
> xref #20613
> 
> AFAICT this doesn't break any tests, was this supposed to have been changed along with the default in `concat` #29786 ?

IIRC this was changed *already* for 1.0, so *may* need to delay this
> IIRC this was changed already for 1.0, so may need to delay this

The last time the relevant part of the docstring was changed was in #20613, the same PR that deprecated the old pd.concat default behavior.
thanks @jbrockmendel must have been thinking about something else
CLN: more consistent error message for ExtensionDtype.construct_from_string
State: closed

message check added to `test_construct_from_string` in `pandas/tests/extension/base/dtype.py` and error messages changed to be consistent with error message on class method in `pandas/core/dtypes/base.py` xref Provide ExtensionDtype.construct_from_string by default #26562

Labels:
name: Error Reporting
description: Incorrect or improved errors from pandas

Comments:
thanks!
ERR: stringify error message from parsers.pyx, closes #29233
State: closed

- [x] closes #29233
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Error Reporting
description: Incorrect or improved errors from pandas
name: IO CSV
description: read_csv, to_csv

Comments:
thanks @jbrockmendel
TST: Add message checks to tests/arrays/sparse/
State: closed

- [ ] xref #23922
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Testing
description: pandas testing functions or related to the test suite

Comments:
LGTM pending green
thanks @simonjayhawkins
TST: Add message checks to tests/arrays/categorical/
State: closed

- [ ] xref #23922
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Testing
description: pandas testing functions or related to the test suite

Comments:
thanks @simonjayhawkins