BrandonFors/pandas-github-issues huggingface dataset:
DOC: Remove absolute urls from the docs
State: closed

- [x] closes #32529
- [ ] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Docs
name: Clean

Comments:
Hi @jorisvandenbossche , I have updated my PR, it would be great if you can review once again. Thanks.
Hi @jorisvandenbossche , I have made the changes, it would be great if you can look into once again. Thanks. 
Sorry for the poor issue description from my side @ArkadeepAdhikari. Looks good, there is just one case that needs to be reverted.

Also, if you can add to `ci/code_checks.sh` a check to make sure the pattern `pandas.pydata.org` doesn't exist in `doc/source` anymore that would be very useful.
Hi @datapythonista, should I be adding the check to not keep `pandas.pydata.org` in `doc/source`? Some of the files like the one I would be reverting are in the same path (for example: `doc/source/whatsnew/v0.21.0.rst`)
Good point. Let's leave it for later, as we'll need to think whether it's worth validating it and how.
> Also, if you can add to ci/code_checks.sh a check to make sure the pattern pandas.pydata.org doesn't exist in doc/source anymore that would be very useful.

Since there are actually cases where we have this url in doc/source (several ones we reverted here in the PR), and given that there were not that much occurences to fix, I personally don't think it is worth the effort to make a code check for this
Hi @datapythonista, @jorisvandenbossche, thanks for the clarifications. I have reverted the final case. Please take a look again. Thanks.
Hi @datapythonista, I have updated my PR, please feel free to review once again.
Thanks @ArkadeepAdhikari !
Thanks for reviewing @jorisvandenbossche, @datapythonista!
ENH: IntegerArray.astype(dt64)
State: closed

- [x] closes #32435
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry

Let's us de-kludge to_datetime code, getting rid of another _ndarray_values usage.

Labels:
name: Dtype Conversions
description: Unexpected or buggy dtype conversions
name: ExtensionArray
description: Extending pandas with custom dtypes or arrays.

Comments:
By my count, after today's slew of values_from_object/internal_get_values PRs, we'll have

- 9 `_ndarray_values` usages
- 3 `values_from_object` usages
- 10 `_internal_get_values` usages
- 1 `Block.to_dense`

(not counting usages in tests, docstrings, or in the each other's implementations)
thanks.

can you add a release note in a followon. (I guess this is a bug fix)
CLN: avoid values_from_object in reshape.merge
State: closed


Labels:
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode
name: Clean

Comments:
> i am thinking this actually enables some cases? 

not sure i follow.  can you give an example?
> > i am thinking this actually enables some cases?
> 
> not sure i follow. can you give an example?

are there any merge cases that work that didn't previously?
> are there any merge cases that work that didn't previously?

there are some dtype combinations that arent tested that looked sketchy.  there's a TODO comment about adding tests for them
> > are there any merge cases that work that didn't previously?
> 
> there are some dtype combinations that arent tested that looked sketchy. there's a TODO comment about adding tests for them

kk
lgtm. merge o ngreen.
hmm, something broke
updated+green
Thanks @jbrockmendel
CLN: remove unnecessary to_dense call
State: closed

This call is not reached in the tests, and if it were the following line would make it superfluous anyway.

I'm pretty sure the comment there is wrong too.

Labels:
name: Clean

Comments:
> This call is not reached in the tests

CategoricalArray and SparseArray have a to_dense methods and therefore hasattr(values, "to_dense") would be True.

However, `format_array` would use ExtensionArrayFormatter and not FloatArrayFormatter for these so hasattr(values, "to_dense") should always be False in FloatArrayFormatter.

ExtensionArrayFormatter dispatches back to format_array with a numpy array. 
Thanks @jbrockmendel
TYP: Add type hint for DataFrame.T and certain array types
State: closed

While updating a large pandas codebase with type coverage to 1.0, I noticed that `DataFrame.transpose()` is annotated to return `DataFrame` while `DataFrame.T` has no such hint. This PR also adds hints in a few places other places where they're trivial.

The definition of `pandas.core.base.IndexOpsMixIn.transpose` (and associated `.T`) currently do not have any type hints and fixing this seems more complicated, meaning `Index.T` and `Series.T` are not fixed by this PR.

Happy to amend to cover that case if anyone has suggestions, but I believe `DataFrame.T` to be the vast majority of usage.

- [ ] closes #xxxx
- [ ] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
thanks @qwhelan
DOC: Update pandas.core.groupby.GroupBy.pipe docstring
State: closed

- [x] closes https://github.com/pandanistas/pandanistas_sprint_ui2020/issues/2
- [ ] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Docs

Comments:
Conflicts and stale, closing. @hsjsjsj009 let me know if you want to keep working on this, or feel free to open a new PR with the required changes.
CI: Update web and docs to OVH with the right structure
State: closed

- [X] closes #32303

I'm uploading all the docs from the NumFOCUS server to the OVH (manually). And I'm changing the CI in this PR to upload the website and the dev docs to the server automatically.

I'll have a look and see if we can upload a new version of the docs automatically to OVH when a new GitHub release is generated. So, we never need to upload the website manually.

After merging this PR, and when we're confident this is working well, we'll have to redirect pandas.pydata.org to the OVH server. The dev docs will be available at pandas.pydata.org/pandas-docs/dev/ (and we can stop using pandas.io and dev.pandas.io).

Labels:
name: Docs
name: CI
description: Continuous Integration
name: Web
description: pandas website

Comments:
> After merging this PR, and when we're confident this is working well, we'll have to redirect pandas.pydata.org to the OVH server. 

Did this already happen? 

And are the dev docs already available on one of the urls? https://dev.pandas.io/docs/ is not updated anymore, https://pandas.io/docs/ also not anymore since last week (when this was merged?), but https://pandas.pydata.org/pandas-docs/dev still points to the old travis-built github-hosted docs url (alendar.google.com/)
> The dev docs will be available at pandas.pydata.org/pandas-docs/dev/ 

Also, could we choose pandas.pydata.org/docs/dev/ instead (a tiny bit shorter)? That url basically never existed (it was always directly redirected), so I think we don't necessarily need to keep it (it could still redirect to the new place)
CLN: simplify get_values usage in groupby
State: closed

This will avoid an object-casting for e.g. IntervalArray-backed ExtensionBlock.

Labels:
name: Groupby
name: Clean

Comments:
Or if overkill just a quick timeit 
> Or if overkill just a quick timeit

```
ts = pd.Timestamp("2016-01-01", tz="UTC")
ii = pd.interval_range(start=ts, end=ts + pd.Timedelta(days=10**4))       
df = pd.DataFrame({"A": ii.right, "B": ii.left, "C": ii, "D": ii.left.year})
gb = df.groupby("D")

In [5]: %timeit gb.count()                                                                                                                                                                            
378 µs ± 6.23 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)  # <-- PR
211 ms ± 18.4 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)  # <-- master
```
CLN: remove SingleBlockManager.get_values
State: closed

It is only used in one place, and its behavior doesn't match `self.blocks[0].get_values()`, which is counter-intuitive.

Labels:
name: Internals
description: Related to non-user accessible pandas implementation
name: Clean

Comments:
@datapythonista do you think it would be viable to make the docbuild output less verbose to make it easier to see what is causing a failure?
> @datapythonista do you think it would be viable to make the docbuild output less verbose to make it easier to see what is causing a failure?

Not sure if it's possible to have more control, but may be we can call it with `-q`. Those are the options I see:
```
console output options:
  -v                increase verbosity (can be repeated)
  -q                no output on stdout, just warnings on stderr
  -Q                no output at all, not even warnings
```
I think it can make sense, I don't think there is anything in the output that I find that useful (surely not all the "reading" and "writing" for the progress). But @jorisvandenbossche should know better if that makes sense. 
If we want that, someone should investigate if all errors we want to see still show up (eg the output of ipython directive might be put on stdout).

But @jbrockmendel, for this specific PR, what was failing in the doc build? (I only see a failure to checkout in the pipeline)
> for this specific PR, what was failing in the doc build?

I never found out.  AFAICT it was unrelated and eventually re-pushing solved it.
> But @jbrockmendel, for this specific PR, what was failing in the doc build? (I only see a failure to checkout in the pipeline)

As far as I looked, the doc build actually didn't fail. It was the checkout phase of the the action that failed
> As far as I looked, the doc build actually didn't fail. It was the checkout phase of the the action that failed

Looking in the logs I see the same thing, maybe past-me intended to ask about this on a different PR.
CLN: removed unused import
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry

---

Not 100% sure about this, can close anytime if not a good PR

Labels:
name: Clean

Comments:
I also benched marked it:

master:

```
In [1]: import pandas as pd                                                                                   

In [2]: %timeit pd.Period("2000-01-01", freq="D")                                                             
165 µs ± 777 ns per loop (mean ± std. dev. of 7 runs, 10000 loops each)
```

---

PR:

```
In [1]: import pandas as pd                                                                                   

In [2]: %timeit pd.Period("2000-01-01", freq="D")                                                             
166 µs ± 449 ns per loop (mean ± std. dev. of 7 runs, 10000 loops each)
```
Why is isort not complaining with the original code? Is it validating that file?
> Why is isort not complaining with the original code? Is it validating that file?

@datapythonista ```isort``` is not checking that file ATM, in the release of ```isort``` 5.0.0 it will begin to support ```.pyx``` files.

I think you ment ```flake8```, and I don't know why it's not catching it (or if it's even checking```.pyx``` files)
@datapythonista @jbrockmendel any outstanding objections?
lgtm
CLN: Suppres compile warnings of pandas/io/sas/sas.pyx
State: closed

- [x] ref #32163
- [ ] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry

---

This is the warning that get removed:

```
pandas/io/sas/sas.c: In function ‘__pyx_f_6pandas_2io_3sas_4_sas_rdc_decompress’:
pandas/io/sas/sas.c:3732:65: warning: ‘__pyx_v_ctrl_bits’ may be used uninitialized in this function [-Wmaybe-uninitialized]
 3732 |     __pyx_t_8 = (((__pyx_v_ctrl_bits & __pyx_v_ctrl_mask) == 0) != 0);
      |                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~
```

Labels:
name: Build
description: Library building on various platforms

Comments:
If you wanted to look at #32163 and have a way to get stricter on warnings without solving all at once would certainly welcome it. Achievable through various GCC flags I think
@WillAyd You think it's possible to have the ```-Werror``` flag only for specific files? (to prevent regression to fixed files)
i was thinking individual warnings but sure to individual files if not overly complicated. There’s only a few left outside of that PR

Sent from my iPhone

> On Mar 7, 2020, at 12:11 PM, MomIsBestFriend <notifications@github.com> wrote:
> 
> ﻿
> @WillAyd You think it's possible to have the -Werror flag only for specific files? (to prevent regression to fixed files)
> 
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub, or unsubscribe.

Thanks @MomIsBestFriend
Deprecate Aliases as orient Argument in DataFrame.to_dict
State: closed

- [ x] closes #32515
- [x ] tests added / passed
- [x ] passes `black pandas`
- [x ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x ] whatsnew entry

Labels:
name: Deprecate
description: Functionality to remove in pandas
name: DataFrame
description: DataFrame data structure

Comments:
On board with the change but I think should wait until 2.0 since this could break things (though yes undocumented)
Should deprecate for 1.1
I see that in c598b32 you reverted some code, ```pytest``` handles DeprecationWarning a bit different than the usual

```python
with pytest.raises(Exception):
    foo()
```

You can read more about how ```pytest``` deals with DeprecationWarning at https://docs.pytest.org/en/latest/warnings.html#ensuring-code-triggers-a-deprecation-warning

---

To sum it up, you catch DeprecationWarning by:

```python

with pytest.deprecated_call():
    foo()
```
@MomIsBestFriend Thanks for the patient review! :clap: 
I've tried to add 
```
- :meth:`DataFrame.to_dict` will not accept short names for ``orient`` in future versions (:issue:`32515`)
```

into the deprecation section of the what's new 1.1.0 and the tests keep failing...
I've removed it.
> I've tried to add
> 
> ```
> - :meth:`DataFrame.to_dict` will not accept short names for ``orient`` in future versions (:issue:`32515`)
> ```
> 
> into the deprecation section of the what's new 1.1.0 and the tests keep failing...
> I've removed it.

The error was just that you had trailing whitespace at the end of the line. Note looks good so if you add back in and delete trailing space should work
/azp run
<samp>
Azure Pipelines successfully started running 1 pipeline(s).<br>

</samp>
Thanks @elmonsomiat - very impressive first contribution
Just reading through some PRs to understand the contribution process better. I see this sentence in the documentation "Abbreviations are allowed. s indicates series and sp indicates split." Is a corresponding doc change needed for this? 

https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.to_dict.html
Oh, yes, this sentence should be removed. This is not the case for `to_json` and I think they should be consistent.
CLN: remove check_series_type
State: closed

Trying to clean up assert_series_equal and avoid usage of _internal_get_values; I'm finding that these functions are _weird_, and in particular categorical dtype handling is opaque

Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Clean

Comments:
this is technically an api change, but not worth leaving it around, but can you add a note in the whatsnew
`assert_series_equal` is a public function, so we should not just remove keyword arguments, I think.

(this broke geopandas CI)
And for subclasses, this keyword can actually be useful. What was the reason you wanted to remove it?
> What was the reason you wanted to remove it?

I'm trying to get rid of the `_internal_get_values` usages in `assert_series_equal` and finding the possible parameter combinations make that difficult.  Since this one was unused internally, it seemed like a good candidate.  If you want to re-introduce it, I have no objection.
How is `_internal_get_values` and `assert_series_equal` related? That is only about checking the class type, but it's not related to the dtype?
> How is _internal_get_values and assert_series_equal related?

_internal_get_values is called a couple of times in assert_series_equal (more before this PR).  Replacing the _internal_get_values usages with e.g. _values is the goal, but it turns out that breaks sometimes when we have Categorical _values.  assert_series_equal has several parameters that change the behavior when we have categorical, and I was trying to narrow those down (see also #32527)
Sorry, that still doesn't explain to me the relation to `check_series_type`, which basically only is this part of the diff:

```patch
-    if check_series_type:
-        # ToDo: There are some tests using rhs is sparse
-        # lhs is dense. Should use assert_class_equal in future
-        assert isinstance(left, type(right))
-        # assert_class_equal(left, right, obj=obj)
+    # TODO: There are some tests using rhs is sparse
+    # lhs is dense. Should use assert_class_equal in future
+    assert isinstance(left, type(right))
+    # assert_class_equal(left, right, obj=obj)
```

so how is that related to (or required by) the `_internal_get_values` changes?


> that still doesn't explain to me the relation to check_series_type

It isnt related, check_series_type was an innocent bystander that happens to be unused internally.  Again, I have no objection if you want to restore it.
Well, having that the title of the PR didn't give the impression of being an innocent bystander .. ;) (also your "I'm trying to get rid of the `_internal_get_values` usages in assert_series_equal and finding the possible parameter combinations make that difficult." seemed to indicate this was related, hence my confusion).
[BUG] add consistency to_numeric on empty list
State: closed

to_numeric should work similarly on empty lists for
downcast=unsigned/float/integer

Addresses: GH32493

- [x] closes #32493
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry


Labels:
name: Dtype Conversions
description: Unexpected or buggy dtype conversions
name: Numeric Operations
description: Arithmetic, Comparison, and Logical operations

Comments:
thanks @mikekutzma
[BUG] Add int-array-like into random state
State: closed

- [x] closes #32503
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [] whatsnew entry


Labels:

Comments:
Hi, I am not sure the cause of docs build failed. My local build seems good
Apologies @xcz011, but it looks like https://github.com/pandas-dev/pandas/pull/32510 was made slightly before yours.
CLN: Avoid bare pytest.raises in computation/test_eval.py 
State: closed

* [x]  ref #30999
 
* [x]  tests added / passed
 
* [x]  passes `black pandas`

* [x]  passes `git diff origin/master -u -- "*.py" | flake8 --diff`

Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Clean

Comments:
Thanks @Vlek
(wip) BUG: groupby with sort=False create buggy multiindex
State: closed

- [ ] closes #32259 
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry

Still messy, just running this through the test suite as my laptop's not great (but please let me know if such behaviour is unwelcome and I won't do it again)

pandas/tests/groupby/test_timegrouper.py
pandas/tests/groupby/test_grouping.py
pandas/tests/groupby/test_categorical.py
pandas/tests/groupby/test_groupby.py
pandas/tests/test_multilevel.py

Labels:

Comments:
This is a tough one, I'll come back to it later
Disallow lossy SparseArray conversion
State: closed

cc @TomAugspurger 

Labels:
name: Sparse
description: Sparse Data Type
name: Warnings
description: Warnings that appear or should be added to pandas

Comments:
> We should likely detect that, warn, and ask users to convert before passing to SparseArray.

This gets hairy in a way that im not sure is worth it.  In particular for tzaware, if instead of raising we fall through to `data = np.array(data)`, we get object-dtype, and fill_value of np.nan instead of NaT.
Updated to warn and maintain current behavior.
thanks, pls add a whatsnew as a followon
Better error message for OOB result
State: closed

Closes #31774


Labels:
name: Error Reporting
description: Incorrect or improved errors from pandas

Comments:
LGTM
The doc build failure is unrelated:

```
2020-03-09T21:49:05.9460052Z WARNING: failed to reach any of the inventories with the following issues:
2020-03-09T21:49:05.9461222Z intersphinx inventory 'https://pandas-gbq.readthedocs.io/en/latest/objects.inv' not fetchable due to <class 'requests.exceptions.HTTPError'>: 502 Server Error: Bad Gateway for url: https://pandas-gbq.readthedocs.io/en/latest/objects.inv
2020-03-09T21:49:09.4756947Z WARNING: failed to reach any of the inventories with the following issues:
2020-03-09T21:49:09.4758680Z intersphinx inventory 'https://dateutil.readthedocs.io/en/latest/objects.inv' not fetchable due to <class 'requests.exceptions.HTTPError'>: 502 Server Error: Bad Gateway for url: https://dateutil.readthedocs.io/en/latest/objects.inv
```
very nice @TomAugspurger