FIX: add support for desc order when ranking infs with nans #19538
State: closed

Please include the output of the validation script below between the "```" ticks:

```
################################################################################
################ Docstring (pandas._libs.algos.rank_1d_object)  ################
################################################################################

Fast NaN-friendly version of scipy.stats.rankdata

################################################################################
################################## Validation ##################################
################################################################################

Errors found:
        Summary does not end with dot
        No extended summary found
        No returns section found
        See Also section not found
        No examples section found
(pandas_dev)

```

Checklist for other PRs (remove this part if you are doing a PR for the pandas documentation sprint):

- [x] closes #19538
- [x] tests added / passed
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry


Labels:
name: Bug
name: Numeric Operations
description: Arithmetic, Comparison, and Logical operations

Comments:
Hello @peterpanmj! Thanks for updating the PR.



Cheers ! There are no PEP8 issues in this Pull Request. :beers: 

##### Comment last updated on March 25, 2018 at 10:11 Hours UTC
# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/20091?src=pr&el=h1) Report
> Merging [#20091](https://codecov.io/gh/pandas-dev/pandas/pull/20091?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/52cffa3b3b2a510c30ed7f8cc8525c03d62e9130?src=pr&el=desc) will **decrease** coverage by `0.02%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/20091/graphs/tree.svg?height=150&width=650&src=pr&token=eZ4WkYLtcO)](https://codecov.io/gh/pandas-dev/pandas/pull/20091?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #20091      +/-   ##
==========================================
- Coverage   91.72%    91.7%   -0.03%     
==========================================
  Files         150      150              
  Lines       49149    49149              
==========================================
- Hits        45083    45071      -12     
- Misses       4066     4078      +12
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `90.08% <ø> (-0.03%)` | :arrow_down: |
| #single | `41.85% <ø> (ø)` | :arrow_up: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/20091?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/plotting/\_converter.py](https://codecov.io/gh/pandas-dev/pandas/pull/20091/diff?src=pr&el=tree#diff-cGFuZGFzL3Bsb3R0aW5nL19jb252ZXJ0ZXIucHk=) | `65.07% <0%> (-1.74%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/20091?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/20091?src=pr&el=footer). Last update [52cffa3...1622515](https://codecov.io/gh/pandas-dev/pandas/pull/20091?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/20091?src=pr&el=h1) Report
> Merging [#20091](https://codecov.io/gh/pandas-dev/pandas/pull/20091?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/63a662df7b599a45e8a7f704334a7b3eec9a5557?src=pr&el=desc) will **decrease** coverage by `0.01%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/20091/graphs/tree.svg?token=eZ4WkYLtcO&height=150&width=650&src=pr)](https://codecov.io/gh/pandas-dev/pandas/pull/20091?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #20091      +/-   ##
==========================================
- Coverage   91.84%   91.82%   -0.02%     
==========================================
  Files         152      152              
  Lines       49259    49249      -10     
==========================================
- Hits        45241    45225      -16     
- Misses       4018     4024       +6
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `90.21% <0%> (-0.02%)` | :arrow_down: |
| #single | `41.89% <0%> (ø)` | :arrow_up: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/20091?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/io/formats/terminal.py](https://codecov.io/gh/pandas-dev/pandas/pull/20091/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL2Zvcm1hdHMvdGVybWluYWwucHk=) | `16.43% <0%> (-4.55%)` | :arrow_down: |
| [pandas/plotting/\_converter.py](https://codecov.io/gh/pandas-dev/pandas/pull/20091/diff?src=pr&el=tree#diff-cGFuZGFzL3Bsb3R0aW5nL19jb252ZXJ0ZXIucHk=) | `65.07% <0%> (-1.74%)` | :arrow_down: |
| [pandas/core/series.py](https://codecov.io/gh/pandas-dev/pandas/pull/20091/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvc2VyaWVzLnB5) | `93.84% <0%> (-0.01%)` | :arrow_down: |
| [pandas/core/panel.py](https://codecov.io/gh/pandas-dev/pandas/pull/20091/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvcGFuZWwucHk=) | `97.29% <0%> (-0.01%)` | :arrow_down: |
| [pandas/core/frame.py](https://codecov.io/gh/pandas-dev/pandas/pull/20091/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZnJhbWUucHk=) | `97.18% <0%> (-0.01%)` | :arrow_down: |
| [pandas/core/strings.py](https://codecov.io/gh/pandas-dev/pandas/pull/20091/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvc3RyaW5ncy5weQ==) | `98.32% <0%> (ø)` | :arrow_up: |
| [pandas/io/formats/format.py](https://codecov.io/gh/pandas-dev/pandas/pull/20091/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL2Zvcm1hdHMvZm9ybWF0LnB5) | `98.24% <0%> (ø)` | :arrow_up: |
| [pandas/core/generic.py](https://codecov.io/gh/pandas-dev/pandas/pull/20091/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZ2VuZXJpYy5weQ==) | `95.85% <0%> (ø)` | :arrow_up: |
| [pandas/core/base.py](https://codecov.io/gh/pandas-dev/pandas/pull/20091/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvYmFzZS5weQ==) | `96.8% <0%> (ø)` | :arrow_up: |
| [pandas/util/\_decorators.py](https://codecov.io/gh/pandas-dev/pandas/pull/20091/diff?src=pr&el=tree#diff-cGFuZGFzL3V0aWwvX2RlY29yYXRvcnMucHk=) | `82.4% <0%> (+0.14%)` | :arrow_up: |
| ... and [2 more](https://codecov.io/gh/pandas-dev/pandas/pull/20091/diff?src=pr&el=tree-more) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/20091?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/20091?src=pr&el=footer). Last update [63a662d...9e47c0b](https://codecov.io/gh/pandas-dev/pandas/pull/20091?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

can you add the simple example from the issue as a test as well.
> The command "ci/lint.sh" exited with 1.

The Travis CI build failed after some modification on "pandas\tests\series\test_rank.py". I have no idea what is the cause. 
somehow lots of commits go there. merge in master and push again.
lint.sh gives error. Don't know why.
@WillAyd if you'd have a look
@peterpanmj nice work - does this work for GroupBy ranking the same way?
@WillAyd  As long as rank_1d_{dtype} is called, it is fine, but group by rank is still broken.
```
In [51]: df = pd.DataFrame([1, np.nan, np.inf, -np.inf, 25])

In [52]: df['key'] = 'foo'

In [53]: df.groupby("key").rank()      # not working properly for infinity
Out[53]:
     0
0  2.0
1  NaN
2  NaN
3  1.0
4  3.0

In [54]: df.rank()     # this is ok
Out[54]:
     0  key
0  2.0  3.0
1  NaN  3.0
2  4.0  3.0
3  1.0  3.0
4  3.0  3.0

In [55]: df.groupby("key").apply(lambda x:x.rank())   # this is also ok
Out[55]:
     0  key
0  2.0  3.0
1  NaN  3.0
2  4.0  3.0
3  1.0  3.0
4  3.0  3.0

```



Do you want to try and tackle the groupby implementation as part of this? I've linked the relevant function below - it isn't that drastically different from the algos rank implementation

https://github.com/pandas-dev/pandas/blob/766a4800b4d9b7e847577b74b24128fcda8eeb55/pandas/_libs/groupby_helper.pxi.in#L415
 I will try but it should take a lot of efforts. 
I don't under stand why _out_ has ndim=2 ? What is the dtype of _labels_ ?
Labels are int64 - each unique grouping will have it's own label. out has `ndim=2` to match the other Cython call signatures, but you'll see in this that only the first dimension ever gets written to (I think this will be cleaned up in the future).
can do the groupby fixes in another PR :> @peterpanmj or @WillAyd can you open an issue?
thanks @peterpanmj 
Opened #20561 for the GroupBy piece. @peterpanmj if you started working on it then by all means continue and let me know if you need help