TST: Fix dtype mismatch on 32bit in IntervalTree get_indexer test
State: closed

xref https://github.com/pandas-dev/pandas/pull/23442#issuecomment-435340042

To address @jreback's comment in the xref:
> `.get_indexer` should be returning platform int - but it’s not ? (maybe just this case)

It looks like `IntervalTree.get_indexer` and any other similar `IntervalTree` methods (`get_loc`, `get_indexer_nonunique`, etc.) will always return int64 as the data comes from a `Int64Vector.to_array()`.  

If returning platform int is expected, I'm not sure that making this change now is the best use of time: I'm hitting these methods as part of the new `IntervalIndex` behavior specs, so probably best enforce platform int as part of that implementation.

Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Interval
description: Interval data type
name: 32bit
description: 32-bit systems

Comments:
Hello @jschendel! Thanks for submitting the PR.

 - There are no PEP8 issues in the file [`pandas/tests/indexes/interval/test_interval_tree.py`](https://github.com/jschendel/pandas/blob/f4372a226a28813139b80df109fbd2218ccc9d13/pandas/tests/indexes/interval/test_interval_tree.py) !


This is called in 1 case in ``get_indexer``. I think this needs wrapping with ``ensure_platform_int`` (could be in get_indexer)

 ```
  def _find_non_overlapping_monotonic_bounds(self, key):
        if isinstance(key, IntervalMixin):
            start = self._searchsorted_monotonic(
                key.left, 'left', exclude_label=key.open_left)
            stop = self._searchsorted_monotonic(
                key.right, 'right', exclude_label=key.open_right)
        elif isinstance(key, slice):
            # slice
            start, stop = key.start, key.stop
            if (key.step or 1) != 1:
                raise NotImplementedError("cannot slice with a slice step")
            if start is None:
                start = 0
            else:
                start = self._searchsorted_monotonic(start, 'left')
            if stop is None:
                stop = len(self)
            else:
                stop = self._searchsorted_monotonic(stop, 'right')
        else:
            # scalar or index-like

            start = self._searchsorted_monotonic(key, 'left')
            stop = self._searchsorted_monotonic(key, 'right')
        return start, stop
```
Merged master, to see if the code-check failure in https://travis-ci.org/pandas-dev/pandas/jobs/450067817#L2756 goes away. It wasn't clear to me what the issue was.
# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/23468?src=pr&el=h1) Report
> Merging [#23468](https://codecov.io/gh/pandas-dev/pandas/pull/23468?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/bd98841b2bc0998565981b793e012c0adb1e8983?src=pr&el=desc) will **decrease** coverage by `<.01%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/23468/graphs/tree.svg?width=650&token=eZ4WkYLtcO&height=150&src=pr)](https://codecov.io/gh/pandas-dev/pandas/pull/23468?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #23468      +/-   ##
==========================================
- Coverage   92.25%   92.24%   -0.01%     
==========================================
  Files         161      161              
  Lines       51181    51224      +43     
==========================================
+ Hits        47217    47254      +37     
- Misses       3964     3970       +6
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `90.63% <ø> (-0.01%)` | :arrow_down: |
| #single | `42.28% <ø> (-0.01%)` | :arrow_down: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/23468?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/core/indexes/timedeltas.py](https://codecov.io/gh/pandas-dev/pandas/pull/23468/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW5kZXhlcy90aW1lZGVsdGFzLnB5) | `89.88% <0%> (-0.84%)` | :arrow_down: |
| [pandas/core/arrays/base.py](https://codecov.io/gh/pandas-dev/pandas/pull/23468/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvYXJyYXlzL2Jhc2UucHk=) | `97.35% <0%> (-0.67%)` | :arrow_down: |
| [pandas/core/internals/blocks.py](https://codecov.io/gh/pandas-dev/pandas/pull/23468/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW50ZXJuYWxzL2Jsb2Nrcy5weQ==) | `93.67% <0%> (-0.36%)` | :arrow_down: |
| [pandas/core/indexes/datetimes.py](https://codecov.io/gh/pandas-dev/pandas/pull/23468/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW5kZXhlcy9kYXRldGltZXMucHk=) | `96.13% <0%> (-0.29%)` | :arrow_down: |
| [pandas/core/arrays/datetimelike.py](https://codecov.io/gh/pandas-dev/pandas/pull/23468/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvYXJyYXlzL2RhdGV0aW1lbGlrZS5weQ==) | `95.9% <0%> (-0.19%)` | :arrow_down: |
| [pandas/core/arrays/categorical.py](https://codecov.io/gh/pandas-dev/pandas/pull/23468/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvYXJyYXlzL2NhdGVnb3JpY2FsLnB5) | `95.09% <0%> (-0.13%)` | :arrow_down: |
| [pandas/core/arrays/sparse.py](https://codecov.io/gh/pandas-dev/pandas/pull/23468/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvYXJyYXlzL3NwYXJzZS5weQ==) | `91.71% <0%> (-0.13%)` | :arrow_down: |
| [pandas/io/json/json.py](https://codecov.io/gh/pandas-dev/pandas/pull/23468/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL2pzb24vanNvbi5weQ==) | `93.11% <0%> (-0.02%)` | :arrow_down: |
| [pandas/core/indexes/base.py](https://codecov.io/gh/pandas-dev/pandas/pull/23468/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW5kZXhlcy9iYXNlLnB5) | `96.45% <0%> (-0.01%)` | :arrow_down: |
| [pandas/core/reshape/reshape.py](https://codecov.io/gh/pandas-dev/pandas/pull/23468/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvcmVzaGFwZS9yZXNoYXBlLnB5) | `99.54% <0%> (-0.01%)` | :arrow_down: |
| ... and [26 more](https://codecov.io/gh/pandas-dev/pandas/pull/23468/diff?src=pr&el=tree-more) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/23468?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/23468?src=pr&el=footer). Last update [bd98841...7e83460](https://codecov.io/gh/pandas-dev/pandas/pull/23468?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

@jschendel are you able to fix this rather than this work-around?
@jreback : will look into this tonight.  sorry, been super bogged down at work.
Modified `IntervalTree` methods to return `'intp'` dtype arrays via an `astype` after calling `Int64Vector.to_array`.
Fixed the linting error (untested).
lgtm. let's merge on green.
thanks @jschendel and @TomAugspurger 
still failing 
https://travis-ci.org/MacPython/pandas-wheels/jobs/452707967
see end of log 


Yeah, of course changing the `IntervalTree` code to return platform int would cascade to other things.  Pretty shortsighted of me.  Should be an easy fix of `'int64'` --> `'intp'` in the `expected` though, so will get that in later today.

It looks like one of the failures is unrelated to my changes though:
```
_______________________ TestSparseGroupBy.test_aggfuncs ________________________
self = <pandas.tests.sparse.test_groupby.TestSparseGroupBy object at 0xde3f08ec>
    def test_aggfuncs(self):
        sparse_grouped = self.sparse.groupby('A')
        dense_grouped = self.dense.groupby('A')
    
        result = sparse_grouped.mean().to_sparse()
        expected = dense_grouped.mean().to_sparse()
    
>       tm.assert_frame_equal(result, expected)
/venv/lib/python3.6/site-packages/pandas/tests/sparse/test_groupby.py:50: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/venv/lib/python3.6/site-packages/pandas/util/testing.py:1185: in assert_extension_array_equal
    assert_numpy_array_equal(left_valid, right_valid)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
left = array([-0.17387645482451206, 0.3414148016424936], dtype=object)
right = array([-0.17387645482451206, 0.3414148016424937], dtype=object)
err_msg = None
    def _raise(left, right, err_msg):
        if err_msg is None:
            if left.shape != right.shape:
                raise_assert_detail(obj, '{obj} shapes are different'
                                    .format(obj=obj), left.shape, right.shape)
    
            diff = 0
            for l, r in zip(left, right):
                # count up differences
                if not array_equivalent(l, r, strict_nan=strict_nan):
                    diff += 1
    
            diff = diff * 100.0 / left.size
            msg = '{obj} values are different ({pct} %)'.format(
                obj=obj, pct=np.round(diff, 5))
>           raise_assert_detail(obj, msg, left, right)
E           AssertionError: numpy array are different
E           
E           numpy array values are different (50.0 %)
E           [left]:  [-0.17387645482451206, 0.3414148016424936]
E           [right]: [-0.17387645482451206, 0.3414148016424937]
/venv/lib/python3.6/site-packages/pandas/util/testing.py:1146: AssertionError
```
This error was not present in [build <span>#</span>763](https://travis-ci.org/MacPython/pandas-wheels/jobs/451198407#L5273) and then appeared in [build <span>#</span>764](https://travis-ci.org/MacPython/pandas-wheels/jobs/451706570#L5281); I had not made any changes between those two builds.