TST: #15752 Add test_drop_duplicates for Categorical dtypes
State: closed

- [x] tests added / passed
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Categorical
description: Categorical Data Type

Comments:
Hello @tmnhat2001! Thanks for updating the PR.



Cheers ! There are no PEP8 issues in this Pull Request. :beers: 

##### Comment last updated on November 06, 2017 at 02:47 Hours UTC
# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/18072?src=pr&el=h1) Report
> Merging [#18072](https://codecov.io/gh/pandas-dev/pandas/pull/18072?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/15fa4bd62c5c7ac7ac618a7e68742b7eb1f90d21?src=pr&el=desc) will **decrease** coverage by `0.01%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/18072/graphs/tree.svg?width=650&height=150&src=pr&token=eZ4WkYLtcO)](https://codecov.io/gh/pandas-dev/pandas/pull/18072?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #18072      +/-   ##
==========================================
- Coverage   91.25%   91.23%   -0.02%     
==========================================
  Files         163      163              
  Lines       50120    50120              
==========================================
- Hits        45737    45727      -10     
- Misses       4383     4393      +10
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `89.04% <ø> (-0.01%)` | :arrow_down: |
| #single | `40.32% <ø> (-0.06%)` | :arrow_down: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/18072?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/io/gbq.py](https://codecov.io/gh/pandas-dev/pandas/pull/18072?src=pr&el=tree#diff-cGFuZGFzL2lvL2dicS5weQ==) | `25% <0%> (-58.34%)` | :arrow_down: |
| [pandas/core/frame.py](https://codecov.io/gh/pandas-dev/pandas/pull/18072?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZnJhbWUucHk=) | `97.75% <0%> (-0.1%)` | :arrow_down: |
| [pandas/core/indexes/datetimes.py](https://codecov.io/gh/pandas-dev/pandas/pull/18072?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW5kZXhlcy9kYXRldGltZXMucHk=) | `95.41% <0%> (-0.1%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/18072?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/18072?src=pr&el=footer). Last update [15fa4bd...289ad35](https://codecov.io/gh/pandas-dev/pandas/pull/18072?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/18072?src=pr&el=h1) Report
> Merging [#18072](https://codecov.io/gh/pandas-dev/pandas/pull/18072?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/15fa4bd62c5c7ac7ac618a7e68742b7eb1f90d21?src=pr&el=desc) will **decrease** coverage by `0.01%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/18072/graphs/tree.svg?token=eZ4WkYLtcO&src=pr&width=650&height=150)](https://codecov.io/gh/pandas-dev/pandas/pull/18072?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #18072      +/-   ##
==========================================
- Coverage   91.25%   91.23%   -0.02%     
==========================================
  Files         163      163              
  Lines       50120    50120              
==========================================
- Hits        45737    45727      -10     
- Misses       4383     4393      +10
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `89.04% <ø> (-0.01%)` | :arrow_down: |
| #single | `40.32% <ø> (-0.06%)` | :arrow_down: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/18072?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/io/gbq.py](https://codecov.io/gh/pandas-dev/pandas/pull/18072?src=pr&el=tree#diff-cGFuZGFzL2lvL2dicS5weQ==) | `25% <0%> (-58.34%)` | :arrow_down: |
| [pandas/core/frame.py](https://codecov.io/gh/pandas-dev/pandas/pull/18072?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZnJhbWUucHk=) | `97.75% <0%> (-0.1%)` | :arrow_down: |
| [pandas/core/indexes/datetimes.py](https://codecov.io/gh/pandas-dev/pandas/pull/18072?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW5kZXhlcy9kYXRldGltZXMucHk=) | `95.41% <0%> (-0.1%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/18072?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/18072?src=pr&el=footer). Last update [15fa4bd...289ad35](https://codecov.io/gh/pandas-dev/pandas/pull/18072?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/18072?src=pr&el=h1) Report
> Merging [#18072](https://codecov.io/gh/pandas-dev/pandas/pull/18072?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/15fa4bd62c5c7ac7ac618a7e68742b7eb1f90d21?src=pr&el=desc) will **increase** coverage by `<.01%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/18072/graphs/tree.svg?width=650&height=150&src=pr&token=eZ4WkYLtcO)](https://codecov.io/gh/pandas-dev/pandas/pull/18072?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #18072      +/-   ##
==========================================
+ Coverage   91.25%   91.26%   +<.01%     
==========================================
  Files         163      163              
  Lines       50120    50122       +2     
==========================================
+ Hits        45737    45743       +6     
+ Misses       4383     4379       -4
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `89.07% <ø> (+0.02%)` | :arrow_up: |
| #single | `40.32% <ø> (-0.06%)` | :arrow_down: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/18072?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/io/gbq.py](https://codecov.io/gh/pandas-dev/pandas/pull/18072?src=pr&el=tree#diff-cGFuZGFzL2lvL2dicS5weQ==) | `25% <0%> (-58.34%)` | :arrow_down: |
| [pandas/core/frame.py](https://codecov.io/gh/pandas-dev/pandas/pull/18072?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZnJhbWUucHk=) | `97.75% <0%> (-0.1%)` | :arrow_down: |
| [pandas/io/parsers.py](https://codecov.io/gh/pandas-dev/pandas/pull/18072?src=pr&el=tree#diff-cGFuZGFzL2lvL3BhcnNlcnMucHk=) | `95.5% <0%> (-0.01%)` | :arrow_down: |
| [pandas/core/indexes/timedeltas.py](https://codecov.io/gh/pandas-dev/pandas/pull/18072?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW5kZXhlcy90aW1lZGVsdGFzLnB5) | `91.19% <0%> (ø)` | :arrow_up: |
| [pandas/core/indexes/category.py](https://codecov.io/gh/pandas-dev/pandas/pull/18072?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW5kZXhlcy9jYXRlZ29yeS5weQ==) | `97.46% <0%> (ø)` | :arrow_up: |
| [pandas/core/reshape/merge.py](https://codecov.io/gh/pandas-dev/pandas/pull/18072?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvcmVzaGFwZS9tZXJnZS5weQ==) | `94.26% <0%> (ø)` | :arrow_up: |
| [pandas/core/indexes/datetimes.py](https://codecov.io/gh/pandas-dev/pandas/pull/18072?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW5kZXhlcy9kYXRldGltZXMucHk=) | `95.51% <0%> (ø)` | :arrow_up: |
| [pandas/core/indexes/period.py](https://codecov.io/gh/pandas-dev/pandas/pull/18072?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW5kZXhlcy9wZXJpb2QucHk=) | `92.87% <0%> (+0.19%)` | :arrow_up: |
| [pandas/plotting/\_converter.py](https://codecov.io/gh/pandas-dev/pandas/pull/18072?src=pr&el=tree#diff-cGFuZGFzL3Bsb3R0aW5nL19jb252ZXJ0ZXIucHk=) | `65.2% <0%> (+1.81%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/18072?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/18072?src=pr&el=footer). Last update [15fa4bd...f9c65c7](https://codecov.io/gh/pandas-dev/pandas/pull/18072?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

Thanks, that makes the test a lot cleaner. While making changes to the test, I noticed an issue with creating a series of datetime64[D] categorical.

```
>>> import pandas as pd
>>> import numpy as np
>>> cat_array = np.array([1, 2, 3, 4, 5], dtype=np.dtype("datetime64[D]"))
>>> cat_array
array(['1970-01-02', '1970-01-03', '1970-01-04', '1970-01-05', '1970-01-06'], dtype='datetime64[D]')
>>> input1 = np.array([1, 2, 3, 3], dtype=np.dtype("datetime64[D]"))
>>> input1
array(['1970-01-02', '1970-01-03', '1970-01-04', '1970-01-04'], dtype='datetime64[D]')
>>> tc1 = pd.Series(pd.Categorical(input1, categories=cat_array, ordered=True))
>>> tc1
0   NaT
1   NaT
2   NaT
3   NaT
dtype: category
Categories (5, datetime64[ns]): [1970-01-02 < 1970-01-03 < 1970-01-04 < 1970-01-05 < 1970-01-06]
```

<details>
<summary>pd.show_versions():</summary>
</br>
INSTALLED VERSIONS</br>
------------------</br>
commit: e1dc3c9475b7da494f511bd92243285662931edb</br>
python: 3.6.3.final.0</br>
python-bits: 64</br>
OS: Windows</br>
OS-release: 10</br>
machine: AMD64</br>
processor: Intel64 Family 6 Model 61 Stepping 4, GenuineIntel</br>
byteorder: little</br>
LC_ALL: None</br>
LANG: None</br>
LOCALE: None.None</br>

pandas: 0.22.0.dev0+41.ge1dc3c9
pytest: 3.2.3
pip: 9.0.1
setuptools: 36.6.0
Cython: 0.27.2
numpy: 1.13.3
scipy: None
pyarrow: None
xarray: None
IPython: None
sphinx: None
patsy: None
dateutil: 2.6.1
pytz: 2017.2
blosc: None
bottleneck: None
tables: None
numexpr: None
feather: None
matplotlib: None
openpyxl: None
xlrd: None
xlwt: None
xlsxwriter: None
lxml: None
bs4: None
html5lib: None
sqlalchemy: None
pymysql: None
psycopg2: None
jinja2: 2.9.6
s3fs: None
fastparquet: None
pandas_gbq: None
pandas_datareader: None
<details>
@tmnhat2001 hmm that looks like a bug.

can you mark that xfail, then we can merge this and can address in a folllowup.
@tmnhat2001 I think this is the issue: https://github.com/pandas-dev/pandas/issues/7996
Yeah, that looks like it. There's no issue if datetime64[h] is used. Can we use that instead?
thanks @tmnhat2001 
certainly take PR's for the fix for xfail here or any other issues you'd like to work on!
thanks for helping!