Repository: pandas-dev/pandas
Pull request: BUG: DataFrame.to_dict when orient=index data loss
- [x] closes #22801 
- [x] tests added / passed
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`


Problem statement:
DataFrame.to_dict can result in data loss when indexes are not unique
#### Code Sample,

```python
df = pd.DataFrame([["A","B","C"],["A","D","E"],["F","G","K"]])
df.set_index(0, inplace=True)
print(df)
```
---1  2
0      
A  B  C
A  D  E
F  G  K
```python
print(df.to_dict(orient="index"))
```
{'A': {1: 'D', 2: 'E'}, 'F': {1: 'G', 2: 'K'}}

#### Problem description

The to_dict method silently tosses away data when indexes are not unique. At least in my impression the rest of the API doesn’t mind non-unique index’s much, so I don’t like this behavior. I think that it should either preserve the data, or raise an an error / warning saying that to_dict requires unique indexes. 

#### Expected Output
'''
{'A': [{1: 'B', 2: 'C'},{1: 'D', 2: 'E'}], 'F': {1: 'G', 2: 'K'}}

or at least throw an error saying ".to_dict(orient="index") does not support non-unique indexes" 

'''
#### Output of ``pd.show_versions()``


INSTALLED VERSIONS
------------------
commit: None
python: 3.5.5.final.0
python-bits: 64
OS: Windows
OS-release: 10
machine: AMD64
processor: Intel64 Family 6 Model 158 Stepping 9, GenuineIntel
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8
LOCALE: None.None

pandas: 0.23.4
pytest: None
pip: 10.0.1
setuptools: 40.0.0
Cython: 0.28.5
numpy: 1.15.0
scipy: None
pyarrow: 0.9.0
xarray: None
IPython: 6.5.0
sphinx: None
patsy: None
dateutil: 2.7.3
pytz: 2018.5
blosc: None
bottleneck: None
tables: None
numexpr: None
feather: None
matplotlib: None
openpyxl: None
xlrd: 1.1.0
xlwt: None
xlsxwriter: None
lxml: None
bs4: None
html5lib: 1.0.1
sqlalchemy: None
pymysql: None
psycopg2: None
jinja2: 2.10
s3fs: None
fastparquet: None
pandas_gbq: None
pandas_datareader: None

Hints from issue discussion:
Yeah, I think we should raise here, ala #4376 for JSON.  Thanks for the report!

```python
In [28]: df.to_json(orient='index')

ValueError: DataFrame index must be unique for orient='index'.
```
dicts in Python do not support duplicate keys so I think we should raise. You can use JSON as an alternate if you need
Looks like @chris-b1 beat me to this, but to clarify my seemingly contradictory point about JSON you can do the following:

```python
In [5]: df.to_json(orient='table')
Out[5]: '{"schema": {"fields":[{"name":0,"type":"string"},{"name":1,"type":"string"},{"name":2,"type":"string"}],"pandas_version":"0.20.0"}, "data": [{"0":"A","1":"B","2":"C"},{"0":"A","1":"D","2":"E"},{"0":"F","1":"G","2":"K"}]}'
```
This issue ties in with an issue I reported a week back: https://github.com/pandas-dev/pandas/issues/22705, rendering `to_dict()` an incomplete replacement for `to_items()`.

Commit message: fix reference to dataframe as self

Patch to review:
diff --git a/pandas/core/frame.py b/pandas/core/frame.py
index 81d5c112885ec..7a15edb16441e 100644
--- a/pandas/core/frame.py
+++ b/pandas/core/frame.py
@@ -1163,8 +1163,13 @@ def to_dict(self, orient='dict', into=dict):
                            for k, v in zip(self.columns, np.atleast_1d(row)))
                     for row in self.values]
         elif orient.lower().startswith('i'):
-            return into_c((t[0], dict(zip(self.columns, t[1:])))
-                          for t in self.itertuples())
+            if self.index.unique().size == self.index.size:
+                return into_c((t[0], dict(zip(self.columns, t[1:])))
+                              for t in self.itertuples())
+            else:
+                raise ValueError(
+                    "DataFrame index must be unique for orient='index'."
+                )
         else:
             raise ValueError("orient '{o}' not understood".format(o=orient))
 
diff --git a/pandas/tests/frame/test_convert_to.py b/pandas/tests/frame/test_convert_to.py
index a0e23d256c25b..185b36e465f11 100644
--- a/pandas/tests/frame/test_convert_to.py
+++ b/pandas/tests/frame/test_convert_to.py
@@ -71,6 +71,10 @@ def test_to_dict_timestamp(self):
         tm.assert_dict_equal(test_data_mixed.to_dict(orient='split'),
                              expected_split_mixed)
 
+    def test_to_dict_index_not_unique_with_index_orient(self):
+        df = DataFrame({'a': [1, 2], 'b': [0.5, 0.75]}, index=['A', 'A'])
+        pytest.raises(ValueError, df.to_dict, orient='index')
+
     def test_to_dict_invalid_orient(self):
         df = DataFrame({'A': [0, 1]})
         pytest.raises(ValueError, df.to_dict, orient='xinvalid')


Code review comments:
diff_hunk: @@ -1163,8 +1163,13 @@ def to_dict(self, orient='dict', into=dict):
                            for k, v in zip(self.columns, np.atleast_1d(row)))
                     for row in self.values]
         elif orient.lower().startswith('i'):
-            return into_c((t[0], dict(zip(self.columns, t[1:])))
-                          for t in self.itertuples())
+            if self.index.unique().size == self.index.size:
path: pandas/core/frame.py
text: @user1:
you can just use ``self.index.is_unique``

diff_hunk: @@ -1163,8 +1163,13 @@ def to_dict(self, orient='dict', into=dict):
                            for k, v in zip(self.columns, np.atleast_1d(row)))
                     for row in self.values]
         elif orient.lower().startswith('i'):
-            return into_c((t[0], dict(zip(self.columns, t[1:])))
-                          for t in self.itertuples())
+            if self.index.unique().size == self.index.size:
+                return into_c((t[0], dict(zip(self.columns, t[1:])))
+                              for t in self.itertuples())
+            else:
path: pandas/core/frame.py
text: @user1:
no need for an else!