Repository: tatsu-lab/alpaca_eval
Issue: ValueError: cannot reindex on an axis with duplicate labels
Root cause: Validation Gap

I have formatted the outputs like "example/outputs.json" and run the command "alpaca_eval --model_outputs xxx.json", then the value error occurred:
> Traceback (most recent call last):
  File "/data/liyang/anaconda3/envs/py311torch212/bin/alpaca_eval", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/data/liyang/anaconda3/envs/py311torch212/lib/python3.11/site-packages/alpaca_eval/main.py", line 611, in main
    fire.Fire(evaluate)
  File "/data/liyang/anaconda3/envs/py311torch212/lib/python3.11/site-packages/fire/core.py", line 135, in Fire
    component_trace = _Fire(component, args, parsed_flag_args, context, name)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/liyang/anaconda3/envs/py311torch212/lib/python3.11/site-packages/fire/core.py", line 468, in _Fire
    component, remaining_args = _CallAndUpdateTrace(
                                ^^^^^^^^^^^^^^^^^^^^
  File "/data/liyang/anaconda3/envs/py311torch212/lib/python3.11/site-packages/fire/core.py", line 684, in _CallAndUpdateTrace
    component = fn(*varargs, **kwargs)
                ^^^^^^^^^^^^^^^^^^^^^^
  File "/data/liyang/anaconda3/envs/py311torch212/lib/python3.11/site-packages/alpaca_eval/main.py", line 179, in evaluate
    leaderboard[name].update(fn_metric_(annotations, **(metric_kwargs or {})))
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/liyang/anaconda3/envs/py311torch212/lib/python3.11/site-packages/alpaca_eval/metrics/glm_winrate.py", line 95, in get_length_controlled_winrate
    df_XY_train, df_X_test, sample_weight = _get_featurized_data(
                                            ^^^^^^^^^^^^^^^^^^^^^
  File "/data/liyang/anaconda3/envs/py311torch212/lib/python3.11/site-packages/alpaca_eval/metrics/glm_winrate.py", line 247, in _get_featurized_data
    df_XY_train, df_X_test = make_dmatrix_for_model(df_gamed_and_m, df_test, formula=formula)
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/liyang/anaconda3/envs/py311torch212/lib/python3.11/site-packages/alpaca_eval/metrics/glm_winrate.py", line 281, in make_dmatrix_for_model
    df_XY_train[col_y_true] = df_train[col_y_true]  # adds the label
    ~~~~~~~~~~~^^^^^^^^^^^^
  File "/data/liyang/anaconda3/envs/py311torch212/lib/python3.11/site-packages/pandas/core/frame.py", line 4311, in __setitem__
    self._set_item(key, value)
  File "/data/liyang/anaconda3/envs/py311torch212/lib/python3.11/site-packages/pandas/core/frame.py", line 4524, in _set_item
    value, refs = self._sanitize_column(value)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/liyang/anaconda3/envs/py311torch212/lib/python3.11/site-packages/pandas/core/frame.py", line 5263, in _sanitize_column
    return _reindex_for_setitem(value, self.index)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/liyang/anaconda3/envs/py311torch212/lib/python3.11/site-packages/pandas/core/frame.py", line 12692, in _reindex_for_setitem
    raise err
  File "/data/liyang/anaconda3/envs/py311torch212/lib/python3.11/site-packages/pandas/core/frame.py", line 12687, in _reindex_for_setitem
    reindexed_value = value.reindex(index)._values
                      ^^^^^^^^^^^^^^^^^^^^
  File "/data/liyang/anaconda3/envs/py311torch212/lib/python3.11/site-packages/pandas/core/series.py", line 5153, in reindex
    return super().reindex(
           ^^^^^^^^^^^^^^^^
  File "/data/liyang/anaconda3/envs/py311torch212/lib/python3.11/site-packages/pandas/core/generic.py", line 5610, in reindex
    return self._reindex_axes(
           ^^^^^^^^^^^^^^^^^^^
  File "/data/liyang/anaconda3/envs/py311torch212/lib/python3.11/site-packages/pandas/core/generic.py", line 5633, in _reindex_axes
    new_index, indexer = ax.reindex(
                         ^^^^^^^^^^^
  File "/data/liyang/anaconda3/envs/py311torch212/lib/python3.11/site-packages/pandas/core/indexes/base.py", line 4429, in reindex
    raise ValueError("cannot reindex on an axis with duplicate labels")
ValueError: cannot reindex on an axis with duplicate labels

and no JSON file was saved.

My xxx.json:
```json
[
  {
    "instruction": "What are the names of some famous actors that started their careers on Broadway?",
    "output": "test",
    "generator": "test",
    "dataset": "helpful_base"
  }
]
```