Repository: allenai/reward-bench
Issue: generative model rewardbench installation problem
Root cause: Documentation Deficiency

### Description
I'm trying to use RewardBench to evaluate my generative reward model. Following the instructions, I installed the `rewardbench` package along with additional packages: `openai`, `anthropic`, `google-generativeai`, and `together`. However, when running the evaluation script, I encountered the following error:

### Steps to Reproduce
1. Installed packages:
   ```bash
   pip install rewardbench
   pip install openai anthropic google-generativeai together
   ```
2. Ran the command:
   ```bash
   python scripts/run_generative_v2.py --model={/path/to/my/model/global_step_100/}
   ```

### Error Log
```
deepspeed@danxin-train2-0:/home/deepspeed/rewardbench$ python scripts/run_generative_v2.py --model={/path/to/my/model/global_step_100/}
/usr/local/lib/python3.10/site-packages/torch/cuda/__init__.py:61: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you.
  import pynvml  # type: ignore[import]
INFO 11-04 11:44:35 __init__.py:190] Automatically detected platform cuda.
[2025-11-04 11:44:36,553] [INFO] [real_accelerator.py:203:get_accelerator] Setting ds_accelerator to cuda (auto detect)
 [WARNING]  async_io requires the dev libaio .so object and headers but these were not found.
 [WARNING]  async_io: please install the libaio-dev package with apt
 [WARNING]  If libaio is already installed (perhaps from source), try setting the CFLAGS and LDFLAGS environment variables to where it can be found.
 [WARNING]  Please specify the CUTLASS repo directory as environment variable $CUTLASS_PATH
 [WARNING]  sparse_attn requires a torch version >= 1.5 and < 2.0 but detected 2.5
 [WARNING]  using untested triton version (3.1.0), only 1.0.0 is known to be compatible
/usr/local/lib/python3.10/site-packages/deepspeed/runtime/zero/linear.py:49: FutureWarning: `torch.cuda.amp.custom_fwd(args...)` is deprecated. Please use `torch.amp.custom_fwd(args..., device_type='cuda')` instead.
  def forward(ctx, input, weight, bias=None):
/usr/local/lib/python3.10/site-packages/deepspeed/runtime/zero/linear.py:67: FutureWarning: `torch.cuda.amp.custom_bwd(args...)` is deprecated. Please use `torch.amp.custom_bwd(args..., device_type='cuda')` instead.
  def backward(ctx, grad_output):
Traceback (most recent call last):
  File "/home/deepspeed/rewardbench/scripts/run_generative_v2.py", line 37, in <module>
    from rewardbench.generative_v2 import (
ImportError: cannot import name 'get_single_rating' from 'rewardbench.generative_v2' (/home/deepspeed/.local/lib/python3.10/site-packages/rewardbench/generative_v2.py)
```

### Investigation and Confusion
After investigating, I suspect this might be a version compatibility issue. I then checked the GitHub documentation and found this experimental note:

> Experimental: Generative RMs can be run from the pip install by running:
> `pip install rewardbench[generative]`

This has left me confused about which package is correct for generative reward model evaluation:

1. Should I install just `rewardbench` or `rewardbench[generative]`?
2. When I tried `pip install rewardbench[generative]`, it failed with an error indicating that this package doesn't exist.

### Request
Could you please provide clear guidance on:
- Which package to install for generative reward model evaluation
- The correct installation and workflow process
- Any version requirements or dependencies I should be aware of

Thank you for your help!

### Environment
- Python version: 3.10
- Installed packages: rewardbench, openai, anthropic, google-generativeai, together
- Platform: Linux