Repository: stanford-crfm/helm
Issue: issues when run `disinformation`
Root cause: Interface Contract Mismatch

when I run the `disinformation` like [run_specs.conf#L538-L540](https://github.com/stanford-crfm/helm/blob/main/src/helm/benchmark/presentation/run_specs.conf#L538-L540), I have two issues：

- `capability=reiteration`：`raise helm.benchmark.executor.ExecutorError: HuggingFace error: num_return_sequences has to be 1, but is 5 when doing greedy search`.  so [the settings](https://github.com/stanford-crfm/helm/blob/main/src/helm/benchmark/run_specs.py#L1128) don't match, this problem also occurs when run `real_toxicity_prompts`.
- `capability=wedging`：`raise ValueError: More than one stop sequence is not supported`. I find [this](https://github.com/stanford-crfm/helm/blob/main/src/helm/benchmark/run_specs.py#L1142) to define `stop_sequence` as a list of two lengths, but [this ](https://github.com/stanford-crfm/helm/blob/main/src/helm/proxy/clients/huggingface_client.py#L146) only allows one length.

When I looked at the corresponding references [this](https://github.com/stanford-crfm/helm/blob/main/src/helm/benchmark/run_specs.py#L1126) [this](https://github.com/stanford-crfm/helm/blob/main/src/helm/benchmark/run_specs.py#L1138) I still wondered how to solve these problems.

> when I run the `disinformation` like [run_specs.conf#L538-L540](https://github.com/stanford-crfm/helm/blob/main/src/helm/benchmark/presentation/run_specs.conf#L538-L540), I have two issues：
> 
> * `capability=reiteration`：`raise helm.benchmark.executor.ExecutorError: HuggingFace error: num_return_sequences has to be 1, but is 5 when doing greedy search`.  so [the settings](https://github.com/stanford-crfm/helm/blob/main/src/helm/benchmark/run_specs.py#L1128) don't match, this problem also occurs when run `real_toxicity_prompts`.
> * `capability=wedging`：`raise ValueError: More than one stop sequence is not supported`. I find [this](https://github.com/stanford-crfm/helm/blob/main/src/helm/benchmark/run_specs.py#L1142) to define `stop_sequence` as a list of two lengths, but [this ](https://github.com/stanford-crfm/helm/blob/main/src/helm/proxy/clients/huggingface_client.py#L146) only allows one length.
> 
> When I looked at the corresponding references [this](https://github.com/stanford-crfm/helm/blob/main/src/helm/benchmark/run_specs.py#L1126) [this](https://github.com/stanford-crfm/helm/blob/main/src/helm/benchmark/run_specs.py#L1138) I still wondered how to solve these problems.

- For the first point, I'm sorry that it was caused by my mistake in choosing the wrong generate mode of huggingface, so it is not a problem; 
- For the second point, I think [that](https://github.com/stanford-crfm/helm/blob/main/src/helm/proxy/clients/huggingface_client.py#L147) if the `length>1`, it should be optimized to not pass the `stop_sequence` as `eos_token`, but should return the result when the `maximum length` or the `eos` of the original tokenizer is reached, and judge whether to truncate outside the `model.generate`. **Is the reason we don't do this because we only want `stop sentence` as `eos` and ignore the original tokenizer's `eos`?**
- Moreover, I found that some tokenizers will tokenize `stop_sequence` (length==1) tokenize into two ids, then [this](https://github.com/stanford-crfm/helm/blob/main/src/helm/proxy/clients/huggingface_client.py#L52) will report an error, which can also be solved by the above method.

If you would give me some answers, I would really appreciate it

This looks like a duplicate of #1501 which was fixed in #1892 i.e. `HuggingFaceClient` failing if the stop sequence is longer than one token.