Repository: huggingface/transformers (Python)
Date: 2023-06-13 11:16:39+00:00
Task type: Bug Fix

Issue:
Error when overriding generation config: GenerationConfig() got multiple values for keyword argument 'num_beams'
### System Info

- `transformers` version: 4.30.0.dev0 (commit: 4aa13224a5bca560147a29c06b2e0597137caf3e)
- Platform: Linux-5.15.0-1013-oracle-x86_64-with-glibc2.31
- Python version: 3.10.11
- Huggingface_hub version: 0.15.1
- Safetensors version: 0.3.1
- PyTorch version (GPU?): 2.0.1+cu117 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: Yes (launching with `accelerate`)

### Who can help?

@gante @sgugger 

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction


Calling `GenerationConfig.from_pretrained` with a model that already defines `num_beams` in its configuration, and attempting to override the `num_beams` parameter (and presumably any other parameter), results in a runtime exception `got multiple values for keyword argument 'num_beams'` 

```python
generation_config: GenerationConfig = GenerationConfig.from_pretrained(
        "My-private-model",
        num_beams=num_beams)
```

Results in : 
```
  File "/app/scripts/fine_tune/./fine_tune_and_evaluate.py", line 1481, in <module>
    main()
  File "/app/scripts/fine_tune/./fine_tune_and_evaluate.py", line 1267, in main
    generation_config: GenerationConfig = GenerationConfig.from_pretrained(
  File "/app/ai_categorize_env/lib/python3.10/site-packages/transformers/generation/configuration_utils.py", line 541, in from_pretrained
    return cls.from_dict(config_dict, **kwargs)
  File "/app/ai_categorize_env/lib/python3.10/site-packages/transformers/generation/configuration_utils.py", line 574, in from_dict
    config = cls(**config_dict, **kwargs)
TypeError: transformers.generation.configuration_utils.GenerationConfig() got multiple values for keyword argument 'num_beams'
```

This appears to be because of this code:
https://github.com/huggingface/transformers/blob/ba695c1efd55091e394eb59c90fb33ac3f9f0d41/src/transformers/generation/configuration_utils.py#L572-L576

That is calling `cls(**config_dict, **kwargs)`, which might pass the same keyword values in twice if the `config_dict` has the property that `kwargs` does, right? I don't see a step where we remove the properties from `config_dict` that are mentioned in `kwargs`, although there is a comment right above that says: `# remove all the arguments that are in the config_dict`

Wouldn't the code need to do something more like this?
```
config_dict_copy = config_dict.copy()
config_dict_copy.update(kwargs)
config = cls(**config_dict_copy)
```

My generation_config.json from my model is this:
```json
{
  "decoder_start_token_id": 0,
  "eos_token_id": 1,
  "length_penalty": 0,
  "max_length": 32,
  "num_beams": 2,
  "num_return_sequences": 2,
  "output_scores": true,
  "pad_token_id": 0,
  "return_dict_in_generate": true,
  "transformers_version": "4.30.0.dev0"
}
```





### Expected behavior

This should not throw an exception:
```python
generation_config: GenerationConfig = GenerationConfig.from_pretrained(
        "My-model",
        num_beams=num_beams)
```



Issue discussion:
Hey @Taytay 👋 

Thank you for raising this issue! This is indeed a bug, I'll open a PR ASAP

Solution patch:
diff --git a/src/transformers/generation/configuration_utils.py b/src/transformers/generation/configuration_utils.py
--- a/src/transformers/generation/configuration_utils.py
+++ b/src/transformers/generation/configuration_utils.py
@@ -288,7 +288,8 @@ def __init__(self, **kwargs):
 
         # Additional attributes without default values
         if not self._from_model_config:
-            # we don't want to copy values from the model config if we're initializing a `GenerationConfig` from a model's default configuration file
+            # we don't want to copy values from the model config if we're initializing a `GenerationConfig` from a
+            # model's default configuration file
             for key, value in kwargs.items():
                 try:
                     setattr(self, key, value)
@@ -569,9 +570,9 @@ def from_dict(cls, config_dict: Dict[str, Any], **kwargs) -> "GenerationConfig":
         if "_commit_hash" in kwargs and "_commit_hash" in config_dict:
             kwargs["_commit_hash"] = config_dict["_commit_hash"]
 
-        # remove all the arguments that are in the config_dict
-
-        config = cls(**config_dict, **kwargs)
+        # The line below allows model-specific config to be loaded as well through kwargs, with safety checks.
+        # See https://github.com/huggingface/transformers/pull/21269
+        config = cls(**{**config_dict, **kwargs})
         unused_kwargs = config.update(**kwargs)
 
         logger.info(f"Generate config {config}")


Test patch:
diff --git a/tests/generation/test_configuration_utils.py b/tests/generation/test_configuration_utils.py
--- a/tests/generation/test_configuration_utils.py
+++ b/tests/generation/test_configuration_utils.py
@@ -93,6 +93,31 @@ def test_initialize_new_kwargs(self):
         generation_config = GenerationConfig.from_model_config(new_config)
         assert not hasattr(generation_config, "foo")  # no new kwargs should be initialized if from config
 
+    def test_kwarg_init(self):
+        """Tests that we can overwrite attributes at `from_pretrained` time."""
+        default_config = GenerationConfig()
+        self.assertEqual(default_config.temperature, 1.0)
+        self.assertEqual(default_config.do_sample, False)
+        self.assertEqual(default_config.num_beams, 1)
+
+        config = GenerationConfig(
+            do_sample=True,
+            temperature=0.7,
+            length_penalty=1.0,
+            bad_words_ids=[[1, 2, 3], [4, 5]],
+        )
+        self.assertEqual(config.temperature, 0.7)
+        self.assertEqual(config.do_sample, True)
+        self.assertEqual(config.num_beams, 1)
+
+        with tempfile.TemporaryDirectory() as tmp_dir:
+            config.save_pretrained(tmp_dir)
+            loaded_config = GenerationConfig.from_pretrained(tmp_dir, temperature=1.0)
+
+        self.assertEqual(loaded_config.temperature, 1.0)
+        self.assertEqual(loaded_config.do_sample, True)
+        self.assertEqual(loaded_config.num_beams, 1)  # default value
+
 
 @is_staging_test
 class ConfigPushToHubTester(unittest.TestCase):