Repository: huggingface/peft

Issue:
Improve config handling
### Feature request

When working on #800, I ran into some strange errors. It turned out that those errors were related to an assumption we make in the code base which I think is dangerous:

At the moment, inside of `PeftModel.__init__`, we pass `self.peft_config` (which is a dict that maps from name to adapter config) to the base model (e.g. the `LoraModel`). Therefore, the base model holds a _reference_ to the same dict. Then, throughout the code, we implicitly make the assumption that these two attributes are the same object, e.g. when adding/setting/deleting/disabling adapters.

IMO, it is dangerous to hold multiple references to the same mutable object and rely on the assumption that it's always the same object. To give an example, look at this line:

https://github.com/huggingface/peft/blob/ed396a69ed6469be87f90e739f98f19ec9973983/src/peft/tuners/tuners_utils.py#L71

When the `isinstance` check returns `True`, the base model's `peft_config` will actually be a _different_ object than the `PeftModel`'s `peft_config` and suddenly the assumption breaks.

I think there are two possible solutions we can take from here:

1. Never rely on the two objects being identical, instead always make them separate. That means that if we do manipulations on the `.peft_config` like adding/setting/deleting/disabling adapters, we have to ensure that the two dicts are kept in sync. This is significant extra work.
2. The `PeftModel` should not have a separate `peft_config` attribute. Instead, `PeftModel` should always use the `base_model.peft_config`, which becomes the single source of truth. This also requires a bit of refactoring.

The more general issue why this could happen in the first place is that we don't have good separation of concerns for adapter handling. Specifically, it is not clear what class is responsible for that. Currently, the responsibility is distributed among a) `PeftModel`, b) `LoraModel` et al, and c) `LoraLayer` et al. Ideally, we would have a single place where all of that is handled. However, this would require a much larger refactor.

Another, more minor issue I found while working on this is that we rely a lot on `PeftModel.__getattr__`. Although it generally works, it is bad for discovery. E.g. it is possible to call `PeftModel.delete_adapter` or `PeftModel.add_weighted_adapter`, but as a user, it is hard to discover that. Since those methods are not directly an attribute of `PeftModel`, they are not listed in `dir(model)`. Therefore, in some (most? all?) editors and in notebooks, they will not show as tab completion. It is also inconsistent, because some similar methods do indeed exist on `PeftModel`, e.g. `add_adapter` and `disable_adapter`.

Therefore, I would suggest to add those methods, including docstring, directly on `PeftModel` instead of relying on `__getattr__`. Of course, we need to raise a good error if the base model does _not_ have those methods.

Honestly, I would even prefer to remove `__getattr__` completely, as it's a source of errors and confusion, instead relying on making the supported methods explicit. But this will probably break existing code.

### Motivation

Making these changes will make the code base less error prone and easier to debug, as we would rely less on "magic".

### Your contribution

I can work on the mentioned issues, but I need confirmation from the other maintainers first that these changes are desired.


Discussion:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.



Resolving patch:
diff --git a/src/peft/peft_model.py b/src/peft/peft_model.py
--- a/src/peft/peft_model.py
+++ b/src/peft/peft_model.py
@@ -101,21 +101,22 @@ class PeftModel(PushToHubMixin, torch.nn.Module):
 
     def __init__(self, model: PreTrainedModel, peft_config: PeftConfig, adapter_name: str = "default"):
         super().__init__()
-        self.base_model = model
-        self.config = getattr(self.base_model, "config", {"model_type": "custom"})
         self.modules_to_save = None
-        self.peft_config = {}
         self.active_adapter = adapter_name
         self.peft_type = peft_config.peft_type
-        if not peft_config.is_prompt_learning:
-            self.peft_config[adapter_name] = peft_config
-            self.base_model = PEFT_TYPE_TO_MODEL_MAPPING[peft_config.peft_type](
-                self.base_model, self.peft_config, adapter_name
-            )
-            self.set_additional_trainable_modules(peft_config, adapter_name)
-        else:
+
+        self._is_prompt_learning = peft_config.is_prompt_learning
+        if self._is_prompt_learning:
+            self._peft_config = {adapter_name: peft_config}
+            self.base_model = model
             self.add_adapter(adapter_name, peft_config)
+        else:
+            self._peft_config = None
+            cls = PEFT_TYPE_TO_MODEL_MAPPING[peft_config.peft_type]
+            self.base_model = cls(model, {adapter_name: peft_config}, adapter_name)
+            self.set_additional_trainable_modules(peft_config, adapter_name)
 
+        self.config = getattr(self.base_model, "config", {"model_type": "custom"})
         if getattr(model, "is_gradient_checkpointing", True):
             model = self._prepare_model_for_gradient_checkpointing(model)
 
@@ -125,6 +126,19 @@ def __init__(self, model: PreTrainedModel, peft_config: PeftConfig, adapter_name
         if hasattr(self.base_model, "config") and hasattr(self.base_model.config, "pretraining_tp"):
             self.base_model.config.pretraining_tp = 1
 
+    @property
+    def peft_config(self) -> Dict[str, PeftConfig]:
+        if self._is_prompt_learning:
+            return self._peft_config
+        return self.base_model.peft_config
+
+    @peft_config.setter
+    def peft_config(self, value: Dict[str, PeftConfig]):
+        if self._is_prompt_learning:
+            self._peft_config = value
+        else:
+            self.base_model.peft_config = value
+
     def save_pretrained(
         self,
         save_directory: str,
@@ -496,10 +510,9 @@ def add_adapter(self, adapter_name: str, peft_config: PeftConfig):
                 f"Found {self.peft_type} and {peft_config.peft_type}."
             )
 
-        self.peft_config[adapter_name] = peft_config
-
         try:
             if peft_config.is_prompt_learning:
+                self.peft_config[adapter_name] = peft_config
                 if hasattr(self.config, "to_dict"):
                     dict_config = self.config.to_dict()
                 else:
@@ -510,9 +523,11 @@ def add_adapter(self, adapter_name: str, peft_config: PeftConfig):
             elif peft_config.is_adaption_prompt:
                 self.base_model.add_adapter(adapter_name, peft_config)
             else:
+                self.peft_config[adapter_name] = peft_config
                 self.base_model.inject_adapter(self, adapter_name)
         except Exception:  # somthing went wrong, roll back
-            del self.peft_config[adapter_name]
+            if adapter_name in self.peft_config:
+                del self.peft_config[adapter_name]
             raise
 
         self.set_additional_trainable_modules(peft_config, adapter_name)
diff --git a/src/peft/tuners/adaption_prompt/model.py b/src/peft/tuners/adaption_prompt/model.py
--- a/src/peft/tuners/adaption_prompt/model.py
+++ b/src/peft/tuners/adaption_prompt/model.py
@@ -45,7 +45,7 @@ def __init__(self, model, configs: Dict, adapter_name: str):
         super().__init__()
         self.model = model
         # Store adapter configs by name.
-        self._configs: Dict[str, AdaptionPromptConfig] = {}
+        self.peft_config: Dict[str, AdaptionPromptConfig] = {}
         # Store lists of the parents of the affected attention modules by adapter name.
         # We keep references to the parents so we can swap the adapters in-and-out of the model.
         self._parents: Dict[str, List[nn.Module]] = {}
@@ -62,7 +62,7 @@ def __init__(self, model, configs: Dict, adapter_name: str):
     def add_adapter(self, adapter_name: str, config: AdaptionPromptConfig) -> None:
         """Add an adapter with the given name and config."""
         config = prepare_config(config, self.model)
-        if adapter_name in self._configs:
+        if adapter_name in self.peft_config:
             raise ValueError(f"Adapter with name '{adapter_name}' already exists.")
 
         parents = []
@@ -87,7 +87,7 @@ def add_adapter(self, adapter_name: str, config: AdaptionPromptConfig) -> None:
         if self._active_adapter is not None and self._enabled:
             self._remove_adapted_attentions(self._active_adapter)
         self._active_adapter = adapter_name
-        self._configs[adapter_name] = config
+        self.peft_config[adapter_name] = config
         self._create_adapted_attentions(config, parents)
         if not self._enabled:
             self._remove_adapted_attentions(self._active_adapter)
@@ -99,7 +99,7 @@ def set_adapter(self, adapter_name: str) -> None:
         """Set the model to use the adapter with the given name."""
         if self._active_adapter == adapter_name:
             return
-        if adapter_name not in self._configs:
+        if adapter_name not in self.peft_config:
             raise ValueError(f"Adapter with name '{adapter_name}' does not exist.")
 
         if self._enabled:
@@ -132,13 +132,13 @@ def _set_adapted_attentions(self, adapter_name: str) -> None:
         """Replace LlamaAttention modules with cached AdaptedAttention modules."""
         cached = self._cached_adapters[adapter_name]
         del self._cached_adapters[adapter_name]
-        config = self._configs[adapter_name]
+        config = self.peft_config[adapter_name]
         for i, par in enumerate(self._parents[adapter_name]):
             setattr(par, config.target_modules, cached[i])
 
     def _remove_adapted_attentions(self, adapter_name: str) -> None:
         """Remove AdaptedAttention modules from the model and store them in the cache."""
-        config = self._configs[adapter_name]
+        config = self.peft_config[adapter_name]
         adapted_attentions = []
         for par in self._parents[adapter_name]:
             attn = getattr(par, config.target_modules)


Test patch: