Repository: sktime/sktime
Pull request: Enable the HF Transformers Forecaster to pass model as object
Fixes https://github.com/sktime/sktime/issues/6561
Here’s a summary of the PR content in bullet points:

- **Extended functionality**: The PR enhances the `HFTransformersForecaster` class.
- **Dual model input**: It now accepts:
  - A **string parameter** for loading a pretrained Hugging Face model.
  - A **transformers model object** for users to pass their own fine-tuned models.
- **Increased flexibility**: Users are no longer limited to using only pretrained models from Hugging Face.
- **Custom model support**: Enables users to work with their own models, which could be fine-tuned or custom models.


Problem statement:
[ENH] Enable the HF Transformers Forecaster to pass object.
**Is your feature request related to a problem? Please describe.**
Currently, it is only possible to pass a string to the sktime Adapter. With this string, the model is downloaded. I would like to have the option to pass an actual hf transformers object. This would enable the users to better fit own models instead of using only pretrained once from hf.

Hints from issue discussion:
@benHeid 
To let the adapter accept the transformers model object along with the string implementation, I propose modifying the core class `HFTransformersForecaster` where it will also be consistent with the backend compatibility of accepting the model name and downloading it.

**Code Changes**

```{python}
## Modifying the __init__ class
from transformers import AutoModelForTimeSeriesForecasting, PreTrainedModel

class HFTransformersForecaster:
    def __init__(self, model, training_args, config):
        if isinstance(model, str):
            # If the model is a string, download the model using the model name
            self.model = AutoModelForTimeSeriesForecasting.from_pretrained(model)
        elif isinstance(model, PreTrainedModel):
            # If the model is an instance of a Hugging Face Transformers model, use it directly
            self.model = model
        else:
            raise ValueError("Model should be either a string (model name) or an instance of a Hugging Face Transformers model")
        
        self.training_args = training_args
        self.config = config

    def fit(self, y):
        # Fitting logic
        pass

    def predict(self, fh):
        # Prediction logic
        pass
```

**Usage Examples**
```{python}
from sktime.forecasting.hf_transformers_forecaster import HFTransformersForecaster
from sktime.datasets import load_airline

# Load data
y = load_airline()

# Create forecaster using a model name
forecaster = HFTransformersForecaster(
    model="huggingface/autoformer-tourism-monthly",
    training_args={
        "num_train_epochs": 20,
        "output_dir": "test_output",
        "per_device_train_batch_size": 32,
    },
    config={
        "lags_sequence": [1, 2, 3],
        "context_length": 2,
        "prediction_length": 4,
        "use_cpu": True,
        "label_length": 2,
    },
)

# Fit the model
forecaster.fit(y)

# Forecasting horizon
fh = [1, 2, 3]

# Make predictions
y_pred = forecaster.predict(fh)
```

> Note
Currently [this](https://www.sktime.net/en/latest/api_reference/auto_generated/sktime.forecasting.hf_transformers_forecaster.HFTransformersForecaster.html#hftransformersforecaster) is the implemented version of calling HF models using strings to attain downloadable models



@benHeid 
I would like to work on this issue!!
I assigned you
@benHeid 
Thanks!
Hi @fkiraly and @Spinachboul is somebody working on this issue?
I can take this up only if the issue is free, otherwise @Spinachboul if you are working on this, can you create a draft PR? I also had some ideas, I can review some things there and probably help you with the process where I could.
Yeah! I'll soon open the draft PR
Original PR had some commit issues
You can then suggest your ideas there
Works?
Glad to hear, I'll wait for your PR then
You can work on this PR
@geetu040 
@benHeid @fkiraly 
The commits for the PR related to this PR were corrupted and I would like to re-work on this issue
For that, I will be opening a new PR with a new branch for the same.
Thanks!!

Currently `HFTransformerForecaster` looks like this
```py
class HFTransformerForecaster:
    def __init__(self, model_path: str, ...):
        ...
```
@benHeid how did you picture it after this issue? While ensuring the backward-compatibility, I think we have 2 options:
1. model_path accepts the model object
```py
class HFTransformerForecaster:
    def __init__(self, model_path: Union[str, PreTrainedModel], ...):
        ...
```
2. introduce a new parameter at the end
```py
class HFTransformerForecaster:
    def __init__(self, model_path: Optional[str] = None, ..., model: Optional[PreTrainedModel] = None):
        ...
```

Commit message: add test_hf_transformers_forecaster

Patch to review:
diff --git a/sktime/forecasting/hf_transformers_forecaster.py b/sktime/forecasting/hf_transformers_forecaster.py
index 5efb7793524..db62f2fbd22 100644
--- a/sktime/forecasting/hf_transformers_forecaster.py
+++ b/sktime/forecasting/hf_transformers_forecaster.py
@@ -30,9 +30,12 @@ class HFTransformersForecaster(BaseForecaster):
 
     Parameters
     ----------
-    model_path : str
-        Path to the huggingface model to use for forecasting. Currently,
-        Informer, Autoformer, and TimeSeriesTransformer are supported.
+    model_path : str or PreTrainedModel
+        Either:
+        - A string specifying the Hugging Face model name or path
+          (e.g., `"huggingface/autoformer-tourism-monthly"`).
+        - An instance of a `PreTrainedModel`, allowing manual initialization
+          and configuration.
     fit_strategy : str, default="minimal"
         Strategy to use for fitting (fine-tuning) the model. This can be one of
         the following:
@@ -43,10 +46,10 @@ class HFTransformersForecaster(BaseForecaster):
           performance but requires more computational power and time.
         - "peft": Applies Parameter-Efficient Fine-Tuning (PEFT) techniques to adapt
           the model with fewer trainable parameters, saving computational resources.
+
           Note: If the 'peft' package is not available, a `ModuleNotFoundError` will
           be raised, indicating that the 'peft' package is required. Please install
           it using `pip install peft` to use this fit strategy.
-
     validation_split : float, default=0.2
         Fraction of the data to use for validation
     config : dict, default={}
@@ -68,64 +71,57 @@ class HFTransformersForecaster(BaseForecaster):
         When `fit_strategy` is set to "peft",
         this will be used to set up PEFT parameters for the model.
         See the `peft` documentation for details.
+    model : transformers.PreTrainedModel, optional
+        An instantiated Hugging Face model object. If provided, this will override
+        the `model_path` and `config` parameters, and the passed model will be used
+        directly for forecasting. In this case, the `model_path` and `config` should
+        be set to None, as they are not needed.
 
     Examples
     --------
+    **Using a Pretrained Model from Hugging Face**
+
     >>> from sktime.forecasting.hf_transformers_forecaster import (
-    ...     HFTransformersForecaster,
+    ...     HFTransformersForecaster
     ... )
     >>> from sktime.datasets import load_airline
     >>> y = load_airline()
     >>> forecaster = HFTransformersForecaster(
-    ...    model_path="huggingface/autoformer-tourism-monthly",
-    ...    training_args ={
-    ...        "num_train_epochs": 20,
-    ...        "output_dir": "test_output",
-    ...        "per_device_train_batch_size": 32,
-    ...    },
-    ...    config={
+    ...     model_path="huggingface/autoformer-tourism-monthly",
+    ...     fit_strategy="minimal",
+    ...     training_args={
+    ...         "num_train_epochs": 10,
+    ...         "output_dir": "output",
+    ...         "per_device_train_batch_size": 4
+    ...     },
+    ...     config={
     ...         "lags_sequence": [1, 2, 3],
     ...         "context_length": 2,
-    ...         "prediction_length": 4,
-    ...         "use_cpu": True,
-    ...         "label_length": 2,
-    ...    },
-    ... ) # doctest: +SKIP
-    >>> forecaster.fit(y) # doctest: +SKIP
-    >>> fh = [1, 2, 3]
-    >>> y_pred = forecaster.predict(fh) # doctest: +SKIP
+    ...         "prediction_length": 4
+    ...     }
+    ... )  # doctest: +SKIP
+    >>> forecaster.fit(y)  # doctest: +SKIP
 
-    >>> from sktime.forecasting.hf_transformers_forecaster import (
-    ...     HFTransformersForecaster,
-    ... )
-    >>> from sktime.datasets import load_airline
-    >>> from peft import LoraConfig
-    >>> y = load_airline()
+    **Using an Initialized Model**
+
+    >>> from transformers import AutoModelForSeq2SeqLM, AutoConfig
+    >>> config = AutoConfig.from_pretrained("huggingface/autoformer-tourism-monthly")
+    >>> model = AutoModelForSeq2SeqLM.from_config(config)
     >>> forecaster = HFTransformersForecaster(
-    ...    model_path="huggingface/autoformer-tourism-monthly",
-    ...    fit_strategy="peft",
-    ...    training_args={
-    ...        "num_train_epochs": 20,
-    ...        "output_dir": "test_output",
-    ...        "per_device_train_batch_size": 32,
-    ...    },
-    ...    config={
+    ...     model_path=model,
+    ...     fit_strategy="minimal",
+    ...     training_args={
+    ...         "num_train_epochs": 10,
+    ...         "output_dir": "output",
+    ...         "per_device_train_batch_size": 4
+    ...     },
+    ...     config={
     ...         "lags_sequence": [1, 2, 3],
     ...         "context_length": 2,
-    ...         "prediction_length": 4,
-    ...         "use_cpu": True,
-    ...         "label_length": 2,
-    ...    },
-    ...    peft_config=LoraConfig(
-    ...        r=8,
-    ...        lora_alpha=32,
-    ...        target_modules=["q_proj", "v_proj"],
-    ...        lora_dropout=0.01,
-    ...    )
-    ... ) # doctest: +SKIP
-    >>> forecaster.fit(y) # doctest: +SKIP
-    >>> fh = [1, 2, 3]
-    >>> y_pred = forecaster.predict(fh) # doctest: +SKIP
+    ...         "prediction_length": 4
+    ...     }
+    ... )  # doctest: +SKIP
+    >>> forecaster.fit(y)  # doctest: +SKIP
     """
 
     _tags = {
@@ -144,7 +140,7 @@ class HFTransformersForecaster(BaseForecaster):
 
     def __init__(
         self,
-        model_path: str,
+        model_path: str = None,
         fit_strategy="minimal",
         validation_split=0.2,
         config=None,
@@ -171,83 +167,79 @@ def __init__(
         self.peft_config = peft_config
 
     def _fit(self, y, X, fh):
-        from transformers import AutoConfig, Trainer, TrainingArguments
-
-        # Load model and extract config
-        config = AutoConfig.from_pretrained(self.model_path)
-
-        # Update config with user provided config
-        _config = config.to_dict()
-        _config.update(self._config)
-        _config["num_dynamic_real_features"] = X.shape[-1] if X is not None else 0
-        _config["num_static_real_features"] = 0
-        _config["num_dynamic_real_features"] = 0
-        _config["num_static_categorical_features"] = 0
-        _config["num_time_features"] = 0 if X is None else X.shape[-1]
-
-        if hasattr(config, "feature_size"):
-            del _config["feature_size"]
-
-        if fh is not None:
-            _config["prediction_length"] = max(
-                *(fh.to_relative(self._cutoff)._values + 1),
-                _config["prediction_length"],
-            )
+        from transformers import AutoConfig, PreTrainedModel, Trainer, TrainingArguments
 
-        config = config.from_dict(_config)
-        import transformers
+        if isinstance(self.model_path, PreTrainedModel):
+            self.model = self.model_path
+            self.config = self.model.config
 
-        prediction_model_class = None
-        if hasattr(config, "architectures") and config.architectures is not None:
-            prediction_model_class = config.architectures[0]
-        elif hasattr(config, "model_type"):
-            prediction_model_class = (
-                "".join(x.capitalize() for x in config.model_type.lower().split("_"))
-                + "ForPrediction"
-            )
         else:
-            raise ValueError(
-                "The model type is not inferable from the config."
-                "Thus, the model cannot be loaded."
-            )
-        # Load model with the updated config
-        self.model, info = getattr(
-            transformers, prediction_model_class
-        ).from_pretrained(
-            self.model_path,
-            config=config,
-            output_loading_info=True,
-            ignore_mismatched_sizes=True,
-        )
+            # Load model and extract config
+            config = AutoConfig.from_pretrained(self.model_path)
+
+            # Update config with user-provided config
+            _config = config.to_dict()
+            _config.update(self._config)
+            _config["num_dynamic_real_features"] = 0
+            _config["num_static_real_features"] = 0
+            _config["num_static_categorical_features"] = 0
+            _config["num_time_features"] = 0 if X is None else X.shape[-1]
+
+            if hasattr(config, "feature_size"):
+                del _config["feature_size"]
+
+            if fh is not None:
+                _config["prediction_length"] = max(
+                    *(fh.to_relative(self._cutoff)._values + 1),
+                    _config.get("prediction_length", 0),
+                )
 
-        # Freeze all loaded parameters
-        for param in self.model.parameters():
-            param.requires_grad = False
-
-        # Clamp all loaded parameters to avoid NaNs due to large values
-        for param in self.model.model.parameters():
-            param.clamp_(-1000, 1000)
-
-        # Reininit the weights of all layers that have mismatched sizes
-        for key, _, _ in info["mismatched_keys"]:
-            _model = self.model
-            for attr_name in key.split(".")[:-1]:
-                _model = getattr(_model, attr_name)
-            _model.weight = torch.nn.Parameter(
-                _model.weight.masked_fill(_model.weight.isnan(), 0.001),
-                requires_grad=True,
+            config = config.from_dict(_config)
+
+            # Load model and info
+            import transformers
+
+            prediction_model_class = None
+            if hasattr(config, "architectures") and config.architectures:
+                prediction_model_class = config.architectures[0]
+            elif hasattr(config, "model_type"):
+                prediction_model_class = (
+                    "".join(x.capitalize() for x in config.model_type.split("_"))
+                    + "ForPrediction"
+                )
+            else:
+                raise ValueError("The model type cannot be inferred from the config.")
+
+            self.model, self.info = getattr(
+                transformers, prediction_model_class
+            ).from_pretrained(
+                self.model_path,
+                config=config,
+                output_loading_info=True,
+                ignore_mismatched_sizes=True,
             )
 
+            # Freeze loaded parameters and reinitialize mismatched layers
+            for param in self.model.parameters():
+                param.requires_grad = False
+            for key, _, _ in self.info["mismatched_keys"]:
+                _model = self.model
+                for attr_name in key.split(".")[:-1]:
+                    _model = getattr(_model, attr_name)
+                _model.weight = torch.nn.Parameter(
+                    _model.weight.masked_fill(_model.weight.isnan(), 0.001),
+                    requires_grad=True,
+                )
+
+        # Dataset preparation
         if self.validation_split is not None:
             split = int(len(y) * (1 - self.validation_split))
-
             train_dataset = PyTorchDataset(
                 y[:split],
                 config.context_length + max(config.lags_sequence),
                 X=X[:split] if X is not None else None,
                 fh=config.prediction_length,
             )
-
             eval_dataset = PyTorchDataset(
                 y[split:],
                 config.context_length + max(config.lags_sequence),
@@ -261,16 +253,17 @@ def _fit(self, y, X, fh):
                 X=X if X is not None else None,
                 fh=config.prediction_length,
             )
-
             eval_dataset = None
 
+        # Prepare training arguments
         training_args = deepcopy(self.training_args)
         training_args["label_names"] = ["future_values"]
         training_args = TrainingArguments(**training_args)
 
+        # Handle fine-tuning strategy
         if self.fit_strategy == "minimal":
-            if len(info["mismatched_keys"]) == 0:
-                return  # No need to fit
+            if self.model is None and len(self.info["mismatched_keys"]) == 0:
+                return
         elif self.fit_strategy == "full":
             for param in self.model.parameters():
                 param.requires_grad = True
@@ -289,11 +282,13 @@ def _fit(self, y, X, fh):
                 ),
             ):
                 from peft import get_peft_model
+
             peft_config = deepcopy(self.peft_config)
             self.model = get_peft_model(self.model, peft_config)
         else:
             raise ValueError("Unknown fit strategy")
 
+        # Train the model
         trainer = Trainer(
             model=self.model,
             args=training_args,
@@ -381,73 +376,53 @@ def get_test_params(cls, parameter_set="default"):
 
         Returns
         -------
-        params : dict or list of dict, default = {}
-            Parameters to create testing instances of the class
-            Each dict are parameters to construct an "interesting" test instance, i.e.,
-            `MyClass(**params)` or `MyClass(**params[i])` creates a valid test instance.
-            `create_test_instance` uses the first (or only) dictionary in `params`
+        params : list of dict
+            Parameters to create testing instances of the class.
+            Each dict contains parameters to construct an "interesting" test instance,
+            i.e., `MyClass(**params)` creates a valid test instance.
+            `create_test_instance` uses the first (or only) dictionary in `params`.
         """
+        base_training_args = {
+            "num_train_epochs": 1,
+            "output_dir": "test_output",
+            "per_device_train_batch_size": 32,
+        }
+
+        base_config = {
+            "lags_sequence": [1, 2, 3],
+            "context_length": 2,
+            "prediction_length": 4,
+        }
+
         test_params = [
+            # General transformer-based test cases
             {
                 "model_path": "huggingface/informer-tourism-monthly",
                 "fit_strategy": "minimal",
-                "training_args": {
-                    "num_train_epochs": 1,
-                    "output_dir": "test_output",
-                    "per_device_train_batch_size": 32,
-                },
-                "config": {
-                    "lags_sequence": [1, 2, 3],
-                    "context_length": 2,
-                    "prediction_length": 4,
-                },
-                "deterministic": True,
-            },
-            {
-                "model_path": "huggingface/autoformer-tourism-monthly",
-                "fit_strategy": "minimal",
-                "training_args": {
-                    "num_train_epochs": 1,
-                    "output_dir": "test_output",
-                    "per_device_train_batch_size": 32,
-                },
-                "config": {
-                    "lags_sequence": [1, 2, 3],
-                    "context_length": 2,
-                    "prediction_length": 4,
-                    "label_length": 2,
-                },
+                "training_args": base_training_args,
+                "config": base_config,
                 "deterministic": True,
-            },
+            }
         ]
 
+        # Add PEFT-specific test case if PEFT is available
         if _check_soft_dependencies("peft", severity="none"):
             from peft import LoraConfig
 
-            test_params.append(
-                {
-                    "model_path": "huggingface/autoformer-tourism-monthly",
-                    "fit_strategy": "peft",
-                    "training_args": {
-                        "num_train_epochs": 1,
-                        "output_dir": "test_output",
-                        "per_device_train_batch_size": 32,
-                    },
-                    "config": {
-                        "lags_sequence": [1, 2, 3],
-                        "context_length": 2,
-                        "prediction_length": 4,
-                        "label_length": 2,
-                    },
-                    "peft_config": LoraConfig(
-                        r=2,
-                        lora_alpha=8,
-                        target_modules=["q_proj"],
-                        lora_dropout=0.01,
-                    ),
-                    "deterministic": True,
-                }
-            )
+            peft_test_case = {
+                "model_path": "huggingface/autoformer-tourism-monthly",
+                "fit_strategy": "peft",
+                "training_args": base_training_args,
+                "config": {**base_config, "label_length": 2},
+                "peft_config": LoraConfig(
+                    r=2,
+                    lora_alpha=8,
+                    target_modules=["q_proj"],
+                    lora_dropout=0.01,
+                ),
+                "deterministic": True,
+            }
+            test_params.append(peft_test_case)
 
         return test_params
 
diff --git a/sktime/forecasting/tests/test_hf_transformers_forecaster.py b/sktime/forecasting/tests/test_hf_transformers_forecaster.py
new file mode 100644
index 00000000000..bcbafc42ccb
--- /dev/null
+++ b/sktime/forecasting/tests/test_hf_transformers_forecaster.py
@@ -0,0 +1,84 @@
+"""Tests for HFTransformersForecaster."""
+
+__author__ = ["Spinachboul"]
+
+import pytest
+from skbase.utils.dependencies import _check_soft_dependencies
+
+from sktime.tests.test_switch import run_test_for_class
+
+# Check if transformers is installed
+TRANSFORMERS_AVAILABLE = _check_soft_dependencies("transformers", severity="none")
+
+if TRANSFORMERS_AVAILABLE:
+    from transformers import AutoConfig, AutoModelForSeq2SeqLM
+
+    from sktime.forecasting.hf_transformers_forecaster import HFTransformersForecaster
+else:
+    pytest.skip(
+        "Skipping HFTransformersForecaster tests since transformers is not installed.",
+        allow_module_level=True,
+    )
+
+
+@pytest.mark.skipif(
+    not run_test_for_class(HFTransformersForecaster),
+    reason="Run test only if soft dependencies are present and incrementally",
+)
+def test_initialized_model():
+    """Test passing an initialized model to HFTransformersForecaster."""
+    model_name = "huggingface/autoformer-tourism-monthly"
+    config = AutoConfig.from_pretrained(model_name)
+    model = AutoModelForSeq2SeqLM.from_config(config)
+
+    forecaster = HFTransformersForecaster(
+        model_path=model,
+        fit_strategy="minimal",
+        training_args={
+            "num_train_epochs": 1,
+            "output_dir": "test_output",
+            "per_device_train_batch_size": 4,
+        },
+        config={
+            "lags_sequence": [1, 2, 3],
+            "context_length": 2,
+            "prediction_length": 4,
+        },
+    )
+
+    assert forecaster.model_path == model
+
+
+@pytest.mark.skipif(
+    not run_test_for_class(HFTransformersForecaster),
+    reason="Run test only if soft dependencies are present and incrementally.",
+)
+def test_forecaster_pipeline():
+    """Test if HFTransformersForecaster integrates with sktime pipeline."""
+    from sktime.forecasting.compose import ForecastingPipeline
+    from sktime.forecasting.naive import NaiveForecaster
+
+    model_name = "huggingface/autoformer-tourism-monthly"
+    config = AutoConfig.from_pretrained(model_name)
+    model = AutoModelForSeq2SeqLM.from_config(config)
+
+    forecaster = HFTransformersForecaster(
+        model_path=model,
+        fit_strategy="minimal",
+        training_args={
+            "num_train_epochs": 1,
+            "output_dir": "test_output",
+            "per_device_train_batch_size": 4,
+        },
+        config={
+            "lags_sequence": [1, 2, 3],
+            "context_length": 2,
+            "prediction_length": 4,
+        },
+    )
+
+    pipeline = ForecastingPipeline(
+        steps=[("naive", NaiveForecaster()), ("hf_transformer", forecaster)]
+    )
+
+    assert isinstance(pipeline, ForecastingPipeline)


Code review comments:
diff_hunk: @@ -0,0 +1,84 @@
+"""Tests for HFTransformersForecaster."""
+
+__author__ = ["Spinachboul"]
+
+import pytest
+from skbase.utils.dependencies import _check_soft_dependencies
+
+from sktime.tests.test_switch import run_test_for_class
+
+# Check if transformers is installed
+TRANSFORMERS_AVAILABLE = _check_soft_dependencies("transformers", severity="none")
+
+if TRANSFORMERS_AVAILABLE:
+    from transformers import AutoConfig, AutoModelForSeq2SeqLM
+
+    from sktime.forecasting.hf_transformers_forecaster import HFTransformersForecaster
+else:
+    pytest.skip(
+        "Skipping HFTransformersForecaster tests since transformers is not installed.",
+        allow_module_level=True,
+    )
+
+
+@pytest.mark.skipif(
+    not run_test_for_class(HFTransformersForecaster),
+    reason="Run test only if soft dependencies are present and incrementally",
+)
+def test_initialized_model():
+    """Test passing an initialized model to HFTransformersForecaster."""
+    model_name = "huggingface/autoformer-tourism-monthly"
+    config = AutoConfig.from_pretrained(model_name)
+    model = AutoModelForSeq2SeqLM.from_config(config)
+
+    forecaster = HFTransformersForecaster(
+        model_path=model,
+        fit_strategy="minimal",
+        training_args={
+            "num_train_epochs": 1,
+            "output_dir": "test_output",
+            "per_device_train_batch_size": 4,
+        },
+        config={
+            "lags_sequence": [1, 2, 3],
+            "context_length": 2,
+            "prediction_length": 4,
+        },
+    )
+
+    assert forecaster.model_path == model
path: sktime/forecasting/tests/test_hf_transformers_forecaster.py
text: @user1:
- can you try to fit and predict the forecaster as well?
- giving the config is meaningless here, it will be ignored
- you may get errors when running fit and predict because of config mismatch with the input data, if you run into errors, let me know, we'll see how to deal with that.

diff_hunk: @@ -0,0 +1,84 @@
+"""Tests for HFTransformersForecaster."""
+
+__author__ = ["Spinachboul"]
+
+import pytest
+from skbase.utils.dependencies import _check_soft_dependencies
+
+from sktime.tests.test_switch import run_test_for_class
+
+# Check if transformers is installed
+TRANSFORMERS_AVAILABLE = _check_soft_dependencies("transformers", severity="none")
+
+if TRANSFORMERS_AVAILABLE:
+    from transformers import AutoConfig, AutoModelForSeq2SeqLM
+
+    from sktime.forecasting.hf_transformers_forecaster import HFTransformersForecaster
+else:
+    pytest.skip(
+        "Skipping HFTransformersForecaster tests since transformers is not installed.",
+        allow_module_level=True,
+    )
+
+
+@pytest.mark.skipif(
+    not run_test_for_class(HFTransformersForecaster),
+    reason="Run test only if soft dependencies are present and incrementally",
+)
+def test_initialized_model():
+    """Test passing an initialized model to HFTransformersForecaster."""
+    model_name = "huggingface/autoformer-tourism-monthly"
+    config = AutoConfig.from_pretrained(model_name)
+    model = AutoModelForSeq2SeqLM.from_config(config)
path: sktime/forecasting/tests/test_hf_transformers_forecaster.py
text: @user1:
are you sure it works with `AutoModelForSeq2SeqLM`? shouldn't it be `AutoModelForPrediction`?

diff_hunk: @@ -30,9 +30,12 @@ class HFTransformersForecaster(BaseForecaster):
 
     Parameters
     ----------
-    model_path : str
-        Path to the huggingface model to use for forecasting. Currently,
-        Informer, Autoformer, and TimeSeriesTransformer are supported.
path: sktime/forecasting/hf_transformers_forecaster.py
text: @user1:
you should not remove this line