Repository: deepset-ai/haystack

Issue:
How to load a local model which is finetuned from huggingface mode? 
**Is your feature request related to a problem? Please describe.**
We finetuned a llama-7B model and tried to load it in following pipeline, but the exception occured.

```
- name: PModel
  params:
    #model_name_or_path: EleutherAI/gpt-j-6B
    model_name_or_path: /home/user/llama-7b-hf
    use_gpu: false
  type: PromptModel
- name: Prompter
  params:
    model_name_or_path: PModel
    default_prompt_template: question-answering
  type: PromptNode

```

Exception:
    self.model_invocation_layer = self.create_invocation_layer(invocation_layer_class=invocation_layer_class)
  File "/opt/conda/lib/python3.9/site-packages/haystack/nodes/prompt/prompt_model.py", line 103, in create_invocation_layer
    raise ValueError(
ValueError: Model /home/user/llama-7b-hf is not supported - no matching invocation layer found. Currently supported invocation layers are: [<class 'haystack.nodes.prompt.invocation_layer.open_ai.OpenAIInvocationLayer'>, <class 'haystack.nodes.prompt.invocation_layer.azure_open_ai.AzureOpenAIInvocationLayer'>, <class 'haystack.nodes.prompt.invocation_layer.chatgpt.ChatGPTInvocationLayer'>, <class 'haystack.nodes.prompt.invocation_layer.azure_chatgpt.AzureChatGPTInvocationLayer'>, <class 'haystack.nodes.prompt.invocation_layer.hugging_face.HFLocalInvocationLayer'>, <class 'haystack.nodes.prompt.invocation_layer.hugging_face_inference.HFInferenceEndpointInvocationLayer'>] You can implement and provide custom invocation layer for /home/user/llama-7b-hf by subclassing PromptModelInvocationLayer.


**Describe the solution you'd like**
The HFLocalInvocationLayer can load the local model.

**Describe alternatives you've considered**
Do not use the supports function to determine whether a specific task is supported. The transformers.pipeline.get_task doen't support the offline model.  In fact, users should know what task_name to use and which tasks their own model can execute should be known. Perhaps haystack should specify which InvocationLayer and task to use in the promptModel of the YAML file.  

**Additional context**
I workaround the supports function to return true alwasy and added the task_name in transformers.pipeine initialization. The HFLocalInvocationLayer can load the local model.



Discussion:
My impression is that this type of model is currently not supported by the `PromptNode`.
I tag @vblagoje who can provide more details.
Is it in Haystack's plan to support this? From me, it's a pretty common use case

Resolving patch:
diff --git a/haystack/nodes/prompt/invocation_layer/hugging_face.py b/haystack/nodes/prompt/invocation_layer/hugging_face.py
--- a/haystack/nodes/prompt/invocation_layer/hugging_face.py
+++ b/haystack/nodes/prompt/invocation_layer/hugging_face.py
@@ -1,5 +1,6 @@
 from typing import Optional, Union, List, Dict
 import logging
+import os
 
 import torch
 from transformers import (
@@ -252,6 +253,9 @@ def _ensure_token_limit(self, prompt: Union[str, List[Dict[str, str]]]) -> Union
     @classmethod
     def supports(cls, model_name_or_path: str, **kwargs) -> bool:
         task_name: Optional[str] = None
+        if os.path.exists(model_name_or_path):
+            return True
+
         try:
             task_name = get_task(model_name_or_path, use_auth_token=kwargs.get("use_auth_token", None))
         except RuntimeError:


Test patch: