valhalla/distilbart-mnli-12-3' is a correct model identifier listed on 'https://huggingface.co/models'
Author: muiPomeranian
Pull request: no

i was using AWS batch transform with this model and worked fine but since after last week, somehow it complains with this 
message:

```valhalla/distilbart-mnli-12-3' is a correct model identifier listed on 'https://huggingface.co/models'```

but I do see that this is a valid model config. Is this awared from your side ? 

Comments:
Hi! Could you post the full stack trace?

Hello! 
this was only related trace i retrived. 

Again, all works except when i try with aws batch transform with given model.tar.gz created from torchServe. 

It was working and somehow it started to fail with this error (cant find this model path from huggingface model) from last week..

if this happens only on AWS then @philschmid  might have some idea.

Appreciate. Please do let me know as this is quiet unexpected failure which blocks me quiet hard :( 

Hey @muiPomeranian, 
what are you using to run your Batch Transform job? 
What is the content of your `model.tar.gz` folder structure? 

We have a working example for batch transformer using the Hugging Face DLCs here: https://github.com/huggingface/notebooks/blob/master/sagemaker/12_batch_transform_inference/sagemaker-notebook.ipynb
Where you don't need to provide a `model.tar.gz` and can directly use the `MODEL_ID` from the Hub.


Thanks for the answer. 

I was providing model.tar.gz file created by torchServe which contains

handler.py
Finetuned model checkpoint

and handler.py uses AutoTokenizer.from_pretrained(path)

this was working fully since last week... but somehow it complains they cant find above given model config path from huggingface. 

Is there any issue regarding to this? We didnt make any changes on our batch transform script..

Not sure about how this works with using `torchserve` native `pytorch` implementation. If you take a look at the sample i provided, which uses a `HuggingFaceModel` you neither need to provide a `model.tar.gz` nor a `inference.py` to run a Batch transform job. 
```python
from sagemaker.huggingface.model import HuggingFaceModel

# Hub Model configuration. <https://huggingface.co/models>
hub = {
    'HF_MODEL_ID':'valhalla/distilbart-mnli-12-3',
    'HF_TASK':'summarization'
}

# create Hugging Face Model Class
huggingface_model = HuggingFaceModel(
   env=hub, # configuration for loading model from Hub
   role=role, # iam role with permissions to create an Endpoint
   transformers_version="4.6", # transformers version used
   pytorch_version="1.7", # pytorch version used
   py_version='py36', # python version used
)

# create Transformer to run our batch job
batch_job = huggingface_model.transformer(
    instance_count=1,
    instance_type='ml.p3.2xlarge',
    output_path=output_s3_path, # we are using the same s3 path to save the output with the input
    strategy='SingleRecord')

# starts batch transform job and uses s3 data as input
batch_job.transform(
    data=s3_file_uri,
    content_type='application/json',    
    split_type='Line')
```

So is there any ongoing issue between aws sagemaker batch transform and huggingface model loading for tokenizer? I am wondering why suddenly i see this weird error....

Also, how can i use finetuned model with your provided example? I have finetuned model at my checkpoint and i packaged it before but seems the example you provided only works with your model in hf. (I cant release my finetuned model due to company policy) :(

> So is there any ongoing issue between aws sagemaker batch transform and huggingface model loading for tokenizer? I am wondering why suddenly i see this weird error....

Not sure what you mean by that, we haven't seen issue with batch transform and hugging face models, when using the `HuggingFaceModel`. 

I thought you were using the weights from the hub. You can provide them like that 
```python
# create Hugging Face Model Class
huggingface_model = HuggingFaceModel(
   model_data=model_uri, # configuration for loading model from Hub
   role=role, # iam role with permissions to create an Endpoint
   transformers_version="4.6", # transformers version used
   pytorch_version="1.7", # pytorch version used
   py_version='py36', # python version used
  env=hub
)
```

`model_uri` is the s3 uri to your `model.tar.gz` and the `model.tar.gz` just needs to contain the required model & tokenizer files, e.g. `pytorch_model.bin` etc. 

You can create the archive like that, expecting `my_model` contains the weights and tokenizer files
```bash
cd my_model && tar zcvf model.tar.gz * 
```

When using your fine-tuned you might still need to provide the `hub` configuration with 
```python
hub = {
    'HF_TASK':'summarization'
}
```

Appreciate for the response again! 

I may have few more follow ups if you dont mind... 
(1)
So instead of providing custom hanlder.py, we can directly use hf internal handler like

hub = {
    'HF_TASK':'classification'
}
And i guess multiRecord will be supported
right? 

(2)
And also is there any way that i can customize handler? Eg, i want to use customized input_id such as third_input_id and third_attention_mask.

also is there any timeout i could customize? 

(3)
Does this mean that i dont need to use torchServe to create model.tar.gz but just directly make tar file with folder which has weights and tokenizer file? 

cd my_model && tar zcvf model.tar.gz * 



thanks again !! 

(1) Yes, we provide a custom hf internal handler which leverages the `pipelines` module and has a similar API as the Inference-API: documentation here: https://huggingface.co/docs/sagemaker/inference#inference-toolkit---api-description

`multiRecord` is currently not supported by the hf internal handler. See (2) on how to leverage multiRecord

(2) Yes you can customize your handler by creating an `inference.py` script in their you have can overwrite the same methods as you know including `input_fn`, `predict_fn`, `output_fn` and `model_fn`. 
There are two option to provide the `inference.py` afterwards, either you bundle it into the `model.tar.gz` but it needs to be in a `code/` or you add it to your `HuggingFaceModel` as `entry_point`: documentation here: https://huggingface.co/docs/sagemaker/inference#user-defined-codemodules

(3) Yes you can easily manually create those `model.tar.gz` for the `HuggingFaceModel` 

Hello again :( sorry but I swtiched tokenizer to `bart-large-mnli` 

and somehow now i am seeing

``` 
W-9069-model_0-stdout MODEL_LOG - requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://huggingface.co/api/models/facebook/bart-large-mnli
```

Although sagemaker endpoint is able to load tokenizer from huggingface, but somehow batch transform only raise this error... 

Why now im seeing 403 error ... ? 

Are you running with some VPC configuration? 
Can you please share the code you use? 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.