transformers pipeline in GCP cloud functions
Author: vijender412
Pull request: no
Labels: name: Help wanted

# ❓ Transformers pipeline in GCP

I am trying to use transformers pipeline in GCP cloud functions. While calling the function the downloading of model is happening everytime. How can we sort this issue. 


Comments:
You could cache the model once in your environment and then load it from there. Just point `from_pretrained` to the directory containing the model and configuration (or tokenizer file if loading the tokenizer) instead of the S3 link.

@LysandreJik Completely understood and right but this is different case here. I am trying to make use of pipeline and load using pipeline (). And now how this can be achieved in GCP. 

The pipeline also accepts directories as models and tokenizers. See the [pipeline documentation](https://huggingface.co/transformers/main_classes/pipelines.html#transformers.pipeline):

- model (str or PreTrainedModel or TFPreTrainedModel, optional, defaults to None) –

  The model that will be used by the pipeline to make predictions. This can be None, **a string checkpoint identifier** or an actual pre-trained model inheriting from PreTrainedModel for PyTorch and TFPreTrainedModel for TensorFlow.

  If None, the default of the pipeline will be loaded.

Thanks @LysandreJik. The documentation link did helped to get better clarity. Will try and get back. 

Tried this way ```
tokenizer = AutoTokenizer.from_pretrained("distilbert-base-cased-distilled-squad")

model = AutoModelForQuestionAnswering.from_pretrained("https://s3.amazonaws.com/models.huggingface.co/bert/distilbert-base-cased-distilled-squad-pytorch_model.bin")

nlp_qa = pipeline('question-answering', model=model, tokenizer=tokenizer)
```
Getting 
`UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 0: invalid start byte`

You can't put a URL like that. It has to be a local file path, like it is shown in the documentation. You can either fetch them and save them to a directory:

```
mkdir local_path
cd local_path

wget https://s3.amazonaws.com/models.huggingface.co/bert/distilbert-base-cased-distilled-squad-pytorch_model.bin
wget https://s3.amazonaws.com/models.huggingface.co/bert/distilbert-base-cased-distilled-squad-config.json
wget https://s3.amazonaws.com/models.huggingface.co/bert/bert-large-cased-vocab.txt
```

or in Python:

```py
model = DistilBertModel.from_pretrained("distilbert-base-cased-distilled-squad")
model.save_pretrained("local_path")

tokenizer = DistilBertTokenizer.from_pretrained("distilbert-base-cased-distilled-squad")
tokenizer.save_pretrained("local_path")
```

You can then access this model/tokenizer:

```py
nlp = pipeline("question-answering", model="local_path", tokenizer="local_path")
```

Thanks @LysandreJik 

@vijender412 may I ask how you got a newer version of pytorch to work on  cloud function? I'm unable to get mine to build with anything later than torch vesrion 1.0.1 which is stopping me from using pipeline :/

I tried to run transformer on Cloud Functions v1 but as expected I could not run it due to the lack of it's resources.

@vijender412 
Did you make it?