Pegasus models load very slowly or do not load at all on initial execution of from_pretrained() when Python is spawned from within a Node.js process
Author: wehriam
Pull request: no
Labels: name: wontfix

## Description

We are using Node.js to coordinate processes, including a Python web server that loads a pretrained model in CPU mode.

The model files are downloaded and cached prior to execution during a container build process.

On initial execution, it appears the model stalls while loading the primary weights file from the local cache. Subsequent executions do not stall at this point and load successfully.

Are there any threading caveats we should be aware of when using a coordinating process? Issue #7516 noticed problems with Celery outside of single-pool mode.

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->
     
- `transformers` version: 3.4.0
- Platform: Docker, RHEL8 UBI Base Image
- Python version: Python 3.8.0 (default, Mar  9 2020, 18:02:46), [GCC 8.3.1 20191121 (Red Hat 8.3.1-5)] on linux
- PyTorch version (GPU?): 1.7.0+cpu, no GPU
- Tensorflow version (GPU?): N/A
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No
- System: AWS t3.large image running Docker 

### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people. -->
 
 Pegasus: @patrickvonplaten
 documentation: @sgugger

### Thank you!

Thank you to the Hugging Face team and contributors to this project. The barriers to entry on complex NLP tasks have been lowered substantially, and it's been a huge amount of fun exploring with these models.

## Information

Model I am using: tuner007/pegasus_paraphrase

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [x] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. Spawn a Python webserver that loads a Pegasus model using `from_pretrained(...)`
2. The initial load takes upwards of 300 seconds or fails
3. Subsequent loads take 60 seconds but succeed

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

Load time of Pegasus models should be consistent.

<!-- A clear and concise description of what you would expect to happen. -->

Docker runs in privileged mode with full access to host memory and network.

## Source Code and Logs

```bash
sudo docker run -d \
    -u 0:0 \
    --ipc=host \
    --privileged \
    --network host \
    --restart always \
    --log-driver json-file \
    --log-opt max-size=100m \
    --log-opt max-file=5 \
    -v /root/.example:/root/.example \
    --name example \
    example:1.0
```

Node.js management script spawns a detached Python process.

```js
// @flow

const uuid = require('uuid');
const { spawn } = require('child_process');
const superagent = require('superagent');
const makeLogger = require('../logger');
const killProcess = require('../lib/kill-process');
const commandExists = require('command-exists');

const pythonWebserverExistsPromise = new Promise((resolve, reject) => {
  commandExists('python3.8', (error:Error, exists:boolean) => {
    if (error) {
      reject(error);
    } else {
      resolve(exists);
    }
  });
});

class PythonWebserverDoesNotExistError extends Error {}

module.exports.PythonWebserverDoesNotExistError = PythonWebserverDoesNotExistError;

module.exports.startPythonWebserver = async (port:number, path:string) => {
  const logger = makeLogger(`Python Webserver ${path}`);
  const exists = await pythonWebserverExistsPromise;
  if (!exists) {
    logger.error('python3.8 does not exist on path');
    throw new PythonWebserverDoesNotExistError('python3.8 does not exist on path');
  }
  const pythonWebserverArgs = [path, `${port}`];
  let isManuallyClosed = false;
  let mainProcess;
  let pid;
  let isReadyPromise = Promise.resolve();
  const spawnPythonWebserver = () => {
    logger.info(`Spawning ${path}`);
    mainProcess = spawn('python3.8', pythonWebserverArgs, { windowsHide: true, detached: true, shell: true, env: Object.assign({}, {}, process.env) });
    pid = mainProcess.pid;
    if (!pid) {
      logger.error('Process did not spawn with PID');
      try {
        mainProcess.kill();
      } catch (error) {
        logger.error('Unable to kill process');
        logger.errorStack(error);
      }
      throw new Error('Python webserver process did not spawn');
    }
    let isClosed = false;
    isManuallyClosed = false;
    mainProcess.stdout.on('data', (data) => {
      data.toString('utf8').trim().split('\n').forEach((line) => logger.info(line));
    });
    mainProcess.stderr.on('data', (data) => {
      data.toString('utf8').trim().split('\n').forEach((line) => logger.error(line));
    });
    mainProcess.on('error', (error) => {
      logger.errorStack(error);
    });
    mainProcess.on('close', (code:number) => {
      if (code === 0 || code === null) {
        logger.info('Process closed');
      } else {
        logger.error(`Failed with exit code ${code}`);
      }
      isClosed = true;
      if (!isManuallyClosed) {
        spawnPythonWebserver();
      }
    });
    logger.info('Listening');
    isReadyPromise = (async () => {
      let lastError;
      const connectionTimeout = Date.now() + 300000;
      const start = Date.now();
      while (true) {
        if (isManuallyClosed && isClosed) {
          return;
        }
        if (isClosed || Date.now() > connectionTimeout) {
          await close();
          logger.error(`Unable to conect to ${path} at http://127.0.0.1:${port} after 300 seconds`);
          if (lastError) {
            logger.errorStack(lastError);
            throw new Error(`Unable to conect to python webserver ${path} at http://127.0.0.1:${port} after 300 seconds: ${lastError.message}`);
          } else {
            throw new Error(`Unable to conect to python webserver ${path} at http://127.0.0.1:${port} after 300 seconds`);
          }
        }
        try {
          await superagent.get(`http://127.0.0.1:${port}/${uuid.v4()}`).timeout({ response: 3000, deadline: 3000 });
          break;
        } catch (error) {
          if (error && error.response && error.response.statusCode === 404) {
            break;
          } else {
            lastError = error;
          }
        }
        await new Promise((resolve) => setTimeout(resolve, 1000));
      }
      logger.info(`Connected to ${path} at http://127.0.0.1:${port} after ${Math.round(10 * (Date.now() - start) / 1000) / 10} seconds`);
    })();
  };
  spawnPythonWebserver();
  const close = async () => {
    isManuallyClosed = true;
    logger.info('Shutting down');
    if (pid) {
      await killProcess(pid, 'python webserver');
    } else {
      logger.warn('PID not found');
    }
    logger.info('Shut down');
  };
  const checkIsReady = () => isReadyPromise;
  return [close, checkIsReady];
};

```

Python uses a Tornado web server to host an inference API.

```python
import tornado.ioloop
import tornado.web
import sys
import gc
import json
import numpy as np
from os.path import expanduser
from transformers import PegasusForConditionalGeneration, PegasusTokenizer, logging
import torch
from multiprocessing import cpu_count

executor = tornado.concurrent.futures.ThreadPoolExecutor(max_workers=2)

def chunks(lst, n):
    """Yield successive n-sized chunks from lst."""
    for i in range(0, len(lst), n):
        yield lst[i:i + n]

class NumpyEncoder(json.JSONEncoder):
  def default(self, obj):
    if isinstance(obj, np.ndarray):
      return obj.tolist()
    return json.JSONEncoder.default(self, obj)

@torch.no_grad()
def generate(tokenizer, model, sentences):
  batch = tokenizer.prepare_seq2seq_batch(sentences, truncation=True, padding='longest', max_length=60)
  translated = model.generate(**batch, num_beams=3, repetition_penalty=2.0, length_penalty=0.4, do_sample=True, temperature=0.8)
  return tokenizer.batch_decode(translated, skip_special_tokens=True)

class Summarization(tornado.web.RequestHandler):
  def initialize(self, model, tokenizer):
      self.model = model
      self.tokenizer = tokenizer

  def set_default_headers(self):
    self.set_header('Content-Type', 'application/json')

  @tornado.gen.coroutine
  def post(self):
    body = json.loads(self.request.body.decode())
    if not isinstance(body, list):
      raise web.HTTPError(400, 'Invalid request body. A valid JSON document containing an array of strings is required.')
    for sentence in body: 
      if not isinstance(sentence, str):
        raise web.HTTPError(400, 'Invalid request body. A valid JSON document containing an array of strings is required.')
    result = []
    sentence_groups = list(chunks(body, 20))
    for sentence_group in sentence_groups: 
      decoded = yield executor.submit(generate, self.tokenizer, self.model, sentence_group)
      result = result + decoded
      gc.collect()
    self.finish(json.dumps(result, cls=NumpyEncoder))    

def make_app(model, tokenizer):
  return tornado.web.Application([
    (r"/summarization", Summarization, dict(model=model, tokenizer=tokenizer))
  ])

if __name__ == "__main__":
  torch.set_grad_enabled(False)
  logging.set_verbosity_debug()
  torch_device = 'cuda' if torch.cuda.is_available() else 'cpu'
  model = PegasusForConditionalGeneration.from_pretrained('tuner007/pegasus_paraphrase').to(torch_device)
  tokenizer = PegasusTokenizer.from_pretrained('tuner007/pegasus_paraphrase')
  model.share_memory()
  app = make_app(model, tokenizer)
  app.listen(sys.argv[1])
  gc.collect()
  tornado.ioloop.IOLoop.current().start()
```

Debug output on initial execution, model fails to load after 300s.

```
Python Webserver /example/python/nlp-tornado-summarization.py - error - loading configuration file https://s3.amazonaws.com/models.huggingface.co/bert/tuner007/pegasus_paraphrase/config.json from cache at /root/.cache/torch/transformers/6aa2f0999c84ce856faa292c839572741e6591fae603fe7245e31c2420c621b1.2e89bfaa32f367525ed659d47352d25c26f87f779656ee16db23d056fe7cfc78
Python Webserver /example/python/nlp-tornado-summarization.py - error - Model config PegasusConfig {
Python Webserver /example/python/nlp-tornado-summarization.py - error -   "activation_dropout": 0.1,
Python Webserver /example/python/nlp-tornado-summarization.py - error -   "activation_function": "relu",
Python Webserver /example/python/nlp-tornado-summarization.py - error -   "add_bias_logits": false,
Python Webserver /example/python/nlp-tornado-summarization.py - error -   "add_final_layer_norm": true,
Python Webserver /example/python/nlp-tornado-summarization.py - error -   "architectures": [
Python Webserver /example/python/nlp-tornado-summarization.py - error -     "PegasusForConditionalGeneration"
Python Webserver /example/python/nlp-tornado-summarization.py - error -   ],
Python Webserver /example/python/nlp-tornado-summarization.py - error -   "attention_dropout": 0.1,
Python Webserver /example/python/nlp-tornado-summarization.py - error -   "bos_token_id": 0,
Python Webserver /example/python/nlp-tornado-summarization.py - error -   "classif_dropout": 0.0,
Python Webserver /example/python/nlp-tornado-summarization.py - error -   "classifier_dropout": 0.0,
Python Webserver /example/python/nlp-tornado-summarization.py - error -   "d_model": 1024,
Python Webserver /example/python/nlp-tornado-summarization.py - error -   "decoder_attention_heads": 16,
Python Webserver /example/python/nlp-tornado-summarization.py - error -   "decoder_ffn_dim": 4096,
Python Webserver /example/python/nlp-tornado-summarization.py - error -   "decoder_layerdrop": 0.0,
Python Webserver /example/python/nlp-tornado-summarization.py - error -   "decoder_layers": 16,
Python Webserver /example/python/nlp-tornado-summarization.py - error -   "do_blenderbot_90_layernorm": false,
Python Webserver /example/python/nlp-tornado-summarization.py - error -   "dropout": 0.1,
Python Webserver /example/python/nlp-tornado-summarization.py - error -   "encoder_attention_heads": 16,
Python Webserver /example/python/nlp-tornado-summarization.py - error -   "encoder_ffn_dim": 4096,
Python Webserver /example/python/nlp-tornado-summarization.py - error -   "encoder_layerdrop": 0.0,
Python Webserver /example/python/nlp-tornado-summarization.py - error -   "encoder_layers": 16,
Python Webserver /example/python/nlp-tornado-summarization.py - error -   "eos_token_id": 1,
Python Webserver /example/python/nlp-tornado-summarization.py - error -   "extra_pos_embeddings": 1,
Python Webserver /example/python/nlp-tornado-summarization.py - error -   "force_bos_token_to_be_generated": false,
Python Webserver /example/python/nlp-tornado-summarization.py - error -   "id2label": {
Python Webserver /example/python/nlp-tornado-summarization.py - error -     "0": "LABEL_0",
Python Webserver /example/python/nlp-tornado-summarization.py - error -     "1": "LABEL_1",
Python Webserver /example/python/nlp-tornado-summarization.py - error -     "2": "LABEL_2"
Python Webserver /example/python/nlp-tornado-summarization.py - error -   },
Python Webserver /example/python/nlp-tornado-summarization.py - error -   "init_std": 0.02,
Python Webserver /example/python/nlp-tornado-summarization.py - error -   "is_encoder_decoder": true,
Python Webserver /example/python/nlp-tornado-summarization.py - error -   "label2id": {
Python Webserver /example/python/nlp-tornado-summarization.py - error -     "LABEL_0": 0,
Python Webserver /example/python/nlp-tornado-summarization.py - error -     "LABEL_1": 1,
Python Webserver /example/python/nlp-tornado-summarization.py - error -     "LABEL_2": 2
Python Webserver /example/python/nlp-tornado-summarization.py - error -   },
Python Webserver /example/python/nlp-tornado-summarization.py - error -   "length_penalty": 0.8,
Python Webserver /example/python/nlp-tornado-summarization.py - error -   "max_length": 60,
Python Webserver /example/python/nlp-tornado-summarization.py - error -   "max_position_embeddings": 60,
Python Webserver /example/python/nlp-tornado-summarization.py - error -   "model_type": "pegasus",
Python Webserver /example/python/nlp-tornado-summarization.py - error -   "normalize_before": true,
Python Webserver /example/python/nlp-tornado-summarization.py - error -   "normalize_embedding": false,
Python Webserver /example/python/nlp-tornado-summarization.py - error -   "num_beams": 8,
Python Webserver /example/python/nlp-tornado-summarization.py - error -   "num_hidden_layers": 16,
Python Webserver /example/python/nlp-tornado-summarization.py - error -   "pad_token_id": 0,
Python Webserver /example/python/nlp-tornado-summarization.py - error -   "scale_embedding": true,
Python Webserver /example/python/nlp-tornado-summarization.py - error -   "static_position_embeddings": true,
Python Webserver /example/python/nlp-tornado-summarization.py - error -   "vocab_size": 96103
Python Webserver /example/python/nlp-tornado-summarization.py - error - }
Python Webserver /example/python/nlp-tornado-summarization.py - error - loading weights file https://cdn.huggingface.co/tuner007/pegasus_paraphrase/pytorch_model.bin from cache at /root/.cache/torch/transformers/387ce6aee5feafa70429f4659a02b7433a17ea8b0a6c5cad24e894cc46c7b88e.37d8caa66cfa802d672246ab9f2f72b886c1a58ac1ba12892a05c17d8b0d421f
Python Webserver /example/python/nlp-tornado-summarization.py - info - Shutting down
Process Killer - info - Sending SIGTERM to python webserver process 38
Python Webserver /example/python/nlp-tornado-summarization.py - info - Process closed
Process Killer - info - Stopped python webserver process 38 with SIGTERM
Python Webserver /example/python/nlp-tornado-summarization.py - info - Shut down
Python Webserver /example/python/nlp-tornado-summarization.py - error - Unable to conect to /example/python/nlp-tornado-summarization.py at http://127.0.0.1:43777 after 300 seconds
Python Webserver /example/python/nlp-tornado-summarization.py - error -  Error: connect ECONNREFUSED 127.0.0.1:43777
Python Webserver /example/python/nlp-tornado-summarization.py - error -      at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1145:16)
Python Webserver /example/python/nlp-tornado-summarization.py - error -  errno: "ECONNREFUSED"
Python Webserver /example/python/nlp-tornado-summarization.py - error -  code: "ECONNREFUSED"
Python Webserver /example/python/nlp-tornado-summarization.py - error -  syscall: "connect"
Python Webserver /example/python/nlp-tornado-summarization.py - error -  address: "127.0.0.1"
Python Webserver /example/python/nlp-tornado-summarization.py - error -  port: 43777
Python Webserver /example/python/nlp-tornado-summarization.py - error -  response: undefined
```
Debug output on second execution, model loads after 60s.

```
Python Webserver /example/python/nlp-tornado-summarization.py - info - Spawning /example/python/nlp-tornado-summarization.py
Python Webserver /example/python/nlp-tornado-summarization.py - info - Listening
Python Webserver /example/python/nlp-tornado-summarization.py - error - loading configuration file https://s3.amazonaws.com/models.huggingface.co/bert/tuner007/pegasus_paraphrase/config.json from cache at /root/.cache/torch/transformers/6aa2f0999c84ce856faa292c839572741e6591fae603fe7245e31c2420c621b1.2e89bfaa32f367525ed659d47352d25c26f87f779656ee16db23d056fe7cfc78
Python Webserver /example/python/nlp-tornado-summarization.py - error - Model config PegasusConfig {
Python Webserver /example/python/nlp-tornado-summarization.py - error -   "activation_dropout": 0.1,
Python Webserver /example/python/nlp-tornado-summarization.py - error -   "activation_function": "relu",
Python Webserver /example/python/nlp-tornado-summarization.py - error -   "add_bias_logits": false,
Python Webserver /example/python/nlp-tornado-summarization.py - error -   "add_final_layer_norm": true,
Python Webserver /example/python/nlp-tornado-summarization.py - error -   "architectures": [
Python Webserver /example/python/nlp-tornado-summarization.py - error -     "PegasusForConditionalGeneration"
Python Webserver /example/python/nlp-tornado-summarization.py - error -   ],
Python Webserver /example/python/nlp-tornado-summarization.py - error -   "attention_dropout": 0.1,
Python Webserver /example/python/nlp-tornado-summarization.py - error -   "bos_token_id": 0,
Python Webserver /example/python/nlp-tornado-summarization.py - error -   "classif_dropout": 0.0,
Python Webserver /example/python/nlp-tornado-summarization.py - error -   "classifier_dropout": 0.0,
Python Webserver /example/python/nlp-tornado-summarization.py - error -   "d_model": 1024,
Python Webserver /example/python/nlp-tornado-summarization.py - error -   "decoder_attention_heads": 16,
Python Webserver /example/python/nlp-tornado-summarization.py - error -   "decoder_ffn_dim": 4096,
Python Webserver /example/python/nlp-tornado-summarization.py - error -   "decoder_layerdrop": 0.0,
Python Webserver /example/python/nlp-tornado-summarization.py - error -   "decoder_layers": 16,
Python Webserver /example/python/nlp-tornado-summarization.py - error -   "do_blenderbot_90_layernorm": false,
Python Webserver /example/python/nlp-tornado-summarization.py - error -   "dropout": 0.1,
Python Webserver /example/python/nlp-tornado-summarization.py - error -   "encoder_attention_heads": 16,
Python Webserver /example/python/nlp-tornado-summarization.py - error -   "encoder_ffn_dim": 4096,
Python Webserver /example/python/nlp-tornado-summarization.py - error -   "encoder_layerdrop": 0.0,
Python Webserver /example/python/nlp-tornado-summarization.py - error -   "encoder_layers": 16,
Python Webserver /example/python/nlp-tornado-summarization.py - error -   "eos_token_id": 1,
Python Webserver /example/python/nlp-tornado-summarization.py - error -   "extra_pos_embeddings": 1,
Python Webserver /example/python/nlp-tornado-summarization.py - error -   "force_bos_token_to_be_generated": false,
Python Webserver /example/python/nlp-tornado-summarization.py - error -   "id2label": {
Python Webserver /example/python/nlp-tornado-summarization.py - error -     "0": "LABEL_0",
Python Webserver /example/python/nlp-tornado-summarization.py - error -     "1": "LABEL_1",
Python Webserver /example/python/nlp-tornado-summarization.py - error -     "2": "LABEL_2"
Python Webserver /example/python/nlp-tornado-summarization.py - error -   },
Python Webserver /example/python/nlp-tornado-summarization.py - error -   "init_std": 0.02,
Python Webserver /example/python/nlp-tornado-summarization.py - error -   "is_encoder_decoder": true,
Python Webserver /example/python/nlp-tornado-summarization.py - error -   "label2id": {
Python Webserver /example/python/nlp-tornado-summarization.py - error -     "LABEL_0": 0,
Python Webserver /example/python/nlp-tornado-summarization.py - error -     "LABEL_1": 1,
Python Webserver /example/python/nlp-tornado-summarization.py - error -     "LABEL_2": 2
Python Webserver /example/python/nlp-tornado-summarization.py - error -   },
Python Webserver /example/python/nlp-tornado-summarization.py - error -   "length_penalty": 0.8,
Python Webserver /example/python/nlp-tornado-summarization.py - error -   "max_length": 60,
Python Webserver /example/python/nlp-tornado-summarization.py - error -   "max_position_embeddings": 60,
Python Webserver /example/python/nlp-tornado-summarization.py - error -   "model_type": "pegasus",
Python Webserver /example/python/nlp-tornado-summarization.py - error -   "normalize_before": true,
Python Webserver /example/python/nlp-tornado-summarization.py - error -   "normalize_embedding": false,
Python Webserver /example/python/nlp-tornado-summarization.py - error -   "num_beams": 8,
Python Webserver /example/python/nlp-tornado-summarization.py - error -   "num_hidden_layers": 16,
Python Webserver /example/python/nlp-tornado-summarization.py - error -   "pad_token_id": 0,
Python Webserver /example/python/nlp-tornado-summarization.py - error -   "scale_embedding": true,
Python Webserver /example/python/nlp-tornado-summarization.py - error -   "static_position_embeddings": true,
Python Webserver /example/python/nlp-tornado-summarization.py - error -   "vocab_size": 96103
Python Webserver /example/python/nlp-tornado-summarization.py - error - }
Python Webserver /example/python/nlp-tornado-summarization.py - error - loading weights file https://cdn.huggingface.co/tuner007/pegasus_paraphrase/pytorch_model.bin from cache at /root/.cache/torch/transformers/387ce6aee5feafa70429f4659a02b7433a17ea8b0a6c5cad24e894cc46c7b88e.37d8caa66cfa802d672246ab9f2f72b886c1a58ac1ba12892a05c17d8b0d421f
Python Webserver /example/python/nlp-tornado-summarization.py - error - All model checkpoint weights were used when initializing PegasusForConditionalGeneration.
Python Webserver /example/python/nlp-tornado-summarization.py - error - All the weights of PegasusForConditionalGeneration were initialized from the model checkpoint at tuner007/pegasus_paraphrase.
Python Webserver /example/python/nlp-tornado-summarization.py - error - If your task is similar to the task the model of the checkpoint was trained on, you can already use PegasusForConditionalGeneration for predictions without further training.
Python Webserver /example/python/nlp-tornado-summarization.py - error - Model name 'tuner007/pegasus_paraphrase' not found in model shortcut name list (google/pegasus-xsum). Assuming 'tuner007/pegasus_paraphrase' is a path, a model identifier, or url to a directory containing tokenizer files.
Python Webserver /example/python/nlp-tornado-summarization.py - error - loading file https://s3.amazonaws.com/models.huggingface.co/bert/tuner007/pegasus_paraphrase/spiece.model from cache at /root/.cache/torch/transformers/fa4532c0035b101d7abcd5c0c9c34a83288902b66c5616034db1a47643e05c75.efce77b8dcd2c57b109b0d10170fcdcd53f23c21286974d4f66706536758ab6e
Python Webserver /example/python/nlp-tornado-summarization.py - error - loading file https://s3.amazonaws.com/models.huggingface.co/bert/tuner007/pegasus_paraphrase/added_tokens.json from cache at None
Python Webserver /example/python/nlp-tornado-summarization.py - error - loading file https://s3.amazonaws.com/models.huggingface.co/bert/tuner007/pegasus_paraphrase/special_tokens_map.json from cache at /root/.cache/torch/transformers/87ea1eeb171e0c2b3d4a7c9dbef4cb9aa4a7251e3673777ff8b756af93bb1e65.d142dfa55f201f5033fe9ee40eb8fe1ca965dcb0f38b175386020492986d507f
Python Webserver /example/python/nlp-tornado-summarization.py - error - loading file https://s3.amazonaws.com/models.huggingface.co/bert/tuner007/pegasus_paraphrase/tokenizer_config.json from cache at /root/.cache/torch/transformers/9ee22427dfb233033bc52ded6b335bbd3dd17b3698f3349e8aecb3c0ec0a99aa.1598fab009ce003f8802a6055c13134aa3be28abc2cca8db6a881bdc1ef0164e
Python Webserver /example/python/nlp-tornado-summarization.py - error - loading file https://s3.amazonaws.com/models.huggingface.co/bert/tuner007/pegasus_paraphrase/tokenizer.json from cache at None
Python Webserver /example/python/nlp-tornado-summarization.py - info - Connected to /example/python/nlp-tornado-summarization.py at http://127.0.0.1:43777 after 282.4 seconds
Python Webserver /example/python/nlp-tornado-encoding.py - info - Spawning /example/python/nlp-tornado-encoding.py
Python Webserver /example/python/nlp-tornado-encoding.py - info - Listening
Python Webserver /example/python/nlp-tornado-summarization.py - error - WARNING:tornado.access:404 GET /458e8657-7e9f-468c-9b87-4d7383e42df8 (127.0.0.1) 0.54ms
Python Webserver /example/python/nlp-tornado-encoding.py - info - Connected to /example/python/nlp-tornado-encoding.py at http://127.0.0.1:43778 after 65.1 seconds
Python Webserver /example/python/nlp-tornado-encoding.py - error - WARNING:tornado.access:404 GET /77f8c1da-9c3d-4ef0-934a-b95f434349f6 (127.0.0.1) 0.54ms
```

Comments:
maybe @julien-c or @Pierrci can answer this question better

This seems very environment-specific. @wehriam in case paid support is an option, we have a technical support program in beta at api-enterprise@huggingface.co (cc @clmnt @jeffboudier)

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.