Repository: huggingface/transformers (Python)
Date: 2023-03-15 18:04:01+00:00
Task type: Bug Fix

Issue:
transformers-cli serve not working
### System Info

System info
``` bash
- `transformers` version: 4.27.0
- Platform: macOS-12.3.1-arm64-arm-64bit
- Python version: 3.8.12
- Huggingface_hub version: 0.13.2
- PyTorch version (GPU?): 2.0.0 (False)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>
```


### Who can help?

_No response_

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction


The following command fails for `transformers[serving]==4.27.0`
```bash
transformers-cli serve --task=fill-mask --model=bert-base-uncased
```
this is the traceback
```bash
Traceback (most recent call last):
  File "venv/bin/transformers-cli", line 8, in <module>
    sys.exit(main())
  File "venv/lib/python3.8/site-packages/transformers/commands/transformers_cli.py", line 54, in main
    service = args.func(args)
  File "venv/lib/python3.8/site-packages/transformers/commands/serving.py", line 49, in serve_command_factory
    nlp = pipeline(
  File "venv/lib/python3.8/site-packages/transformers/pipelines/__init__.py", line 976, in pipeline
    return pipeline_class(model=model, framework=framework, task=task, **kwargs)
  File "venv/lib/python3.8/site-packages/transformers/pipelines/base.py", line 773, in __init__
    self.model = self.model.to(device=device)
  File "venv/lib/python3.8/site-packages/transformers/modeling_utils.py", line 1811, in to
    return super().to(*args, **kwargs)
  File "venv/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1126, in to
    device, dtype, non_blocking, convert_to_format = torch._C._nn._parse_to(*args, **kwargs)
RuntimeError: Device index must not be negative
```

### Expected behavior

However, downgrading to `transformers[serving]==4.26.1` fixes the issue

```bash
INFO:     Started server process [22054]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://localhost:8888 (Press CTRL+C to quit)
```


Issue discussion:
cc @Narsil 

Solution patch:
diff --git a/src/transformers/pipelines/base.py b/src/transformers/pipelines/base.py
--- a/src/transformers/pipelines/base.py
+++ b/src/transformers/pipelines/base.py
@@ -769,8 +769,8 @@ def __init__(
         self.modelcard = modelcard
         self.framework = framework
 
-        if self.framework == "pt" and device is not None:
-            self.model = self.model.to(device=device)
+        if self.framework == "pt" and device is not None and not (isinstance(device, int) and device < 0):
+            self.model.to(device)
 
         if device is None:
             # `accelerate` device map


Test patch:
diff --git a/tests/pipelines/test_pipelines_common.py b/tests/pipelines/test_pipelines_common.py
--- a/tests/pipelines/test_pipelines_common.py
+++ b/tests/pipelines/test_pipelines_common.py
@@ -484,6 +484,14 @@ def add(number, extra=0):
         outputs = list(dataset)
         self.assertEqual(outputs, [[{"id": 2}, {"id": 3}, {"id": 4}, {"id": 5}]])
 
+    def test_pipeline_negative_device(self):
+        # To avoid regressing, pipeline used to accept device=-1
+        classifier = pipeline("text-generation", "hf-internal-testing/tiny-random-bert", device=-1)
+
+        expected_output = [{"generated_text": ANY(str)}]
+        actual_output = classifier("Test input.")
+        self.assertEqual(expected_output, actual_output)
+
     @slow
     @require_torch
     def test_load_default_pipelines_pt(self):