Repository: flairNLP/flair

Issue:
CUDA Error during CharacterEmbeddings
**My code**
```python
from flair.embeddings import CharacterEmbeddings

sentence = Sentence('La casa es muy bonita.', use_tokenizer=True)

embedding = CharacterEmbeddings()
embedding.embed(sentence)

for token in sentence:
    print(token)
    print(token.embedding)
```

**Errors**
![error](https://user-images.githubusercontent.com/18166309/51676189-296d4480-1fd6-11e9-885f-12ba930e925c.png)


**Environment:**
 - OS: Ubuntu 18.04.1
 - Version: code from *master* branch
 - Nvidia:
![nvidia](https://user-images.githubusercontent.com/18166309/51676559-38a0c200-1fd7-11e9-831a-380c3f3df58f.png)






Discussion:
Thanks for reporting :) I think this could be fixed by:

Current [code](https://github.com/zalandoresearch/flair/blob/master/flair/embeddings.py#L386):

```python
# chars for rnn processing
chars = torch.LongTensor(tokens_mask)
chars = chars.to(flair.device)

character_embeddings = self.char_embedding(chars).transpose(0, 1)
```

Fix:

```python
# chars for rnn processing
chars = torch.LongTensor(tokens_mask)
chars = chars.to(flair.device)
chars = chars.detach().cpu()   # <-- added

character_embeddings = self.char_embedding(chars).transpose(0, 1)
```

Thank you. It solves the problem.
Could you also try this fix? 

```python
# chars for rnn processing
chars = torch.LongTensor(tokens_mask)
chars = chars.cpu()   # <-- added (don't detach!)

character_embeddings = self.char_embedding(chars).transpose(0, 1)
```

I think that if you detach the vector, the gradients cannot flow into the character model during training and features are not trained. I.e. the character features would stay random. 

If you do not detach with the code above, training will be slower but this way the character features are always trained on the downstream task like proposed by Lample et al., 2016. 

It seemed to be fixed but now I get the same error while executing the real program.

(I tested it with .detach() and without .detach() -> same error)
I think same error occurred when I tried the BERT tutorial example. 
It is also about "RuntimeError: Expected object of backend CPU but got backend CUDA for argument #3 'index'".
Hello @lisette-garciamoya what do you mean by executing the real program? 
Hi @lisette-garciamoya - I was able to understand where the error is coming from. In fact, the original code of the CharacterEmbeddings class is correct. 

However, when you instantiate the CharacterEmbeddings, by default it is only instantiated on CPU. The ModelTrainer then puts it on GPU which is why the training works. But if you instantiate the CharacterEmbeddings yourself, it is only on CPU even if you are on a GPU machine, which causes the error. 

For now, the simplest fix is to do this: 

```python
 from flair.embeddings import CharacterEmbeddings

sentence = Sentence('La casa es muy bonita.', use_tokenizer=True)

embedding = CharacterEmbeddings()
embeddings = embeddings.cuda() # add this line to put the embeddings on CUDA
embedding.embed(sentence)

for token in sentence:
    print(token)
    print(token.embedding)
```

Could you test if this works for you?

We will also set up a PR that fixes this behavior. Default behavior should be that embeddings are instantiated on cuda if available.

Thanks for finding this error and reporting it!

Resolving patch:
diff --git a/flair/embeddings.py b/flair/embeddings.py
--- a/flair/embeddings.py
+++ b/flair/embeddings.py
@@ -17,7 +17,6 @@
 from .data import Dictionary, Token, Sentence
 from .file_utils import cached_path
 
-
 log = logging.getLogger('flair')
 
 
@@ -329,13 +328,14 @@ def extra_repr(self):
     def __str__(self):
         return self.name
 
+
 class CharacterEmbeddings(TokenEmbeddings):
     """Character embeddings of words, as proposed in Lample et al., 2016."""
 
     def __init__(self, path_to_char_dict: str = None):
         """Uses the default character dictionary if none provided."""
 
-        super(CharacterEmbeddings, self).__init__()
+        super().__init__()
         self.name = 'Char'
         self.static_embeddings = False
 
@@ -353,6 +353,8 @@ def __init__(self, path_to_char_dict: str = None):
 
         self.__embedding_length = self.char_embedding_dim * 2
 
+        self.to(flair.device)
+
     @property
     def embedding_length(self) -> int:
         return self.__embedding_length
@@ -909,8 +911,7 @@ def _add_embeddings_internal(self, sentences: List[Sentence]) -> List[Sentence]:
         all_input_masks = all_input_masks.to(flair.device)
 
         # put encoded batch through BERT model to get all hidden states of all encoder layers
-        if torch.cuda.is_available():
-            self.model.cuda()
+        self.model.to(flair.device)
         self.model.eval()
         all_encoder_layers, _ = self.model(all_input_ids, token_type_ids=None, attention_mask=all_input_masks)
 


Test patch: