Repository: flairNLP/flair

Issue:
add column-name-based embeddings
**Is your feature/enhancement request related to a problem? Please describe.**
There are other token-level types that can be embedded, e.g. part of speech tags or UMLS medical concepts. Right now we can only use embeddings for the token column in the CoNLL format.

**Describe the solution you'd like**
This class roughly implements what I'd like. Not sure whether it should be its own class though because it can also be implemented by adding a few lines to existing classes.

```
class FieldEmbeddings(WordEmbeddings):

    def __init__(self, path : str, field : str):
        super().__init__(path)
        self.field = field

    def _add_embeddings_internal(self, sentences: List[Sentence]) -> List[Sentence]:
        for i, sentence in enumerate(sentences):
            for token, token_idx in zip(sentence.tokens, range(len(sentence.tokens))):
                token: Token = token
                text = token.get_tag(self.field).value
                if text in self.precomputed_word_embeddings:
                    word_embedding = self.precomputed_word_embeddings[text]
                    # self.hits += 1
                elif text.lower() in self.precomputed_word_embeddings:
                    word_embedding = self.precomputed_word_embeddings[text.lower()]
                elif re.sub(r'\d', '#', text.lower()) in self.precomputed_word_embeddings:
                    word_embedding = self.precomputed_word_embeddings[re.sub(r'\d', '#', text.lower())]
                elif re.sub(r'\d', '0', text.lower()) in self.precomputed_word_embeddings:
                    word_embedding = self.precomputed_word_embeddings[re.sub(r'\d', '0', text.lower())]
                else:
                    word_embedding = np.zeros(self.embedding_length, dtype='float')
                word_embedding = torch.FloatTensor(word_embedding)
                token.set_embedding(self.name, word_embedding)
        return sentences
```
And used with:
```
    column_format = {0:'word', 1:'pos', 2:'cui', 3:'ner'}
    embeddings = FieldEmbeddings("resources/embeddings/cui_embeddings.gensim", 'cui')




Discussion:
Hi @tmills that's a good idea - we should add this to Flair. I think we could probably add this as an extra field to the `WordEmbeddings` class, i.e. an optional `field` parameter that is set to `text` per default. 

Would you be interested in making a PR for this?

Also, do you know publicly available pretrained PoS embeddings or similar that we could include as an example in the documentation on how to use the field parameter in the class?

Resolving patch:
diff --git a/flair/embeddings.py b/flair/embeddings.py
--- a/flair/embeddings.py
+++ b/flair/embeddings.py
@@ -144,7 +144,7 @@ def __str__(self):
 class WordEmbeddings(TokenEmbeddings):
     """Standard static word embeddings, such as GloVe or FastText."""
 
-    def __init__(self, embeddings: str):
+    def __init__(self, embeddings: str, field: str = None):
         """
         Initializes classic word embeddings. Constructor downloads required files if not there.
         :param embeddings: one of: 'glove', 'extvec', 'crawl' or two-letter language code.
@@ -206,6 +206,8 @@ def __init__(self, embeddings: str):
 
         self.precomputed_word_embeddings = gensim.models.KeyedVectors.load(str(embeddings))
 
+        self.field = field
+
         self.__embedding_length: int = self.precomputed_word_embeddings.vector_size
         super().__init__()
 
@@ -220,14 +222,19 @@ def _add_embeddings_internal(self, sentences: List[Sentence]) -> List[Sentence]:
             for token, token_idx in zip(sentence.tokens, range(len(sentence.tokens))):
                 token: Token = token
 
-                if token.text in self.precomputed_word_embeddings:
-                    word_embedding = self.precomputed_word_embeddings[token.text]
-                elif token.text.lower() in self.precomputed_word_embeddings:
-                    word_embedding = self.precomputed_word_embeddings[token.text.lower()]
-                elif re.sub(r'\d', '#', token.text.lower()) in self.precomputed_word_embeddings:
-                    word_embedding = self.precomputed_word_embeddings[re.sub(r'\d', '#', token.text.lower())]
-                elif re.sub(r'\d', '0', token.text.lower()) in self.precomputed_word_embeddings:
-                    word_embedding = self.precomputed_word_embeddings[re.sub(r'\d', '0', token.text.lower())]
+                if 'field' not in self.__dict__ or self.field is None:
+                    word = token.text
+                else:
+                    word = token.get_tag(self.field).value
+
+                if word in self.precomputed_word_embeddings:
+                    word_embedding = self.precomputed_word_embeddings[word]
+                elif word.lower() in self.precomputed_word_embeddings:
+                    word_embedding = self.precomputed_word_embeddings[word.lower()]
+                elif re.sub(r'\d', '#', word.lower()) in self.precomputed_word_embeddings:
+                    word_embedding = self.precomputed_word_embeddings[re.sub(r'\d', '#', word.lower())]
+                elif re.sub(r'\d', '0', word.lower()) in self.precomputed_word_embeddings:
+                    word_embedding = self.precomputed_word_embeddings[re.sub(r'\d', '0', word.lower())]
                 else:
                     word_embedding = np.zeros(self.embedding_length, dtype='float')
 


Test patch: