opened
johann-petrak
username_0
The current code assigns the lemmata in the __call__ method of spaCyIWNLP to every token which is fine.

However, when registering the extension, a getter is defined as well which will then ignore what has been stored in the "_" dict and instead re-calculate the lemmas and return those. 

Unless I misunderstand something important here, I think this is a problem, because it will always invoke the wrapper whenever the lemmas for a token are retrieved.

Also, it makes it impossible for a client pipeline to update or set the iwnlp_lemmas field, should it not return anything or return something wrong (e.g. the IWNLP lemmatizer does not create a lemma for punctuation while Spacy does, so in order to always have some lemma, one could set the iwnlp_lemmas attribute to `[token.lemma_]`. Indeed it is possible to set the attribute to this, but it is not possible to retrieve this because it will instead recalculate the value and return that.
Why assign the lemma and re-calculate it using the getter?
issue
created
Liebeck
username_1
Hi @username_0,
thanks for you feedback! I wrote the extension when spaCy v2 was still in alpha / was just released. As far as I recall, the __call__ method is only invoked when creating the pipeline. Did this behavior change?

I've never thought of making the lemma settable but we can of course change that implementation as there is no disadvantage to make it settable. Can you create a pull request? ;-)
comment
created
johann-petrak
username_0
No, the getter is invoked whenever the value of an attribute is retrieved, either through `token._.attrname` or through `token._.get("attrname")`. So in your code, setting the value in `__call__` is superfluous because the getter will get invoked very time and call the wrapper anyways. 

The difference is this: if you define the getter (and `__call__` does nothing), then the lemma field can be retrieved as soon as the spaCyIWNLP object has been created, it does not even need to be part of the Spacy pipeline! On the other hand if you do not set the getter and keep the code in `__call__`, the lemma attribute can only be retrieved after the spaCyWNLP object as been called as part of a Spacy pipeline. 

I think the second solution is more intuitive, and probably faster if the attribute can get accessed many times.

One additional detail is that if you add the "default" parameter to the `set_extension` call, the given default value (which could be None or an empty list in your case) is returned even before the pipeline is run and the actual value has been set by the wrapper. 

So my suggestion would be to simply change the code so that line 10 becomes:
```
Token.set_extension('iwnlp_lemmas', force=True, default=None)
```
comment