Repository: nltk/nltk
Pull request: Make Wordnet interoperable with various taggers and tagged corpora
Fix #3392 by adding a function that uses Universal Tags (Petrov et al., 2012), to convert the tags available in _nltk_data/taggers/universal_tagset_ to Wordnet Part-of-Speech. This allows to restrict Wordnet searches to the relevant grammatical category.

```
from nltk.corpus import wordnet as wn
from nltk.corpus import brown
form,tag = brown.tagged_sents()[0][17]
print((form, tag))
```
('evidence', 'NN')

Restricting the Wordnet search to _n_ discards the verbal senses:

`
print(wn.synsets(form, wn.tag2pos(tag, brown._tagset)))
`

[Synset('evidence.n.01'), Synset('evidence.n.02'), Synset('evidence.n.03')]


Problem statement:
Restrict Wordnet searches when Penn Treebank Tags are available
The NLTK Part-of-Speech tagger outputs Penn Treebank tags, which are often neglected when looking the results up in Wordnet. When a word has senses in different grammatical categories, this may lead to considering lots of spurious senses pertaining to irrelevant parts-of-speech, which could be avoided by leveraging the available _en-ptb_ tags.

But unfortunately, Wordnet searches can only be restricted by using the Wordnet parts-of-speech tags (n, v, a, r), although it would be relatively easy to provide a function that converts _en-ptb_ tags to Wordnet _pos_, by extending the conversion between en-ptb and Universal Tags (Petrov et al., 2012), which is already available through the _nltk.tag.mapping_ module.

Hints from issue discussion:
FWIW, here is one implementation of lemmatization with WordNet based on PTB tags:

https://github.com/nschneid/pysupersensetagger/blob/4783c54a28d8e439beb9d115508abb2dc305fbf3/src/morph.py#L24-L47
Thanks @nschneid, L44 in your code is a concise shortcut of the approach I have in mind. You avoid the detour through Universal Tags by just mapping the first letter of the input tag, but PTB also has 'MD' (modal verb) as the only exception.

The scope of the Universal Tags approach is broader, since it allows to support all the tagsets available in _nltk_data/taggers/universal_tagset_, and can thus provide an interface between Wordnet and other annotated corpora, such as wsj, brown, etc...

Commit message: Update nltk/corpus/reader/wordnet.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Patch to review:
diff --git a/nltk/corpus/reader/wordnet.py b/nltk/corpus/reader/wordnet.py
index f74006b90b..4cd919309b 100644
--- a/nltk/corpus/reader/wordnet.py
+++ b/nltk/corpus/reader/wordnet.py
@@ -2108,6 +2108,23 @@ def filter_forms(forms):
         # 2. Return all that are in the database (and check the original too)
         return filter_forms([form] + forms)
 
+    def tag2pos(self, tag, tagset="en-ptb") -> Optional[str]:
+        """
+        Convert a tag from one of the tagsets in nltk_data/taggers/universal_tagset, to a
+        WordNet Part-of-Speech, using Universal Tags (Petrov et al., 2012) as intermediary.
+        Return None when WordNet does not cover that Pos.
+
+        >>> import nltk
+        >>> tagged = nltk.tag.pos_tag(nltk.tokenize.word_tokenize("Banks check books."))
+        >>> print([(word, tag, nltk.corpus.wordnet.tag2pos(tag)) for word,tag in tagged])
+        [('Banks', 'NNS', 'n'), ('check', 'VBP', 'v'), ('books', 'NNS', 'n'), ('.', '.', None)]
+        """
+
+        from nltk.tag import map_tag
+
+        utag2wnpos = {'NOUN': 'n', 'VERB': 'v', 'ADJ': 'a', 'ADV': 'r'}
+        return utag2wnpos.get(map_tag(tagset, "universal", tag), None)
+
     #############################################################
     # Create information content from corpus
     #############################################################


Code review comments:
diff_hunk: @@ -2108,6 +2108,23 @@ def filter_forms(forms):
         # 2. Return all that are in the database (and check the original too)
         return filter_forms([form] + forms)
 
+    def tag2pos(self, tag, tagset="en-ptb") -> Optional[str]:
+        """
+        Convert a tag from one of the tagsets in nltk_data/taggers/universal_tagset, to a
+        WordNet Part-of-Speech, using Universal Tags (Petrov et al., 2012) as intermediary.
+        Return None when WordNet does not cover that Pos.
+
+        >>> import nltk
+        >>> tagged = nltk.tag.pos_tag(nltk.tokenize.word_tokenize("Banks check books."))
+        >>> print([(word, tag, nltk.corpus.wordnet.tag2pos(tag)) for word,tag in tagged])
+        [('Banks', 'NNS', 'n'), ('check', 'VBP', 'v'), ('books', 'NNS', 'n'), ('.', '.', None)]
+        """
+
+        from nltk.tag import map_tag
+
+        utag2wnpos = {'NOUN': 'n', 'VERB': 'v', 'ADJ': 'a', 'ADV': 'r'}
+        return utag2wnpos.get(map_tag(tagset, "universal", tag), None)
path: nltk/corpus/reader/wordnet.py
text: @user1:
[nitpick] Consider promoting this literal to a module-level constant (e.g., `_UTAG2WN_POS`) so the mapping isn't recreated on each call and follows naming conventions for constants.
```suggestion
        return UTAG2WN_POS.get(map_tag(tagset, "universal", tag), None)
```