Repository: goodmami/wn (language: python)

Issue:
Add progress bar to lmf.load (#46)
Add a progress bar to `lmf.load` (#46 ) function. Since we use generators, as an heuristic we just count lexicon starting tags from the XML source (to reduce memory footprint and to avoid processing twice the generators). For the progress bar I had to add a dependency to `tqdm`.

Here is how it looks like:

![Alt Text](https://media.giphy.com/media/SUOaCW9iRWXPl7yuG5/giphy.gif)
Fetching proposed ILIs from the database is broken
Thanks to a user who emailed a report, it appears that fetching ILIs for proposed ILIs is currently broken:

```python
>>> import wn
>>> wn.synset('ewn-09067337-n').ili
ILI('i84182')
>>> wn.synset('ewn-09067337-n').hyponyms()[1]
Synset('ewn-90010441-n')
>>> wn.synset('ewn-09067337-n').hyponyms()[1].ili
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/goodmami/postdoc/wn/wn/_core.py", line 511, in ili
    row = next(find_proposed_ilis(synset_id=self._id), None)
TypeError: find_proposed_ilis() got an unexpected keyword argument 'synset_id'
```

Web API
Some might find it useful to have a web API to complement the Python API of Wn. It could be developed as a separate project, or built into Wn. I think the built-in solution is appealing except that it incurs a dependency for the web framework library, so it can be an optional requirement (i.e., `pip install wn[web]`).

Since Wn is Python 3.6+, we can use one of the ASGI frameworks like [Starlette](https://www.starlette.io/) or [FastAPI](https://fastapi.tiangolo.com/).

Example endpoints:

    /lexicons
    /lexicons/ewn:2020/
    /lexicons/ewn:2020/
    /lexicons/ewn:2020/synsets?form="dog"&pos="n"
    /lexicons/ewn:2020/synsets/ewn-02086723-n/
    /lexicons/ewn:2020/synsets/ewn-02086723-n/definitions
    /synsets?lexicon="ewn:2020%20frawn:1.3+omw"&form="chat"

One potential issue is that lexicon specifiers use `:` (e.g., `ewn:2020`), which, while valid in the path portion of a URL, can [cause issues](https://www.mediawiki.org/wiki/Manual_talk:Short_URL/Archive_1#Colon_issue_with_Very_Short_URLs_on_Windows) if a Windows machine hosts the server.
Web API
Some might find it useful to have a web API to complement the Python API of Wn. It could be developed as a separate project, or built into Wn. I think the built-in solution is appealing except that it incurs a dependency for the web framework library, so it can be an optional requirement (i.e., `pip install wn[web]`).

Since Wn is Python 3.6+, we can use one of the ASGI frameworks like [Starlette](https://www.starlette.io/) or [FastAPI](https://fastapi.tiangolo.com/).

Example endpoints:

    /lexicons
    /lexicons/ewn:2020/
    /lexicons/ewn:2020/
    /lexicons/ewn:2020/synsets?form="dog"&pos="n"
    /lexicons/ewn:2020/synsets/ewn-02086723-n/
    /lexicons/ewn:2020/synsets/ewn-02086723-n/definitions
    /synsets?lexicon="ewn:2020%20frawn:1.3+omw"&form="chat"

One potential issue is that lexicon specifiers use `:` (e.g., `ewn:2020`), which, while valid in the path portion of a URL, can [cause issues](https://www.mediawiki.org/wiki/Manual_talk:Short_URL/Archive_1#Colon_issue_with_Very_Short_URLs_on_Windows) if a Windows machine hosts the server.
Option to disable queries of non-lemma forms
In the NLTK implementation(s) of Wordnet, it has been requested (by me, for one: nltk/nltk#2421) to be able to search for lemmatic forms without any morphological processing. While this library does not yet have Morphy or any other lemmatizer, the lexicons do have non-lemmatic forms as alternate `<Form>` elements on a `<LexicalEntry>`, and these are searched by default:

```xml
    <LexicalEntry id="ewn-goose-n">
      <Lemma partOfSpeech="n" writtenForm="goose" />
      <Form writtenForm="geese" />
```

Therefore it seems that queries implicitly search for morphological variations. Of course, this depends on the data source. The Japanese wordnet, for instance, does not list morphological inflections but alternate writing systems (original, hiragana, katakana, romaji), and other wordnets don't use `<Form>` at all. There should be a way to disable searching the `<Form>` and only use the `<Lemma>`.

One way to do this is with a default "lemmatizer" (although the term is no longer accurate) that adds all `<Form>` elements as "exceptions" (in the Morphy sense). Thus, by setting `lemmatizer=None` we could disable such queries. This would have a cost, though, as all of those forms need to be pulled into memory upon initializing a Wordnet object. Another option is a flag, e.g.:

```python
wn.Wordnet(..., search_alternate_forms=False)
```



Discussion / hints:

Fix patch:
diff --git a/CHANGELOG.md b/CHANGELOG.md
index f39466a..8537634 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -2,6 +2,42 @@
 
 ## [Unreleased]
 
+## [v0.7.0]
+
+**Release date: 2021-06-09**
+
+### Added
+
+* Support for approximate word searches; on by default, configurable
+  only by instantiating a `wn.Wordnet` object ([#105])
+* `wn.morphy` ([#19])
+* `wn.Wordnet.lemmatizer` attribute ([#8])
+* `wn.web` ([#116])
+* `wn.Sense.relations()` ([#82])
+* `wn.Synset.relations()` ([#82])
+
+### Changed
+
+* `wn.lmf.load()` now takes a `progress_handler` parameter ([#46])
+* `wn.lmf.scan_lexicons()` no longer returns sets of relation types or
+  lexfiles; `wn.add()` now gets these from loaded lexicons instead
+* `wn.util.ProgressHandler`
+  - Now has a `refresh_interval` parameter; updates only trigger a
+    refresh after the counter hits the threshold set by the interval
+  - The `update()` method now takes a `force` parameter to trigger a
+    refresh regardless of the refresh interval
+* `wn.Wordnet`
+  - Initialization now takes a `normalizer` parameter ([#105])
+  - Initialization now takes a `lemmatizer` parameter ([#8])
+  - Initialization now takes a `search_all_forms` parameter ([#115])
+  - `Wordnet.words()`, `Wordnet.senses()` and `Wordnet.synsets()` now
+    use any specified lemmatization or normalization functions to
+    expand queries on word forms ([#105])
+
+### Fixed
+
+* `wn.Synset.ili` for proposed ILIs now works again (#117)
+
 
 ## [v0.6.2]
 
@@ -311,6 +347,7 @@ the https://github.com/nltk/wordnet/ code which had been effectively
 abandoned, but this is an entirely new codebase.
 
 
+[v0.7.0]: ../../releases/tag/v0.7.0
 [v0.6.2]: ../../releases/tag/v0.6.2
 [v0.6.1]: ../../releases/tag/v0.6.1
 [v0.6.0]: ../../releases/tag/v0.6.0
@@ -325,9 +362,12 @@ abandoned, but this is an entirely new codebase.
 [unreleased]: ../../tree/main
 
 [#7]: https://github.com/goodmami/wn/issues/7
+[#8]: https://github.com/goodmami/wn/issues/8
 [#15]: https://github.com/goodmami/wn/issues/15
 [#17]: https://github.com/goodmami/wn/issues/17
+[#19]: https://github.com/goodmami/wn/issues/19
 [#23]: https://github.com/goodmami/wn/issues/23
+[#46]: https://github.com/goodmami/wn/issues/46
 [#47]: https://github.com/goodmami/wn/issues/47
 [#58]: https://github.com/goodmami/wn/issues/58
 [#59]: https://github.com/goodmami/wn/issues/59
@@ -348,6 +388,7 @@ abandoned, but this is an entirely new codebase.
 [#78]: https://github.com/goodmami/wn/issues/78
 [#79]: https://github.com/goodmami/wn/issues/79
 [#81]: https://github.com/goodmami/wn/issues/81
+[#82]: https://github.com/goodmami/wn/issues/82
 [#83]: https://github.com/goodmami/wn/issues/83
 [#86]: https://github.com/goodmami/wn/issues/86
 [#87]: https://github.com/goodmami/wn/issues/87
@@ -362,3 +403,6 @@ abandoned, but this is an entirely new codebase.
 [#105]: https://github.com/goodmami/wn/issues/105
 [#106]: https://github.com/goodmami/wn/issues/106
 [#108]: https://github.com/goodmami/wn/issues/108
+[#115]: https://github.com/goodmami/wn/issues/115
+[#116]: https://github.com/goodmami/wn/issues/116
+[#117]: https://github.com/goodmami/wn/issues/117
diff --git a/docs/api/wn.constants.rst b/docs/api/wn.constants.rst
index 2112965..bc8420c 100644
--- a/docs/api/wn.constants.rst
+++ b/docs/api/wn.constants.rst
@@ -140,6 +140,8 @@ Sense Relations
    - ``other``
 
 
+.. _parts-of-speech:
+
 Parts of Speech
 ---------------
 
diff --git a/docs/api/wn.morphy.rst b/docs/api/wn.morphy.rst
new file mode 100644
index 0000000..5675e28
--- /dev/null
+++ b/docs/api/wn.morphy.rst
@@ -0,0 +1,106 @@
+
+wn.morphy
+=========
+
+.. automodule:: wn.morphy
+
+.. seealso::
+
+   The Princeton WordNet `documentation
+   <https://wordnet.princeton.edu/documentation/morphy7wn>`_ describes
+   the original implementation of Morphy.
+
+   The :doc:`../guides/lemmatization` guide describes how Wn handles
+   lemmatization in general.
+
+
+Initialized and Uninitialized Morphy
+------------------------------------
+
+There are two ways of using Morphy in Wn: initialized and
+uninitialized.
+
+Unintialized Morphy is a simple callable that returns lemma
+*candidates* for some given wordform. That is, the results might not
+be valid lemmas, but this is not a problem in practice because
+subsequent queries against the database will filter out the invalid
+ones. This callable is obtained by creating a :class:`Morphy` object
+with no arguments:
+
+>>> from wn import morphy
+>>> m = morphy.Morphy()
+
+As an uninitialized Morphy cannot predict which lemmas in the result
+are valid, it always returns the original form and any transformations
+it can find for each part of speech:
+
+>>> m('lemmata', pos='n')  # exceptional form
+{'n': {'lemmata'}}
+>>> m('lemmas', pos='n')   # regular morphology with part-of-speech
+{'n': {'lemma', 'lemmas'}}
+>>> m('lemmas')            # regular morphology for any part-of-speech
+{None: {'lemmas'}, 'n': {'lemma'}, 'v': {'lemma'}}
+>>> m('wolves')            # invalid forms may be returned
+{None: {'wolves'}, 'n': {'wolf', 'wolve'}, 'v': {'wolve', 'wolv'}}
+
+
+This lemmatizer can also be used with a :class:`wn.Wordnet` object to
+expand queries:
+
+>>> import wn
+>>> ewn = wn.Wordnet('ewn:2020')
+>>> ewn.words('lemmas')
+[]
+>>> ewn = wn.Wordnet('ewn:2020', lemmatizer=morphy.Morphy())
+>>> ewn.words('lemmas')
+[Word('ewn-lemma-n')]
+
+An initialized Morphy is created with a :class:`wn.Wordnet` object as
+its argument. It then uses the wordnet to build lists of valid lemmas
+and exceptional forms (this takes a few seconds). Once this is done,
+it will only return lemmas it knows about:
+
+>>> ewn = wn.Wordnet('ewn:2020')
+>>> m = morphy.Morphy(ewn)
+>>> m('lemmata', pos='n')  # exceptional form
+{'n': {'lemma'}}
+>>> m('lemmas', pos='n')   # regular morphology with part-of-speech
+{'n': {'lemma'}}
+>>> m('lemmas')            # regular morphology for any part-of-speech
+{'n': {'lemma'}}
+>>> m('wolves')            # invalid forms are pre-filtered
+{'n': {'wolf'}}
+
+In order to use an initialized Morphy lemmatizer with a
+:class:`wn.Wordnet` object, it must be assigned to the object after
+creation:
+
+>>> ewn = wn.Wordnet('ewn:2020')  # default: lemmatizer=None
+>>> ewn.words('lemmas')
+[]
+>>> ewn.lemmatizer = morphy.Morphy(ewn)
+>>> ewn.words('lemmas')
+[Word('ewn-lemma-n')]
+
+There is little to no difference in the results obtained from a
+:class:`wn.Wordnet` object using an initialized or uninitialized
+:class:`Morphy` object, but there may be slightly different
+performance profiles for future queries.
+
+
+Default Morphy Lemmatizer
+-------------------------
+
+As a convenience, an uninitialized Morphy lemmatizer is provided in
+this module via the :data:`morphy` member.
+
+.. data:: morphy
+
+   A :class:`Morphy` object created without a :class:`wn.Wordnet`
+   object.
+
+
+The Morphy Class
+----------------
+
+.. autoclass:: Morphy
diff --git a/docs/api/wn.rst b/docs/api/wn.rst
index 242c5c1..3084b65 100644
--- a/docs/api/wn.rst
+++ b/docs/api/wn.rst
@@ -286,7 +286,7 @@ Wn's data storage and retrieval can be configured through the
 
 .. seealso::
 
-   :doc:`../guides/setup` describes how to configure Wn using the
+   :doc:`../setup` describes how to configure Wn using the
    :data:`wn.config` instance.
 
 .. autodata:: config
diff --git a/docs/api/wn.web.rst b/docs/api/wn.web.rst
new file mode 100644
index 0000000..18e199a
--- /dev/null
+++ b/docs/api/wn.web.rst
@@ -0,0 +1,431 @@
+
+wn.web
+======
+
+.. automodule:: wn.web
+
+This module provides a RESTful API with `JSON:API`_ responses to
+queries against a Wn database. This API implements the primary queries
+of the Python API (see :ref:`primary-queries`). For instance, to
+search all words in the ``ewn:2020`` lexicon with the form *jet* and
+part-of-speech *v*, we can perform the following query::
+
+  /lexicons/ewn:2020/words?form=jet&pos=v
+
+This query would return the following response:
+
+.. code-block:: javascript
+
+   {
+     "data": [
+       {
+         "id": "ewn-jet-v",
+         "type": "word",
+         "attributes": {
+           "pos": "v",
+           "lemma": "jet",
+           "forms": ["jet", "jetted", "jetting"]
+         },
+         "links": {
+           "self": "http://example.com/lexicons/ewn:2020/words/ewn-jet-v"
+         },
+         "relationships": {
+           "senses": {
+             "links": {"related": "http://example.com/lexicons/ewn:2020/words/ewn-jet-v/senses"}
+           },
+           "synsets": {
+             "data": [
+               {"type": "synset", "id": "ewn-01518922-v"},
+               {"type": "synset", "id": "ewn-01946093-v"}
+             ]
+           },
+           "lexicon": {
+             "links": {"related": "http://example.com/lexicons/ewn:2020"}
+           }
+         },
+         "included": [
+           {
+             "id": "ewn-01518922-v",
+             "type": "synset",
+             "attributes": {"pos": "v", "ili": "i29306"},
+             "links": {"self": "http://example.com/lexicons/ewn:2020/synsets/ewn-01518922-v"}
+           },
+           {
+             "id": "ewn-01946093-v",
+             "type": "synset",
+             "attributes": {"pos": "v", "ili": "i31432"},
+             "links": {"self": "http://example.com/lexicons/ewn:2020/synsets/ewn-01946093-v"}
+           }
+         ]
+       }
+     ],
+     "meta": {"total": 1}
+   }
+
+Currently, only ``GET`` requests are handled.
+
+.. _JSON\:API: https://jsonapi.org
+
+
+Installing Dependencies
+-----------------------
+
+By default, Wn does not install the requirements needed for this
+module. Install them with the ``[web]`` extra:
+
+.. code-block:: bash
+
+   $ pip install wn[web]
+
+
+Running and Deploying the Server
+--------------------------------
+
+This module does not provide an ASGI server, so one will need to be
+installed and ran separately. Any ASGI-compliant server should
+work.
+
+For example, the `Uvicorn <https://www.uvicorn.org/>`_ server may be
+used directly for local development, optionally with the ``--reload``
+option for hot reloading:
+
+.. code-block:: bash
+
+   $ uvicorn --reload wn.web:app
+
+For production, see Uvicorn's `documentation about deployment
+<https://www.uvicorn.org/deployment/>`_.
+
+
+Requests: API Endpoints
+-----------------------
+
+The module provides the following endpoints:
+
+.. table::
+   :width: 100%
+
+   ======================================  ========================================================
+   Endpoint                                Description
+   ======================================  ========================================================
+   ``/words``                              List words in all available lexicons
+   ``/senses``                             List senses in all available lexicons
+   ``/synsets``                            List synsets in all available lexicons
+   ``/lexicons``                           List available lexicons
+   ``/lexicons/:lex``                      Get lexicon with specifier ``:lex``
+   ``/lexicons/:lex/words``                List words for lexicon with specifier ``:lex``
+   ``/lexicons/:lex/words/:id/senses``     List senses for word ``:id`` in lexicon ``:lex``
+   ``/lexicons/:lex/words/:id``            Get word with ID ``:id`` in lexicon ``:lex``
+   ``/lexicons/:lex/senses``               List senses for lexicon with specifier ``:lex``
+   ``/lexicons/:lex/senses/:id``           Get sense with ID ``:id`` in lexicon ``:lex``
+   ``/lexicons/:lex/synsets``              List synsets for lexicon with specifier ``:lex``
+   ``/lexicons/:lex/synsets/:id``          Get synset with ID ``:id`` in lexicon ``:lex``
+   ``/lexicons/:lex/synsets/:id/members``  Get member senses for synset ``:id`` in lexicon ``:lex``
+   ======================================  ========================================================
+
+
+Requests: Query Parameters
+--------------------------
+
+``lang``
+''''''''
+
+Specifies the language in `BCP 47`_ of the lexicon(s) from which
+results are returned.
+
+.. _BCP 47: https://en.wikipedia.org/wiki/IETF_language_tag
+
+Example::
+
+  /words?lang=fr
+
+Valid for::
+
+  /lexicons
+  /words
+  /senses
+  /synsets
+
+``form``
+''''''''
+
+Specifies the word form of the objects that are returned.
+
+Example::
+
+  /words?form=chat
+
+Valid for::
+
+  /words
+  /senses
+  /synsets
+  /lexicon/:lex/words
+  /lexicon/:lex/senses
+  /lexicon/:lex/synsets
+
+``pos``
+'''''''
+
+Specifies the part-of-speech of the objects that are returned. Valid
+values are given in :ref:`parts-of-speech`.
+
+Example::
+
+  /words?pos=v
+
+Valid for::
+
+  /words
+  /senses
+  /synsets
+  /lexicon/:lex/words
+  /lexicon/:lex/senses
+  /lexicon/:lex/synsets
+
+``ili``
+'''''''
+
+Specifies the interlingual index of a synset.
+
+Example::
+
+  /synsets?ili=i57031
+
+Valid for::
+
+  /synsets
+  /lexicon/:lex/synsets
+
+``page[offset]`` and ``page[limit]``
+''''''''''''''''''''''''''''''''''''
+
+Used for pagination: ``page[offset]`` specifies the starting index of
+a set of results, and ``page[limit]`` specifies how many results from
+the offset will be returned.
+
+Example::
+
+  /words?page[offset]=150
+
+Valid for::
+
+  /words
+  /senses
+  /synsets
+  /lexicon/:lex/words
+  /lexicon/:lex/senses
+  /lexicon/:lex/synsets
+
+
+Responses
+---------
+
+Responses are JSON data following the `JSON:API`_ specification. A
+full description of JSON:API is left to the linked specification, but
+a brief walkthrough is provided here. First, the top-level structure
+of "to-one" responses (e.g., getting a single synset) is:
+
+.. code-block:: javascript
+
+   {
+     "data": { ... },  // primary response data as a JSON object
+     "meta": { ... }   // metadata for the response
+   }
+
+For "to-many" responses (e.g., getting a list of matching synsets), it
+is the same as above except the ``data`` key maps to an array and it
+includes pagination links:
+
+.. code-block:: javascript
+
+   {
+     "data": [{ ... }, ...],  // primary response data as an array of objects
+     "links": { ... },        // pagination links
+     "meta": { ... }          // metadata; e.g., total number of results
+   }
+
+Each JSON:API *resource object* (the primary data given by the
+``data`` key) has the following structure:
+
+.. code-block:: javascript
+
+   {
+     "id": "...",               // Lexicon specifier or entity ID
+     "type": "...",             // "lexicon", "word", "sense", or "synset"
+     "attributes": { ... },     // Basic resource information
+     "links": { "self": ... },  // URL for this specific resource
+     "relationships": { ... },  // Word senses, synset members, other relations
+     "included": [ ... ],       // Data for related resources
+   }
+
+
+Lexicons
+''''''''
+
+.. code-block:: javascript
+
+   {
+     "id": "ewn:2020",
+     "type": "lexicon",
+     "attributes": {
+       "version": "2020",
+       "label": "English WordNet",
+       "language": "en",
+       "license": "https://creativecommons.org/licenses/by/4.0/"
+     },
+     "links": {"self": "http://example.com/lexicons/ewn:2020"},
+     "relationships": {
+       "words": {"links": {"related": "http://example.com/lexicons/ewn:2020/words"}},
+       "synsets": {"links": {"related": "http://example.com/lexicons/ewn:2020/synsets"}},
+       "senses": {"links": {"related": "http://example.com/lexicons/ewn:2020/senses"}}
+     }
+   }
+
+
+Words
+'''''
+
+.. code-block:: javascript
+
+   {
+     "id": "ewn-brick-v",
+     "type": "word",
+     "attributes": {"pos": "v", "lemma": "brick", "forms": ["brick"]},
+     "links": {"self": "http://example.com/lexicons/ewn:2020/words/ewn-brick-v"},
+     "relationships": {
+       "senses": {"links": {"related": "http://example.com/lexicons/ewn:2020/words/ewn-brick-v/senses"}},
+       "synsets": {"data": [{"type": "synset", "id": "ewn-90011761-v"}]},
+       "lexicon": {"links": {"related": "http://example.com/lexicons/ewn:2020"}}
+     },
+     "included": [
+       {
+         "id": "ewn-90011761-v",
+         "type": "synset",
+         "attributes": {"pos": "v", "ili": null},
+         "links": {"self": "http://example.com/lexicons/ewn:2020/synsets/ewn-90011761-v"}
+       }
+     ]
+   }
+
+
+Senses
+''''''
+
+.. code-block:: javascript
+
+   {
+     "id": "ewn-explain-v-00941308-01",
+     "type": "sense",
+     "links": {"self": "http://example.com/lexicons/ewn:2020/senses/ewn-explain-v-00941308-01"},
+     "relationships": {
+       "word": {"links": {"related": "http://example.com/lexicons/ewn:2020/words/ewn-explain-v"}},
+       "synset": {"links": {"related": "http://example.com/lexicons/ewn:2020/synsets/ewn-00941308-v"}},
+       "lexicon": {"links": {"related": "http://example.com/lexicons/ewn:2020"}},
+       "derivation": {
+         "data": [
+           {"type": "sense", "id": "ewn-explanatory-s-01327635-01"},
+           {"type": "sense", "id": "ewn-explanation-n-07247081-01"}
+         ]
+       }
+     },
+     "included": [
+       {
+         "id": "ewn-explanatory-s-01327635-01",
+         "type": "sense",
+         "links": {"self": "http://example.com/lexicons/ewn:2020/senses/ewn-explanatory-s-01327635-01"}
+       },
+       {
+         "id": "ewn-explanation-n-07247081-01",
+         "type": "sense",
+         "links": {"self": "http://example.com/lexicons/ewn:2020/senses/ewn-explanation-n-07247081-01"}
+       }
+     ]
+   }
+
+
+Synsets
+'''''''
+
+.. code-block:: javascript
+
+   {
+     "id": "ewn-03204585-n",
+     "type": "synset",
+     "attributes": {"pos": "n", "ili": "i52917"},
+     "links": {"self": "http://example.com/lexicons/ewn:2020/synsets/ewn-03204585-n"},
+     "relationships": {
+       "members": {"links": {"related": "http://example.com/lexicons/ewn:2020/synsets/ewn-03204585-n/members"}},
+       "words": {
+         "data": [
+           {"type": "word", "id": "ewn-dory-n"},
+           {"type": "word", "id": "ewn-rowboat-n"},
+           {"type": "word", "id": "ewn-dinghy-n"}
+         ]
+       },
+       "lexicon": {"links": {"related": "http://example.com/lexicons/ewn:2020"}},
+       "hypernym": {"data": [{"type": "synset", "id": "ewn-04252125-n"}]},
+       "mero_part": {
+         "data": [
+           {"type": "synset", "id": "ewn-03911849-n"},
+           {"type": "synset", "id": "ewn-04439177-n"}
+         ]
+       },
+       "hyponym": {
+         "data": [
+           {"type": "synset", "id": "ewn-04122550-n"},
+           {"type": "synset", "id": "ewn-04584425-n"}
+         ]
+       }
+     },
+     "included": [
+       {
+         "id": "ewn-dory-n",
+         "type": "word",
+         "attributes": {"pos": "n", "lemma": "dory", "forms": ["dory"]},
+         "links": {"self": "http://example.com/lexicons/ewn:2020/words/ewn-dory-n"}
+       },
+       {
+         "id": "ewn-rowboat-n",
+         "type": "word",
+         "attributes": {"pos": "n", "lemma": "rowboat", "forms": ["rowboat"]},
+         "links": {"self": "http://example.com/lexicons/ewn:2020/words/ewn-rowboat-n"}
+       },
+       {
+         "id": "ewn-dinghy-n",
+         "type": "word",
+         "attributes": {"pos": "n", "lemma": "dinghy", "forms": ["dinghy"]},
+         "links": {"self": "http://example.com/lexicons/ewn:2020/words/ewn-dinghy-n"}
+       },
+       {
+         "id": "ewn-04252125-n",
+         "type": "synset",
+         "attributes": {"pos": "n", "ili": "i59107"},
+         "links": {"self": "http://example.com/lexicons/ewn:2020/synsets/ewn-04252125-n"}
+       },
+       {
+         "id": "ewn-03911849-n",
+         "type": "synset",
+         "attributes": {"pos": "n", "ili": "i57094"},
+         "links": {"self": "http://example.com/lexicons/ewn:2020/synsets/ewn-03911849-n"}
+       },
+       {
+         "id": "ewn-04439177-n",
+         "type": "synset",
+         "attributes": {"pos": "n", "ili": "i60240"},
+         "links": {"self": "http://example.com/lexicons/ewn:2020/synsets/ewn-04439177-n"}
+       },
+       {
+         "id": "ewn-04122550-n",
+         "type": "synset",
+         "attributes": {"pos": "n", "ili": "i58319"},
+         "links": {"self": "http://example.com/lexicons/ewn:2020/synsets/ewn-04122550-n"}
+       },
+       {
+         "id": "ewn-04584425-n",
+         "type": "synset",
+         "attributes": {"pos": "n", "ili": "i61103"},
+         "links": {"self": "http://example.com/lexicons/ewn:2020/synsets/ewn-04584425-n"}
+       }
+     ]
+   }
diff --git a/docs/guides/cli.rst b/docs/cli.rst
similarity index 100%
rename from docs/guides/cli.rst
rename to docs/cli.rst
diff --git a/docs/conf.py b/docs/conf.py
index 65e4e43..1635b79 100644
--- a/docs/conf.py
+++ b/docs/conf.py
@@ -58,6 +58,9 @@
    :class: highlight
 """
 
+# smartquotes = False
+smartquotes_action = 'De'  # D = en- and em-dash; e = ellipsis
+
 # -- Options for HTML output -------------------------------------------------
 
 # The theme to use for HTML and HTML Help pages.  See the documentation for
@@ -80,7 +83,7 @@
 html_logo = "_static/wn-logo.svg"
 
 pygments_style = 'manni'
-pygments_dark_style = 'manni'
+pygments_dark_style = 'monokai'
 
 # Add any paths that contain custom static files (such as style sheets) here,
 # relative to this directory. They are copied after the builtin static files,
diff --git a/docs/guides/basic.rst b/docs/guides/basic.rst
index 2bb57b2..14214b3 100644
--- a/docs/guides/basic.rst
+++ b/docs/guides/basic.rst
@@ -50,6 +50,8 @@ In fact, the simple queries above implicitly create such a
 lexicons.
 
 
+.. _primary-queries:
+
 Primary Queries
 ---------------
 
@@ -226,7 +228,7 @@ Filtering by Language
 ---------------------
 
 The ``lang`` parameter of :func:`wn.words()`, :func:`wn.senses()`,
-:func:`wn.synsets()`, and :class:`~wn.Wordnet` allows a single `BCP-47
+:func:`wn.synsets()`, and :class:`~wn.Wordnet` allows a single `BCP 47
 <https://en.wikipedia.org/wiki/IETF_language_tag>`_ language
 code. When this parameter is used, only entries in the specified
 language will be returned.
diff --git a/docs/guides/lemmatization.rst b/docs/guides/lemmatization.rst
new file mode 100644
index 0000000..c111b7d
--- /dev/null
+++ b/docs/guides/lemmatization.rst
@@ -0,0 +1,269 @@
+
+Lemmatization and Normalization
+===============================
+
+Wn provides two methods for expanding queries: lemmatization_ and
+normalization_\ . Wn also has a setting that allows `alternative forms
+<alternative-forms_>`_ stored in the database to be included in
+queries.
+
+.. seealso::
+
+   The :mod:`wn.morphy` module is a basic English lemmatizer included
+   with Wn.
+
+.. _lemmatization:
+
+Lemmatization
+-------------
+
+When querying a wordnet with wordforms from natural language text, it
+is important to be able to find entries for inflected forms as the
+database generally contains only lemmatic forms, or *lemmas* (or
+*lemmata*, if you prefer irregular plurals).
+
+>>> import wn
+>>> ewn = wn.Wordnet('ewn:2020')
+>>> ewn.words('plurals')  # no results
+[]
+>>> ewn.words('plural')
+[Word('ewn-plural-n'), Word('ewn-plural-a')]
+
+Lemmas are sometimes called *citation forms* or *dictionary forms* as
+they are often used as the head words in dictionary entries. In
+Natural Language Processing (NLP), *lemmatization* is a technique
+where a possibly inflected word form is transformed to yield a
+lemma. In Wn, this concept is generalized somewhat to mean a
+transformation that yields a form matching wordforms stored in the
+database. For example, the English word *sparrows* is the plural
+inflection of *sparrow*, while the word *leaves* is ambiguous between
+the plural inflection of the nouns *leaf* and *leave* and the
+3rd-person singular inflection of the verb *leave*.
+
+For tasks where high-accuracy is needed, wrapping the wordnet queries
+with external tools that handle tokenization, lemmatization, and
+part-of-speech tagging will likely yield the best results as this
+method can make use of word context. That is, something like this:
+
+.. code-block:: python
+
+   for lemma, pos in fancy_shmancy_analysis(corpus):
+       synsets = w.synsets(lemma, pos=pos)
+
+For modest needs, however, Wn provides a way to integrate basic
+lemmatization directly into the queries.
+
+Lemmatization in Wn works as follows: if a :class:`wn.Wordnet` object
+is instantiated with a *lemmatizer* argument, then queries involving
+wordforms (e.g., :meth:`wn.Wordnet.words`, :meth:`wn.Wordnet.senses`,
+:meth:`wn.Wordnet.synsets`) will first lemmatize the wordform and then
+check all resulting wordforms and parts of speech against the
+database as successive queries.
+
+Lemmatization Functions
+'''''''''''''''''''''''
+
+The *lemmatizer* argument of :class:`wn.Wordnet` is a callable that
+takes two string arguments: (1) the original wordform, and (2) a
+part-of-speech or :python:`None`. It returns a dictionary mapping
+parts-of-speech to sets of lemmatized wordforms. The signature is as
+follows:
+
+.. code-block:: python
+
+   lemmatizer(s: str, pos: Optional[str]) -> Dict[Optional[str], Set[str]]
+
+The part-of-speech may be used by the function to determine which
+morphological rules to apply. If the given part-of-speech is
+:python:`None`, then it is not specified and any rule may apply. A
+lemmatizer that only deinflects should not change any specified
+part-of-speech, but this is not a requirement, and a function could be
+provided that undoes derivational morphology (e.g., *democratic* →
+*democracy*).
+
+Querying With Lemmatization
+'''''''''''''''''''''''''''
+
+As the needs of lemmatization differs from one language to another, Wn
+does not provide a lemmatizer by default, and therefore it is
+unavailable to the convenience functions :func:`wn.words`,
+:func:`wn.senses`, and :func:`wn.synsets`. A lemmatizer can be added
+to a :class:`wn.Wordnet` object. For example, using :mod:`wn.morphy`:
+
+>>> import wn
+>>> from wn.morphy import Morphy
+>>> ewn = wn.Wordnet('ewn:2020', lemmatizer=Morphy())
+>>> ewn.words('sparrows')
+[Word('ewn-sparrow-n')]
+>>> ewn.words('leaves')
+[Word('ewn-leaf-n'), Word('ewn-leave-n'), Word('ewn-leave-v')]
+
+
+Querying Without Lemmatization
+''''''''''''''''''''''''''''''
+
+When lemmatization is not used, inflected terms may not return any
+results:
+
+>>> ewn = wn.Wordnet('ewn:2020')
+>>> ewn.words('sparrows')
+[]
+>>> ewn.words('leaves')
+[]
+
+Depending on the lexicon, there may be situations where results are
+returned for inflected lemmas, such as when the inflected form is
+lexicalized as its own entry:
+
+>>> ewn.words('glasses')
+[Word('ewn-glasses-n')]
+
+Or if the lexicon lists the inflected form as an alternative form. For
+example, the English Wordnet lists irregular inflections as
+alternative forms:
+
+>>> ewn.words('lemmata')
+[Word('ewn-lemma-n')]
+
+See below for excluding alternative forms from such queries.
+
+.. _alternative-forms:
+
+Alternative Forms in the Database
+---------------------------------
+
+A lexicon may include alternative forms in addition to lemmas for each
+word, and by default these are included in queries. What exactly is
+included as an alternative form depends on the lexicon. The English
+Wordnet, for example, adds irregular inflections (or "exceptional
+forms"), while the Japanese Wordnet includes the same word in multiple
+orthographies (original, hiragana, katakana, and two romanizations).
+For the English Wordnet, this means that you might get basic
+lemmatization for irregular forms only:
+
+>>> ewn = wn.Wordnet('ewn:2020')
+>>> ewn.words('learnt', pos='v')
+[Word('ewn-learn-v')]
+>>> ewn.words('learned', pos='v')
+[]
+
+If this is undesirable, the alternative forms can be excluded from
+queries with the *search_all_forms* parameter:
+
+>>> ewn = wn.Wordnet('ewn:2020', search_all_forms=False)
+>>> ewn.words('learnt', pos='v')
+[]
+>>> ewn.words('learned', pos='v')
+[]
+
+
+.. _normalization:
+
+Normalization
+-------------
+
+While lemmatization deals with morphological variants of words,
+normalization handles minor orthographic variants. Normalized forms,
+however, may be invalid as wordforms in the target language, and as
+such they are only used behind the scenes for query expansion and not
+presented to users. For instance, a user might attempt to look up
+*résumé* in the English wordnet, but the wordnet only contains the
+form without diacritics: *resume*. With strict string matching, the
+entry would not be found using the wordform in the query. By
+normalizing the query word, the entry can be found. Similarly in the
+Spanish wordnet, *año* (year) and *ano* (anus) are two different
+words. A user who types *año* likely does not want to get results for
+*ano*, but one who types *ano* may be a non-Spanish speaker who is
+unaware of the missing diacritic or does not have an input method that
+allows them to type the diacritic, so this query would return both
+entries by matching against the normalized forms in the database. Wn
+handles all of these use cases.
+
+When a lexicon is added to the database, potentially two wordforms are
+inserted for every one in the lexicon: the original wordform and a
+normalized form. When querying against the database, the original
+query string is first compared with the original wordforms and, if
+normalization is enabled, with the normalized forms in the database as
+well. If this first attempt yields no results and if normalization is
+enabled, the query string is normalized and tried again.
+
+Normalization Functions
+'''''''''''''''''''''''
+
+The normalized form is obtained from a *normalizer* function, passed
+as an argument to :class:`wn.Wordnet`, that takes a single string
+argument and returns a string. That is, a function with the following
+signature:
+
+.. code-block:: python
+
+   normalizer(s: str) -> str
+
+While custom *normalizer* functions could be used, in practice the
+choice is either the default normalizer or :python:`None`. The default
+normalizer works by downcasing the string and performing NFKD_
+normalization to remove diacritics. If the normalized form is the same
+as the original, only the original is inserted into the database.
+
+.. table:: Examples of normalization
+   :align: center
+
+   =============  ===============
+   Original Form  Normalized Form
+   =============  ===============
+   résumé         resume
+   año            ano
+   San José       san jose
+   ハラペーニョ   ハラヘーニョ
+   =============  ===============
+
+.. _NFKD: https://en.wikipedia.org/wiki/Unicode_equivalence#Normal_forms
+
+Querying With Normalization
+'''''''''''''''''''''''''''
+
+By default, normalization is enabled when a :class:`wn.Wordnet` is
+created. Enabling normalization does two things: it allows queries to
+check the original wordform in the query against the normalized forms
+in the database and, if no results are returned in the first step, it
+allows the queried wordform to be normalized as a back-off technique.
+
+>>> ewn = wn.Wordnet('ewn:2020')
+>>> ewn.words('résumé')
+[Word('ewn-resume-v'), Word('ewn-resume-n')]
+>>> spa = wn.Wordnet('spawn:1.3+omw')
+>>> spa.words('año')
+[Word('spawn-lex57514')]
+>>> spa.words('ano')
+[Word('spawn-lex34109'), Word('spawn-lex57514')]
+
+.. note::
+
+   Users may supply a custom *normalizer* function to the
+   :class:`wn.Wordnet` object, but currently this is discouraged as
+   the result is unlikely to match normalized forms in the database
+   and there is not yet a way to customize the normalization of forms
+   added to the database.
+
+Querying Without Normalization
+''''''''''''''''''''''''''''''
+
+Normalization can be disabled by passing :python:`None` as the
+argument of the *normalizer* parameter of :class:`wn.Wordnet`. The
+queried wordform will not be checked against normalized forms in the
+database and neither will it be normalized as a back-off technique.
+
+>>> ewn = wn.Wordnet('ewn:2020', normalizer=None)
+>>> ewn.words('résumé')
+[]
+>>> spa = wn.Wordnet('spawn:1.3+omw', normalizer=None)
+>>> spa.words('año')
+[Word('spawn-lex57514')]
+>>> spa.words('ano')
+[Word('spawn-lex34109')]
+
+.. note::
+
+   It is not possible to disable normalization for the convenience
+   functions :func:`wn.words`, :func:`wn.senses`, and
+   :func:`wn.synsets`.
diff --git a/docs/guides/lexicons.rst b/docs/guides/lexicons.rst
index 5337b9b..46d78cc 100644
--- a/docs/guides/lexicons.rst
+++ b/docs/guides/lexicons.rst
@@ -88,7 +88,11 @@ Adding Local Lexicons
 
 Lexicons can be added from local files with :py:func:`wn.add`:
 
->>> wn.add('~/data/omw/nobwn/')
+>>> wn.add('~/data/omw/nob/nobwn.xml')
+
+Or, with the directory as a package:
+
+>>> wn.add('~/data/omw/nob/')
 
 Listing Installed Lexicons
 --------------------------
@@ -112,11 +116,9 @@ Lexicons can be removed from the database with :py:func:`wn.remove`:
 
 Note that this removes a single lexicon and not a project, so if, for
 instance, you've installed a multi-lexicon project like ``omw``, you
-will need to remove each lexicon individually:
+will need to remove each lexicon individually or use a star specifier:
 
->>> for lex in wn.lexicons():
-...     if lex.version == '1.3+omw':
-...         wn.remove(f'{lex.id}:{lex.version}')
+>>> wn.remove('*:1.3+omw')
 
 WN-LMF Files, Packages, and Collections
 ---------------------------------------
diff --git a/docs/index.rst b/docs/index.rst
index 48dcc15..6156b4b 100644
--- a/docs/index.rst
+++ b/docs/index.rst
@@ -36,18 +36,19 @@ Contents
 .. toctree::
    :maxdepth: 2
 
+   setup.rst
+   cli.rst
    faq.rst
 
 .. toctree::
    :caption: Guides
    :maxdepth: 2
 
-   guides/setup.rst
-   guides/cli.rst
    guides/lexicons.rst
    guides/basic.rst
    guides/interlingual.rst
    guides/wordnet.rst
+   guides/lemmatization.rst
    guides/nltk-migration.rst
 
 .. toctree::
@@ -58,6 +59,8 @@ Contents
    api/wn.rst
    api/wn.constants.rst
    api/wn.lmf.rst
+   api/wn.morphy.rst
    api/wn.project.rst
    api/wn.similarity.rst
    api/wn.util.rst
+   api/wn.web.rst
diff --git a/docs/requirements.txt b/docs/requirements.txt
index cc8b60d..dd976fe 100644
--- a/docs/requirements.txt
+++ b/docs/requirements.txt
@@ -1,4 +1,4 @@
-sphinx
+sphinx>=3.5,<4
 furo
 sphinx-copybutton
 .
diff --git a/docs/guides/setup.rst b/docs/setup.rst
similarity index 93%
rename from docs/guides/setup.rst
rename to docs/setup.rst
index d14642a..9fcff25 100644
--- a/docs/guides/setup.rst
+++ b/docs/setup.rst
@@ -4,11 +4,11 @@ Installation and Configuration
 .. seealso::
 
    This guide is for installing and configuring the Wn software. For
-   adding lexicons to the database, see :doc:`lexicons`.
+   adding lexicons to the database, see :doc:`guides/lexicons`.
 
 
-Installation
-------------
+Installing from PyPI
+--------------------
 
 Install the latest release from `PyPI <https://pypi.org/project/wn>`_:
 
@@ -16,6 +16,13 @@ Install the latest release from `PyPI <https://pypi.org/project/wn>`_:
 
    pip install wn
 
+To get the dependencies for the :mod:`wn.web` module, use the ``web``
+installation extra:
+
+.. code-block:: bash
+
+   pip install wn[web]
+
 
 The Data Directory
 ------------------
diff --git a/noxfile.py b/noxfile.py
index 7e09b74..8580878 100644
--- a/noxfile.py
+++ b/noxfile.py
@@ -3,8 +3,7 @@
 
 @nox.session
 def lint(session):
-    session.install('flake8', 'mypy')
-    session.install('.')
+    session.install('.[test]')
     session.run('flake8', '--max-line-length', '88', 'wn')
     session.run('mypy', 'wn')
 
diff --git a/pyproject.toml b/pyproject.toml
index 9ff9979..e913130 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -35,14 +35,20 @@ classifiers = [
 Documentation = "https://wn.readthedocs.io"
 
 [tool.flit.metadata.requires-extra]
+web = [
+    "starlette",
+]
 test = [
     "pytest",
     "mypy",
     "flake8",
     "nox",
+    # typing stubs
+    'types-requests',
+    'types-toml',
 ]
 doc = [
-    "sphinx",
+    "sphinx>=3.5,<4",
     "furo",
     "sphinx-copybutton",
 ]
diff --git a/wn/__init__.py b/wn/__init__.py
index 2fe67e7..6135dbc 100644
--- a/wn/__init__.py
+++ b/wn/__init__.py
@@ -49,4 +49,4 @@
     Wordnet
 )
 
-__version__ = '0.6.2'
+__version__ = '0.7.0'
diff --git a/wn/_add.py b/wn/_add.py
index 5053cb4..94cd6ff 100644
--- a/wn/_add.py
+++ b/wn/_add.py
@@ -97,7 +97,7 @@ def add(
     try:
         for package in iterpackages(source):
             if package.type == _WORDNET:
-                _add_lmf(package.resource_file(), progress)
+                _add_lmf(package.resource_file(), progress, progress_handler)
             elif package.type == _ILI:
                 _add_ili(package.resource_file(), progress)
             else:
@@ -109,6 +109,7 @@ def add(
 def _add_lmf(
     source,
     progress: ProgressHandler,
+    progress_handler: Type[ProgressHandler],
 ) -> None:
     with connect() as conn:
         cur = conn.cursor()
@@ -134,14 +135,15 @@ def _add_lmf(
             return
 
         # all clear, try to add them
-        progress.flash('Updating lookup tables')
-        _update_lookup_tables(all_infos, cur)
-
         progress.flash(f'Reading {source!s}')
-        for lexicon, info in zip(lmf.load(source), all_infos):
+
+        for lexicon, info in zip(lmf.load(source, progress_handler), all_infos):
             if 'skip' in info:
                 continue
 
+            progress.flash('Updating lookup tables')
+            _update_lookup_tables(lexicon, cur)
+
             progress.set(count=0, total=_sum_counts(info))
             synsets = lexicon.synsets
             entries = lexicon.lexical_entries
@@ -188,22 +190,20 @@ def _precheck(source, cur):
 
 def _sum_counts(info) -> int:
     counts = info['counts']
-    return sum(counts.get(name, 0) for name in
-               ('LexicalEntry', 'ExternalLexicalEntry',
-                'Lemma', 'Form', 'Pronunciation', 'Tag',
-                'Sense', 'ExternalSense',
-                'SenseRelation', 'Example', 'Count',
-                'SyntacticBehaviour',
-                'Synset', 'ExternalSynset',
-                'Definition',  # 'ILIDefinition',
-                'SynsetRelation'))
-
-
-def _update_lookup_tables(all_infos, cur):
-    reltypes = set(rt for info in all_infos for rt in info['relations'])
+    return sum(counts.get(name, 0) for name in lmf.LEXICON_INFO_ATTRIBUTES)
+
+
+def _update_lookup_tables(lexicon, cur):
+    reltypes = set(rel.type
+                   for ss in lexicon.synsets
+                   for rel in ss.relations)
+    reltypes.update(rel.type
+                    for e in lexicon.lexical_entries
+                    for s in e.senses
+                    for rel in s.relations)
     cur.executemany('INSERT OR IGNORE INTO relation_types VALUES (null,?)',
                     [(rt,) for rt in sorted(reltypes)])
-    lexfiles = set(lf for info in all_infos for lf in info['lexfiles'])
+    lexfiles = set(ss.lexfile for ss in lexicon.synsets) - {None}
     cur.executemany('INSERT OR IGNORE INTO lexfiles VALUES (null,?)',
                     [(lf,) for lf in sorted(lexfiles)])
 
@@ -667,13 +667,15 @@ def remove(
     The *lexicon* argument is a :ref:`lexicon specifier
     <lexicon-specifiers>`. Note that this removes a lexicon and not a
     project, so the lexicons of projects containing multiple lexicons
-    will need to be removed individually.
+    will need to be removed individually or, if applicable, a star
+    specifier.
 
     The *progress_handler* parameter takes a subclass of
     :class:`wn.util.ProgressHandler`. An instance of the class will be
     created, used, and closed by this function.
 
-    >>> wn.remove('ewn:2019')
+    >>> wn.remove('ewn:2019')  # removes a single lexicon
+    >>> wn.remove('*:1.3+omw')  # removes all lexicons with version 1.3+omw
 
     """
     if progress_handler is None:
@@ -697,7 +699,7 @@ def remove(
                 extra = f' (and {len(extensions)} extension(s))' if extensions else ''
                 progress.set(status=f'{spec}', count=0)
                 conn.execute('DELETE from lexicons WHERE rowid = ?', (rowid,))
-                progress.flash(f'Removed {spec}{extra}')
+                progress.flash(f'Removed {spec}{extra}\n')
 
     finally:
         progress.close()
diff --git a/wn/_core.py b/wn/_core.py
index 534d84e..8ee1d18 100644
--- a/wn/_core.py
+++ b/wn/_core.py
@@ -1,10 +1,25 @@
 
-from typing import TypeVar, Optional, List, Tuple, Dict, Set, Iterator
+from typing import (
+    Type,
+    TypeVar,
+    Callable,
+    Optional,
+    List,
+    Tuple,
+    Dict,
+    Set,
+    Sequence,
+    Iterator,
+)
 import warnings
 
 import wn
-from wn._types import Metadata
-from wn._util import flatten
+from wn._types import (
+    Metadata,
+    NormalizeFunction,
+    LemmatizeFunction,
+)
+from wn._util import flatten, normalize_form
 from wn._db import NON_ROWID
 from wn._queries import (
     find_lexicons,
@@ -425,6 +440,9 @@ def __init__(
         super().__init__(_lexid=_lexid, _id=_id, _wordnet=_wordnet)
         self.id = id
 
+    def relations(self: T, *args: str) -> Dict[str, List[T]]:
+        raise NotImplementedError
+
     def get_related(self: T, *args: str) -> List[T]:
         raise NotImplementedError
 
@@ -495,7 +513,7 @@ def ili(self):
         if self._ili:
             row = next(find_ilis(id=self._ili), None)
         else:
-            row = next(find_proposed_ilis(synset_id=self._id), None)
+            row = next(find_proposed_ilis(synset_rowid=self._id), None)
         if row is not None:
             return ILI(*row)
         return None
@@ -583,15 +601,64 @@ def lemmas(self) -> List[Form]:
         """
         return [w.lemma() for w in self.words()]
 
+    def relations(self, *args: str) -> Dict[str, List['Synset']]:
+        """Return a mapping of relation names to lists of synsets.
+
+        One or more relation names may be given as positional
+        arguments to restrict the relations returned. If no such
+        arguments are given, all relations starting from the synset
+        are returned.
+
+        See :meth:`get_related` for getting a flat list of related
+        synsets.
+
+        Example:
+
+            >>> button_rels = wn.synsets('button')[0].relations()
+            >>> for relname, sslist in button_rels.items():
+            ...     print(relname, [ss.lemmas() for ss in sslist])
+            ...
+            hypernym [['fixing', 'holdfast', 'fastener', 'fastening']]
+            hyponym [['coat button'], ['shirt button']]
+
+        """
+        d: Dict[str, List['Synset']] = {}
+        for relname, ss in self._get_relations(args):
+            if relname in d:
+                d[relname].append(ss)
+            else:
+                d[relname] = [ss]
+        return d
+
     def get_related(self, *args: str) -> List['Synset']:
-        targets: List['Synset'] = []
+        """Return the list of related synsets.
+
+        One or more relation names may be given as positional
+        arguments to restrict the relations returned. If no such
+        arguments are given, all relations starting from the synset
+        are returned.
+
+        This method does not preserve the relation names that lead to
+        the related synsets. For a mapping of relation names to
+        related synsets, see :meth:`relations`.
+
+        Example:
+
+            >>> fulcrum = wn.synsets('fulcrum')[0]
+            >>> [ss.lemmas() for ss in fulcrum.get_related()]
+            [['pin', 'pivot'], ['lever']]
+        """
+        return [ss for _, ss in self._get_relations(args)]
+
+    def _get_relations(self, args: Sequence[str]) -> List[Tuple[str, 'Synset']]:
+        targets: List[Tuple[str, 'Synset']] = []
 
         lexids = self._get_lexicon_ids()
 
         # first get relations from the current lexicon(s)
         if self._id != NON_ROWID:
-            relations = get_synset_relations({self._id}, args, lexids)
-            targets.extend(Synset(*row[2:], self._wordnet)
+            relations = list(get_synset_relations({self._id}, args, lexids))
+            targets.extend((row[0], Synset(*row[2:], self._wordnet))
                            for row in relations
                            if row[5] in lexids)
 
@@ -602,25 +669,26 @@ def get_related(self, *args: str) -> List['Synset']:
             # get expanded relation
             expss = find_synsets(ili=self._ili, lexicon_rowids=expids)
             rowids = {rowid for _, _, _, _, rowid in expss} - {self._id, NON_ROWID}
-            relations = get_synset_relations(rowids, args, expids)
+            relations = list(get_synset_relations(rowids, args, expids))
             ilis = {row[4] for row in relations} - {None}
 
             # map back to target lexicons
-            seen = {ss._id for ss in targets}
+            seen = {ss._id for _, ss in targets}
             for row in get_synsets_for_ilis(ilis, lexicon_rowids=lexids):
                 if row[-1] not in seen:
-                    targets.append(Synset(*row, self._wordnet))
+                    targets.append((row[0], Synset(*row, self._wordnet)))
 
             # add empty synsets for ILIs without a target in lexids
-            for ili in (ilis - {tgt._ili for tgt in targets}):
-                targets.append(
-                    Synset.empty(
+            unseen_ilis = ilis - {tgt._ili for _, tgt in targets}
+            for rel_row in relations:
+                if rel_row[4] in unseen_ilis:
+                    ss = Synset.empty(
                         id=_INFERRED_SYNSET,
-                        ili=ili,
+                        ili=rel_row[4],
                         _lexid=self._lexid,
                         _wordnet=self._wordnet
                     )
-                )
+                    targets.append((rel_row[0], ss))
 
         return targets
 
@@ -970,6 +1038,26 @@ def metadata(self) -> Metadata:
         """Return the sense's metadata."""
         return get_metadata(self._id, 'senses')
 
+    def relations(self, *args: str) -> Dict[str, List['Sense']]:
+        """Return a mapping of relation names to lists of senses.
+
+        One or more relation names may be given as positional
+        arguments to restrict the relations returned. If no such
+        arguments are given, all relations starting from the sense
+        are returned.
+
+        See :meth:`get_related` for getting a flat list of related
+        senses.
+
+        """
+        d: Dict[str, List['Sense']] = {}
+        for relname, s in self._get_relations(args):
+            if relname in d:
+                d[relname].append(s)
+            else:
+                d[relname] = [s]
+        return d
+
     def get_related(self, *args: str) -> List['Sense']:
         """Return a list of related senses.
 
@@ -987,10 +1075,13 @@ def get_related(self, *args: str) -> List['Sense']:
             incoherent
 
         """
+        return [s for _, s in self._get_relations(args)]
+
+    def _get_relations(self, args: Sequence[str]) -> List[Tuple[str, 'Sense']]:
         lexids = self._get_lexicon_ids()
         iterable = get_sense_relations(self._id, args, lexids)
-        return [Sense(sid, eid, ssid, lexid, rowid, self._wordnet)
-                for _, _, sid, eid, ssid, lexid, rowid in iterable
+        return [(relname, Sense(sid, eid, ssid, lexid, rowid, self._wordnet))
+                for relname, _, sid, eid, ssid, lexid, rowid in iterable
                 if lexids is None or lexid in lexids]
 
     def get_related_synsets(self, *args: str) -> List[Synset]:
@@ -1022,16 +1113,21 @@ def translate(self, lexicon: str = None, *, lang: str = None) -> List['Sense']:
                 for t_sense in t_synset.senses()]
 
 
+# Useful for factory functions of Word, Sense, or Synset
+C = TypeVar('C', Word, Sense, Synset)
+
+
 class Wordnet:
+
     """Class for interacting with wordnet data.
 
     A wordnet object acts essentially as a filter by first selecting
     matching lexicons and then searching only within those lexicons
-    for later queries. On instantiation, a *lang* argument is a BCP47
-    language code that restricts the selected lexicons to those whose
-    language matches the given code. A *lexicon* argument is a
+    for later queries. On instantiation, a *lang* argument is a `BCP
+    47`_ language code that restricts the selected lexicons to those
+    whose language matches the given code. A *lexicon* argument is a
     space-separated list of lexicon specifiers that more directly
-    select lexicons by their ID and version; this is preferable when
+    selects lexicons by their ID and version; this is preferable when
     there are multiple lexicons in the same language or multiple
     version with the same ID.
 
@@ -1039,15 +1135,56 @@ class Wordnet:
     wordnet, namely the Princeton WordNet, and then relying on the
     larger wordnet for structural relations. An *expand* argument is a
     second space-separated list of lexicon specifiers which are used
-    for traversing relations, but not as the results of queries.
+    for traversing relations, but not as the results of
+    queries. Setting *expand* to an empty string (:python:`expand=''`)
+    disables expand lexicons.
+
+    The *normalizer* argument takes a callable that normalizes word
+    forms in order to expand the search. The default function
+    downcases the word and removes diacritics via NFKD_ normalization
+    so that, for example, searching for *san josé* in the English
+    WordNet will find the entry for *San Jose*. Setting *normalizer*
+    to :python:`None` disables normalization and forces exact-match
+    searching.
+
+    The *lemmatizer* argument may be :python:`None`, which is the
+    default and disables lemmatizer-based query expansion, or a
+    callable that takes a word form and optional part of speech and
+    returns base forms of the original word. To support lemmatizers
+    that use the wordnet for instantiation, such as :mod:`wn.morphy`,
+    the lemmatizer may be assigned to the :attr:`lemmatizer` attribute
+    after creation.
+
+    If the *search_all_forms* argument is :python:`True` (the
+    default), searches of word forms consider all forms in the
+    lexicon; if :python:`False`, only lemmas are searched. Non-lemma
+    forms may include, depending on the lexicon, morphological
+    exceptions, alternate scripts or spellings, etc.
+
+    .. _BCP 47: https://en.wikipedia.org/wiki/IETF_language_tag
+    .. _NFKD: https://en.wikipedia.org/wiki/Unicode_equivalence#Normal_forms
+
+    Attributes:
+
+        lemmatizer: A lemmatization function or :python:`None`.
 
     """
 
     __slots__ = ('_lexicons', '_lexicon_ids', '_expanded', '_expanded_ids',
-                 '_default_mode')
+                 '_default_mode', '_normalizer', 'lemmatizer',
+                 '_search_all_forms',)
     __module__ = 'wn'
 
-    def __init__(self, lexicon: str = None, *, lang: str = None, expand: str = None):
+    def __init__(
+        self,
+        lexicon: str = None,
+        *,
+        lang: str = None,
+        expand: str = None,
+        normalizer: Optional[NormalizeFunction] = normalize_form,
+        lemmatizer: Optional[LemmatizeFunction] = None,
+        search_all_forms: bool = True,
+    ):
         # default mode means any lexicon is searched or expanded upon,
         # but relation traversals only target the source's lexicon
         self._default_mode = (not lexicon and not lang)
@@ -1081,6 +1218,10 @@ def __init__(self, lexicon: str = None, *, lang: str = None, expand: str = None)
             self._expanded = tuple(map(_to_lexicon, find_lexicons(lexicon=expand)))
         self._expanded_ids: Tuple[int, ...] = tuple(lx._id for lx in self._expanded)
 
+        self._normalizer = normalizer
+        self.lemmatizer = lemmatizer
+        self._search_all_forms = search_all_forms
+
     def lexicons(self):
         """Return the list of lexicons covered by this wordnet."""
         return self._lexicons
@@ -1106,8 +1247,7 @@ def words(self, form: str = None, pos: str = None) -> List[Word]:
         restricts words by their part of speech.
 
         """
-        iterable = find_entries(form=form, pos=pos, lexicon_rowids=self._lexicon_ids)
-        return [Word(*word_data, self) for word_data in iterable]
+        return _find_helper(self, Word, find_entries, form, pos)
 
     def synset(self, id: str) -> Synset:
         """Return the first synset in this wordnet with identifier *id*."""
@@ -1131,10 +1271,7 @@ def synsets(
         select a unique synset within a single lexicon.
 
         """
-        iterable = find_synsets(
-            form=form, pos=pos, ili=ili, lexicon_rowids=self._lexicon_ids,
-        )
-        return [Synset(*synset_data, self) for synset_data in iterable]
+        return _find_helper(self, Synset, find_synsets, form, pos, ili=ili)
 
     def sense(self, id: str) -> Sense:
         """Return the first sense in this wordnet with identifier *id*."""
@@ -1153,8 +1290,7 @@ def senses(self, form: str = None, pos: str = None) -> List[Sense]:
         *pos* restricts senses by their word's part of speech.
 
         """
-        iterable = find_senses(form=form, pos=pos, lexicon_rowids=self._lexicon_ids)
-        return [Sense(*sense_data, self) for sense_data in iterable]
+        return _find_helper(self, Sense, find_senses, form, pos)
 
     def ili(self, id: str) -> ILI:
         """Return the first ILI in this wordnet with identifer *id*."""
@@ -1165,6 +1301,11 @@ def ili(self, id: str) -> ILI:
             raise wn.Error(f'no such ILI: {id}')
 
     def ilis(self, status: str = None) -> List[ILI]:
+        """Return the list of ILIs in this wordnet.
+
+        If *status* is given, only return ILIs with a matching status.
+
+        """
         iterable = find_ilis(status=status, lexicon_rowids=self._lexicon_ids)
         return [ILI(*ili_data) for ili_data in iterable]
 
@@ -1185,6 +1326,63 @@ def _to_lexicon(data) -> Lexicon:
     )
 
 
+def _find_helper(
+    w: Wordnet,
+    cls: Type[C],
+    query_func: Callable,
+    form: Optional[str],
+    pos: Optional[str],
+    ili: str = None
+) -> List[C]:
+    """Return the list of matching wordnet entities.
+
+    If the wordnet has a normalizer and the search includes a word
+    form, the original word form is searched against both the
+    original and normalized columns in the database. Then, if no
+    results are found, the search is repeated with the normalized
+    form. If the wordnet does not have a normalizer, only exact
+    string matches are used.
+
+    """
+    kwargs: Dict = {
+        'lexicon_rowids': w._lexicon_ids,
+        'search_all_forms': w._search_all_forms,
+    }
+    if ili is not None:
+        kwargs['ili'] = ili
+
+    # easy case is when there is no form
+    if form is None:
+        return [cls(*data, w)  # type: ignore
+                for data in query_func(pos=pos, **kwargs)]
+
+    # if there's a form, we may need to lemmatize and normalize
+    lemmatize = w.lemmatizer
+    normalize = w._normalizer
+    kwargs['normalized'] = bool(normalize)
+
+    forms = lemmatize(form, pos) if lemmatize else {}
+    # if no lemmatizer or word not covered by lemmatizer, back off to
+    # the original form and pos
+    if not forms:
+        forms = {pos: {form}}
+
+    results = [
+        cls(*data, w)  # type: ignore
+        for _pos, _forms in forms.items()
+        for data in query_func(forms=_forms, pos=_pos, **kwargs)
+    ]
+    if not results and normalize:
+        results = [
+            cls(*data, w)  # type: ignore
+            for _pos, _forms in forms.items()
+            for data in query_func(
+                forms=[normalize(f) for f in _forms], pos=_pos, **kwargs
+            )
+        ]
+    return results
+
+
 def projects() -> List[Dict]:
     """Return the list of indexed projects.
 
diff --git a/wn/_queries.py b/wn/_queries.py
index bc9b254..94595b1 100644
--- a/wn/_queries.py
+++ b/wn/_queries.py
@@ -3,7 +3,7 @@
 """
 
 from typing import (
-    Optional, Any, Dict, Set, List, Tuple, Collection, Iterator, Sequence
+    Optional, Dict, Set, List, Tuple, Collection, Iterator, Sequence
 )
 import itertools
 import sqlite3
@@ -280,132 +280,170 @@ def find_proposed_ilis(
 
 
 def find_entries(
-        id: str = None,
-        form: str = None,
-        pos: str = None,
-        lexicon_rowids: Sequence[int] = None,
+    id: str = None,
+    forms: Sequence[str] = None,
+    pos: str = None,
+    lexicon_rowids: Sequence[int] = None,
+    normalized: bool = False,
+    search_all_forms: bool = False,
 ) -> Iterator[_Word]:
     conn = connect()
-    query_parts = [
-        'SELECT DISTINCT e.lexicon_rowid, e.rowid, e.id, e.pos,'
-        '                f.form, f.id, f.script, f.rowid',
-        '  FROM entries AS e',
-        '  JOIN forms AS f ON f.entry_rowid = e.rowid',
-    ]
-
-    params: Dict[str, Any] = {'id': id, 'form': form, 'pos': pos}
+    cte = ''
+    params: List = []
     conditions = []
     if id:
-        conditions.append('e.id = :id')
-    if form:
-        conditions.append('e.rowid IN'
-                          ' (SELECT entry_rowid FROM forms WHERE form = :form)')
+        conditions.append('e.id = ?')
+        params.append(id)
+    if forms:
+        cte = f'WITH wordforms(s) AS (VALUES {_vs(forms)})'
+        or_norm = 'OR normalized_form IN wordforms' if normalized else ''
+        and_rank = '' if search_all_forms else 'AND rank = 0'
+        conditions.append(f'''
+            e.rowid IN
+               (SELECT entry_rowid
+                  FROM forms
+                 WHERE (form IN wordforms {or_norm}) {and_rank})
+        '''.strip())
+        params.extend(forms)
     if pos:
-        conditions.append('e.pos = :pos')
+        conditions.append('e.pos = ?')
+        params.append(pos)
     if lexicon_rowids:
-        kws = {f'lex{i}': rowid for i, rowid in enumerate(lexicon_rowids, 1)}
-        params.update(kws)
-        conditions.append(f'e.lexicon_rowid IN ({_kws(kws)})')
+        conditions.append(f'e.lexicon_rowid IN ({_qs(lexicon_rowids)})')
+        params.extend(lexicon_rowids)
 
+    condition = ''
     if conditions:
-        query_parts.append(' WHERE ' + '\n   AND '.join(conditions))
+        condition = 'WHERE ' + '\n           AND '.join(conditions)
 
-    query_parts.append(' ORDER BY e.rowid, e.id, f.rank')
+    query = f'''
+          {cte}
+        SELECT DISTINCT e.lexicon_rowid, e.rowid, e.id, e.pos,
+                        f.form, f.id, f.script, f.rowid
+          FROM entries AS e
+          JOIN forms AS f ON f.entry_rowid = e.rowid
+         {condition}
+         ORDER BY e.rowid, e.id, f.rank
+    '''
 
-    query = '\n'.join(query_parts)
     rows: Iterator[
         Tuple[int, int, str, str, str, Optional[str], Optional[str], int]
     ] = conn.execute(query, params)
     groupby = itertools.groupby
     for key, group in groupby(rows, lambda row: row[0:4]):
         lexid, rowid, id, pos = key
-        forms = [(row[4], row[5], row[6], row[7]) for row in group]
-        yield (id, pos, forms, lexid, rowid)
+        wordforms = [(row[4], row[5], row[6], row[7]) for row in group]
+        yield (id, pos, wordforms, lexid, rowid)
 
 
 def find_senses(
-        id: str = None,
-        form: str = None,
-        pos: str = None,
-        lexicon_rowids: Sequence[int] = None,
+    id: str = None,
+    forms: Sequence[str] = None,
+    pos: str = None,
+    lexicon_rowids: Sequence[int] = None,
+    normalized: bool = False,
+    search_all_forms: bool = False,
 ) -> Iterator[_Sense]:
     conn = connect()
-    query_parts = [
-        'SELECT DISTINCT s.id, e.id, ss.id, s.lexicon_rowid, s.rowid'
-        '  FROM senses AS s'
-        '  JOIN entries AS e ON e.rowid = s.entry_rowid'
-        '  JOIN synsets AS ss ON ss.rowid = s.synset_rowid'
-    ]
-
-    params: Dict[str, Any] = {'id': id, 'form': form, 'pos': pos}
+    cte = ''
+    params: List = []
     conditions = []
     if id:
-        conditions.append('s.id = :id')
-    if form:
-        conditions.append('s.entry_rowid IN'
-                          ' (SELECT entry_rowid FROM forms WHERE form = :form)')
+        conditions.append('s.id = ?')
+        params.append(id)
+    if forms:
+        cte = f'WITH wordforms(s) AS (VALUES {_vs(forms)})'
+        or_norm = 'OR normalized_form IN wordforms' if normalized else ''
+        and_rank = '' if search_all_forms else 'AND rank = 0'
+        conditions.append(f'''
+            s.entry_rowid IN
+               (SELECT entry_rowid
+                  FROM forms
+                 WHERE (form IN wordforms {or_norm}) {and_rank})
+        '''.strip())
+        params.extend(forms)
     if pos:
-        conditions.append('e.pos = :pos')
+        conditions.append('e.pos = ?')
+        params.append(pos)
     if lexicon_rowids:
-        kws = {f'lex{i}': rowid for i, rowid in enumerate(lexicon_rowids, 1)}
-        params.update(kws)
-        conditions.append(f's.lexicon_rowid IN ({_kws(kws)})')
+        conditions.append(f's.lexicon_rowid IN ({_qs(lexicon_rowids)})')
+        params.extend(lexicon_rowids)
 
+    condition = ''
     if conditions:
-        query_parts.append(' WHERE ' + '\n   AND '.join(conditions))
+        condition = 'WHERE ' + '\n           AND '.join(conditions)
+
+    query = f'''
+          {cte}
+        SELECT DISTINCT s.id, e.id, ss.id, s.lexicon_rowid, s.rowid
+          FROM senses AS s
+          JOIN entries AS e ON e.rowid = s.entry_rowid
+          JOIN synsets AS ss ON ss.rowid = s.synset_rowid
+         {condition}
+    '''
 
-    query = '\n'.join(query_parts)
     rows: Iterator[_Sense] = conn.execute(query, params)
     yield from rows
 
 
 def find_synsets(
-        id: str = None,
-        form: str = None,
-        pos: str = None,
-        ili: str = None,
-        lexicon_rowids: Sequence[int] = None,
+    id: str = None,
+    forms: Sequence[str] = None,
+    pos: str = None,
+    ili: str = None,
+    lexicon_rowids: Sequence[int] = None,
+    normalized: bool = False,
+    search_all_forms: bool = False,
 ) -> Iterator[_Synset]:
     conn = connect()
-    query_parts = [
-        'SELECT DISTINCT ss.id, ss.pos,',
-        '                (SELECT ilis.id FROM ilis WHERE ilis.rowid=ss.ili_rowid),',
-        '                ss.lexicon_rowid, ss.rowid',
-        '  FROM synsets AS ss',
-    ]
-
-    params: Dict[str, Any] = {'id': id, 'form': form, 'pos': pos, 'ili': ili}
+    cte = ''
+    join = ''
     conditions = []
+    order = ''
+    params: List = []
     if id:
-        conditions.append('ss.id = :id')
-    if form:
-        query_parts.extend([
-            '  JOIN (SELECT _s.synset_rowid, _s.entry_rowid, _s.entry_rank',
-            '          FROM senses AS _s',
-            '          JOIN forms AS f',
-            '            ON f.entry_rowid = _s.entry_rowid',
-            '         WHERE f.form = :form) AS s',
-            '    ON s.synset_rowid = ss.rowid',
-        ])
+        conditions.append('ss.id = ?')
+        params.append(id)
+    if forms:
+        cte = f'WITH wordforms(s) AS (VALUES {_vs(forms)})'
+        or_norm = 'OR normalized_form IN wordforms' if normalized else ''
+        and_rank = '' if search_all_forms else 'AND rank = 0'
+        join = f'''\
+          JOIN (SELECT _s.entry_rowid, _s.synset_rowid, _s.entry_rank
+                  FROM forms AS f
+                  JOIN senses AS _s ON _s.entry_rowid = f.entry_rowid
+                 WHERE (f.form IN wordforms {or_norm}) {and_rank}) AS s
+            ON s.synset_rowid = ss.rowid
+        '''.strip()
+        params.extend(forms)
+        order = 'ORDER BY s.entry_rowid, s.entry_rank'
     if pos:
-        conditions.append('ss.pos = :pos')
+        conditions.append('ss.pos = ?')
+        params.append(pos)
     if ili:
-        query_parts.extend([
-            '  JOIN (SELECT _i.rowid FROM ilis AS _i WHERE _i.id=:ili) AS ili',
-            '    ON ss.ili_rowid = ili.rowid',
-        ])
+        conditions.append(
+            'ss.ili_rowid IN (SELECT ilis.rowid FROM ilis WHERE ilis.id = ?)'
+        )
+        params.append(ili)
     if lexicon_rowids:
-        kws = {f'lex{i}': rowid for i, rowid in enumerate(lexicon_rowids, 1)}
-        params.update(kws)
-        conditions.append(f'ss.lexicon_rowid IN ({_kws(kws)})')
+        conditions.append(f'ss.lexicon_rowid IN ({_qs(lexicon_rowids)})')
+        params.extend(lexicon_rowids)
 
+    condition = ''
     if conditions:
-        query_parts.append(' WHERE ' + '\n   AND '.join(conditions))
+        condition = 'WHERE ' + '\n           AND '.join(conditions)
 
-    if form:
-        query_parts.append(' ORDER BY s.entry_rowid, s.entry_rank')
+    query = f'''
+          {cte}
+        SELECT DISTINCT ss.id, ss.pos,
+                        (SELECT ilis.id FROM ilis WHERE ilis.rowid=ss.ili_rowid),
+                        ss.lexicon_rowid, ss.rowid
+          FROM synsets AS ss
+          {join}
+         {condition}
+         {order}
+    '''
 
-    query = '\n'.join(query_parts)
     rows: Iterator[_Synset] = conn.execute(query, params)
     yield from rows
 
@@ -581,7 +619,7 @@ def get_sense_relations(
 ) -> Iterator[_Sense_Relation]:
     params: List = []
     constraint = ''
-    if '*' not in relation_types:
+    if relation_types and '*' not in relation_types:
         constraint = f'WHERE type IN ({_qs(relation_types)})'
         params.extend(relation_types)
     params.append(source_rowid)
@@ -743,4 +781,5 @@ def get_lexfile(synset_rowid: int) -> Optional[str]:
 
 
 def _qs(xs: Collection) -> str: return ','.join('?' * len(xs))
+def _vs(xs: Collection) -> str: return ','.join(['(?)'] * len(xs))
 def _kws(xs: Collection) -> str: return ','.join(f':{x}' for x in xs)
diff --git a/wn/_types.py b/wn/_types.py
index a589f90..ef153fb 100644
--- a/wn/_types.py
+++ b/wn/_types.py
@@ -1,5 +1,7 @@
 
-from typing import Union, Mapping, Sequence, Dict, Any
+from typing import (
+    Optional, Union, Callable, Mapping, Sequence, Dict, Set, Any,
+)
 from pathlib import Path
 
 # For functions taking a filesystem path as a str or a pathlib.Path
@@ -10,3 +12,14 @@
 
 # User-facing metadata representation
 Metadata = Dict[str, Any]
+
+# A callable that returns a normalized word form for a given word form
+NormalizeFunction = Callable[[str], str]
+
+# Lemmatization returns a mapping of parts of speech (or None) to
+# lists of wordforms that are potential lemmas for some query word
+LemmatizeResult = Dict[Optional[str], Set[str]]
+
+# A callable that returns a LemmatizationResult for a given word form
+# and optional part of speech
+LemmatizeFunction = Callable[[str, Optional[str]], LemmatizeResult]
diff --git a/wn/lmf.py b/wn/lmf.py
index a1c098e..1f880ca 100644
--- a/wn/lmf.py
+++ b/wn/lmf.py
@@ -4,6 +4,7 @@
 """
 
 from typing import (
+    Type,
     Container,
     List,
     Tuple,
@@ -31,6 +32,19 @@
     PARTS_OF_SPEECH,
     LEXICOGRAPHER_FILES,
 )
+from wn.util import ProgressHandler, ProgressBar
+
+
+LEXICON_INFO_ATTRIBUTES = {
+    'LexicalEntry', 'ExternalLexicalEntry',
+    'Lemma', 'Form', 'Pronunciation', 'Tag',
+    'Sense', 'ExternalSense',
+    'SenseRelation', 'Example', 'Count',
+    'SyntacticBehaviour',
+    'Synset', 'ExternalSynset',
+    'Definition',  # 'ILIDefinition',
+    'SynsetRelation'
+}
 
 
 class LMFError(wn.Error):
@@ -80,8 +94,13 @@ class LMFWarning(Warning):
 
 class XMLEventIterator:
     """etree.iterparse() event iterator with lookahead"""
-    def __init__(self, iterator: Iterator[Tuple[str, ET.Element]]):
+    def __init__(
+        self,
+        iterator: Iterator[Tuple[str, ET.Element]],
+        progress: ProgressHandler
+    ):
         self.iterator = iterator
+        self.progress = progress
         self._next = next(iterator, (None, None))
 
     def __iter__(self):
@@ -89,7 +108,9 @@ def __iter__(self):
 
     def __next__(self):
         _next = self._next
+        event, elem = _next
         if _next == (None, None):
+            self.progress.set(status="Complete")
             raise StopIteration
         self._next = next(self.iterator, (None, None))
         return _next
@@ -114,6 +135,7 @@ def end(self, *tags: str) -> ET.Element:
             raise LMFError(f'expected </{"|".join(tags)}>, got <{elem.tag}>')
         if elem.tag not in tags:
             raise LMFError(f'expected </{"|".join(tags)}>, got </{elem.tag}>')
+        self.progress.update()
         return elem
 
 
@@ -526,18 +548,15 @@ def scan_lexicons(source: AnyPath) -> List[Dict]:
     def start(name, attrs):
         if name in ('Lexicon', 'LexiconExtension'):
             attrs['counts'] = {}
-            attrs['relations'] = set()
-            attrs['lexfiles'] = set()
             infos.append(attrs)
         elif name == 'Extends':
             infos[-1]['extends'] = attrs['id'], attrs['version']
         elif infos:
-            if name in ('SynsetRelation', 'SenseRelation'):
-                infos[-1]['relations'].add(attrs['relType'])
-            elif name == 'Synset' and 'lexfile' in attrs:
-                infos[-1]['lexfiles'].add(attrs['lexfile'])
             counts = infos[-1]['counts']
-            counts[name] = counts.get(name, 0) + 1
+            if name in counts:
+                counts[name] += 1
+            else:
+                counts[name] = 1
 
     p = xml.parsers.expat.ParserCreate()
     p.StartElementHandler = start
@@ -550,18 +569,33 @@ def start(name, attrs):
     return infos
 
 
-def load(source: AnyPath) -> LexicalResource:
+def load(
+    source: AnyPath,
+    progress_handler: Optional[Type[ProgressHandler]] = ProgressBar
+) -> LexicalResource:
     """Load wordnets encoded in the WN-LMF format.
 
     Args:
         source: path to a WN-LMF file
     """
+    if progress_handler is None:
+        progress_handler = ProgressHandler
+
     source = Path(source).expanduser()
 
     with source.open('rb') as fh:
         version = _read_header(fh)
+        # _read_header() only reads the first 2 lines
+        remainder = fh.read()
+        total_elements = remainder.count(b'</') + remainder.count(b'/>')
 
-    events = XMLEventIterator(ET.iterparse(source, events=('start', 'end')))
+    progress = progress_handler(
+        message='Read', total=total_elements, refresh_interval=10000
+    )
+    events = XMLEventIterator(
+        ET.iterparse(source, events=('start', 'end')),
+        progress
+    )
     root = events.start('LexicalResource')
 
     lexicons: List[Lexicon] = []
@@ -590,9 +624,13 @@ def _load_lexicon(events, version) -> Lexicon:
             requires.append(_load_dependency(events, 'Requires'))
 
     attrs = lex_root.attrib
+    events.progress.set(message=f'Read {attrs["id"]}:{attrs["version"]}')
+
+    events.progress.set(status='Lexical Entries')
     entries, frames, sbmap = _load_lexical_entries(
         events, extension, version, lex_root
     )
+    events.progress.set(status='Synsets')
     synsets = _load_synsets(
         events, extension, version, lex_root
     )
diff --git a/wn/morphy.py b/wn/morphy.py
new file mode 100644
index 0000000..6f7eeca
--- /dev/null
+++ b/wn/morphy.py
@@ -0,0 +1,165 @@
+
+"""A simple English lemmatizer that finds and removes known suffixes.
+
+"""
+
+from typing import Optional, Dict, Set, List, Tuple
+from enum import Flag, auto
+
+import wn
+from wn._types import LemmatizeResult
+from wn.constants import NOUN, VERB, ADJ, ADJ_SAT, ADV, PARTS_OF_SPEECH
+
+POSExceptionMap = Dict[str, Set[str]]
+ExceptionMap = Dict[str, POSExceptionMap]
+
+
+class _System(Flag):
+    """Flags to track suffix rules in various implementations of Morphy."""
+    PWN = auto()
+    NLTK = auto()
+    WN = auto()
+    ALL = PWN | NLTK | WN
+
+
+_PWN = _System.PWN
+_NLTK = _System.NLTK
+_WN = _System.WN
+_ALL = _System.ALL
+
+
+Rule = Tuple[str, str, _System]
+
+DETACHMENT_RULES: Dict[str, List[Rule]] = {
+    NOUN: [
+        ("s",    "",    _ALL),
+        ("ces",  "x",   _WN),
+        ("ses",  "s",   _ALL),
+        ("ves",  "f",   _NLTK | _WN),
+        ("ives", "ife", _WN),
+        ("xes",  "x",   _ALL),
+        ("xes",  "xis", _WN),
+        ("zes",  "z",   _ALL),
+        ("ches", "ch",  _ALL),
+        ("shes", "sh",  _ALL),
+        ("men",  "man", _ALL),
+        ("ies",  "y",   _ALL),
+    ],
+    VERB: [
+        ("s",   "",  _ALL),
+        ("ies", "y", _ALL),
+        ("es",  "e", _ALL),
+        ("es",  "",  _ALL),
+        ("ed",  "e", _ALL),
+        ("ed",  "",  _ALL),
+        ("ing", "e", _ALL),
+        ("ing", "",  _ALL),
+    ],
+    ADJ: [
+        ("er",  "",  _ALL),
+        ("est", "",  _ALL),
+        ("er",  "e", _ALL),
+        ("est", "e", _ALL),
+    ],
+    ADV: [],
+}
+DETACHMENT_RULES[ADJ_SAT] = DETACHMENT_RULES[ADJ]
+
+
+class Morphy:
+    """The Morphy lemmatizer class.
+
+    Objects of this class are callables that take a wordform and an
+    optional part of speech and return a dictionary mapping parts of
+    speech to lemmas. If objects of this class are not created with a
+    :class:`wn.Wordnet` object, the returned lemmas may be invalid.
+
+    Arguments:
+        wordnet: optional :class:`wn.Wordnet` instance
+
+    Example:
+
+        >>> import wn
+        >>> from wn.morphy import Morphy
+        >>> ewn = wn.Wordnet('ewn:2020')
+        >>> m = Morphy(ewn)
+        >>> m('axes', pos='n')
+        {'n': {'axe', 'ax', 'axis'}}
+        >>> m('geese', pos='n')
+        {'n': {'goose'}}
+        >>> m('gooses')
+        {'n': {'goose'}, 'v': {'goose'}}
+        >>> m('goosing')
+        {'v': {'goose'}}
+
+    """
+
+    def __init__(self, wordnet: Optional[wn.Wordnet] = None):
+        self._rules = {
+            pos: [rule for rule in rules if rule[2] & _System.WN]
+            for pos, rules in DETACHMENT_RULES.items()
+        }
+        exceptions: ExceptionMap = {pos: {} for pos in PARTS_OF_SPEECH}
+        all_lemmas: Dict[str, Set[str]] = {pos: set() for pos in PARTS_OF_SPEECH}
+        if wordnet:
+            for word in wordnet.words():
+                pos = word.pos
+                pos_exc = exceptions[pos]
+                lemma, *others = word.forms()
+                # store every lemma whether it has other forms or not
+                all_lemmas[pos].add(lemma)
+                # those with other forms map to the original lemmas
+                for other in others:
+                    if other in pos_exc:
+                        pos_exc[other].add(lemma)
+                    else:
+                        pos_exc[other] = {lemma}
+            self._initialized = True
+        else:
+            self._initialized = False
+        self._exceptions = exceptions
+        self._all_lemmas = all_lemmas
+
+    def __call__(self, form: str, pos: Optional[str] = None) -> LemmatizeResult:
+        result = {}
+        if not self._initialized:
+            result[pos] = {form}  # always include original when not initialized
+
+        if pos is None:
+            pos_list = list(DETACHMENT_RULES)
+        elif pos in DETACHMENT_RULES:
+            pos_list = [pos]
+        else:
+            pos_list = []  # not handled by morphy
+
+        no_pos_forms = result.get(None, set())  # avoid unnecessary duplicates
+        for _pos in pos_list:
+            candidates = self._morphstr(form, _pos) - no_pos_forms
+            if candidates:
+                result.setdefault(_pos, set()).update(candidates)
+
+        return result
+
+    def _morphstr(self, form: str, pos: str) -> Set[str]:
+        candidates: Set[str] = set()
+
+        initialized = self._initialized
+        if initialized:
+            all_lemmas = self._all_lemmas[pos]
+            if form in all_lemmas:
+                candidates.add(form)
+            candidates.update(self._exceptions[pos].get(form, set()))
+        else:
+            all_lemmas = set()
+
+        for suffix, repl, _ in self._rules[pos]:
+            # avoid applying rules that perform full suppletion
+            if form.endswith(suffix) and len(suffix) < len(form):
+                candidate = f'{form[:-len(suffix)]}{repl}'
+                if not initialized or candidate in all_lemmas:
+                    candidates.add(candidate)
+
+        return candidates
+
+
+morphy = Morphy()
diff --git a/wn/util.py b/wn/util.py
index 55a4959..2d50acc 100644
--- a/wn/util.py
+++ b/wn/util.py
@@ -1,5 +1,4 @@
 """Wn utility classes."""
-
 from typing import TextIO
 import sys
 
@@ -31,6 +30,7 @@ def __init__(
         message: str = '',
         count: int = 0,
         total: int = 0,
+        refresh_interval: int = 0,
         unit: str = '',
         status: str = '',
         file: TextIO = sys.stderr,
@@ -39,18 +39,23 @@ def __init__(
         self.kwargs = {
             'count': count,
             'total': total,
+            'refresh_interval': refresh_interval,
             'message': message,
             'unit': unit,
             'status': status,
         }
+        self._refresh_quota: int = refresh_interval
 
-    def update(self, n: int = 1) -> None:
+    def update(self, n: int = 1, force: bool = False) -> None:
         """Update the counter with the increment value *n*.
 
         This method should update the ``count`` key of :attr:`kwargs`
         with the increment value *n*. After this, it is expected to
         update some user-facing progress indicator.
 
+        If *force* is :python:`True`, any indicator will be refreshed
+        regardless of the value of the refresh interval.
+
         """
         self.kwargs['count'] += n  # type: ignore
 
@@ -63,7 +68,7 @@ def set(self, **kwargs) -> None:
 
         """
         self.kwargs.update(**kwargs)
-        self.update(0)
+        self.update(0, force=True)
 
     def flash(self, message: str) -> None:
         """Issue a message unrelated to the current counter.
@@ -100,13 +105,16 @@ class ProgressBar(ProgressHandler):
     #: The default formatting template.
     FMT = '\r{message}{bar}{counter}{status}'
 
-    def update(self, n: int = 1) -> None:
+    def update(self, n: int = 1, force: bool = False) -> None:
         """Increment the count by *n* and print the reformatted bar."""
         self.kwargs['count'] += n  # type: ignore
-        s = self.format()
-        if self.file:
-            print('\r\033[K', end='', file=self.file)
-            print(s, end='', file=self.file)
+        self._refresh_quota -= n
+        if force or self._refresh_quota <= 0:
+            self._refresh_quota = self.kwargs['refresh_interval']  # type: ignore
+            s = self.format()
+            if self.file:
+                print('\r\033[K', end='', file=self.file)
+                print(s, end='', file=self.file)
 
     def format(self) -> str:
         """Format and return the progress bar.
diff --git a/wn/web.py b/wn/web.py
new file mode 100644
index 0000000..adca6f7
--- /dev/null
+++ b/wn/web.py
@@ -0,0 +1,296 @@
+
+"""Web interface for Wn databases."""
+
+from typing import Optional, Union
+from functools import wraps
+from urllib.parse import urlsplit, parse_qs, urlencode
+
+from starlette.applications import Starlette  # type: ignore
+from starlette.responses import JSONResponse  # type: ignore
+from starlette.routing import Route  # type: ignore
+from starlette.requests import Request  # type: ignore
+
+import wn
+
+DEFAULT_PAGINATION_LIMIT = 50
+
+
+def paginate(proto):
+
+    def paginate_wrapper(func):
+
+        @wraps(func)
+        async def _paginate_wrapper(request: Request) -> JSONResponse:
+            url = str(request.url)
+            query = dict(request.query_params)
+            offset = abs(int(query.pop('page[offset]', 0)))
+            limit = abs(int(query.pop('page[limit]', DEFAULT_PAGINATION_LIMIT)))
+
+            obj = await func(request)
+            total = len(obj['data'])
+            prev = max(0, offset - limit)
+            next = offset + limit
+            last = (total//limit)*limit
+
+            obj['data'] = [proto(x, request) for x in obj['data'][offset:offset+limit]]
+            obj.setdefault('meta', {}).update(total=total)
+
+            links = {}
+            if offset > 0:
+                links['first'] = replace_query_params(url, **{'page[offset]': 0})
+                links['prev'] = replace_query_params(url, **{'page[offset]': prev})
+            if next < total:
+                links['next'] = replace_query_params(url, **{'page[offset]': next})
+                links['last'] = replace_query_params(url, **{'page[offset]': last})
+            if links:
+                obj.setdefault('links', {}).update(links)
+
+            return JSONResponse(obj)
+
+        return _paginate_wrapper
+
+    return paginate_wrapper
+
+
+def replace_query_params(url: str, **params) -> str:
+    u = urlsplit(url)
+    q = parse_qs(u.query)
+    q.update(params)
+    qs = urlencode(q, doseq=True)
+    return u._replace(query=qs).geturl()
+
+
+# Data-making functions
+
+def _url_for_obj(
+    request: Request,
+    name: str,
+    obj: Union[wn.Word, wn.Sense, wn.Synset],
+    lexicon: Optional[str] = None,
+) -> str:
+    if lexicon is None:
+        lexicon = obj.lexicon().specifier()
+    kwargs = {
+        'lexicon': lexicon,
+        name: obj.id
+    }
+    return request.url_for(name, **kwargs)
+
+
+def make_lexicon(lex: wn.Lexicon, request: Request) -> dict:
+    spec = lex.specifier()
+    return {
+        'id': spec,
+        'type': 'lexicon',
+        'attributes': {
+            # cannot have 'id' as JSON:API disallows it
+            'version': lex.version,
+            'label': lex.label,
+            'language': lex.language,
+            'license': lex.license,
+        },
+        'links': {
+            'self': request.url_for('lexicon', lexicon=spec)
+        },
+        'relationships': {
+            'words': {
+                'links': {'related': request.url_for('words', lexicon=spec)},
+            },
+            'synsets': {
+                'links': {'related': request.url_for('synsets', lexicon=spec)},
+            },
+            'senses': {
+                'links': {'related': request.url_for('senses', lexicon=spec)},
+            },
+        }
+    }
+
+
+def make_word(w: wn.Word, request: Request, basic: bool = False) -> dict:
+    lex_spec = w.lexicon().specifier()
+    d: dict = {
+        'id': w.id,
+        'type': 'word',
+        'attributes': {
+            'pos': w.pos,
+            'lemma': w.lemma(),
+            'forms': w.forms(),
+        },
+        'links': {
+            'self': _url_for_obj(request, 'word', w, lexicon=lex_spec)
+        }
+    }
+    if not basic:
+        synsets = w.synsets()
+        lex_link = request.url_for('lexicon', lexicon=lex_spec)
+        senses_link = request.url_for('senses', word=w.id, lexicon=lex_spec)
+        d.update({
+            'relationships': {
+                'senses': {'links': {'related': senses_link}},
+                'synsets': {'data': [dict(type='synset', id=ss.id) for ss in synsets]},
+                'lexicon': {'links': {'related': lex_link}}
+            },
+            'included': [make_synset(ss, request, basic=True) for ss in synsets]
+        })
+    return d
+
+
+def make_sense(s: wn.Sense, request: Request, basic: bool = False) -> dict:
+    lex_spec = s.lexicon().specifier()
+    d: dict = {
+        'id': s.id,
+        'type': 'sense',
+        'links': {
+            'self': _url_for_obj(request, 'sense', s, lexicon=lex_spec)
+        }
+    }
+    if not basic:
+        w = s.word()
+        ss = s.synset()
+        lex_link = request.url_for('lexicon', lexicon=lex_spec)
+        word_link = request.url_for('word', word=w.id, lexicon=lex_spec)
+        synset_link = request.url_for('synset', synset=ss.id, lexicon=lex_spec)
+        relationships: dict = {
+            'word': {'links': {'related': word_link}},
+            'synset': {'links': {'related': synset_link}},
+            'lexicon': {'links': {'related': lex_link}}
+        }
+        included = []
+        for relname, slist in s.relations().items():
+            relationships[relname] = {
+                'data': [dict(type='sense', id=_s.id) for _s in slist]
+            }
+            included.extend([make_sense(_s, request, basic=True) for _s in slist])
+        d.update({'relationships': relationships, 'included': included})
+    return d
+
+
+def make_synset(ss: wn.Synset, request: Request, basic: bool = False) -> dict:
+    lex_spec = ss.lexicon().specifier()
+    d: dict = {
+        'id': ss.id,
+        'type': 'synset',
+        'attributes': {
+            'pos': ss.pos,
+            'ili': ss._ili,
+        },
+        'links': {
+            'self': _url_for_obj(request, 'synset', ss, lexicon=lex_spec)
+        }
+    }
+    if not basic:
+        words = ss.words()
+        lex_link = request.url_for('lexicon', lexicon=lex_spec)
+        members_link = request.url_for('senses', synset=ss.id, lexicon=lex_spec)
+        relationships: dict = {
+            'members': {'links': {'related': members_link}},
+            'words': {'data': [dict(type='word', id=w.id) for w in words]},
+            'lexicon': {'links': {'related': lex_link}}
+        }
+        included = [make_word(w, request, basic=True) for w in words]
+        for relname, sslist in ss.relations().items():
+            relationships[relname] = {
+                'data': [dict(type='synset', id=_s.id) for _s in sslist]
+            }
+            included.extend([make_synset(_s, request, basic=True) for _s in sslist])
+        d.update({'relationships': relationships, 'included': included})
+    return d
+
+
+@paginate(make_lexicon)
+async def lexicons(request):
+    query = request.query_params
+    _lexicons = wn.lexicons(
+        lexicon=query.get('lexicon'),
+        lang=query.get('lang'),
+    )
+    return {'data': _lexicons}
+
+
+async def lexicon(request):
+    path_params = request.path_params
+    lex = wn.lexicons(lexicon=path_params['lexicon'])[0]
+    return JSONResponse({'data': make_lexicon(lex, request)})
+
+
+@paginate(make_word)
+async def words(request):
+    query = request.query_params
+    lexicon = request.path_params.get('lexicon') or query.get('lexicon')
+    _words = wn.words(
+        form=query.get('form'),
+        pos=query.get('pos'),
+        lexicon=lexicon,
+        lang=query.get('lang'),
+    )
+    return {'data': _words}
+
+
+async def word(request):
+    path_params = request.path_params
+    word = wn.word(path_params['word'], lexicon=path_params['lexicon'])
+    return JSONResponse({'data': make_word(word, request)})
+
+
+@paginate(make_sense)
+async def senses(request):
+    query = request.query_params
+    path = request.path_params
+    lexicon = path.get('lexicon') or query.get('lexicon')
+    if 'word' in path:
+        _senses = wn.word(path['word'], lexicon=lexicon).senses()
+    elif 'synset' in path:
+        _senses = wn.synset(path['synset'], lexicon=lexicon).members()
+    else:
+        _senses = wn.senses(
+            form=query.get('form'),
+            pos=query.get('pos'),
+            lexicon=lexicon,
+            lang=query.get('lang'),
+        )
+    return {'data': _senses}
+
+
+async def sense(request):
+    path_params = request.path_params
+    sense = wn.sense(path_params['sense'], lexicon=path_params['lexicon'])
+    return JSONResponse({'data': make_sense(sense, request)})
+
+
+@paginate(make_synset)
+async def synsets(request):
+    query = request.query_params
+    lexicon = request.path_params.get('lexicon') or query.get('lexicon')
+    _synsets = wn.synsets(
+        form=query.get('form'),
+        pos=query.get('pos'),
+        ili=query.get('ili'),
+        lexicon=lexicon,
+        lang=query.get('lang'),
+    )
+    return {'data': _synsets}
+
+
+async def synset(request):
+    path_params = request.path_params
+    synset = wn.synset(path_params['synset'], lexicon=path_params['lexicon'])
+    return JSONResponse({'data': make_synset(synset, request)})
+
+
+routes = [
+    Route('/lexicons', endpoint=lexicons),
+    Route('/lexicons/{lexicon}', endpoint=lexicon),
+    Route('/lexicons/{lexicon}/words', endpoint=words),
+    Route('/lexicons/{lexicon}/words/{word}', endpoint=word),
+    Route('/lexicons/{lexicon}/words/{word}/senses', endpoint=senses),
+    Route('/lexicons/{lexicon}/senses', endpoint=senses),
+    Route('/lexicons/{lexicon}/senses/{sense}', endpoint=sense),
+    Route('/lexicons/{lexicon}/synsets', endpoint=synsets),
+    Route('/lexicons/{lexicon}/synsets/{synset}', endpoint=synset),
+    Route('/lexicons/{lexicon}/synsets/{synset}/members', endpoint=senses),
+    Route('/words', endpoint=words),
+    Route('/senses', endpoint=senses),
+    Route('/synsets', endpoint=synsets),
+]
+
+app = Starlette(debug=True, routes=routes)


Test patch:
diff --git a/tests/data/mini-lmf-1.0.xml b/tests/data/mini-lmf-1.0.xml
index e54564a..a29da4e 100644
--- a/tests/data/mini-lmf-1.0.xml
+++ b/tests/data/mini-lmf-1.0.xml
@@ -81,6 +81,7 @@ Spanish:
 
     <LexicalEntry id="test-en-datum-n">
       <Lemma partOfSpeech="n" writtenForm="datum" />
+      <Form writtenForm="data" />
       <Sense id="test-en-datum-n-0006-01" synset="test-en-0006-n" />
     </LexicalEntry>
 
diff --git a/tests/morphy_test.py b/tests/morphy_test.py
new file mode 100644
index 0000000..64baa92
--- /dev/null
+++ b/tests/morphy_test.py
@@ -0,0 +1,40 @@
+
+import pytest
+
+import wn
+from wn import morphy
+
+
+def test_morphy_uninitialized():
+    # An unintialized Morphy isn't very bright, but it starts up
+    # fast. It relies on the database to filter bad items.
+    m = morphy.Morphy()
+    assert m('example', 'n') == {'n': {'example'}}
+    assert m('examples', 'n') == {'n': {'examples', 'example'}}
+    assert m('examples', 'v') == {'v': {'examples', 'example', 'exampl'}}
+    assert m('exemplifying', 'n') == {'n': {'exemplifying'}}
+    assert m('exemplifying', 'v') == {'v': {'exemplifying', 'exemplify', 'exemplifye'}}
+    assert m('data', 'n') == {'n': {'data'}}
+    assert m('datums', 'n') == {'n': {'datums', 'datum'}}  # expected false positive
+    assert m('examples', None) == {None: {'examples'},
+                                   'n': {'example'},
+                                   'v': {'example', 'exampl'}}
+    assert m('exemplifying', None) == {None: {'exemplifying'},
+                                       'v': {'exemplify', 'exemplifye'}}
+    assert m('data', None) == {None: {'data'}}
+
+
+@pytest.mark.usefixtures('mini_db')
+def test_morphy_initialized():
+    w = wn.Wordnet('test-en:1')
+    m = morphy.Morphy(wordnet=w)
+    assert m('example', 'n') == {'n': {'example'}}
+    assert m('examples', 'n') == {'n': {'example'}}
+    assert m('examples', 'v') == {}
+    assert m('exemplifying', 'n') == {}
+    assert m('exemplifying', 'v') == {'v': {'exemplify'}}
+    assert m('data', 'n') == {'n': {'datum'}}
+    assert m('datums', 'n') == {'n': {'datum'}}  # expected false positive
+    assert m('examples', None) == {'n': {'example'}}
+    assert m('exemplifying', None) == {'v': {'exemplify'}}
+    assert m('data', None) == {'n': {'datum'}}
diff --git a/tests/relations_test.py b/tests/relations_test.py
index 7d71251..735de79 100644
--- a/tests/relations_test.py
+++ b/tests/relations_test.py
@@ -40,6 +40,31 @@ def test_synset_hypernyms_expand_specified():
     ]
 
 
+@pytest.mark.usefixtures('mini_db')
+def test_synset_relations():
+    w = wn.Wordnet(lang='en')
+    assert w.synset('test-en-0002-n').relations() == {
+        'hypernym': [w.synset('test-en-0001-n')],
+        'hyponym': [w.synset('test-en-0004-n')]
+    }
+
+
+@pytest.mark.usefixtures('mini_db')
+def test_sense_get_related():
+    w = wn.Wordnet('test-en')
+    assert w.sense('test-en-example-n-0002-01').get_related() == [
+        w.sense('test-en-exemplify-v-0003-01')
+    ]
+
+
+@pytest.mark.usefixtures('mini_db')
+def test_sense_relations():
+    w = wn.Wordnet('test-en')
+    assert w.sense('test-en-example-n-0002-01').relations() == {
+        'derivation': [w.sense('test-en-exemplify-v-0003-01')]
+    }
+
+
 @pytest.mark.usefixtures('mini_db_1_1')
 def test_extension_relations():
     # default mode
diff --git a/tests/secondary_query_test.py b/tests/secondary_query_test.py
index 39ae397..9ac9e97 100644
--- a/tests/secondary_query_test.py
+++ b/tests/secondary_query_test.py
@@ -83,6 +83,16 @@ def test_synset_lemmas():
     assert wn.synset('test-es-0003-v').lemmas() == ['ejemplificar', 'ilustrar']
 
 
+@pytest.mark.usefixtures('mini_db')
+def test_synset_ili():
+    assert isinstance(wn.synset('test-en-0001-n').ili, wn.ILI)
+    assert wn.synset('test-en-0001-n').ili.id == 'i67447'
+    assert wn.synset('test-en-0001-n').ili.status == 'presupposed'
+    assert wn.synset('test-en-0005-n-fake').ili is None
+    assert wn.synset('test-en-0007-v').ili.id is None
+    assert wn.synset('test-en-0007-v').ili.status == 'proposed'
+
+
 @pytest.mark.usefixtures('mini_db')
 def test_synset_definition():
     assert wn.synset('test-en-0001-n').definition() == 'something that informs'
diff --git a/tests/wordnet_test.py b/tests/wordnet_test.py
new file mode 100644
index 0000000..a659115
--- /dev/null
+++ b/tests/wordnet_test.py
@@ -0,0 +1,82 @@
+
+import pytest
+
+import wn
+
+
+@pytest.mark.usefixtures('mini_db_1_1')
+def test_wordnet_lexicons():
+    en = wn.Wordnet('test-en')
+    assert len(en.lexicons()) == 1
+    assert len(en.expanded_lexicons()) == 0
+
+    en1 = wn.Wordnet('test-en:1')
+    assert en.lexicons() == en1.lexicons()
+    assert en.expanded_lexicons() == en1.expanded_lexicons()
+
+    en2 = wn.Wordnet(lang='en')
+    assert len(en2.lexicons()) == 2
+    assert len(en2.expanded_lexicons()) == 0
+
+    es = wn.Wordnet('test-es')
+    assert len(es.lexicons()) == 1
+    assert len(es.expanded_lexicons()) == 0
+
+    es2 = wn.Wordnet('test-es', expand='test-en')
+    assert len(es2.lexicons()) == 1
+    assert len(es2.expanded_lexicons()) == 1
+
+    ja = wn.Wordnet('test-ja')
+    assert len(ja.lexicons()) == 1
+    assert len(ja.expanded_lexicons()) == 1
+
+    ja2 = wn.Wordnet('test-ja', expand='')
+    assert len(ja2.lexicons()) == 1
+    assert len(ja2.expanded_lexicons()) == 0
+
+
+@pytest.mark.usefixtures('mini_db')
+def test_wordnet_normalize():
+    es = wn.Wordnet('test-es')
+    assert es.words('Informacion') == es.words('información')
+    assert es.words('ínfórmácíón') == es.words('información')
+    es = wn.Wordnet('test-es', normalizer=None)
+    assert es.words('informacion') == []
+    assert es.words('Información') == []
+
+    # The following doesn't necessarily work because any non-None
+    # normalizer causes the normalized form column to be tested with
+    # the original form
+    # es = wn.Wordnet('test-es', normalizer=str.lower)
+    # assert es.words('informacion') == []
+    # assert es.words('Información') == es.words('información')
+
+
+@pytest.mark.usefixtures('mini_db')
+def test_wordnet_lemmatize():
+    # default lemmatizer compares alternative forms
+    en = wn.Wordnet('test-en')
+    assert en.words('examples') == []
+    assert en.words('exemplifying') == en.words('exemplify')
+    assert en.words('data') == en.words('datum')
+
+    en = wn.Wordnet('test-en', search_all_forms=False)
+    assert en.words('examples') == []
+    assert en.words('exemplifying') == []
+    assert en.words('data') == []
+
+    def morphy_lite(form, pos):
+        result = {pos: {form}}
+        if pos in ('n', None) and form.endswith('s'):
+            result.setdefault('n', set()).add(form[:-1])
+        return result
+
+    en = wn.Wordnet('test-en', lemmatizer=morphy_lite, search_all_forms=False)
+    assert en.words('examples', pos='n') == en.words('example')
+    assert en.words('examples') == en.words('example')
+    assert en.words('exemplifying') == []
+    assert en.words('data') == []
+
+    en = wn.Wordnet('test-en', lemmatizer=morphy_lite, search_all_forms=True)
+    assert en.words('data') == en.words('datum')
+    assert en.words('exemplifying') == en.words('exemplify')