Repository: goodmami/wn

Issue:
Return the lemmas in the order specified in the XML
**Describe the bug**
The order of the lemmas returned by the `.lemma()`/`.words()` function does not match that specified in the wordnet XML file.

**To Reproduce**

```pycon
>>> import wn
>>> wn.download("oewn:2024")
>>> wordnet = wn.Wordnet("oewn:2024")
>>> synset = wordnet.synset("oewn-08301402-n")
>>> synset.words()
[Word('oewn-Gymnasium-n'),
 Word('oewn-lycee-n'),
 Word('oewn-lyceum-n'),
 Word('oewn-middle_school-n'),
 Word('oewn-secondary_school-n')]
```

**Expected behavior**
The order of the lemmas of a synset is specifed by the `members` attribute of the synset

```xml
    <Synset id="oewn-08301402-n" ili="i80680" 
members="oewn-secondary_school-n oewn-lyceum-n oewn-lycee-n oewn-Gymnasium-n oewn-middle_school-n"
 partOfSpeech="n" lexfile="noun.group">
```

The result should be ordered according to this attribute

**Environment**

```
Python 3.11.12
Wn 0.12.0
```

Discussion:
@goodmami: Thanks for the report, @jmccrae

The `senses` table does encode the "rank" (i.e., order) of the sense both in the entry (`entry_rank`) and in the synset members list (`synset_rank`), and queries are ordered by this rank. We can see that the `omw-en` lexicon does return the expected order:

```python
>>> wn.synset("omw-en-08284481-n", lexicon="omw-en:1.4").words()
[Word('omw-en-secondary_school-n'),
 Word('omw-en-lyceum-n'),
 Word('omw-en-lycee-n'),
 Word('omw-en-Gymnasium-n'),
 Word('omw-en-middle_school-n')]
```

It appears there was some confusion as to whether the `members` attribute should encode the sense IDs (which `omw-en` does) or the word IDs (which OEWN does). From https://globalwordnet.github.io/schemas/ (emphasis added):

> ```xml
>         <Synset id="example-en-10161911-n" ili="i90287" partOfSpeech="n"
>             members="example-en-10161911-n-1 example-en-1-n-1">
>             <Definition>
>                 the father of your father or mother
>             </Definition>
> ```
> 
> The members property gives the **list of senses** in order.

However, in https://aclanthology.org/2021.gwc-1.11.pdf the prose and the example are in disagreement (section 3.1, emphasis added, xml reformatted):

> [...] Resources such as the open English WordNet were preserving [the order of words in a synset] by means of encoding it within the **sense identifiers** with a new attribute `members`. [...]
>
> ```xml
> <LexicalEntry id="ex-rabbit-n">
>   <Lemma writtenForm="rabbit" partOfSpeech="n"/>
>   <Sense id="ex-rabbit-n-1" synset="ex-synset-1"/>
>   <Sense id="ex-rabbit-n-2" synset="ex-synset-2"/>
> </LexicalEntry>
> <Synset id="ex-synset-1" members="ex-rabbit-n ex-bunny-n"/>
> ```

Wn expects a list of sense IDs and calculates the rank based on this. If there is a lookup failure, the sense assigns a default value to `synset_rank`.

Since a sense uniquely identifies both a word and a synset, the two are effectively equivalent. We could therefore change the lexicon-adding code to look up a sense's synset rank either by the sense ID or the word ID and update the schema documentation to state that `members` can be either a list of sense IDs or word IDs.@goodmami: @jmccrae see #256 for a fix. This will require you to re-add the affected lexicons to the database, but the next release will probably require a full DB rebuild, anyway (see #254).

I'll leave the PR open for a little bit so you can comment.@jmccrae: Looks good to me and thanks for the quick response.

Solution patch:
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 1d711ec..4cd315a 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -2,6 +2,11 @@
 
 ## [Unreleased][unreleased]
 
+## Changed
+
+* `wn.add()` allows synset members to be lexical entry IDs for rank
+  calculations ([#255])
+
 
 ## [v0.12.0]
 
@@ -771,3 +776,4 @@ abandoned, but this is an entirely new codebase.
 [#241]: https://github.com/goodmami/wn/issues/241
 [#246]: https://github.com/goodmami/wn/issues/246
 [#250]: https://github.com/goodmami/wn/issues/250
+[#255]: https://github.com/goodmami/wn/issues/255
diff --git a/wn/_add.py b/wn/_add.py
index 5e955a1..09bdc32 100644
--- a/wn/_add.py
+++ b/wn/_add.py
@@ -665,9 +665,11 @@ def _insert_senses(
     progress: ProgressHandler
 ) -> None:
     progress.set(status='Senses')
-    ssrank = {s: i
-              for ss in _local_synsets(synsets)
-              for i, s in enumerate(ss.get('members', []))}
+    ssrank = {
+        (ss['id'], _id): i
+        for ss in _local_synsets(synsets)
+        for i, _id in enumerate(ss.get('members', []))
+    }
     query = f'''
         INSERT INTO senses
         VALUES (null,
@@ -682,14 +684,23 @@ def _insert_senses(
     '''
     for batch in _batch(entries):
         data = [
-            (sense['id'],
-             lexid,
-             entry['id'], lexidmap.get(entry['id'], lexid),
-             i,
-             sense['synset'], lexidmap.get(sense['synset'], lexid),
-             ssrank.get(sense['id'], DEFAULT_MEMBER_RANK),
-             sense.get('lexicalized', True),
-             sense['meta'])
+            (
+                sense['id'],
+                lexid,
+                entry['id'], lexidmap.get(entry['id'], lexid),
+                i,
+                sense['synset'], lexidmap.get(sense['synset'], lexid),
+                # members can be sense or entry IDs
+                ssrank.get(
+                    (sense['synset'], sense['id']),
+                    ssrank.get(
+                        (sense['synset'], entry['id']),
+                        DEFAULT_MEMBER_RANK
+                    )
+                ),
+                sense.get('lexicalized', True),
+                sense['meta']
+            )
             for entry in batch
             for i, sense in enumerate(_local_senses(_senses(entry)))
         ]


Test patch:
diff --git a/tests/data/sense-member-order.xml b/tests/data/sense-member-order.xml
new file mode 100644
index 0000000..ece2ea7
--- /dev/null
+++ b/tests/data/sense-member-order.xml
@@ -0,0 +1,33 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE LexicalResource SYSTEM "http://globalwordnet.github.io/schemas/WN-LMF-1.1.dtd">
+<LexicalResource xmlns:dc="http://purl.org/dc/elements/1.1/">
+
+<!-- duplicate ID in synsets -->
+
+  <Lexicon id="test"
+           label="Testing Sense Member Orders"
+           language="en"
+           email="maintainer@example.com"
+           license="https://creativecommons.org/licenses/by/4.0/"
+           version="1">
+
+    <LexicalEntry id="test-foo-n">
+      <Lemma partOfSpeech="n" writtenForm="foo" />
+      <Sense id="test-01-foo-n" synset="test-01-n" />
+      <Sense id="test-02-foo-n" synset="test-02-n" />
+    </LexicalEntry>
+
+    <LexicalEntry id="test-bar-n">
+      <Lemma partOfSpeech="n" writtenForm="bar" />
+      <Sense id="test-02-bar-n" synset="test-02-n" />
+      <Sense id="test-01-bar-n" synset="test-01-n" />
+    </LexicalEntry>
+
+    <!-- sense IDs as members -->
+    <Synset id="test-01-n" ili="i12345" partOfSpeech="n" members="test-01-bar-n test-01-foo-n"/>
+    <!-- word IDs as members -->
+    <Synset id="test-02-n" ili="i12346" partOfSpeech="n" members="test-bar-n test-foo-n" />
+
+  </Lexicon>
+
+</LexicalResource>
diff --git a/tests/secondary_query_test.py b/tests/secondary_query_test.py
index 0164286..c2b094c 100644
--- a/tests/secondary_query_test.py
+++ b/tests/secondary_query_test.py
@@ -146,3 +146,25 @@ def test_synset_lexicalized():
 def test_synset_translate():
     assert len(wn.synset('test-en-0001-n').translate(lang='es')) == 1
     assert len(wn.synset('test-es-0001-n').translate(lang='en')) == 1
+
+
+@pytest.mark.usefixtures('uninitialized_datadir')
+def test_word_sense_order(datadir):
+    wn.add(datadir / 'sense-member-order.xml')
+    assert [s.id for s in wn.word('test-foo-n').senses()] == [
+        "test-01-foo-n", "test-02-foo-n",
+    ]
+    assert [s.id for s in wn.word('test-bar-n').senses()] == [
+        "test-02-bar-n", "test-01-bar-n",
+    ]
+
+
+@pytest.mark.usefixtures('uninitialized_datadir')
+def test_synset_member_order(datadir):
+    wn.add(datadir / 'sense-member-order.xml')
+    assert [s.id for s in wn.synset('test-01-n').senses()] == [
+        "test-01-bar-n", "test-01-foo-n",
+    ]
+    assert [s.id for s in wn.synset('test-02-n').senses()] == [
+        "test-02-bar-n", "test-02-foo-n",
+    ]