Repository: radomirbosak/duden (language: python)

Issue:
Documentation of synonyms doesn't match code
In the usage document it looks like `DudenWord.synonyms` is just a single string but the code actually returns a list with strings.  Or always just *one* string, as the fix to #115 suggests.


Discussion / hints:
Hi @Marrin , thank you for creating this issue! You are absolutely right: the documentation does not match the real output.

I fixed the usage document in https://github.com/radomirbosak/duden/pull/182 which now looks [like this](https://github.com/radomirbosak/duden/blob/master/docs/usage.md)
![image](https://user-images.githubusercontent.com/4988740/202057650-4f0d1a1d-a0b1-4f88-aeb1-b33a0b6d3cee.png)

![image](https://user-images.githubusercontent.com/4988740/202057590-562829d1-ed7d-4ba4-b16e-1ca532b5d17d.png)

I also thought about modifying the implementation to always return the single (first) item of the array, but I had some reservations (below), so I modified just the documentation for now. Reservations:
* breaking any code which expect DudenWord.synonyms an array
* the current implementation of `.synonyms` actually can return both a simple string as well as list of strings (or  list of lists of strings, see [recursively_extract](https://github.com/radomirbosak/duden/blob/498d8df7d487051d00d1d67f7957dc1a1aa8d5ae/duden/common.py#L10) function), based on the source HTML code.
  * the HTML always seems to contain a single `<li>` element under array `<ul>`, which is semantically a 1-item array, but I don't know if the website administrators plan to expand the array in the future. (Though I think they just forgot to remove the `<ul><li>` structure from there)
![image](https://user-images.githubusercontent.com/4988740/202058395-08414428-f55e-46be-9ef6-60a19afa7c85.png)

But if the content remains as a 1-item list, it might make practical sense to change the implementation and just return the first element.

What do you think?

Fix patch:
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 8ca37bc..2d88e2d 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -10,6 +10,7 @@ Breaking:
 New features:
 
 * Add `pronunciation_audio_url` property #183 by @mundanevision20
+* Synonyms are now properly split #195
 
 Bugfixes:
 
diff --git a/docs/usage.md b/docs/usage.md
index ab71b64..a897ebc 100644
--- a/docs/usage.md
+++ b/docs/usage.md
@@ -36,7 +36,7 @@ This example showcases the most useful functions of the `DudenWord` class.
 'barmherziges Wesen, Verhalten'
 
 > w.synonyms
-'[Engels]güte, Milde, Nachsicht, Nachsichtigkeit; (gehoben) Herzensgüte, Mildtätigkeit, Seelengüte; (bildungssprachlich) Humanität, Indulgenz; (veraltend) Wohltätigkeit; (Religion) Gnade'
+['[Engels]güte', 'Milde', 'Nachsicht', 'Nachsichtigkeit']
 
 > w.origin
 'mittelhochdeutsch barmherzekeit, barmherze, althochdeutsch armherzi, nach (kirchen)lateinisch misericordia'
diff --git a/duden/word.py b/duden/word.py
index 60fecc6..c8d58bf 100644
--- a/duden/word.py
+++ b/duden/word.py
@@ -239,16 +239,17 @@ def synonyms(self):
         """
         Return the structure with word synonyms
         """
-        try:
-            section = self.soup.find("div", id="synonyme")
-            section = copy.copy(section)
-            if section.header:
-                section.header.extract()
-            return recursively_extract(
-                section, maxdepth=2, exfun=lambda x: x.text.strip()
-            )
-        except AttributeError:
+        section = self.soup.find("div", id="synonyme")
+        if section is None:
             return None
+        section = copy.copy(section)
+        if section.header:
+            section.header.extract()
+        more_nav = section.find("nav", class_="more")
+        if more_nav:
+            more_nav.extract()
+
+        return split_synonyms(section.text.strip())
 
     @property
     def origin(self):
@@ -430,3 +431,34 @@ def alternative_spellings(self):
             return None
 
         return [spelling.get_text() for spelling in alternative_spellings]
+
+
+def split_synonyms(text):
+    """
+    Properly split strings like
+
+    meaning1, (commonly) meaning2; (formal, distant) meaning3
+    """
+    # split by ',' and ';'
+    comma_splits = text.split(",")
+    fine_splits = []
+    for split in comma_splits:
+        fine_splits.extend(split.split(";"))
+
+    # now join back parts which are inside of parentheses
+    final_splits = []
+    inside_parens = False
+    for split in fine_splits:
+        if inside_parens:
+            final_splits[-1] = final_splits[-1] + "," + split
+        else:
+            final_splits.append(split)
+
+        if "(" in split and ")" in split:
+            inside_parens = split.index("(") > split.index("(")
+        elif "(" in split:
+            inside_parens = True
+        elif ")" in split:
+            inside_parens = False
+
+    return [split.strip() for split in final_splits]


Test patch:
diff --git a/tests/test_data/Barmherzigkeit.yaml b/tests/test_data/Barmherzigkeit.yaml
index 50f69b7..92b1105 100644
--- a/tests/test_data/Barmherzigkeit.yaml
+++ b/tests/test_data/Barmherzigkeit.yaml
@@ -16,7 +16,10 @@ origin: mittelhochdeutsch barmherzekeit, barmherze, althochdeutsch armherzi, nac
 grammar_overview: 'die Barmherzigkeit; Genitiv: der Barmherzigkeit'
 compounds: null
 synonyms:
-- '[Engels]güte, Milde, Nachsicht, Nachsichtigkeit'
+- '[Engels]güte'
+- Milde
+- Nachsicht
+- Nachsichtigkeit
 words_before:
 - barmen
 - Barmen
diff --git a/tests/test_data/Feiertag.yaml b/tests/test_data/Feiertag.yaml
index 9ae6488..5edeb22 100644
--- a/tests/test_data/Feiertag.yaml
+++ b/tests/test_data/Feiertag.yaml
@@ -43,7 +43,9 @@ compounds:
   - verbringen
   - öffnen
 synonyms:
-- Festtag, Gedenktag, Ehrentag
+- Festtag
+- Gedenktag
+- Ehrentag
 words_before:
 - Feierlichkeit
 - Feiermodus
diff --git a/tests/test_data/Kragen.yaml b/tests/test_data/Kragen.yaml
index 1bcc139..8be889a 100644
--- a/tests/test_data/Kragen.yaml
+++ b/tests/test_data/Kragen.yaml
@@ -21,7 +21,8 @@ grammar_overview: 'der Kragen; Genitiv: des Kragens, Plural: die Kragen, süddeu
   österreichisch, schweizerisch: Krägen'
 compounds: null
 synonyms:
-- Gurgel, Kehle
+- Gurgel
+- Kehle
 words_before:
 - Kraftwerksbetreiberin
 - Kraftwort
diff --git a/tests/test_data/Petersilie.yaml b/tests/test_data/Petersilie.yaml
index fd0bff0..12ccd60 100644
--- a/tests/test_data/Petersilie.yaml
+++ b/tests/test_data/Petersilie.yaml
@@ -17,8 +17,9 @@ origin: mittelhochdeutsch pētersil(je), althochdeutsch petersilie, petrasile <
 grammar_overview: 'die Petersilie; Genitiv: der Petersilie, Plural: die Petersilien'
 compounds: null
 synonyms:
-- (schweizerisch) Peterli; (bayrisch, österreichisch umgangssprachlich) Petersil;
-  (südwestdeutsch und schweizerisch mundartlich) Peterle
+- (schweizerisch) Peterli
+- (bayrisch, österreichisch umgangssprachlich) Petersil
+- (südwestdeutsch und schweizerisch mundartlich) Peterle
 words_before:
 - Peter-Paul-Kirche
 - Petersburg
diff --git a/tests/test_data/einfach.yaml b/tests/test_data/einfach.yaml
index 87d6e2d..df22e24 100644
--- a/tests/test_data/einfach.yaml
+++ b/tests/test_data/einfach.yaml
@@ -45,7 +45,10 @@ compounds:
   - stimmen
   - werden
 synonyms:
-- einmal, nicht doppelt, nicht mehrfach, bequem
+- einmal
+- nicht doppelt
+- nicht mehrfach
+- bequem
 words_before:
 - Eineuromünze
 - Eineurostück
diff --git a/tests/test_data/laufen.yaml b/tests/test_data/laufen.yaml
index c931c1b..7afcbf9 100644
--- a/tests/test_data/laufen.yaml
+++ b/tests/test_data/laufen.yaml
@@ -63,7 +63,10 @@ compounds:
   - Vertrag
   - Vorbereitung
 synonyms:
-- eilen, fegen, hetzen, jagen
+- eilen
+- fegen
+- hetzen
+- jagen
 words_before:
 - Laufbekleidung
 - Laufbrett
diff --git a/tests/test_word.py b/tests/test_word.py
new file mode 100644
index 0000000..0d0af5a
--- /dev/null
+++ b/tests/test_word.py
@@ -0,0 +1,12 @@
+"""Test word functions"""
+
+from duden.word import split_synonyms
+
+
+def test_split_synonyms():
+    """Test one-line list splitting"""
+    assert split_synonyms("") == [""]
+    assert split_synonyms("a, b ,c") == ["a", "b", "c"]
+
+    expected = ["a", "b (b, c)", "d (d, e, f) g", "h"]
+    assert split_synonyms("a, b (b, c); d (d; e, f) g, h") == expected