Repository: OCHA-DAP/hdx-python-country

Issue:
Country.simplify_countryname() exceptions with partial brackets
This is a bit of an odd situation, but easy for users to get into.

Given a country with a formal name
And the formal name uses the bracketed notation (e.g. `Korea (the Republic of)`)
And the formal name has punctuation stripped from the start and end of the string (e.g. to remove trailing fullstops - `country = input_string.strip(strings.punctuation)`)
When `Country.simplify_countryname()` processes the country
Then it should not raise `ValueError: list.remove(x): x not in list`

What seems to happen is:
* [Bracket removal](https://github.com/OCHA-DAP/hdx-python-country/blob/main/src/hdx/location/country.py#L698) doesn't detect the bracketed part of the formal name because we removed the trailing character
* The [simplifications strip "the republic of"](https://github.com/OCHA-DAP/hdx-python-country/blob/main/src/hdx/location/country.py#L701-L712) leaving `countryupper` as `Korea (`
* [`get_words_in_sentence()` strips the punctuation and spaces and returns one word](https://github.com/OCHA-DAP/hdx-python-country/blob/main/src/hdx/location/country.py#L715) and [so doesn't overwrite `countryupper`](https://github.com/OCHA-DAP/hdx-python-country/blob/main/src/hdx/location/country.py#L716)
* `countryupper` still contains trailing punctuation, [which is cleaned up](https://github.com/OCHA-DAP/hdx-python-country/blob/main/src/hdx/location/country.py#L719), but leaves `Country[space]`, which isn't in the list of words and so errors on `remove()`

It might be possible to fix this just by adding whitespace to the list of characters to strip on line 719. I can't see how it could have a negative impact. But there may be other "partial punctuation" situations that aren't handled that I haven't seen.

Discussion:
@mcarans: Thanks @StuartBertram for submitting this bug. Please can you see if https://github.com/OCHA-DAP/hdx-python-country/pull/71 resolves it for your test cases and let me know.@StuartBertram: Yeah, that seems to fix it. Thanks.

Solution patch:
diff --git a/.github/workflows/publish.yaml b/.github/workflows/publish.yaml
index bff0ddf..5498fad 100644
--- a/.github/workflows/publish.yaml
+++ b/.github/workflows/publish.yaml
@@ -10,7 +10,7 @@ jobs:
 
     environment:
       name: pypi
-      url: https://pypi.org/p/hdx-python-utilities
+      url: https://pypi.org/p/hdx-python-country
 
     permissions:
       id-token: write  # IMPORTANT: mandatory for trusted publishing
diff --git a/README.md b/README.md
index 27643e1..8889461 100755
--- a/README.md
+++ b/README.md
@@ -83,16 +83,16 @@ To run the tests and view coverage, execute:
 
 ## Packages
 
-[uv](https://github.com/astral-sh/uv) is used for package management.  If 
-you’ve introduced a new package to the source code (i.e. anywhere in `src/`), 
-please add it to the `project.dependencies` section of `pyproject.toml` with 
+[uv](https://github.com/astral-sh/uv) is used for package management.  If
+you’ve introduced a new package to the source code (i.e. anywhere in `src/`),
+please add it to the `project.dependencies` section of `pyproject.toml` with
 any known version constraints.
 
 To add packages required only for testing, add them to the `test` section under
 `[project.optional-dependencies]`.
 
 Any changes to the dependencies will be automatically reflected in
-`requirements.txt` with `pre-commit`, but you can re-generate the file without 
+`requirements.txt` with `pre-commit`, but you can re-generate the file without
 committing by executing:
 
 ```shell
diff --git a/pyproject.toml b/pyproject.toml
index 8cf9290..081e199 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -34,7 +34,7 @@ classifiers = [
 requires-python = ">=3.8"
 
 dependencies = [
-    "hdx-python-utilities>=3.8.5",
+    "hdx-python-utilities>=3.8.6",
     "libhxl>=5.2.2",
     "tenacity",
 ]
diff --git a/requirements.txt b/requirements.txt
index afe75c8..917a135 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -2,7 +2,7 @@
 #    uv pip compile pyproject.toml --resolver=backtracking --all-extras -o requirements.txt
 annotated-types==0.7.0
     # via pydantic
-astdoc==1.1.5
+astdoc==1.2.1
     # via mkapi
 attrs==25.3.0
     # via
@@ -14,13 +14,13 @@ babel==2.17.0
     # via mkdocs-material
 backrefs==5.8
     # via mkdocs-material
-certifi==2025.1.31
+certifi==2025.4.26
     # via requests
 cfgv==3.4.0
     # via pre-commit
 chardet==5.2.0
     # via frictionless
-charset-normalizer==3.4.1
+charset-normalizer==3.4.2
     # via requests
 click==8.1.8
     # via
@@ -28,23 +28,25 @@ click==8.1.8
     #   typer
 colorama==0.4.6
     # via mkdocs-material
-coverage==7.7.1
+coverage==7.8.0
     # via pytest-cov
+cydifflib==1.2.0
+    # via hdx-python-utilities
 distlib==0.3.9
     # via virtualenv
 et-xmlfile==2.0.0
     # via openpyxl
 filelock==3.18.0
     # via virtualenv
-frictionless==5.18.0
+frictionless==5.18.1
     # via hdx-python-utilities
 ghp-import==2.1.0
     # via mkdocs
-hdx-python-utilities==3.8.5
+hdx-python-utilities==3.8.6
     # via hdx-python-country (pyproject.toml)
-humanize==4.12.2
+humanize==4.12.3
     # via frictionless
-identify==2.6.9
+identify==2.6.10
     # via pre-commit
 idna==3.10
     # via requests
@@ -68,21 +70,20 @@ jsonschema==4.23.0
     # via
     #   frictionless
     #   tableschema-to-template
-jsonschema-specifications==2024.10.1
+jsonschema-specifications==2025.4.1
     # via jsonschema
 libhxl==5.2.2
     # via hdx-python-country (pyproject.toml)
 loguru==0.7.3
     # via hdx-python-utilities
-markdown==3.7
+markdown==3.8
     # via
-    #   astdoc
     #   mkdocs
     #   mkdocs-material
     #   pymdown-extensions
 markdown-it-py==3.0.0
     # via rich
-marko==2.1.2
+marko==2.1.3
     # via frictionless
 markupsafe==3.0.2
     # via
@@ -94,7 +95,7 @@ mergedeep==1.3.4
     # via
     #   mkdocs
     #   mkdocs-get-deps
-mkapi==4.1.3
+mkapi==4.3.2
     # via hdx-python-country (pyproject.toml)
 mkdocs==1.6.1
     # via
@@ -102,7 +103,7 @@ mkdocs==1.6.1
     #   mkdocs-material
 mkdocs-get-deps==0.2.0
     # via mkdocs
-mkdocs-material==9.6.9
+mkdocs-material==9.6.12
     # via mkapi
 mkdocs-material-extensions==1.3.1
     # via mkdocs-material
@@ -110,7 +111,7 @@ nodeenv==1.9.1
     # via pre-commit
 openpyxl==3.1.5
     # via hdx-python-utilities
-packaging==24.2
+packaging==25.0
     # via
     #   mkdocs
     #   pytest
@@ -118,7 +119,7 @@ paginate==0.5.7
     # via mkdocs-material
 pathspec==0.12.1
     # via mkdocs
-petl==1.7.15
+petl==1.7.16
     # via frictionless
 platformdirs==4.3.7
     # via
@@ -132,15 +133,15 @@ ply==3.11
     #   libhxl
 pre-commit==4.2.0
     # via hdx-python-country (pyproject.toml)
-pydantic==2.10.6
+pydantic==2.11.4
     # via frictionless
-pydantic-core==2.27.2
+pydantic-core==2.33.2
     # via pydantic
 pygments==2.19.1
     # via
     #   mkdocs-material
     #   rich
-pymdown-extensions==10.14.3
+pymdown-extensions==10.15
     # via mkdocs-material
 pyphonetics==0.5.3
     # via hdx-python-utilities
@@ -148,7 +149,7 @@ pytest==8.3.5
     # via
     #   hdx-python-country (pyproject.toml)
     #   pytest-cov
-pytest-cov==6.0.0
+pytest-cov==6.1.1
     # via hdx-python-country (pyproject.toml)
 python-dateutil==2.9.0.post0
     # via
@@ -187,11 +188,9 @@ requests-file==2.1.0
     # via hdx-python-utilities
 rfc3986==2.0.0
     # via frictionless
-rich==13.9.4
-    # via
-    #   mkapi
-    #   typer
-rpds-py==0.23.1
+rich==14.0.0
+    # via typer
+rpds-py==0.24.0
     # via
     #   jsonschema
     #   referencing
@@ -205,38 +204,39 @@ simpleeval==1.0.3
     # via frictionless
 six==1.17.0
     # via python-dateutil
-stringcase==1.2.0
-    # via frictionless
-structlog==25.2.0
+structlog==25.3.0
     # via libhxl
 tableschema-to-template==0.0.13
     # via hdx-python-utilities
 tabulate==0.9.0
     # via frictionless
-tenacity==9.0.0
+tenacity==9.1.2
     # via hdx-python-country (pyproject.toml)
 text-unidecode==1.3
     # via python-slugify
-typer==0.15.2
+typer==0.15.3
     # via frictionless
-typing-extensions==4.12.2
+typing-extensions==4.13.2
     # via
     #   frictionless
     #   pydantic
     #   pydantic-core
     #   referencing
     #   typer
-unidecode==1.3.8
+    #   typing-inspection
+typing-inspection==0.4.0
+    # via pydantic
+unidecode==1.4.0
     # via
     #   libhxl
     #   pyphonetics
-urllib3==2.3.0
+urllib3==2.4.0
     # via
     #   libhxl
     #   requests
-validators==0.34.0
+validators==0.35.0
     # via frictionless
-virtualenv==20.29.3
+virtualenv==20.31.1
     # via pre-commit
 watchdog==6.0.0
     # via mkdocs
@@ -248,7 +248,7 @@ xlrd3==1.1.0
     # via libhxl
 xlsx2csv==0.8.4
     # via hdx-python-utilities
-xlsxwriter==3.2.2
+xlsxwriter==3.2.3
     # via tableschema-to-template
 xlwt==1.3.0
     # via hdx-python-utilities
diff --git a/src/hdx/location/country.py b/src/hdx/location/country.py
index aad980f..864c9ff 100755
--- a/src/hdx/location/country.py
+++ b/src/hdx/location/country.py
@@ -709,7 +709,7 @@ def simplify_countryname(cls, country: str) -> (str, List[str]):
             for simplification2 in simplifications:
                 remove.append(simplification2)
         remove = "|".join(remove)
-        regex = re.compile(r"\b(" + remove + r")\b", flags=re.IGNORECASE)
+        regex = re.compile(r"[^\w\s]|\b(" + remove + r")\b", flags=re.IGNORECASE)
         countryupper = regex.sub("", countryupper)
         countryupper = countryupper.strip()
         countryupper_words = get_words_in_sentence(countryupper)


Test patch:
diff --git a/tests/hdx/location/test_country.py b/tests/hdx/location/test_country.py
index a835930..e3c1b94 100755
--- a/tests/hdx/location/test_country.py
+++ b/tests/hdx/location/test_country.py
@@ -528,6 +528,14 @@ def test_simplify_countryname(self):
         assert Country.simplify_countryname(
             "Democratic People's Republic of Korea"
         ) == ("KOREA", ["DEMOCRATIC", "PEOPLE'S", "REPUBLIC", "OF"])
+        assert Country.simplify_countryname("Korea (the Republic of))") == (
+            "KOREA",
+            ["THE", "REPUBLIC", "OF"],
+        )
+        assert Country.simplify_countryname("Korea (the Republic of") == (
+            "KOREA",
+            ["THE", "REPUBLIC", "OF"],
+        )
         assert Country.simplify_countryname(
             "The former Yugoslav Republic of Macedonia"
         ) == ("MACEDONIA", ["THE", "FORMER", "YUGOSLAV", "REPUBLIC", "OF"])