Repository: python-babel/babel
Pull request #1188 (2025-02-04 23:17:31)

Validate all plurals in python format checker
Previously, the format checker would only validate the first 2 plurals. If a locale had more than two plurals, those would not be checked. This PR fixes that.

codecov[bot]: ## [Codecov](https://app.codecov.io/gh/python-babel/babel/pull/1188?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=python-babel) Report
All modified and coverable lines are covered by tests :white_check_mark:
> Project coverage is 90.83%. Comparing base [(`27e7303`)](https://app.codecov.io/gh/python-babel/babel/commit/27e7303549831c84ffda090ec51204b214407e96?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=python-babel) to head [(`48ce7bf`)](https://app.codecov.io/gh/python-babel/babel/commit/48ce7bf1f24754b9b6e0d5c5df205e08efaf9e99?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=python-babel).

<details><summary>Additional details and impacted files</summary>


```diff
@@            Coverage Diff             @@
##           master    #1188      +/-   ##
==========================================
- Coverage   91.68%   90.83%   -0.85%     
==========================================
  Files          27       27              
  Lines        4689     4692       +3     
==========================================
- Hits         4299     4262      -37     
- Misses        390      430      +40     
```

| [Flag](https://app.codecov.io/gh/python-babel/babel/pull/1188/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=python-babel) | Coverage Δ | |
|---|---|---|
| [macos-14-3.10](https://app.codecov.io/gh/python-babel/babel/pull/1188/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=python-babel) | `90.72% <100.00%> (+<0.01%)` | :arrow_up: |
| [macos-14-3.11](https://app.codecov.io/gh/python-babel/babel/pull/1188/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=python-babel) | `?` | |
| [macos-14-3.12](https://app.codecov.io/gh/python-babel/babel/pull/1188/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=python-babel) | `?` | |
| [macos-14-3.13](https://app.codecov.io/gh/python-babel/babel/pull/1188/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=python-babel) | `?` | |
| [macos-14-3.8](https://app.codecov.io/gh/python-babel/babel/pull/1188/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=python-babel) | `?` | |
| [macos-14-3.9](https://app.codecov.io/gh/python-babel/babel/pull/1188/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=python-babel) | `?` | |
| [macos-14-pypy3.10](https://app.codecov.io/gh/python-babel/babel/pull/1188/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=python-babel) | `?` | |
| [ubuntu-24.04-3.10](https://app.codecov.io/gh/python-babel/babel/pull/1188/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=python-babel) | `90.75% <100.00%> (+<0.01%)` | :arrow_up: |
| [ubuntu-24.04-3.11](https://app.codecov.io/gh/python-babel/babel/pull/1188/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=python-babel) | `?` | |
| [ubuntu-24.04-3.12](https://app.codecov.io/gh/python-babel/babel/pull/1188/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=python-babel) | `?` | |
| [ubuntu-24.04-3.13](https://app.codecov.io/gh/python-babel/babel/pull/1188/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=python-babel) | `?` | |
| [ubuntu-24.04-3.8](https://app.codecov.io/gh/python-babel/babel/pull/1188/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=python-babel) | `?` | |
| [ubuntu-24.04-3.9](https://app.codecov.io/gh/python-babel/babel/pull/1188/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=python-babel) | `90.68% <100.00%> (+<0.01%)` | :arrow_up: |
| [ubuntu-24.04-pypy3.10](https://app.codecov.io/gh/python-babel/babel/pull/1188/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=python-babel) | `?` | |
| [windows-2022-3.10](https://app.codecov.io/gh/python-babel/babel/pull/1188/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=python-babel) | `?` | |
| [windows-2022-3.11](https://app.codecov.io/gh/python-babel/babel/pull/1188/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=python-babel) | `?` | |
| [windows-2022-3.12](https://app.codecov.io/gh/python-babel/babel/pull/1188/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=python-babel) | `?` | |
| [windows-2022-3.13](https://app.codecov.io/gh/python-babel/babel/pull/1188/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=python-babel) | `?` | |
| [windows-2022-3.8](https://app.codecov.io/gh/python-babel/babel/pull/1188/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=python-babel) | `?` | |
| [windows-2022-3.9](https://app.codecov.io/gh/python-babel/babel/pull/1188/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=python-babel) | `?` | |
| [windows-2022-pypy3.10](https://app.codecov.io/gh/python-babel/babel/pull/1188/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=python-babel) | `?` | |

Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=python-babel#carryforward-flags-in-the-pull-request-comment) to find out more.

</details>

[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/python-babel/babel/pull/1188?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=python-babel).   
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=python-babel).


Patch:
diff --git a/babel/messages/catalog.py b/babel/messages/catalog.py
index f84a5bd..a35647f 100644
--- a/babel/messages/catalog.py
+++ b/babel/messages/catalog.py
@@ -217,10 +217,17 @@ class Message:
         return self.__dict__ == other.__dict__
 
     def clone(self) -> Message:
-        return Message(*map(copy, (self.id, self.string, self.locations,
-                                   self.flags, self.auto_comments,
-                                   self.user_comments, self.previous_id,
-                                   self.lineno, self.context)))
+        return Message(
+            id=copy(self.id),
+            string=copy(self.string),
+            locations=copy(self.locations),
+            flags=copy(self.flags),
+            auto_comments=copy(self.auto_comments),
+            user_comments=copy(self.user_comments),
+            previous_id=copy(self.previous_id),
+            lineno=self.lineno,  # immutable (str/None)
+            context=self.context,  # immutable (str/None)
+        )
 
     def check(self, catalog: Catalog | None = None) -> list[TranslationError]:
         """Run various validation checks on the message.  Some validations
diff --git a/babel/messages/checkers.py b/babel/messages/checkers.py
index df7c3ca..072a48a 100644
--- a/babel/messages/checkers.py
+++ b/babel/messages/checkers.py
@@ -54,9 +54,12 @@ def python_format(catalog: Catalog | None, message: Message) -> None:
     if not isinstance(msgstrs, (list, tuple)):
         msgstrs = (msgstrs,)
 
-    for msgid, msgstr in zip(msgids, msgstrs):
-        if msgstr:
-            _validate_format(msgid, msgstr)
+    if msgstrs[0]:
+        _validate_format(msgids[0], msgstrs[0])
+    if message.pluralizable:
+        for msgstr in msgstrs[1:]:
+            if msgstr:
+                _validate_format(msgids[1], msgstr)
 
 
 def _validate_format(format: str, alternative: str) -> None:
@@ -116,13 +119,15 @@ def _validate_format(format: str, alternative: str) -> None:
                                            'and named placeholders')
         return bool(positional)
 
-    a, b = map(_parse, (format, alternative))
+    a = _parse(format)
+    b = _parse(alternative)
 
     if not a:
         return
 
     # now check if both strings are positional or named
-    a_positional, b_positional = map(_check_positional, (a, b))
+    a_positional = _check_positional(a)
+    b_positional = _check_positional(b)
     if a_positional and not b_positional and not b:
         raise TranslationError('placeholders are incompatible')
     elif a_positional != b_positional:
diff --git a/babel/messages/extract.py b/babel/messages/extract.py
index 7f4230f..dcf62f2 100644
--- a/babel/messages/extract.py
+++ b/babel/messages/extract.py
@@ -108,7 +108,7 @@ def _strip_comment_tags(comments: MutableSequence[str], tags: Iterable[str]):
             if line.startswith(tag):
                 return line[len(tag):].strip()
         return line
-    comments[:] = map(_strip, comments)
+    comments[:] = [_strip(c) for c in comments]
 
 
 def default_directory_filter(dirpath: str | os.PathLike[str]) -> bool:
diff --git a/babel/messages/jslexer.py b/babel/messages/jslexer.py
index 5fc4956..7a2ba6a 100644
--- a/babel/messages/jslexer.py
+++ b/babel/messages/jslexer.py
@@ -53,7 +53,7 @@ _rules: list[tuple[str | None, re.Pattern[str]]] = [
         (0x[a-fA-F0-9]+)
     )''', re.VERBOSE)),
     ('jsx_tag', re.compile(r'(?:</?[^>\s]+|/>)', re.I)),  # May be mangled in `get_rules`
-    ('operator', re.compile(r'(%s)' % '|'.join(map(re.escape, operators)))),
+    ('operator', re.compile(r'(%s)' % '|'.join(re.escape(op) for op in operators))),
     ('template_string', re.compile(r'''`(?:[^`\\]*(?:\\.[^`\\]*)*)`''', re.UNICODE)),
     ('string', re.compile(r'''(
         '(?:[^'\\]*(?:\\.[^'\\]*)*)'  |
diff --git a/babel/messages/pofile.py b/babel/messages/pofile.py
index 2bb0c77..3afdd60 100644
--- a/babel/messages/pofile.py
+++ b/babel/messages/pofile.py
@@ -73,8 +73,7 @@ def denormalize(string: str) -> str:
         escaped_lines = string.splitlines()
         if string.startswith('""'):
             escaped_lines = escaped_lines[1:]
-        lines = map(unescape, escaped_lines)
-        return ''.join(lines)
+        return ''.join(unescape(line) for line in escaped_lines)
     else:
         return unescape(string)
 
@@ -144,7 +143,7 @@ class _NormalizedString:
         self._strs.append(s.strip())
 
     def denormalize(self) -> str:
-        return ''.join(map(unescape, self._strs))
+        return ''.join(unescape(s) for s in self._strs)
 
     def __bool__(self) -> bool:
         return bool(self._strs)
diff --git a/babel/plural.py b/babel/plural.py
index 085209e..63a081e 100644
--- a/babel/plural.py
+++ b/babel/plural.py
@@ -586,7 +586,8 @@ class _GettextCompiler(_Compiler):
             if item[0] == item[1]:
                 rv.append(f"({expr} == {self.compile(item[0])})")
             else:
-                min, max = map(self.compile, item)
+                min = self.compile(item[0])
+                max = self.compile(item[1])
                 rv.append(f"({expr} >= {min} && {expr} <= {max})")
         return f"({' || '.join(rv)})"
 
diff --git a/docs/dates.rst b/docs/dates.rst
index 0c2c17f..ee23505 100644
--- a/docs/dates.rst
+++ b/docs/dates.rst
@@ -124,7 +124,7 @@ the `Locale Data Markup Language specification`_. The following table is just a
 relatively brief overview.
 
  .. _`Locale Data Markup Language specification`:
-    https://unicode.org/reports/tr35/#Date_Format_Patterns
+    https://unicode.org/reports/tr35/tr35-dates.html#Date_Format_Patterns
 
 Date Fields
 -----------
diff --git a/pyproject.toml b/pyproject.toml
index e68b6d5..98ab326 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -30,3 +30,25 @@ ignore = [
 
 [tool.ruff.lint.per-file-ignores]
 "scripts/import_cldr.py" = ["E402"]
+
+[tool.pytest.ini_options]
+norecursedirs = [
+    "venv*",
+    ".*",
+    "_*",
+    "scripts",
+    "{args}"
+]
+doctest_optionflags = [
+    "ELLIPSIS",
+    "NORMALIZE_WHITESPACE",
+    "ALLOW_UNICODE",
+    "IGNORE_EXCEPTION_DETAIL"
+]
+markers = [
+    "all_locales: parameterize test with all locales"
+]
+filterwarnings = [
+    # The doctest for format_number would raise this, but we don't really want to see it.
+    "ignore:babel.numbers.format_decimal:DeprecationWarning"
+]
diff --git a/setup.cfg b/setup.cfg
index ac49ca5..8183238 100644
--- a/setup.cfg
+++ b/setup.cfg
@@ -1,11 +1,2 @@
-[tool:pytest]
-norecursedirs = venv* .* _* scripts {args}
-doctest_optionflags = ELLIPSIS NORMALIZE_WHITESPACE ALLOW_UNICODE IGNORE_EXCEPTION_DETAIL
-markers =
-    all_locales: parameterize test with all locales
-filterwarnings =
-    # The doctest for format_number would raise this, but we don't really want to see it.
-    ignore:babel.numbers.format_decimal:DeprecationWarning
-
 [metadata]
 license_files = LICENSE
diff --git a/setup.py b/setup.py
index c8cdb9d..93e0bb7 100755
--- a/setup.py
+++ b/setup.py
@@ -52,8 +52,10 @@ setup(
         'Programming Language :: Python :: 3.10',
         'Programming Language :: Python :: 3.11',
         'Programming Language :: Python :: 3.12',
+        'Programming Language :: Python :: 3.13',
         'Programming Language :: Python :: Implementation :: CPython',
         'Programming Language :: Python :: Implementation :: PyPy',
+        'Topic :: Software Development :: Internationalization',
         'Topic :: Software Development :: Libraries :: Python Modules',
     ],
     python_requires='>=3.8',


Test patch:
diff --git a/tests/messages/test_checkers.py b/tests/messages/test_checkers.py
index bba8f14..e4559f7 100644
--- a/tests/messages/test_checkers.py
+++ b/tests/messages/test_checkers.py
@@ -337,6 +337,8 @@ class TestPythonFormat:
         (('foo %s', 'bar'), ('foo', 'bar')),
         (('foo', 'bar %s'), ('foo', 'bar')),
         (('foo %s', 'bar'), ('foo')),
+        (('foo %s', 'bar %d'), ('foo %s', 'bar %d', 'baz')),
+        (('foo %s', 'bar %d'), ('foo %s', 'bar %d', 'baz %d', 'qux')),
     ])
     def test_python_format_invalid(self, msgid, msgstr):
         msg = Message(msgid, msgstr)
@@ -346,9 +348,13 @@ class TestPythonFormat:
     @pytest.mark.parametrize(('msgid', 'msgstr'), [
         ('foo', 'foo'),
         ('foo', 'foo %s'),
+        ('foo %s', ''),
         (('foo %s', 'bar %d'), ('foo %s', 'bar %d')),
-        (('foo %s', 'bar %d'), ('foo %s', 'bar %d', 'baz')),
+        (('foo %s', 'bar %d'), ('foo %s', 'bar %d', 'baz %d')),
         (('foo', 'bar %s'), ('foo')),
+        (('foo', 'bar %s'), ('', '')),
+        (('foo', 'bar %s'), ('foo', '')),
+        (('foo %s', 'bar %d'), ('foo %s', '')),
     ])
     def test_python_format_valid(self, msgid, msgstr):
         msg = Message(msgid, msgstr)
diff --git a/tests/messages/test_setuptools_frontend.py b/tests/messages/test_setuptools_frontend.py
index a623efd..5c3f443 100644
--- a/tests/messages/test_setuptools_frontend.py
+++ b/tests/messages/test_setuptools_frontend.py
@@ -45,6 +45,13 @@ def test_extract_distutils_keyword_arg_388(kwarg, expected):
     assert set(cmdinst.add_comments) == {"Bar", "Foo"}
 
 
+@pytest.mark.xfail(
+    # Python 3.10.16[pypy-7.3.19-final] in GHA fails with "unsupported locale setting"
+    # in the subprocesses this test spawns.  Hard to say why because it doesn't do that
+    # locally.
+    condition=(sys.implementation.name == "pypy" and "BABEL_TOX_INI_DIR" in os.environ),
+    reason="Test will likely fail with 'unsupported locale setting' in subprocesses; see comment",
+)
 def test_setuptools_commands(tmp_path, monkeypatch):
     """
     Smoke-tests all of the setuptools versions of the commands in turn.
diff --git a/tests/test_localedata.py b/tests/test_localedata.py
index 6911cbd..1879934 100644
--- a/tests/test_localedata.py
+++ b/tests/test_localedata.py
@@ -16,7 +16,6 @@ import random
 import sys
 import tempfile
 import unittest
-from operator import methodcaller
 
 import pytest
 
@@ -94,14 +93,13 @@ def test_unique_ids():
     all_ids = localedata.locale_identifiers()
     assert len(all_ids) == len(set(all_ids))
     # Check locale IDs don't collide after lower-case normalization.
-    lower_case_ids = list(map(methodcaller('lower'), all_ids))
+    lower_case_ids = [id.lower() for id in all_ids]
     assert len(lower_case_ids) == len(set(lower_case_ids))
 
 
 def test_mixedcased_locale():
     for locale in localedata.locale_identifiers():
-        locale_id = ''.join([
-            methodcaller(random.choice(['lower', 'upper']))(c) for c in locale])
+        locale_id = ''.join(c.lower() if random.random() < 0.5 else c.upper() for c in locale)
         assert localedata.exists(locale_id)