Repository: ckan/ckan

Issue:
Chinese (zh_CN and zh_TW) can not be used on a source install
CKAN Version if known: master

Setting up `ckan.locale_default = zh_CN` fails with `AssertionError: default language "zh_CN" not available`.

`zh_CN` and `zh_TW` are the locale codes used on Transifex.

[This check](https://github.com/ckan/ckan/blob/master/ckan/lib/i18n.py#L51:L53) fails. The folder where babel keeps the locale data files (`<virtualenv>/lib/python2.7/site-packages/babel/locale-data`) has different codes for the Chinese locales. Bizarrely this works on the package install (2.5.x), ie the `zh_CN.dat` files are there.

Not sure if it's related to the recent upgrade to Babel (#3004) or something specific with the package install (seems unlikely)

Probably related to https://github.com/python-babel/babel/pull/25



Discussion:

Resolving patch:
diff --git a/ckan/lib/i18n.py b/ckan/lib/i18n.py
--- a/ckan/lib/i18n.py
+++ b/ckan/lib/i18n.py
@@ -1,7 +1,9 @@
 import os
 
-from babel import Locale, localedata
-from babel.core import LOCALE_ALIASES, get_locale_identifier
+from babel import Locale
+from babel.core import (LOCALE_ALIASES,
+                        get_locale_identifier,
+                        UnknownLocaleError)
 from babel.support import Translations
 from paste.deploy.converters import aslist
 from pylons import config
@@ -48,7 +50,18 @@ def _get_locales():
         i18n_path = os.path.join(config.get('ckan.i18n_directory'), 'i18n')
     else:
         i18n_path = os.path.dirname(ckan.i18n.__file__)
-    locales += [l for l in os.listdir(i18n_path) if localedata.exists(l)]
+
+    # For every file in the ckan i18n directory see if babel can understand
+    # the locale. If yes, add it to the available locales
+    for locale in os.listdir(i18n_path):
+        try:
+            Locale.parse(locale)
+            locales.append(locale)
+        except (ValueError, UnknownLocaleError):
+            # Babel does not know how to make a locale out of this.
+            # This is fine since we are passing all files in the
+            # ckan.i18n_directory here which e.g. includes the __init__.py
+            pass
 
     assert locale_default in locales, \
         'default language "%s" not available' % locale_default
@@ -123,11 +136,21 @@ def get_available_locales():
     e.g.  [ Locale('en'), Locale('de'), ... ] '''
     global available_locales
     if not available_locales:
-        available_locales = map(Locale.parse, get_locales())
-    # Add the full identifier (eg `pt_BR`) to the locale classes, as it does
-    # not offer a way of accessing it directly
-    for locale in available_locales:
-        setattr(locale, 'identifier', get_identifier_from_locale_class(locale))
+        available_locales = []
+        for locale in get_locales():
+            # Add the short names for the locales. This equals the filename
+            # of the ckan translation files as opposed to the long name
+            # that includes the script which is generated by babel
+            # so e.g. `zn_CH` instead of `zn_Hans_CH` this is needed
+            # to properly construct urls with url_for
+            parsed_locale = Locale.parse(locale)
+            parsed_locale.short_name = locale
+
+            # Add the full identifier (eg `pt_BR`) to the locale classes,
+            # as it does not offer a way of accessing it directly
+            parsed_locale.identifier = \
+                get_identifier_from_locale_class(parsed_locale)
+            available_locales.append(parsed_locale)
     return available_locales
 
 


Test patch: