django/django — pull request #1912 by Bouke (2013-11-12 19:59:38+00:00)
Title: Fixed #21389 -- Accept most valid language codes

By removing the 'supported' keyword from the detection methods and only relying
on a cached settings.LANGUAGES, the speed of said methods has been improved;
around 4x raw performance. This allows us to stop checking Python's incomplete
list of locales, and rely on a less restrictive regular expression for
accepting certain locales.

HTTP `Accept-Language` is defined as being case-insensitive, based on this fact
extra performance improvements have been made; it wouldn't make sense to
check for case differences.

----

There is a backwards incompatibility involved with the current code.

1. Both `get_language_from_path` and `get_supported_language_variant` have a `supported` argument, to override which languages those methods should inspect. However nowhere the list is overridden and the default (`settings.LANGUAGES`) is being used. As `lru_cache` doesn't support caching with dictionary arguments, the two don't work together.
2. `parse_accept_lang_header` now returns lowercase locales, instead of the case provided. This is because the `Accept-Language` header is case insensitive and not checking for case speeds up locale detection.

Note that backwards compatibility can be added if needed, but I'd rather have some input on the proposed changes and on the need of backwards compatibility in this case.

----

Using a new benchmark ([see PR for djangobench](https://github.com/django/djangobench/pull/15)) there is also a nice performance gain with these changes:

$ djangobench locale_from_request --control=upstream/master --experiment=tickets/21389 -t 500
Running benchmarks: locale_from_request
Control: Django 1.7.dev20131112174535 (in git branch upstream/master)
Experiment: Django 1.7.dev20131112193851 (in git branch tickets/21389)

Running 'locale_from_request' benchmark ...
Min: 0.003650 -> 0.000661: 5.5231x faster
Avg: 0.003864 -> 0.000687: 5.6250x faster
Significant (t=155.648026)
Stddev: 0.00045 -> 0.00009: 5.0473x smaller (N = 500)