Package: precis-i18n
PRECIS-i18n: Internationalized Usernames and PasswordsIf you want your application to accept Unicode user names and passwords,
you must be careful in how you validate and compare them. The PRECIS
framework makes internationalized user names and passwords safer for use
by applications. PRECIS profiles transform Unicode strings into a
canonical form, suitable for comparison.This module implements the PRECIS Framework as described in:PRECIS Framework: Preparation, Enforcement, and Comparison of
Internationalized Strings in Application Protocols (RFC
8264)Preparation, Enforcement, and Comparison of Internationalized
Strings Representing Usernames and Passwords (RFC
8265)Preparation, Enforcement, and Comparison of Internationalized
Strings Representing Nicknames (RFC
8266)Requires Python 3.5 or later.UsageUse theget_profilefunction to obtain a profile object, then use itsenforcemethod. Theenforcemethod returns a Unicode string.>>>fromprecis_i18nimportget_profile>>>username=get_profile('UsernameCaseMapped')>>>username.enforce('Kevin')'kevin'>>>username.enforce('\u212Aevin')'kevin'>>>username.enforce('\uFF2Bevin')'kevin'>>>username.enforce('\U0001F17Aevin')Traceback (most recent call last):...UnicodeEncodeError:'UsernameCaseMapped' codec can't encode character '\U0001f17a' in position 0: DISALLOWED/symbolsAlternatively, you can use the Pythonstr.encodeAPI. Import theprecis_i18n.codecmodule to register the PRECIS codec names. Now you
can use thestr.encodemethod with any Unicode string. The result will
be a UTF-8 encoded byte string or aUnicodeEncodeErrorif the string
is disallowed.>>>importprecis_i18n.codec>>>'Kevin'.encode('UsernameCasePreserved')b'Kevin'>>>'\u212Aevin'.encode('UsernameCasePreserved')b'Kevin'>>>'\uFF2Bevin'.encode('UsernameCasePreserved')b'Kevin'>>>'\u212Aevin'.encode('UsernameCaseMapped')b'kevin'>>>'\uFF2Bevin'.encode('OpaqueString')b'\xef\xbc\xabevin'>>>'\U0001F17Aevin'.encode('UsernameCasePreserved')Traceback (most recent call last):...UnicodeEncodeError:'UsernameCasePreserved' codec can't encode character '\U0001f17a' in position 0: DISALLOWED/symbolsAlternative Unicode VersionsTheget_profilefunction uses whatever version ofunicodedatais
provided by the Python runtime. The Unicode version is usually tied to
the major version of the Python runtime. Python 3.7.x uses Unicode 11.0.
Python 3.6.x uses Unicode 10.0.To use an alternativeunicodedataimplementation, pass theunicodedatakeyword argument toget_profile.For example, you could separately install version 12.0 of theunicodedata2module from PyPI. Then, pass it to get_profile to
retrieve a profile that uses Unicode 12.0.>>>importunicodedata2>>>fromprecis_i18nimportget_profile>>>username=get_profile('UsernameCaseMapped',unicodedata=unicodedata2)>>>username.enforce('Kevin')'kevin'Supported Profiles and CodecsEach PRECIS profile has a corresponding codec name. TheCaseMappedvariant converts the string to lower case for implementing
case-insensitive comparison.UsernameCasePreservedUsernameCaseMappedOpaqueStringNicknameCasePreservedNicknameCaseMappedTheCaseMappedprofiles use UnicodeToLowerper the latest RFC.
Previous versions of this package used Unicode Default Case Folding.
There are CaseMapped variants for different case transformations. These
profile names are deprecated:UsernameCaseMapped:ToLowerUsernameCaseMapped:CaseFoldNicknameCaseMapped:ToLowerNicknameCaseMapped:CaseFoldThe PRECIS base string classes are also available as codecs:IdentifierClassFreeFormClassUserparts and Space Delimited UsernamesThe Username profiles in this implementation do not allow spaces. The
Username profiles correspond to the definition of "userparts" in RFC
8265. If you want to allow spaces in your application's user names, you
must split the string first.defenforce_app_username(name):profile=precis_i18n.get_profile('UsernameCasePreserved')userparts=[profile.enforce(userpart)foruserpartinname.split(' ')]return' '.join(userparts)Be aware that a username constructed this way can contain bidirectional
text in the separate userparts.Error MessagesA PRECIS profile raises aUnicodeEncodeErrorexception if a string is
disallowed. Thereasonfield specifies the kind of error.ReasonExplanationDISALLOWED/arabic_indicArabic-Indic digits cannot be mixed with Extended Arabic-Indic Digits. (Context)DISALLOWED/bidi_ruleRight-to-left string cannot contain left-to-right characters due to the "Bidi" rule. (Context)DISALLOWED/controlsControl character is not allowed.DISALLOWED/emptyAfter applying the profile, the result cannot be empty.DISALLOWED/exceptionsException character is not allowed.DISALLOWED/extended_arabic_indicExtended Arabic-Indic digits cannot be mixed with Arabic-Indic Digits. (Context)DISALLOWED/greek_keraiaGreek keraia must be followed by a Greek character. (Context)DISALLOWED/has_compatCompatibility characters are not allowed.DISALLOWED/hebrew_punctuationHebrew punctuation geresh or gershayim must be preceded by Hebrew character. (Context)DISALLOWED/katakana_middle_dotKatakana middle dot must be accompanied by a Hiragana, Katakana, or Han character. (Context)DISALLOWED/middle_dotMiddle dot must be surrounded by the letter 'l'. (Context)DISALLOWED/not_idempotentAfter reapplying the profile, the result is not stable.DISALLOWED/old_hangul_jamoConjoining Hangul Jamo is not allowed.DISALLOWED/otherOther character is not allowed.DISALLOWED/other_letter_digitsNon-traditional letter or digit is not allowed.DISALLOWED/precis_ignorable_propertiesDefault ignorable or non-character is not allowed.DISALLOWED/punctuationNon-ASCII punctuation character is not allowed.DISALLOWED/spacesSpace character is not allowed.DISALLOWED/symbolsNon-ASCII symbol character is not allowed.DISALLOWED/unassignedUnassigned Unicode character is not allowed.DISALLOWED/zero_width_joinerZero width joiner must immediately follow a combining virama. (Context)DISALLOWED/zero_width_nonjoinerZero width non-joiner must immediately follow a combining virama, or appear where it breaks a cursive connection in a formally cursive script. (Context)The Nickname Profile and White SpaceWhen PRECIS processes a string using theNicknameprofile, one of the
enforcement steps silently removes leading and trailing white space.
Starting with version 1.1, this library uses a morerestrictivedefinition ofwhite spacein theNicknameprofile.1.1 and lateronlyinclude Unicode categoryZs. If you try to
enforce a Nickname that contains white space characters like'\n',
you will get a UnicodeEncodeErrorDISALLOWED/controls.1.0.5 and earlier included control characters such as'\n','\t', and'\r'when removing leading/trailing white space from
Nicknames. The result treated these legacy white space characters
the same asZsand stripped them.In all versions,internalwhite space (not leading or trailing)
matches Unicode categoryZsonly.The trimming of white space is specific to the Nickname profile only.
Here is an example of the current behavior:>>>fromprecis_i18nimportget_profile>>>nickname=get_profile('NicknameCaseMapped')>>>nickname.enforce('Kevin\n')Traceback (most recent call last):...UnicodeEncodeError:'NicknameCaseMapped' codec can't encode character '\x0a' in position 5: DISALLOWED/controlsIn version 1.0.5 and earlier, theNicknameCaseMappedprofile enforced"Kevin\n"as"kevin".Unicode Version Update ProcedureWhen Unicode releases a new version, take the following steps to update
internal tables and pass unit tests:Under a version of Python that supports the new Unicode version, run
the tests usingpython -m unittest discoverand check that thetest_derived_propstest FAILS due to a missing file.Generate a newderived-propsfile by runningPYTHONPATH=. python test/test_derived_props.py > derived-props-VERSION.txt.
Rename the file using the Unicode version, and re-run the tests. The
unit tests will further check that no derived properties in the new
file contradict the previous values.Check for changes to internal tables used for context rules by
runningPYTHONPATH=. python tools/check_codepoints.py. Update the
corresponding tables in precis_i18n/unicode.py if necessary.