Package: lingua
What is lingua?Lingua is a package with tools to extract translatable texts from
your code, and to check existing translations. It replaces the use
of thexgettextcommand from gettext, orpybabelfrom Babel.Message extractionThe simplest way to extract all translatable messages is to point thepot-createtool at the root of your source tree.$ pot-create srcThis will create amessages.potfile containing all found messages.Specifying input filesThere are three ways to tell lingua which files you want it to scan:Specify filenames directly on the command line. For example:$ pot-create main.py utils.pySpecify a directory on the command line. Lingua will recursively scan that
directory for all files it knows how to handle.$ pot-create srcUse the--files-fromparameter to point to a file with a list of
files to scan. Lines starting with#and empty lines will be ignored.$ pot-create --files-from=POTFILES.inYou can also use the--directory=PATHparameter to add the given path to the
list of directories to check for files. This may sound confusing, but can be
useful. For example this command will look formain.pyandutils.pyin
the current directory, and if they are not found there in the../srcdirectory:$ pot-create --directory=../src main.py utils.pyConfigurationIn its default configuration lingua will use its python extractor for.pyfiles, its XML extractor for.ptand.zptfiles and its ZCML extractor
for.zcmlfiles. If you use different extensions you setup a configuration
file which tells lingua how to process files. This file uses a simple ini-style
format.There are two types of configuration that can be set in the configuration file:
which extractor to use for a file extension, and the configuration for a single
extractor.File extensions are configured in theextensionssection. Each entry in
this section maps a file extension to an extractor name. For example to
tell lingua to use its XML extractor for files with a.htmlextension
you can use this configuration:[extensions]
.html = xmlTo find out which extractors are available use the-list-extractorsoption.$ bin/pot-create --list-extractors
chameleon         Chameleon templates (defaults to Python expressions)
python            Python sources
xml               Chameleon templates (defaults to Python expressions)
zcml              Zope Configuration Markup Language (ZCML)
zope              Zope templates (defaults to TALES expressions)A section namedextractor:<name>can be used to configure a specific
extractor. For example to tell the XML extractor that the default language
used for expressions is TALES instead of Python:[extractor:xml]
default-engine = talesEither place a global configuration file named.config/linguato your
home folder or use the--configoption to point lingua to your
configuration file.$ pot-create -c lingua.cfg srcDomain filteringWhen working with large systems you may use multiple translation domains
in a single source tree. Lingua can support that by filtering messages by
domain when scanning sources. To enable domain filtering use the-doption:$ pot-create -d mydomain srcLingua will always include messages for which it can not determine the domain.
For example, take this Python code:print(gettext(u'Hello, World'))
print(dgettext('mydomain', u'Bye bye'))The first hello-message does not specify its domain and will always be
included. The second line usesdgettextto explicitly
specify the domain. Lingua will use this information when filtering domains.Including commentsYou can add comments to messages to help translators, for example to explain
how a text is used, or provide hints on how it should be translated. For
chameleon templates this can be done using thei18n:commentattribute:<label i18n:comment="This is a form label" i18n:translate="">Password</label>Comments are inherited, so you can put them on a parent element as well.<form i18n:comment="This is used in the password reset form">
  <label i18n:translate="">Password</label>
  <button i18n:translate="">Change</button>
</form>For Python code you can tell lingua to include comments by using the--add-commentsoption. This will make Linua include all comments on the
line(s)immediately preceeding(there may be no empty line in between) a
translation call.# This text should address the user directly.
return _('Thank you for using our service.')Alternatively you can also put a comment at the end of the line starting your
translation function call.return _('Thank you for using our service.')  # Address the user directlyIf you do not want all comments to be included but only specific ones you can
add a keyword to the--add-commentsoption, for example--add-comments=I18N.# I18N This text should address the user directly, and use formal addressing.
return _('Thank you for using our service')Setting message flags in commentsMessages can haveflags. These are to indicate what format a message has, and
are typically used by validation tools to check if a translation does not break
variable references or template syntax. Lingua does a reasonable job to detect
strings using C and Python formatting, but sometimes you may need to set flags
yourself. This can be done with a[flag, flag]marker in a comment.# I18N [markdown,c-format]
header =  _(u'# Hello *%s*')Specifying keywordsWhen looking for messages a lingua parser uses a default list of keywords
to identify translation calls. You can add extra keywords via the--keywordoption. If you have your ownmygettextfunction which takes a string
to translate as its first parameter you can use this:$ pot-create --keyword=mygettextIf your function takes more parameters you will need to tell lingua about them.
This can be done in several ways:If the translatable text is not the first parameter you can specify the
parameter number with<keyword>:<parameternumber>. For example if
you usei18n_log(level, msg)the keyword specifier would bei18n_log:2If you support plurals you can specify the parameter used for the plural message
by specifying the parameter number for both the singular and plural text. For
example if your function signature isshow_result(single, plural)the
keyword specifier isshow_result:1,2If you use message contexts you can specify the parameter used for the context
by adding acto the parameter number. For example the keyword specifier forpgettextispgettext:1c,2.If your function takes the domain as a parameter you can specify which parameter
is used for the domain by adding adto the parameter number. For example
the keyword specifier fordgettextisdgettext:1d,2. This is a
lingua-specified extension.You can specify the exact number of parameters a function call must have
using thetpostfix. For example if a functionmusthave four parameters
to be a valid call, the specifier could bemyfunc:1,4t.ExtractorsLingua includes a number of extractors:python: handles Python source code.chameleon: handlesChameleonfiles,
using theZope i18n syntaxzcml: handles Zope Configuration Markup Language (ZCML) files.zope: a variant of the chameleon extractor, which assumes the defaultexpression language isTALESinstead of Python.xml: old name for thechameleonextractor. This name should not be used
anymore and is only supported for backwards compatibility.Babel extractorsThere are several packages with plugins forBabel’s message extraction tool. Lingua can use those
plugins as well. The plugin names will be prefixed withbabel-to
distinguish them from lingua extractors.For example, if you have thePyBabel-jsonpackage installed you can
instruct lingua to use it for .json files by adding this to your configuration
file:[extensions]
.json = babel-jsonSome Babel plugins require you to specify comment tags. This can be set with
thecomment-tagsoption.[extractor:babel-mako]
comment-tags = TRANSLATOR:Comparison to other toolsDifferences compared toGNU gettext:Support for file formats such as Zope Page Templates (popular inPyramid,Chameleon,PloneandZope).Better support for detecting format strings used in Python.No direct support for C, C++, Perl, and many other languages. Lingua focuses
on languages commonly used in Python projects, although support for other
languages can be added via plugins.Differences compared toBabel:More reliable detection of Python format strings.Lingua includes plural support.Support for only extracting texts for a given translation domain. This is
often useful for extensible software where you use multiple translation
domains in a single application.Validating translationsLingua includes a simplepolinttool which performs a few basic checks on
PO files. Currently implemented tests are:duplicated message ids (can also be checked with GNU gettext’smsgfmt).
These should never happen and are usually a result of a bug in the message
extraction logic.identical translations used for multiple canonical texts. This can happen
for valid reasons, for example when the original text is not spelled
consistently.To check a po file simply runpolintwith the po file as argument:$ polint nl.po

Translation:
    ${val} ist keine Zeichenkette
Used for 2 canonical texts:
1       ${val} is not a string
2       "${val}" is not a stringWriting custom extractorsFirst we need to create the custom extractor:from lingua.extractors import Extractor
from lingua.extractors import Message

class MyExtractor(Extractor):
    '''One-line description for --list-extractors'''
    extensions = ['.txt']

    def __call__(self, filename, options):
        return [Message(None, 'msgid', None, [], u'', u'', (filename, 1))]Hooking up extractors to lingua is done bylingua.extractorsentry points
insetup.py:setup(name='mypackage',
      ...
      install_requires=[
          'lingua',
      ],
      ...
      entry_points='''
      [lingua.extractors]
      my_extractor = mypackage.extractor:MyExtractor
      '''
      ...)Note - the registered extractor must be a class derived from theExtractorbase class.After installingmypackagelingua will automatically detect the new custom
extractor.Helper ScriptThere exists a helper shell script for managing translations of packages indocs/examplesnamedi18n.sh. Copy it to package root where you want to
work on translations, edit the configuration params inside the script and use:./i18n.sh langfor initial catalog creation and:./i18n.shfor updating translation and compiling the catalog.