LATEX News
Issue 24, February 2016 (LATEX release 2016-02-01)

Contents
LuaTEX support

1Unicode data

1

More support for east European accents

2

Changes in Graphics

2

Changes in Tools

2

Improving support for Unicode engines

2

LuaTEX support
This release refines the LuaTEX support introduced inthe 2015/10/01 release. A number of patches have been
added to improve the behavior of ltluatex (thankslargely to code review by Philipp Gesang). The kernel
code has been adjusted to allow for changes in LuaTEX
v0.85–v0.88. Most notably, newer LuaTEX releases allow
more than 16 write streams and these are now enabled
for use by \newwrite, but also the experimental
newtoken Lua library has been renamed back to token
which required small adjustments in the LuaTEX setup.
The biggest change in LuaTEX v0.85–v0.87 compared
to previous versions is that all the primitives (originallydefined in pdfTEX) dealing with the PDF “back end”
are no longer defined, being replaced by a much smaller
set of new primitives. This does not directly affect thecore LATEX files in this release but has required majorchanges to the .ini files used by TEX Live and similar
distributions to set up the format files. These changes
in the LuaTEX engine will affect any packages using
these back end commands (packages such as graphics,color, hyperref, etc.). Until all contributed packages are
updated to the new syntax users may need to add
aliases for the old pdfTEX commands. A new
luapdftexalias package has been contributed to CTAN
(not part of the core LATEX release) that may be usedfor this purpose.
See also the sections below for related changes in thetools and graphics bundles.

Unicode data
As noted in LATEX News 22, the 2015/01/01 release of
LATEX introduced built-in support for extended TEX
systems. In particular, the kernel now loads appropriate

data from the Unicode Consortium to set \lccode,\uccode, \catcode and \sfcode values in anautomated fashion for the entire Unicode range.
The initial approach taken by the team was toincorporate the existing model used by (plain) XETEX
and to pre-process the “raw” Unicode data into a
ready-to-use form as unicode-letters.def. However,
the relationship between the Unicode Consortium files
and TEX data structures is non-trivial and still being
explored. As such, it is preferable to directly parse theoriginal (.txt) files at point of use. The team has
therefore “spun-out” both the data and the loading to a
new generic package, unicode-data. This package makes
the original Unicode Consortium data files available in
the texmf tree (in tex/generic/unicode-data) andprovides generic loaders suitable for reading this data
into the plain, LATEX 2ε , and other, formats.
At present, the following data files are included inthis new package:• CaseFolding.txt• EastAsianWidth.txt• LineBreak.txt• MathClass.txt• SpecialCasing.txt• UnicodeData.txt
These files are used either by LATEX 2ε or by expl3(i.e. they represent the set currently required by theteam). The Unicode Consortium provides various otherdata files and we would be happy to add these to thegeneric package, as it is intended to provide a single
place to collect this material in the texmf tree.
Such requests can be mailed to the teamas usual or logged at the package home page:https://github.com/latex3/unicode-data.
The new approach extends use of Unicode data insetting TEX information in two ways. First, the\sfcode of all end-of-quotation/closing punctuation is
now set to 0 (transparent to TEX). Second, \Umathcode
values are now set using MathClass.txt rather thansetting up only letters (which was done using anarbitrary plane 0/plane 1 separation). There are also
minor refinements to the existing code setting,particularly splitting the concepts of case andletter/non-letter category codes.
For XETEX, users should note that \xtxHanGlue and\xtxHanSpace are no longer defined, that noLATEX News, and the LATEX software, are brought to you by the LATEX Project Team; Copyright 2016, all rights reserved.

assignments are made to \XeTeXinterchartoks andthat no \XeTeXintercharclass data is loaded into theformat. The values which were previously inheritedfrom the plain XETEX setup files are not suitable forproperly typesetting East Asian text. There arethird-party packages addressing this area well, notably
those in the CTeX bundle. Third-party packages mayneed adjustment to load the data themselves; see theunicode-data package for one possible loader.

More support for east European accents
As noted in LATEX News 23, comma accent support wasadded for s and t in the 2015/10/01 release. In thisrelease a matching \textcommaabove accent has been‘
added for U+0123 (\c{g}, g)which is the lower case ofU+0122 (\c{G}, G, ). In the OT1 and T1 encodings thecombinations are declared as composites with the \ccommand, which matches the Unicode names “latin(capital|small) letter g with cedilla” and also allows\MakeUppercase{\c{g}} to produce \c{G}, asrequired. In T1 encoding, the composite of \c with k, l,n and r are also declared to use the comma below
accent rather than cedilla to match the conventional useof these letters.
The UTF-8 inputenc option utf8 has been extended
to support all latin combinations that can be reasonably
constructed with a (single) accent command an a base‘character for the T1 encoding so g,u˛ and similar
characters may be directly input using UTF-8 encoding.

Changes in Graphics
The changes in LuaTEX v0.87 mean that the color andgraphics packages no longer share the pdftex.def file
between LuaTEX and pdfTEX. A separate fileluatex.def (distributed separately) has been produced,and distributions are encouraged to modify
graphics.cfg and color.cfg configuration files todefault to the luatex option if LuaTEX v0.87 or later isbeing used. The team has contributed suitable .cfg
files to CTAN to be used as models.
Normally it is best to let the local graphics.cfg
automatically supply the right option depending on theTEX engine being used; however the color and graphics(and so graphicx) packages have been extended to havean explicit luatex option comparable to the existing
pdftex and xetex options.
The trig package has been updated so thatpre-computed values such as sin(90) now expand todigits (1 rather than the internal token \@one in thiscase). This allows them to be used directly in PDF
literal strings.

Changes in Tools
LuaTEX from version v0.87 no longer supports the\write18 syntax to access system commands. A newpackage shellesc has been added to tools that defines anew command \ShellEscape that may be used in all
TEX variants to provide a consistent access to systemcommands. The package also defines \write18 in
LuaTEX so that it continues to access system commands
as before; see the package documentation for details.

Improving support for Unicode engines
Stability concerns are always paramountwhen considering any change to the LATEX 2ε kernel.
At the same time, it is important that the format
remains usable and gives reliable results for users. For
the Unicode TEX engines XETEX and LuaTEX there areimportant differences in behavior from classical (8-bit)
TEX engines which mean that identical default behaviors
are not appropriate. Over the past 18 months the teamhas addressed the most pressing of these considerations(as detailed above and in LATEX News 22 and23), primarily by integrating existing patches into thekernel. There are, though, important areas which stillneed consideration, and which may result in refinements
to kernel support in this area in future releases.
The default font setup in LATEX 2ε at present is to use
the OT1 encoding. This assumes that hyphenation
patterns have been read using appropriate codes: the T1
encoding is assumed. The commonly-used hyphenation
patterns today, hyph-utf8, are set up in this way for
8-bit engines (pdfTEX) but for Unicode engines use
Unicode code points. This means that hyphenation will
be incorrect with Unicode engines unless a Unicode fontis loaded. This requires a concept of a Unicode fontencoding, which is currently provided by the fontspec
package in two versions, EU1 and EU2. The team isworking to fully understand what is meant by a“Unicode font encoding”, as unlike a classical TEX
encoding it is essentially impossible to know whatglyphs will be provided (though each slot is always
defined with the same meaning). There is also anoverlap between this area and ideas of language andwriting system, most obviously in documents featuring
mixed scripts (for example Latin and Cyrillic).
As well as these font considerations, the team is also
exploring to what extent it is possible to allow existing(8-bit) documents to compile directly with Unicode
engines without requiring changes in the sources.
Whether this is truly possible remains an open question.
It is important to stress that changes will only bemade in this area where they do not affect documents
processed with ε-TEX/pdfTEX (i.e. documents which arewritten for “classical” 8-bit TEX engines). Changes will
also be made only where they clearly address
deficiencies in the current setup for Unicode engines(i.e. where current behaviors are wrong).

–2