LATEX3 News
Issue 12, January 2020

ContentsIntroduction

1

New features in expl3
A new argument specifier: e-type . . . . . . . .
New functions . . . . . . . . . . . . . . . . . . .
String conversion moves to expl3 . . . . . . . .
Case changing of text . . . . . . . . . . . . . .

1
1
1
1
2

Notable fixes and changes
File name parsing . . . . . . . . . . . . . . . . .
Message formatting . . . . . . . . . . . . . . . .
Key inheritance . . . . . . . . . . . . . . . . . .
Floating point juxtaposition . . . . . . . . . . .
Changing box dimensions . . . . . . . . . . . .
More functions moved to stable . . . . . . . . .
Deprecations . . . . . . . . . . . . . . . . . . .

222
2
2
2
22

Internal improvements
Cross-module functions . . . . . . . . . . . . .
The backend . . . . . . . . . . . . . . . . . . .

22
2

Better support for (u)pTEX

2Options

3Engine requirements

3

Documentation
News . . . . . . . . . . . . . . . . . . . . . . . .
ChangeLog . . . . . . . . . . . . . . . . . . . .

3
3
3

Changes in xparse

3New experimental modules3l3build changes

3Introduction
There has been quite a gap since the last LATEX3 News(Issue 11, February 2018), and so there is quite a bit tocover here. Luckily, one of the things there is to coveris that we are using a more formalised approach forlogging changes, so writing up what has happened isa bit easier. (By mistake LATEX3 News 11 itself did not
get published when written, but is now available: wehave kept the information it contains separate as it is agood summary of the work that had happened in 2017.)
Work has continued apace across the LATEX3
codebase in the last (nearly) two years. A lot of thisis ultimately focussed on making the core of expl3 evenmore stable: squeezing out more experimental ideas,
refining ones we have and making it a serious option forcore LATEX programming.
As a result of these activities, the LATEX3
programming layer will be available as part of thekernel of LATEX 2ε from 2020-02-02 onwards, i.e., canbe used without explicitly loading expl3. See LATEXNews 31 [2] for more details on this.

New features in expl3
A new argument specifier: e-type
During 2018, the team worked with the TEX Live,XETEX and (u)pTEX developers to add the \expandedprimitive to pdfTEX, XETEX and (u)pTEX. Thisprimitive was originally suggested for pdfTEX v1.50(never released), and was present in LuaTEX from thestart of that project.
Adding \expanded lets us create a new argument
specifier: e-type expansion. This is almost the same as
x-type, but is itself expandable. (It also doesn’t needdoubled # tokens.) That’s incredibly useful for creating
function-like macros: you can ensure that everything isexpanded in an argument before you go near it, withnot an \expandafter in sight.
New functions
New programming tools have appeared in various
places across expl3. The highlights are• Shuffling of sequences to allow randomization• Arrays of integers and floating point values; thesehave constant-time access• Functions to return values after system shell usage• Expandable access to file information, including filesize, MD5 hash and modification date
For the latter, we have revised handling of file names
considerably. There is now support for finding files
in expansion contexts (by using the \(pdf)filesize
primitive). Spaces and quotes in file names are now
fully normalised, in a similar manner to the approach
used by the latest LATEX 2ε kernel.
String conversion moves to expl3
In addition to entirely new functions, the team have
moved the l3str-convert module from the l3experimental
bundle into the expl3 core. This module is essential for
dealing with the need to produce UTF-16 and UTF-32
strings in some contexts, and also offers built-in escapefor url and PDF strings.

LATEX3 News, and the LATEX software, are brought to you by the LATEX3 Project Team; Copyright 2020, all rights reserved.

Case changing of text
Within expl3, the team have renamed and reworked theideas from \tl_upper_case:n and so on, creating a
new module l3text. This is a “final” home for functions
to manipulate text; token lists that can reasonably beexpected to expand to plain text plus limited markup,
for example emphasis and labels/references. Moving
these functions, we have also made a small number ofchanges in other modules to give consistent names tofunctions: see the change log for full details.
Over time, we anticipate that functions for othertextual manipulation will be added to this module.

Notable fixes and changes
File name parsing
The functions for parsing file names have been entirely
rewritten, partly as this is required for the expandable
access to file information mentioned above. The newcode correctly deals with spaces and quote marks in filenames and splits the path/name/extension.
Message formatting
The format of messages in expl3 was originally quite
text-heavy, the idea being that they would stand out
in the .log file. However, this made them hard to findby a regular expression search, and was very different
from the LATEX 2ε message approach. The formattingof expl3 messages has been aligned with that from theLATEX 2ε kernel, such that IDE scripts and similar will
be able to find and extract them directly.
Key inheritance
A number of changes have been made to theinheritance code for keys, to allow inheritance to work
“as expected” in (almost) all cases.
Floating point juxtaposition
Implicit multiplication by juxtaposition, such as 2pi,
is now handled separately from parenthetic values.
Thus for example 1in/1cm is treated as equal to(1in)/(1cm) and thus yields 2.54, and 1/2(pi+pi)
is equal to pi.
Changing box dimensions
TEX’s handling of boxes is subtly different from otherregisters, and this shows up in particular when youwant to resize a box. To bring treatment of boxes, orrather the grouping behavior of boxes, into line withother registers, we have made some internal changes tohow functions such as \box_set_wd:N are implemented.
This will be transparent for “well-behaved” use cases ofthese functions.
More functions moved to stable
A large number of functions which were introduced as
candidates have been evaluated and moved to stablestatus. The team hopes to move all functions in expl3

to stable status, or move them out of the core, over thecoming months.
Deprecations
There have been two notable sets of deprecations
over the past 18 months. First, we have rationalised
all of the “raw” primitive names to the form \tex_<name>:D. This means that the older names, starting\pdftex_..., \xetex_..., etc., have been removed.
Secondly, the use of integer constants, which dates
back to the earliest days of expl3, is today more likely
to make the code harder to read than anything else.
Speed improvements in engines mean that the tinyenhancements in reading such constants are no longer
required. Thus for example \c_two is deprecated infavour of simply using 2.
In parallel with this, a number of older .sty files
have been removed. These older files provided legacy
stubs for files which have now been integrated in theexpl3 core. They have now had sufficient time to allow
users to update their code.

Internal improvements
Cross-module functions
The team introduced the idea of internal module
functions some time ago. Within the kernel, there areplaces where functions need to be used in multiple
modules. To make the nature of the kernel interactions
clearer, we have worked on several aspects• Reducing as far as possible cross-module functions• Making more generally-useful functions public, forexample scan marks• Creating an explicit cross-kernel naming
convention for functions which are internal but are
essential to use in multiple kernel modules
The backend
Creating graphics, working with color, setting up
hyperlinks and so on require backend-specific code.
Here, backends are for example dvips, xdvipdfmx andthe direct PDF mode in pdfTEX and LuaTEX. These
functions are needed across the LATEX3 codebase andhave to be updated separately from the expl3 core.
To facilitate that, we have split those sources into aseparate bundle, which can be updated as required.
At the same time, the code these files contain isvery low-level and is best described as internal. We
have re-structured how the entire set of functions arereferred to such that they are now internal for the area
they implement, for example image inclusion, box affine
transformations, etc.

Better support for (u)pTEX
The developers behind (u)pTEX (Japanese TEX) haverecently enhanced their English documentation (seehttps://github.com/texjporg/ptex-manual). Using
this new information, we have been able to makeinternal adjustments to expl3 to better support theseengines.

Options
A new option undo-recent-deprecations is now
available for cases where a document (or package)requires some expl3 functions that have been formally
removed after deprecation. This is to allow temporary
work-arounds for documents to be compiled whilst code
is begin updated.
The “classical” options for selecting backends (dvips,pdftex, etc.) are now recognised in addition to thenative key–value versions. This should make it mucheasier to use the expl3 image and color support as it isbrought up to fully-workable standards.

Engine requirements
The minimum engine versions needed to use expl3 havebeen incremented a little:• pdfTEX v1.40• XETEX v0.99992• LuaTEX v0.95• ε-(u)pTEX mid-2012
The team have also worked with the XETEX
and (u)pTEX developers to standardise the set ofpost-ε-TEX utility primitives that are available: theso-called “pdfTEX utilities”. These are now available inall supported engines, and in time will all be required.
This primarily impacts XETEX, which gained most ofthese primitives in the 2019 TEX Live cycle. (Examples
are the random number primitives and expandable filedata provision.) See LATEX News 31 [2] for more.

Documentation
NewsThe LATEX3 News files were until recently only used tocreate PDF files on the team website [1]. We have now
integrated those into the l3kernel (expl3 core) bundle.
The news files cover all of LATEX3 files, as the core files
are always available.
ChangeLog
Since the start of 2018, the team have commenced acomprehensive change log for each of the bundles whichmake up the LATEX3 code. These are simple Markdown
text files, which means that they can be displayedformatted in web views.

Changes in xparse
A number of new features have been added to xparse.
To allow handling of the fact that skipping spaces
may be required only in some cases when searchingfor optional arguments, a new modifier ! is available
in argument specifiers. This causes xparse to requirethat an optional argument follows immediately with nointervening spaces.
There is a new argument type purely forenvironments: b-type for collecting a \begin...\end
pair, i.e., collecting the body of an environment. Thisis similar in concept to the environ package, but isintegrated directly into xparse.
Finally, it is now possible to refer to one argument as
the default for another optional one, for example\NewDocumentCommand{\caption}{O{#2} m} ...

New experimental modules
A number of new experimental modules have been
added within the l3experimental bundle:
l3benchmark Performance-testing system using thetiming function in modern TEX engines
l3cctab Category code tables for all engines, not justLuaTEX
l3color Color support, similar in interface to xcolor
l3draw Creation of drawings, inspired by pgf, but using
the LATEX3 FPU for calculations
l3pdf Support for PDF features such as compression,
hyperlinks, etc.
l3sys-shell Shell escape functions for file manipulation

l3build changes
The l3build tool for testing and releasing TEX packages
has seen a number of incremental improvements. It
is now available directly as a script in TEX Live andMiKTEX, meaning you can call it simply as
l3build target
Accompanying this, we have added support forinstalling scripts and script man files.
There is a new upload target that can take a zip fileand send it to CTAN: you just have to fill in release
information for this upload at the prompts.
Testing using PDF files rather than logs has beenheavily revised: this is vital for work on PDF tagging.
There is also better support for complex directory
structures, including the ability to manuallyspecify TDS location for all installed files. This isparticularly targeted at packages with both generic andformat-specific files to install.

References[1] LATEX Project Website.
https://latex-project.org/[2] LATEX 2ε release newsletters on the LATEX ProjectWebsite. https://latex-project.org/news/
latex2e-news/