LATEX3 News
Issue 9, March 2014

Contents
Hiatus?

1

expl3 in the community

1

Logo for the LATEX3 Programming Language

2

Recent activity

2Work in progress
Uppercasing and lowercasing . . . . . . . . . .
Space-skipping in xparse . . . . . . . . . . . . .

22
3. . . and for 2014 onwards

3

What can you do for the LATEX3 project?
Programming Layer . . . . . . . . . . . . . . .
Design Layer . . . . . . . . . . . . . . . . . . .
Document Interface Layer . . . . . . . . . . . .
In Summary . . . . . . . . . . . . . . . . . . . .
And something else . . . . . . . . . . . . . . . .

4
4
4
5
5
5

Hiatus?
Well, it’s been a busy couple of years. Work has slowedon the LATEX3 codebase as all active members of theteam have been — shall we say — busily occupied withmore pressing concerns in their day-to-day activities.
Nonetheless, Joseph and Bruno have continued
to fine-tune the LATEX3 kernel and add-on packages.
Browsing through the commit history shows bug fixes
and improvements to documentation, test files, andinternal code across the entire breadth of the codebase.
Members of the team have presented at two TUG
conferences since the last LATEX3 news. (Has it reallybeen so long?) In July 2012, Frank and Will travelled
to Boston; Frank discussed the challenges faced in thepast and continuing to the present day due to the limitsof the various TEX engines; and, Frank and Will together covered a brief history and recent developmentsof the expl3 code.
In 2013, Joseph and Frank wrote a talk on complex
layouts, and the “layers” ideas discussed in LATEX3;Frank went to Tokyo in October to present the work.
Slides of and recordings from these talks are available
on the LATEX3 website.
These conferences are good opportunities to introduce the expl3 language to a wider group of people;
in many cases, explaining the rationale behind why
expl3 looks a little strange at first helps to convince

the audience that it’s not so weird after all. In our experience, anyone that’s been exposed to some of themore awkward expansion aspects of TEX programming
appreciates how expl3 makes life much easier for us.

expl3 in the community
While things have been slightly quieter for the team,more and more people are adopting expl3 for their own
use. A search on the TEX Stack Exchange website for
either “expl3” or “latex3” at time of writing yieldaround one thousand results each.
In order to help standardise the prefixes used in expl3
modules, we have developed a registration procedure
for package authors (which amounts to little more than
notifying us that their package uses a specific prefix,
which will often be the name of the package itself).
Please contact us via the latex-l mailing list to register your module prefixes and package names; we ask
that you avoid using package names that begin withl3... since expl3 packages use this internally. Someauthors have started using the package prefix lt3...
as a way of indicating their package builds on expl3 insome way but is not maintained by the LATEX3 team.
In the prefix database at present, some thirty package prefixes are registered by fifteen separate individuals (unrelated to the LATEX3 project — the numberof course grows if you include packages by membersof the team). These packages cover a broad range offunctionality:
acro Interface for creating (classes of) acronyms
hobby Hobby’s algorithm in PGF/TiKZ for drawing
optimally smooth curves.
chemmacros Typesetting in the field of chemistry.
classics Traditional-style citations for the classics.
conteq Continued (in)equalities in mathematics.
ctex A collection of macro packages and documentclasses for Chinese typesetting.
endiagram Draw potential energy curve diagrams.
enotez Support for end-notes.
exsheets Question sheets and exams with metadata.
lt3graph A graph data structure.
newlfm The venerable class for memos and letters.
fnpct Interaction between footnotes and punctuation.

LATEX3 News, and the LATEX software, are brought to you by the LATEX3 Project Team; Copyright 2014, all rights reserved.

GS1 Barcodes and so forth.
hobete Beamer theme for the Univ. of Hohenheim.
kantlipsum Generate sentences in Kant’s style.
lualatex-math Extended support for mathematics in
LuaLATEX.
media9 Multimedia inclusion for Adobe Reader.
pkgloader Managing the options and loading order ofother packages.
substances Lists of chemicals, etc., in a document.

2. Joseph’s refinement of the experimental galley code
now allows separation of paragraph shapes frommargins/cutouts. This still needs some testing!
3. For some time now expl3 has provided “native”
drivers although they have not been selected bydefault in most cases. These have been revised to
improve robustness, which makes them probably
ready to enable by default. The improvements
made to the drivers have also fed back to more“general” LATEX code.

withargs Ephemeral macro use.
xecjk Support for CJK documents in XELATEX.

Work in progress

xpatch, regexpatch Patch command definitions.

Some of these packages are marked by their authors as
experimental, but it is clear that these packages havebeen developed to solve specific needs for typesetting
and document production.
The expl3 language has well and truly gained traction
after many years of waiting patiently.

We’re still actively discussing a variety of areas totackle next. We are aware of various “odds and ends”
in expl3 that still need sorting out. In particular, someexperimental functions have been working quite welland it’s time to assess moving them into the “stable”
modules, in particular the l3str module for dealing with
catcode-twelve token lists more commonly known in
expl3 as strings.
Areas of active discussion including issues arounduppercasing and lowercasing (and the esoteric waysthat this can be achieved in TEX) and space skipping(or not) in commands and environments with optional
arguments. These two issues are discussed next.

A logo for the LATEX3 Programming Language

Uppercasing and lowercasing

xpeek Commands that peek ahead in the input stream.
xpinjin Automatically add pinyin to Chinese characterszhnumber Typeset Chinese representations of numbers
zxjatype Standards-conforming typesetting of Japanesefor XELATEX.

To show that expl3 is ready for general use Paulo
Cereda drew up a nice logo for us, showing ahummingbird (agile and fast — but needs huge amountsof energy) picking at “l3”. Big thanks to Paulo!

The commands \tl_to_lowercase:n and\tl_to_uppercase:n have long been overdue for agood hard look. From a traditional TEX viewpoint,these commands are simply the primitive \lowercase
and \uppercase, and in practice it’s well known that
there are various limitations and peculiarities associated with them. We know these commands are good, to
one extent or another, for three things:
1. Uppercasing text for typesetting purposes such as
all-uppercase titles.

Recent activity

2. Lowercasing text for normalisation in sorting andother applications such as filename comparisons.

LATEX3 work has only slowed, not ground to a halt.
While changes have tended to be minor in recent times,
there are a number of improvements worth discussing
explicitly.

3. Achieving special effects, in concert with manipulating \uccode and the like, such as defining
commands that contain characters with different
catcodes than usual.

1. Bruno has extended the floating point code tocover additional functions such as inverse trigonometric functions. These additions round out thefunctionality well and make it viable for use inmost cases needing floating point mathematics.

We are working on providing a set of commands toachieve all three of these functions in a more direct andeasy-to-use fashion, including support for Unicode in
LuaLATEX and XELATEX.

Space-skipping in xparse. . . and for 2014 onwards

We have also re-considered the behaviour of spaceskipping in xparse. Consider the following examples:

There is one (understandable) misconception thatshows up once in a while with people claiming that\begin{dmath}[x y z] = [1 2 3]\end{dmath}\begin{dmath}[label=foo]x^2 + y^2 = z^2\end{dmath}

In the first case, we are typesetting some mathematics
that contains square brackets. In the second, we areassigning a label to the equation using an optional argument, which also uses brackets. The fact that both
work correctly is due to behaviour that is specificallyprogrammed into the workings of the dmath environment of breqn: spaces before an optional argument are
explicitly forbidden. At present, this is also how commands and environments defined using xparse behave.
But consider a pgfplots environment:\begin{pgfplot}[% plot options]\begin{axis}[% axis options]...
\end{axis}\end{pgfplot}
This would seem like quite a natural way to write suchenvironments, but with the current state of xparse this
syntax would be incorrect. One would have to write
either of these instead:\begin{pgfplot}%[% plot options]\begin{pgfplot}[% plot options]

Is this an acceptable compromise? We’re not entirely
sure here — we’re in a corner because the humble [ hasended up being part of both the syntax and semantics
of a LATEX document.
Despite the current design covering most regular usecases, we have considered adding a further option toxparse to define the space-skipping behaviour as desiredby a package author. But at this very moment we’ve
rejected adding this additional complexity, because environments that change their parsing behaviour basedon their intended use make a LATEX-based language
more difficult to predict; one could imagine such behaviour causing difficulties down the road for automaticsyntax checkers and so forth. However, we don’t makesuch decisions in a vacuum and we’re always happy tocontinue to discuss such matters.

expl3 = LATEX3.
However, the correct relation would be a subset,
expl3 ⊂ LATEX3,
with expl3 forming the Core Language Layer on which
there will eventually be several other layers on top thatprovide• higher-level concepts for typesetting (Typesetting
Foundation Layer),• a designer interface for specifying document structures and layouts (Designer Layer),• and finally a Document Representation Layer that
implements document level syntax.
Of those four layers, the lowest one — expl3 — is available for use and with xparse we have an instance of
the Document Representation Layer modeled largely
after LATEX 2ε syntax (there could be others). Bothcan be successfully used within the current LATEX 2εframework and as mentioned above this is increasingly
happening.
The middle layers, however, where the rubber meetsthe road, are still at the level of prototypes and ideas(templates, ldb, galley, xor and all the good stuff) thatneed to be revised and further developed to arrive at a
LATEX3 environment that can stand on its own and thatis to where we want to return in 2014.
An overview on this can be found in the answer to“What can *I* do to help the LATEX3 project?” on
Stack Exchange,1 which is reproduced below in slightly
abridged form. This is of course not the first time that
we have discussed such matters, and you can find similar material in other publications such as those at
http://latex-project.org; e.g., the architecture talkgiven at the TUG 2011 conference.

1 http://tex.stackexchange.com/questions/45838

What can you do for the LATEX3 project?
By Frank Mittelbach
My vision of LATEX3 is really a system with multiple
layers that provide interfaces for different kinds of roles.
These layers are• the underlying engine (some TEX variant)• the programming layer (the core language, i.e.,
expl3)• the typesetting foundation layer (providing higherlevel concepts for typesetting)• the typesetting element layer (templates for all
types of document elements)• the designer interface foundation layer• the class designer layer (where instances of document elements with specific settings are defined)• document representation layer (that provides the
input syntax, i.e., how the author uses elements)
If you look at it from the perspective of user roles
then there are at least three or four roles that you canclearly distinguish:• The Programmer (template and functionality
provider)• The Document Type Designer (defines whichelements are available; abstract syntax and semantics)• The Designer (typography and layout)• The Author (content)
As a consequence the LATEX3 Project needs different
kinds of help depending on what layer or role we arelooking at.
The “Author” is using, say, list structures by specifying something like \begin{itemize} \item in his
documents. Or perhaps by writing <ul> ... </ul> orwhatever the UI representation offers to him.
The “Document Type Designer” defines what kindof abstract document elements are available, and what
attributes or arguments those elements provide at theauthor level. E.g., he may specify that a certain classof documents provides the display lists “enumerate”,“itemize” and “description”.
The “Programmer” on the other hand implements
templates (that offer customizations) for such document elements, e.g., for display lists. What kind ofcustomization possibilities should be provided by the“Programmer” is the domain of the “Document Designer”; he drives what kind of flexibility he needs forthe design. In most cases the “Document Designer”
should be able to simply select templates (already written) from a template library and only focus on thedesign, i.e., instantiating the templates with values
so that the desired layout for “itemize” lists, etc., iscreated.
In real life a single person may end up playing morethan one role, but it is important to recognise that allof them come with different requirements with respect
to interfaces and functionality.
Programming Layer
The programming layer consists of a core language
layer (called expl3 (EXP erimental L aTeX 3) for historical reasons and now we are stuck with it :-))and two more components: the “Typesetting Foundation Layer” that we are currently working on and the“Typesetting Element Layer” that is going to provide
customizable objects for the design layer. While expl3
is in many parts already fairly complete and usable theother two are under construction.
Help is needed for the programming layer in• helping by extending and completing the regression
test suite for expl3• helping with providing good or better documentation, including tutorials• possibly helping in coding additional core functionality — but that requires, in contrast to the firsttwo points, a good amount of commitment andexperience with the core language as otherwise thedanger is too high that the final results will end up
being inconsistentOnce we are a bit further along with the “Typesetting Foundation Layer” we would need help in providing higher-level functionality, perhaps rewriting
existing packages/code for elements making use of extended possibilities. Two steps down the road (once the“Designer Layer” is closer to being finalized) we would
need help with developing templates for all kinds ofelements.
In summary for this part, we need help from peopleinterested in programming in TEX and expl3 and/orinterested in providing documentation (but for this athorough understanding of the programming concepts
is necessary too).
Design Layer
The intention of the design layer is to provide interfacesthat allow specifying layout and typography styles ina declarative way. On the implementation side there
are a number of prototypes (most notably xtemplate
and the recent reimplementation of ldb). These need to
be unified into a common model which requires somemore experimentation and probably also some further
thoughts.

But the real importance of this layer is not the implementation of its interfaces but the conceptual viewof it: provisioning a rich declarative method (or methods) to describe design and layout. I.e., enabling adesigner to think not in programs but in visual representations and relationships.
So here is the big area where people who do not feelthey can or want to program TEX’s bowels can help.
What would be extremely helpful (and in fact not justfor LATEX3) would be• collecting and classifying a huge set of layouts anddesigns– designs for individual document elements(such as headings, TOCs, etc)
– document designs that include relationships
between document elements• thinking about good, declarative ways to specify
such designs– what needs to be specified– to what extent and with what flexibility
I believe that this is a huge task (but rewarding in itself) and already the first part of collecting existing
design specifications will be a major undertaking andwill need coordination and probably a lot of work. But
it will be a huge asset towards testing any implementations and interfaces for this layer later on.
Document Interface Layer
If we get the separation done correctly, then this layer
should effectively offer nothing more than a front end
for parsing the document syntax and transforming it
into an internal standardised form. This means that on
this layer one should not see any (or not much) coding
or computation.
It is envisioned that alternative document syntax
models can be provided. At the moment we have adraft solution in xparse. This package offers a document
syntax in the style of LATEX 2ε , that is with *-forms,
optional arguments in brackets, etc., but with a few
more bells and whistles such as a more generalized concept of default values, support for additional delimiters
for arguments, verbatim-style arguments, and so on.
It is fairly conventional though. In addition when it
was written the clear separation of layers wasn’t welldefined and so the package also contains components
for conditional programming that I no longer thinkshould be there.
Bottom line on what is needed for this layer is to• think about good syntax for providing document
content from “the author” perspective• think about good syntax for providing document
content from an “application to typesetting” per-spective, i.e., the syntax and structure for automated typesetting where the content is preparedby a system/application rather than by a human
These two areas most likely need strict structure (asautomation works much better with structures that do
not have a lot of alternative possibilities and shortcuts,etc.) and even when just looking at the human author
a lot of open questions need answering. And theseanswers may or may not be to painfully stick withexisting LATEX 2ε conventions in all cases (or perhaps
with any?).
None of this requires coding or expl3 experience.
What it requires is familiarity with existing input concepts, a feel for where the pain points currently are andthe willingness to think and discuss what alternatives
and extensions could look like.
In Summary
Basically help is possible on any level and it doesn’t
need to involve programming. Thoughts are sprinkled throughout this article, but here are a few more
highlights:• help with developing/improving the core programming layer by– joining the effort to improve the test suite– help improving the existing (or not existing)documentation– joining the effort to produce core or auxiliarycode modules• help on the design layer by– collecting and classifying design tasks– thinking and suggesting ways to describe
layout requirements in a declarative manner• help on shaping the document interface layer
These concepts, as well as their implementation, areunder discussion on the list latex-l.2 The list has
only a fairly low level of traffic right now as actual
implementation and development tasks are typically
discussed directly among the few active implementors.
But this might change if more people join.
And something else . . .
The people on the LATEX3 team are also committed
to keeping LATEX 2ε stable and even while there isn’t
that much to do these days there remains the need toresolve bug reports (if they concern the 2e core), provide new distributions once in a while, etc. All this iswork that takes effort or remains undone or incomplete.
Thus here too, it helps the LATEX3 efforts if we get help
to free up resources.
2 Instructions for joining and browsing archives at:
http://latex-project.org/code.html