TUGboat, Volume 43 (2022), No. 2165
Case changing: LATEX reaches Unicode-land
Joseph Wright, LATEX Project Team1Introduction

The concept of letters having case is familiar tospeakers of several languages, most obviously thosefrom Europe using Latin, Greek or Cyrillic scripts.
The ability to convert between upper and lower case,
case changing, is something we might take for granted
both for people and for computer systems. However,there are subtleties that a careful implementationneeds to take into account.
The Unicode Consortium have defined four different case-related operations that are required tosupport the range of applications that come up inpractical use cases:• Upper casing• Lower casing• Title casing• Case folding
Here, title casing means making the first letter (broadly speaking) upper case, then making the rest of thetext lower case. Case folding means removing case,and is needed for programmers: often lower casing
is used, but this is not appropriate for true caselesstext comparisons.
Unicode have also identified that case changing
is not a simple fixed operation: depending on thecontext around a character, the right outcome canvary, while different languages can have different
rules. All of this means that the TEX primitives\lowercase and \uppercase are not suitable forcase changing with the variety of text we might seetoday.
2An expl3 implementation

In 2015, I spoke at the TUG meeting about an expl3
implementation that sought to provide a full set ofUnicode-compliance case changing tools. At thattime, only Unicode engines (XETEX and LuaTEX)
were supported. However, the code could provide allof the requirements of Unicode in an approach thatworks by expansion: this meant that one could casechange text inside an \edef.
Since that talk, the LATEX team have refinedideas about the future of LATEX, meaning that thecase changer needed to be extended to work withpdfTEX. It also needed to be able to carry out
something similar to \protected@edef by expansion. Both of those ideas have been covered over thepast few years.
The expl3 implementation now also incorporates
ideas from David Carlisle’s textcase package. This isCase changing: LATEX reaches Unicode-landdoi.org/10.47397/tb/43-2/tb134wright-case166TUGboat, Volume 43 (2022), No. 2

mainly about being able to exclude content from casechanging: math mode material should never be casechanged, and it’s important to be able to mark individual items as unaffected by case operations. Bothof those ideas are relatively easy to do by expansion,as we need to examine each token anyway.
3Bring it to LATEX 2εSince \uppercase and \lowercase have long been
known to have limitations, I am not the first personto look at supporting the needs of different languages.
There have been a number of clever approaches togetting the required mappings from implementations using the TEX primitives. However, the newcode provides a single consistent interface: it canhandle different languages, multiple scripts and soon without needing to load potentially incompatible
code.
More importantly for the team, we needed tolook again at how active characters are handled in
pdfTEX. The change, to use engine protection forthese active bytes, makes life a lot easier in general
but breaks the previous approaches to case changing
these characters. The expl3 code, in contrast, works
fine with the protected definitions. So this drove thechange.
For users, the long-standing \MakeUppercase
and \MakeLowercase commands stay unchanged:
only the implementation has been adjusted. Titlecasing has some subtleties that mean it needs a
dedicated document command, so now it has one:\MakeTitlecase. The command \NoCaseChange,
originally defined by textcase, is also now integrated
into the kernel.

For package authors, we have added a command \AddToNoCaseChangeList to add commands
to the set which are skipped for case changing: thingslike \ref and \label are already there. We havealso provided \CaseSwitch, a command that selects
between different outcomes (no change, upper, lower,title case): this is useful if you have something that
doesn’t expand to text but where you want a different
result inside and outside of case changing. Finally,
we have added \DeclareCaseChangeEquivalent forsituations where a package author needs to entirely
swap the functionality of a command inside a casechanging context.
4The data: pdfTEX

One minor wrinkle is the data support, particularlyfor pdfTEX. For the Unicode engines, we can read
all of the data automatically. For pdfTEX, we don’t
have \lccode and \uccode storage for the whole ofUnicode, only for the 8-bit range. That means thatmost of the case changing data has to be stored inmacros. To avoid wasting a lot of memory, only code
points that are typically typeset with 8-bit engines
are included here. That does mean that it’s possiblea few get missed: if there is something to add, please
let us know.
5Looking ahead

For the Spring 2022 release, we have not included
support for language variation in the document command interface. But that’s on the agenda, and likely
to appear in the Fall 2022 LATEX 2ε update. The mostlikely approach there is as an optional argument to\MakeUppercase, etc.: watch this space.
⋄ Joseph WrightNorthampton, United Kingdom
joseph dot wright (at)morningstar2.co.uk⋄ LATEX Project Team

Joseph Wright, LATEX Project Team