Package: east-asian-spacing
East Asian Contextual SpacingThis directory contains tools for
the OpenType Contextual Half-width Spacing feature
for Japanese/Chinese/Korean typography.This feature enables the typography described inJLREQ 3.1.2 Positioning of Punctuation Marks (Commas, Periods and Brackets)句読点や，括弧類などの基本的な配置方法for Japanese,
andCLREQ 3.1.6.1 Punctuation Adjustment Space标点符号的调整空间 標點符號的調整空間for Chinese.
Following is a figure from JLREQ:An early discussion atAdobe CJK Type blog articlemay help to understand the feature better.DemoYou can findsample text here.
This sample page uses fonts built with this tool.OpenType Font FeaturesOpenType defines 4 feature tags
for fonts to support this feature:The "chws" feature tag,
and the "vchw" feature tag as its vertical flow counterpart.The "halt" feature tag,
and the "vhal" feature tag as its vertical flow counterpart.All 4 features are desired,
as each feature is applied in different context.This package adds these features to any OpenType/TrueType fonts
when they are missing,
by computing the feature tables from data
such as Unicode code points and glyph outlines.Adding the features to your fontsInstallYou can install this tool bypiporpipx.pipxinstalleast-asian-spacingpipinstalleast-asian-spacingPlease be aware that,
if you install withpipin the global environment,
its dependencies may cause conflicts with other packages.
If all what you need is the command line tool,pipxcan install it globally
while still isolating it in a virtual environment.Please also see theinstall packagesection
if you want to use this package from your Python program,
or theclone and installsection
if you want to diagnose fonts or the code in more details.Command Line UsagesThe following example adds the feature toinput-font-fileand saves it to thebuilddirectory.east-asian-spacing-obuildinput-font-fileThetestingsection has resources forchecking the differencesand
testing fonts you built.For other options and usages,
the--helpoption can show the full list of options.Supported FontsThealgorithmis applicable to any CJK fonts.
Following fonts are tested on each release:Noto CJKMeiryoBIZ UDGothicCJK fonts atfonts.google.comare tested in thechws_toolpackage.
Several other fonts were also tested during the development.When adding the features to your fonts,
thetest HTMLis a handy tool to check the results.
If you encounter any problems with your fonts,
please report toissues.Please also see theAdvanced Topicsbelow
if you want to customize the default behaviors for your fonts.TrueType Collection (TTC)When the input font file is a TrueType Collection (TTC),
this tool adds the feature to all fonts in the TTC by default.If you want to add the feature to only some of fonts in the TTC,
you can specify a comma-separated list of font indices.
The following example adds the feature to the font index 0 and 1,
but not to other fonts in the TTC.east-asian-spacing--index=0,1input-font-file.ttcAPIInstall PackageYou can install this package
using your favorite package management tools
such aspoetry,pipenv, orpip.pipinstalleast-asian-spacingpipenvinstalleast-asian-spacingpoetryaddeast-asian-spacingPlease also see theclone and installsection
if you want to diagnose fonts or the code in more details.Sample CodeThe following example creates a font with the features
in the "build" directory if the features are applicable.importeast_asian_spacingasyncdefmain_async():builder=east_asian_spacing.Builder("fonts/input.otf")output_path=awaitbuilder.build_and_save("build")ifoutput_path:print(f"Saved to{output_path}")else:print("Skipped")TestingTest HTMLAtest HTML pageis available
to check the behavior of fonts on browsers.It can test fonts you built locally.Save the page to your local drive.
The HTML is a single file, saving the HTML file should work.Add your font files to the "fonts" list
at the beginning of the<script>block.Open it in your browser and choose your font.Note, when you want to test a TTC (TrueType Collection)
but your browser can load only the first font in the TTC,
the following command extracts all OpenType fonts (.otf or .ttf)
from an OpenType Collection font file (.ttc or .otc).east-asian-spacingttcbuild/NotoSansCJK-Regular.ttcDumpThedumpsub-command can create various types of text dump files.The most simple usage is to show a list of tables.
This is similar to the "-l" option ofTTX,
except for TrueType Collections (TTC),
this tool can show tables of all fonts in the TTC,
along with which tables are shared with which fonts.east-asian-spacingdumpbuild/NotoSansCJK-Regular.ttcThe "-o" option creates table list files in the specified directory:east-asian-spacingdump-obuild/dumpbuild/*.ttcThe "--ttx" option createsTTXtext dumps of all tables
in addition to the table list files.
This is similar to the "-s" option ofTTX,
except that it can dump all tables in TrueType Collections (TTC).east-asian-spacingdump-obuild/dump--ttxbuild/*.ttcDiffThedumpsub-command can also createdumpfiles of two font files and compare them.
This helps visualizing differences in two fonts,
specifically, the font files you created from the original font files.east-asian-spacingdump-obuild/diff--diffsource_fonts_dirbuild/NotoSansCJK.ttcThe example above
computes the differences betweensource_fonts_dir/NotoSansCJK.ttcandbuild/NotoSansCJK.ttcby creating following 3 sets of files:The table list and TTX text dump files forbuild/NotoSansCJK.ttcin thebuild/diff/dumpdirectory.The table list and TTX text dump files forsource_fonts_dir/NotoSansCJK.ttcin thebuild/diff/srcdirectory.Diff files of the two sets of dump files in thebuild/diffdirectory.Note:
The "--diff" option is more efficient than doing all these,
especially for large fonts,
because it skips creating TTX of tables when they are binary-equal.The-ooption is optional. When it is omitted,
the sub-command outputs the diff tostdout.east-asian-spacingdump--diffsource_fonts_dirbuild/NotoSansCJK.ttc|lessTo create diff files for all fonts you built,
you can pipe the output as below:east-asian-spacing-p*.otf|east-asian-spacingdump-obuild/diff-The "-p" option prints the input and output font paths tostdoutin the tab-separated-values format.
Thedumpsub-command with the "-" argument reads this list fromstdin,
and creates their text dump and diff files in thebuild/diffdirectory.
The "--diff" option is not necessary in this case,
because the source font paths are provided from the pipe.ReferencesOnce you reviewed thedifffiles created above,
or tested fonts you build,
you can copy the diff files into thereferencesdirectory.
Then when you want to build them again,
such as when the fonts are updated or when the build environment is changed,
you can compare the diff files with the reference files
to know how new fonts are different from previous builds.With the "-r" option, thedumpsub-command
createsdifffiles between two font files,
and compare the diff files
with once-reviewed diff files in thereferencesdirectory.The typical usage of this option is as below:east-asian-spacing-p-g=build/glyphs*.otf|east-asian-spacingdump-o=build/diff-r=references-Please see theDiffsection for the "-p" option and piping.Thebuild*.shscriptsinclude this option.Shape TestThe shape testing shapes test strings
and checks whether the contextual spacing is applied or not.The--testoption sets the level of the shape testing.east-asian-spacing--test2-v-obuildinput-font-fileThe level 0 disables the shape testing.
The level 1 runs a smoke test using a small set of samples.
The level 2 runs the shape testing using a large set of test strings.
The default value is 1.Advanced TopicsAlgorithmThe algorithm is language agnostic and is applicable to any CJK fonts.This package determines the glyph pairs to adjust spacings
by a set of Unicode code points
defined in theConfigclass.Then for each pair, it checks if the spacings are applicable
by examining glyph outlines and computing ink bounding boxes of glyphs.
For example, when glyphs are very thick,
glyphs may not have enough internal spacings,
and applying the spacings may cause glyphs to collide.
This package automatically detects such cases and
avoids applying spacings to such pairs.This automatic behavior can be disabled
by specifying thelanguagesbelow,
or by settingConfig.use_ink_boundstoFalsein your Python program.LanguagesThere are language-specific conventions
for where punctuation characters are placed in the glyph spaces.
For example,
U+3002 IDEOGRAPHIC FULL STOP
should be placed at the left-bottom corner of the glyph space in Japanese,
while it should be placed at the center in Traditional Chinese.By default,
this package determines such differences from glyph outlines
as described in theAlgorithmsection above.
But you can specify theOpenType language system tagto let this package follow the language convention
instead of using glyph outlines.
The following example
disables the automatic determination by glyph outlines,
and specifies that the font is a Japanese font.east-asian-spacing--language=JANinput-font-fileFor TrueType Collections (TTC),
the language option applies to all fonts in the TTC by default.
When you want to specify different languages to each font in the TTC,
it accepts a comma-separated list.
The following example specifies
Korean for the font index 1,
Simplified Chinese for the font index 2,
and automatic for all other fonts.east-asian-spacing--language=,KOR,ZHSinput-font-file.ttcYou can combine these two options.
The following example appliesJANto the index 2,
andZHSto the index 3.
Other fonts in the TTC are not changed.east-asian-spacing--index=2,3--language=JAN,ZHSinput-font-file.ttcCharacter-PairsYou may want to adjust which character-pairs should adjust spacings,
in cases such as when
your fonts may not have expected spacings for some characters.
Currently, this is possible only from Python programs.For a simple example, please see thetest_configfunction
intests/config_test.py.Thechws_toolproject is an actual example of customizing this package.HarfBuzzThis package uses theHarfBuzzshaping engine
by using a Cython bindingsuharfbuzz.If you want to use a specific build of theHarfBuzz,
this tool can invoke the externalhb-shapecommand line tool instead
by setting theSHAPERenvironment variable.exportSHAPER=hb-shapeTo installhb-shapefor Linux:sudoaptgetlibharfbuzz-binTo installhb-shapefor Mac withhomebrew:brewinstallharfbuzzInstructions for other platforms may be available atcommand-not-found.com.Clone and InstallIf you may need to diagnose fonts or the code,
cloning and installing usingpoetryis recommended:gitclonehttps://github.com/kojiishi/east_asian_spacingcdeast_asian_spacing
poetryinstall
poetryshellThis method has following advantages:Installs the exact versions of dependencies.Installs in the editable mode
(i.e.,pip "-e" optionorsetuptools"development mode").Installs testing tools too.
You can rununit teststo verify your installation if needed.Creates the virtual environment automatically.You can also install the cloned directory usingpipif you prefer:gitclonehttps://github.com/kojiishi/east_asian_spacingcdeast_asian_spacing
pipinstall.Unit TestsThis repository contains unit tests usingpytest.
The unit tests cover the basic functionalities
includingshape tests,
adding the feature to a test font,
and comparing it withreferences.If you followed theclone and installsection,
tools for unit testing are already installed.
Before you run them first time,
you need to download fonts for testing:./tests/download_fonts.pyYou can then run the tests by:pytestor run them with multiple versions of Python usingtox:toxScriptsThescriptsdirectory has some small shell scripts.build*.shscripts are useful to build fonts,
computedifffrom source fonts,
and compare the diff files withreferences.
Followings are example usages../scripts/build.shinput-font-file.otf-v
./scripts/build-noto-cjk.sh~/fonts/noto-cjk-v