Package: charex
charexis a Unicode and character set explorer for understanding
issues with character set translation and Unicode normalization.Why Did I Make This?I find the ambiguity of text data interesting. In memory it’s all ones
and zeros. There is nothing inherent to the data that makes0x20mean
a space character, but we’ve mostly agreed that it does. That “mostly”
part is what’s interesting to me, and it’s where a lot of fun problems lie.How Do I Use This?It’s in PyPI, so you can install it withpip, as long as you are
using Python 3.11 or higher:pip install charexcharexhas four modes of operation:Direct command line invocation,An interactive shell,A graphical user interface (GUI),An application programming interface (API).Command LineTo get help for direct invocation from the command line:$ charex -hInteractive ShellTo launch the interactive shell:$ charexThat will bring you to thecharexshell:Welcome to the charex shell.
Press ? for a list of comands.

charex>From here you can type?to see the list of available commands:Welcome to the charex shell.
Press ? for a list of comands.

charex> ?
The following commands are available:

  * cd: Decode the given address in all codecs.
  * ce: Encode the given character in all codecs.
  * cl: List registered character sets.
  * clear: Clear the terminal.
  * ct: Count denormalization results.
  * dm: Build a denormalization map.
  * dn: Perform denormalizations.
  * dt: Display details for a code point.
  * el: List the registered escape schemes.
  * es: Escape a string using the given scheme.
  * fl: List registered normalization forms.
  * nl: Perform normalizations.
  * sh: Run in an interactive shell.
  * up: List the Unicode properties.
  * uv: List the valid values for a Unicode property.

For help on individual commands, use "help {command}".

charex>And then typehelpthen a name of one of the commands to learn what
it does:charex> help dn
usage: charex dn [-h] [-m MAXDEPTH] [-n NUMBER] [-r] [-s SEED] form base

Denormalize a string.

positional arguments:
  form                  The normalization form for the denormalization. Valid
                        options are: casefold, nfc, nfd, nfkc, nfkd.
  base                  The base normalized string.

options:
  -h, --help            show this help message and exit
  -m MAXDEPTH, --maxdepth MAXDEPTH
                        Maximum number of reverse normalizations to use for
                        each character.
  -n NUMBER, --number NUMBER
                        Maximum number of results to return.
  -r, --random          Randomize the denormalization.
  -s SEED, --seed SEED  Seed the randomized denormalization.

charex>GUITo launch thecharexGUI:$ charex guiAPITo importcharexinto your Python script to get a summary of a
Unicode character:>>> import charex
>>>
>>>
>>> value = 'a'
>>> char = charex.Character(value)
>>> print(char.summarize())
a U+0061 (LATIN SMALL LETTER A)What Is Left To Do?The following features are planned for the v0.2.1 or later releases:(Done.) Add every Unihan property to charex.Character.(Done.) Fix names for CJK Ideographs.(Done.) Move all Y/N properties to simple lists.(Done.) Move the resource file registry tosources.json.Support for more versions of Unicode.Emoji combiner.Basic doctests for all public classes and functions.Web API.Registration for character set codecs.The list of Unicode properties can be found here:IndexThe list of Unihan properties is here:tr38How Do I Run the Tests?charexis using thepytestpackage for unit testing. It also
comes with a makefile that automates testing. So, to run the
tests:Installpipenv:pip install pipenvInstall the development dependencies:pipenv install –devTo run just the unit tests:make testTo run the full test suite:make pre