cs-lex 20250914
Lexical analysis functions, tokenisers, transcribers: an arbitrary assortment of lexical and tokenisation functions useful for writing recursive descent parsers, of which I have several. There are also some transcription functions for producing text from various objects, such as `hexify` and `unctrl`.
Keywords: python3

Lexical analysis functions, tokenisers, transcribers:
an arbitrary assortment of lexical and tokenisation functions useful
for writing recursive descent parsers, of which I have several.
There are also some transcription functions for producing text
from various objects, such as `hexify` and `unctrl`.

*Latest release 20250914*:
* New single_space(s[,sep=]) function to split a string and rejoin, with a single space by default.
* Obsolete the @has_format_attributes decorator, how covered by FormatableMixin automatically.
* New FormatMapping(obj,base_format_mapping[,missing]) class presenting a mapping where some entries are callable, used by FormatableMixin.format_as.
* FStr: rename lc to lc_ to reflect the use of the lc_() function.
* format_as: new optional `missing(mapping,key)` parameter to supply values for keys not in `format_mapping`.
* FormatAsError: annotate the error with .key, .format_s, .format_mapping.
* Many small internal changes.

Generally the get_* functions accept a source string and an offset
(usually optional, default `0`) and return a token and the new offset,
raising `ValueError` on failed tokenisation.

Short summary:
* `as_lines`: Generator yielding complete lines from arbitrary pieces of text from the iterable of `str` `chunks`.
* `BaseToken`: A mixin for token dataclasses.
* `camelcase`: Convert a snake cased string `snakecased` into camel case.
* `common_prefix`: Return the common prefix of the strings `strs`.
* `common_suffix`: Return the common suffix of the strings `strs`.
* `CoreTokens`: A mixin for token dataclasses whose subclasses include `Identifier`, 'NumericValue` and `QuotedString`.
* `cropped`: If the length of `s` exceeds `max_length` (default `32`), replace enough of the tail with `ellipsis` and the last `roffset` (default `1`) characters of `s` to fit in `max_length` characters.
* `cropped_repr`: Compute a cropped `repr()` of `obj`.
* `cutprefix`: Remove `prefix` from the front of `s` if present. Return the suffix if `s.startswith(prefix)`, else `s`. As with `str.startswith`, `prefix` may be a `str` or a `tuple` of `str`. If a tuple, the first matching prefix from the tuple will be removed.
* `cutsuffix`: Remove `suffix` from the end of `s` if present. Return the prefix if `s.endswith(suffix)`, else `s`. As with `str.endswith`, `suffix` may be a `str` or a `tuple` of `str`. If a tuple, the first matching suffix from the tuple will be removed.
* `FFloat`: Formattable `float`.
* `FInt`: Formattable `int`.
* `FNumericMixin`: A `FormatableMixin` subclass.
* `format_as`: Format the string `format_s` using `Formatter.vformat`, return the formatted result. This is a wrapper for `str.format_map` which raises a more informative `FormatAsError` exception on failure.
* `format_attribute`: A decorator to mark a method as available as a format method. This sets `method.is_format_attribute=True`.
* `format_escape`: Escape `{}` characters in a string to protect them from `str.format`.
* `format_recover`: Decorator for `__format__` methods which replaces failed formats with `{self:format_spec}`.
* `FormatableFormatter`: A `string.Formatter` subclass interacting with objects which inherit from `FormatableMixin`.
* `FormatableMixin`: A subclass of `FormatableFormatter` which  provides 2 main features: - a `__format__` method which parses the `format_spec` string into multiple colon separated terms whose results chain - a `format_as` method which formats a format string using `str.format_map` with a suitable mapping derived from the instance via its `format_kwargs` method (whose default is to return the instance itself).
* `FormatAsError`: Subclass of `LookupError` for use by `format_as`.
* `FormatMapping`: A `Mapping` subclass based on an object and a mapping intended for use by the `FormatableMixin.format_as` method. The mapping maps field names to values, where the values may be callables accepting an object. Fetching a value from the mapping will call `value(obj)` if the value is callable. Some additonal extra field names are provided if not already present in the mapping: - `self`: the object.
* `FStr`: A `str` subclass with the `FormatableMixin` methods, particularly its `__format__` method which uses `str` method names as valid formats.
* `get_chars`: Scan the string `s` for characters in `gochars` starting at `offset`. Return `(match,new_offset)`.
* `get_decimal`: Scan the string `s` for decimal characters starting at `offset` (default `0`). Return `(dec_string,new_offset)`.
* `get_decimal_or_float_value`: Fetch a decimal or basic float (nnn.nnn) value from the str `s` at `offset` (default `0`). Return `(value,new_offset)`.
* `get_decimal_value`: Scan the string `s` for a decimal value starting at `offset` (default `0`). Return `(value,new_offset)`.
* `get_delimited`: Collect text from the string `s` from position `offset` up to the first occurence of delimiter `delim`; return the text excluding the delimiter and the offset after the delimiter.
* `get_dotted_identifier`: Scan the string `s` for a dotted identifier (by default an ASCII letter or underscore followed by letters, digits or underscores) with optional trailing dot and another dotted identifier, starting at `offset` (default `0`). Return `(match,new_offset)`.
* `get_envvar`: Parse a simple environment variable reference to $varname or $x where "x" is a special character.
* `get_hexadecimal`: Scan the string `s` for hexadecimal characters starting at `offset` (default `0`). Return `(hex_string,new_offset)`.
* `get_hexadecimal_value`: Scan the string `s` for a hexadecimal value starting at `offset` (default `0`). Return `(value,new_offset)`.
* `get_identifier`: Scan the string `s` for an identifier (by default an ASCII letter or underscore followed by letters, digits or underscores) starting at `offset` (default 0). Return `(match,new_offset)`.
* `get_ini_clause_entryname`: Parse a `[`*clausename*`]`*entryname* string from `s` at `offset` (default `0`). Return `(clausename,entryname,new_offset)`.
* `get_ini_clausename`: Parse a `[`*clausename*`]` string from `s` at `offset` (default `0`). Return `(clausename,new_offset)`.
* `get_nonwhite`: Scan the string `s` for characters not in `string.whitespace` starting at `offset` (default `0`). Return `(match,new_offset)`.
* `get_other_chars`: Scan the string `s` for characters not in `stopchars` starting at `offset` (default `0`). Return `(match,new_offset)`.
* `get_prefix_n`: Strip a leading `prefix` and numeric value `n` from the string `s` starting at `offset` (default `0`). Return the matched prefix, the numeric value and the new offset. Returns `(None,None,offset)` on no match.
* `get_qstr`: Get quoted text with slosh escapes and optional environment substitution.
* `get_qstr_or_identifier`: Parse a double quoted string or an identifier.
* `get_sloshed_text`: Collect slosh escaped text from the string `s` from position `offset` (default `0`) and return the decoded unicode string and the offset of the completed parse.
* `get_suffix_part`: Strip a trailing "part N" suffix from the string `s`. Return the matched suffix and the number part number. Retrn `(None,None)` on no match.
* `get_tokens`: Parse the string `s` from position `offset` using the supplied tokeniser functions `getters`. Return the list of tokens matched and the final offset.
* `get_uc_identifier`: Scan the string `s` for an identifier as for `get_identifier`, but require the letters to be uppercase.
* `get_white`: Scan the string `s` for characters in `string.whitespace` starting at `offset` (default `0`). Return `(match,new_offset)`.
* `has_format_attributes`: OBSOLETE version of has_format_attributes, suggestion: @has_format_attributes is no longer needed.
* `hexify`: A flavour of `binascii.hexlify` returning a `str`.
* `htmlify`: Convert a string for safe transcription in HTML.
* `htmlquote`: Quote a string for use in HTML.
* `Identifier`: A dotted identifier.
* `indent`: Return the `paragraph` indented by `line_indent` (default `"  "`).
* `is_dotted_identifier`: Test if the string `s` is an identifier from position `offset` onward.
* `is_identifier`: Test if the string `s` is an identifier from position `offset` (default `0`) onward.
* `is_uc_identifier`: Test if the string `s` is an uppercase identifier from position `offset` (default `0`) onward.
* `isUC_`: Check that a string matches the regular expression `^[A-Z][A-Z_0-9]*$`.
* `jsquote`: Quote a string for use in JavaScript.
* `lc_`: Return `value.lower()` with `'-'` translated into `'_'` and `' '` translated into `'-'`.
* `match_tokens`: Wrapper for `get_tokens` which catches `ValueError` exceptions and returns `(None,offset)`.
* `NumericValue`: An `int` or `float` literal.
* `parseUC_sAttr`: Take an attribute name `attr` and return `(key,is_plural)`.
* `phpquote`: Quote a string for use in PHP code.
* `printt`: A wrapper for `tabulate()` to print the results. Each positional argument is a table row.
* `QuotedString`: A double quoted string.
* `r`: Like `typed_str` but using `repr` instead of `str`. This is available as both `typed_repr` and `r`.
* `s`: Return "type(obj).__name__:str(obj)" for some object `obj`. This is available as both `typed_str` and `s`.
* `single_space`: Return the string `s` stripped and with internal whitespace replaced by `sep` (default `" "`).
* `skipwhite`: Convenience routine for skipping past whitespace; returns the offset of the next nonwhitespace character.
* `slosh_mapper`: Return a string to replace backslash-`c`, or `None`.
* `slosh_quote`: Quote a string `raw_s` with quote character `q`.
* `snakecase`: Convert a camel cased string `camelcased` into snake case.
* `split_remote_path`: OBSOLETE version of split_remote_path, suggestion: cs.fs.RemotePath.from_str.
* `stripped_dedent`: Slightly smarter dedent which ignores a string's opening indent.
* `strlist`: Convert an iterable to strings and join with `sep` (default `', '`).
* `tabpadding`: Compute some spaces to use a tab padding at an offfset.
* `tabulate`: A generator yielding lines of values from `rows` aligned in columns.
* `texthexify`: Transcribe the bytes `bs` to text using compact text runs for some common text values.
* `titleify_lc`: Translate `'-'` into `' '` and `'_'` translated into `'-'`, then titlecased.
* `typed_repr`: Like `typed_str` but using `repr` instead of `str`. This is available as both `typed_repr` and `r`.
* `typed_str`: Return "type(obj).__name__:str(obj)" for some object `obj`. This is available as both `typed_str` and `s`.
* `unctrl`: Return the string `s` with `TAB`s expanded and control characters replaced with printable representations.
* `untexthexify`: Decode a textual representation of binary data into binary data.

Module contents:
- <a name="as_lines"></a>`as_lines(chunks, partials=None)`: Generator yielding complete lines from arbitrary pieces of text from
  the iterable of `str` `chunks`.

  After completion, any remaining newline-free chunks remain
  in the partials list; they will be unavailable to the caller
  unless the list is presupplied.
- <a name="BaseToken"></a>`class BaseToken(cs.deco.Promotable)`: A mixin for token dataclasses.

  Presently I use this in `cs.app.tagger.rules` and `cs.app.pilfer.parse`.

*`BaseToken.__post_init__(self)`*:
An omitted `offset,end_offset` means the token is the whole `source_text`.

*`BaseToken.from_str(text: str) -> 'BaseToken'`*:
Parse `test` as a token of type `cls`, return the token.
Raises `SyntaxError` on a parse failure.
This is a wrapper for the `parse` class method.

*`BaseToken.matched_text`*:
The text from `self.source_text` which matches this token.

*`BaseToken.parse(text: str, offset: int = 0, *, skip=False) -> Tuple[ForwardRef('BaseToken'), int]`*:
Parse a token from `test` at `offset` (default `0`).
Return a `BaseToken` subclass instance.
Raise `SyntaxError` if no subclass parses it.
Raise `EOFError` if at the end of the `text`,
checked after any whitespace if `skip` is true.
The returned token's `.end_offset` is the next parse point.

This base class method attempts the `.parse` method of all
the public subclasses.

Parameters:
* `text`: the text being parsed
* `offset`: the offset within the `text` of the the parse cursor
* `skip`: if true (default `False`), skip any leading
  whitespace before matching

*`BaseToken.scan(text: str, offset: int = 0, *, skip=True) -> Iterable[ForwardRef('BaseToken')]`*:
Scan `text`, parsing tokens using `BaseToken.parse` and yielding them.
Parameters are as for `BaseToken.parse` except as follows:
- encountering end of text end the iteration instead of raising `EOFError`
- `skip` defaults to `True` to allow whitespace between tokens

*`BaseToken.token_classes()`*:
Return the `baseToken` subclasses to consider when parsing a token stream.
- <a name="camelcase"></a>`camelcase(snakecased: str, first_letter_only: bool = False) -> str`: Convert a snake cased string `snakecased` into camel case.

  Parameters:
  * `snakecased`: the snake case string to convert
  * `first_letter_only`: optional flag (default `False`);
    if true then just ensure that the first character of a word
    is uppercased, otherwise use `str.title`

  Example:

      >>> camelcase('abc_def')
      'abcDef'
      >>> camelcase('ABc_def')
      'abcDef'
      >>> camelcase('abc_dEf')
      'abcDef'
      >>> camelcase('abc_dEf', first_letter_only=True)
      'abcDEf'
- <a name="common_prefix"></a>`common_prefix(*strs)`: Return the common prefix of the strings `strs`.

  Examples:

      >>> common_prefix('abc', 'def')
      ''
      >>> common_prefix('abc', 'abd')
      'ab'
      >>> common_prefix('abc', 'abcdef')
      'abc'
      >>> common_prefix('abc', 'abcdef', 'abz')
      'ab'
      >>> # contrast with cs.fileutils.common_path_prefix
      >>> common_prefix('abc/def', 'abc/def1', 'abc/def2')
      'abc/def'
- <a name="common_suffix"></a>`common_suffix(*strs)`: Return the common suffix of the strings `strs`.
- <a name="CoreTokens"></a>`class CoreTokens(BaseToken)`: A mixin for token dataclasses whose subclasses include `Identifier`,
  'NumericValue` and `QuotedString`.
- <a name="cropped"></a>`cropped(s: str, max_length: int = 32, roffset: int = 1, ellipsis: str = '...')`: If the length of `s` exceeds `max_length` (default `32`),
  replace enough of the tail with `ellipsis`
  and the last `roffset` (default `1`) characters of `s`
  to fit in `max_length` characters.
- <a name="cropped_repr"></a>`cropped_repr(obj, roffset=1, max_length=32, inner_max_length=None)`: Compute a cropped `repr()` of `obj`.

  Parameters:
  * `obj`: the object to represent
  * `max_length`: the maximum length of the representation, default `32`
  * `inner_max_length`: the maximum length of the representations
    of members of `obj`, default `max_length//2`
  * `roffset`: the number of trailing characters to preserve, default `1`
- <a name="cutprefix"></a>`cutprefix(s, prefix)`: Remove `prefix` from the front of `s` if present.
  Return the suffix if `s.startswith(prefix)`, else `s`.
  As with `str.startswith`, `prefix` may be a `str` or a `tuple` of `str`.
  If a tuple, the first matching prefix from the tuple will be removed.

  Example:

      >>> abc_def = 'abc.def'
      >>> cutprefix(abc_def, 'abc.')
      'def'
      >>> cutprefix(abc_def, 'zzz.')
      'abc.def'
      >>> cutprefix(abc_def, '.zzz') is abc_def
      True
      >>> cutprefix('this_that', ('this', 'thusly'))
      '_that'
      >>> cutprefix('thusly_that', ('this', 'thusly'))
      '_that'
- <a name="cutsuffix"></a>`cutsuffix(s, suffix)`: Remove `suffix` from the end of `s` if present.
  Return the prefix if `s.endswith(suffix)`, else `s`.
  As with `str.endswith`, `suffix` may be a `str` or a `tuple` of `str`.
  If a tuple, the first matching suffix from the tuple will be removed.

  Example:

      >>> abc_def = 'abc.def'
      >>> cutsuffix(abc_def, '.def')
      'abc'
      >>> cutsuffix(abc_def, '.zzz')
      'abc.def'
      >>> cutsuffix(abc_def, '.zzz') is abc_def
      True
      >>> cutsuffix('this_that', ('that', 'tother'))
      'this_'
      >>> cutsuffix('this_tother', ('that', 'tother'))
      'this_'
- <a name="FFloat"></a>`class FFloat(FNumericMixin, builtins.float)`: Formattable `float`.
- <a name="FInt"></a>`class FInt(FNumericMixin, builtins.int)`: Formattable `int`.
- <a name="FNumericMixin"></a>`class FNumericMixin(FormatableMixin)`: A `FormatableMixin` subclass.

*`FNumericMixin.localtime(self)`*:
Treat this as a UNIX timestamp and return a localtime `datetime`.

*`FNumericMixin.utctime(self)`*:
Treat this as a UNIX timestamp and return a UTC `datetime`.
- <a name="format_as"></a>`format_as(format_s: str, format_mapping, formatter=None, *, error_sep=None, missing: Optional[Callable[[Mapping, Any], Any]] = None, strict=False)`: Format the string `format_s` using `Formatter.vformat`,
  return the formatted result.
  This is a wrapper for `str.format_map`
  which raises a more informative `FormatAsError` exception on failure.

  Parameters:
  * `format_s`: the format string to use as the template
  * `format_mapping`: the mapping of available replacement fields
  * `formatter`: an optional `string.Formatter`-like instance
    with a `.vformat(format_string,args,kwargs)` method,
    usually a subclass of `string.Formatter`;
    if not specified then `FormatableFormatter` is used
  * `error_sep`: optional separator for the multipart error message,
    default from `FormatAsError.DEFAULT_SEPARATOR`:
    `'; '`
  * `missing`: an optional callable to turn a key missing from
    `format_mapping` into a value to interpolate
- <a name="format_attribute"></a>`format_attribute(method)`: A decorator to mark a method as available as a format method.
  This sets `method.is_format_attribute=True`.

  For example,
  the `FormatableMixin.json` method is defined like this:

      @format_attribute
      def json(self):
          return self.FORMAT_JSON_ENCODER.encode(self)

  which allows a `FormatableMixin` subclass instance
  to be used in a format string like this:

      {instance:json}

  to insert a JSON transcription of the instance.

  It is recommended that methods marked with `@format_attribute`
  have no side effects and do not modify state,
  as they are intended for use in ad hoc format strings
  supplied by an end user.
- <a name="format_escape"></a>`format_escape(s)`: Escape `{}` characters in a string to protect them from `str.format`.
- <a name="format_recover"></a>`format_recover(*da, **dkw)`: Decorator for `__format__` methods which replaces failed formats
  with `{self:format_spec}`.
- <a name="FormatableFormatter"></a>`class FormatableFormatter(string.Formatter)`: A `string.Formatter` subclass interacting with objects
  which inherit from `FormatableMixin`.

*`FormatableFormatter.format_field(value, format_spec: str)`*:
Format a value using `format_field`, returning an `FStr`
(a `str` subclass with additional `format_spec` features).

We actually recognise colon separated chains of formats
and apply each format to the previously converted value.
The final result is promoted to an `FStr` before return.

*`FormatableFormatter.get_arg_name(field_name)`*:
The default initial arg_name is an identifier.

Returns `(prefix,offset)`, and `('',0)` if there is no arg_name.

*`FormatableFormatter.get_field(self, field_name, args, kwargs)`*:
Get the object referenced by the field text `field_name`.
Raises `KeyError` for an unknown `field_name`.

*`FormatableFormatter.get_format_subspecs(format_spec)`*:
Parse a `format_spec` as a sequence of colon separated components,
return a list of the components.

*`FormatableFormatter.get_subfield(value, subfield_text: str)`*:
Resolve `value` against `subfield_text`,
the remaining field text after the term which resolved to `value`.

For example, a format `{name.blah[0]}`
has the field text `name.blah[0]`.
A `get_field` implementation might initially
resolve `name` to some value,
leaving `.blah[0]` as the `subfield_text`.
This method supports taking that value
and resolving it against the remaining text `.blah[0]`.

For generality, if `subfield_text` is the empty string
`value` is returned unchanged.

*`FormatableFormatter.get_value(self, arg_name, args, kwargs)`*:
Get the object with index `arg_name`.

This default implementation returns `(kwargs[arg_name],arg_name)`.
- <a name="FormatableMixin"></a>`class FormatableMixin(FormatableFormatter)`: A subclass of `FormatableFormatter` which  provides 2 main features:
  - a `__format__` method which parses the `format_spec` string
    into multiple colon separated terms whose results chain
  - a `format_as` method which formats a format string using `str.format_map`
    with a suitable mapping derived from the instance
    via its `format_kwargs` method
    (whose default is to return the instance itself)

  The `format_as` method is like an inside out `str.format` or
  `object.__format__` method.

  The `str.format` method is designed for formatting a string
  from a variety of other objects supplied in the keyword arguments.

  The `object.__format__` method is for filling out a single `str.format`
  replacement field from a single object.

  By contrast, `format_as` is designed to fill out an entire format
  string from the current object.

  For example, the `cs.tagset.TagSet` class subclasses
  `FormatableMixin` to provide a `format_as` method whose
  replacement fields are derived from the tags in the tag set.

  Subclasses wanting to provide additional `format_spec` terms
  should:
  - override `FormatableFormatter.format_field1` to implement
    terms with no colons, letting `format_field` do the split into terms
  - override `FormatableFormatter.get_format_subspecs` to implement
    the parse of `format_spec` into a sequence of terms.
    This might recognise a special additional syntax
    and quietly fall back to `super().get_format_subspecs`
    if that is not present.

*`FormatableMixin.__format__(self, format_spec)`*:
Format `self` according to `format_spec`.

This implementation calls `self.format_field`.
As such, a `format_spec` is considered
a sequence of colon separated terms.

Classes wanting to implement additional format string syntaxes
should either:
- override `FormatableFormatter.format_field1` to implement
  terms with no colons, letting `format_field1` do the split into terms
- override `FormatableFormatter.get_format_subspecs` to implement
  the term parse.

The default implementation of `__format1__` just calls `super().__format__`.
Implementations providing specialised formats
should implement them in `__format1__`
with fallback to `super().__format1__`.

*`FormatableMixin.__init_subclass__(**kw)`*:
Prefill the `cls.format_attributes` mapping from the
superclass and any format attributes of `cls`.

*`FormatableMixin.convert_field(self, value, conversion)`*:
The default converter for fields calls `Formatter.convert_field`.

This is a tiny shim to transmute the `''` conversion to `None`
which is what `Formatter.convert_field` expects.

*`FormatableMixin.convert_via_method_or_attr(self, format_spec) -> Tuple[Any, int]`*:
Apply a method or attribute name based conversion to `self`
where `format_spec` starts with a method or attribute name.
Return `(converted,offset)`
being the converted value and the offset after the method name.

Note that if there is not a leading identifier on `format_spec`
then this method returns `(self,0)`.

The converted value is obtained from `getattr(self,name)`;
if this raises an `AttributeError` a second attempt is made with
`getattr(FStr(self),attr)` if `self` is not already an `FStr`
(this provides the common utility methods on other types).

If the value is callable but does not have a true
`.is_format_attribute` a `TypeError` is raised, otherwise
the value is called to complete the conversion.
(The `.is_format_attribute` is usually set by decorating a
method with the `@format_attribute` decorator.)

The motivating example was a `PurePosixPath`,
which does not JSON transcribe;
this tweak supports both
`posixpath:basename` via the pathlib stuff
and `posixpath:json` via `FStr`
even though a `PurePosixPath` does not subclass `FStr`.

*`FormatableMixin.format_as(self, format_s: str, *, error_sep: Optional[str] = None, missing: Optional[Callable[[Mapping, Any], Any]] = None, strict=True, **format_kwargs_kw)`*:
Return the string `format_s` formatted using the mapping
returned by `self.format_kwargs(**format_kwargs_kw)`.

If a class using the mixin has no `format_kwargs()` method
to provide a mapping for `str.format_map`
then the instance itself is used as the mapping.

*`FormatableMixin.json(self)`*:
The value transcribed as compact JSON.
- <a name="FormatAsError"></a>`class FormatAsError(builtins.LookupError)`: Subclass of `LookupError` for use by `format_as`.
- <a name="FormatMapping"></a>`class FormatMapping(collections.abc.Mapping)`: A `Mapping` subclass based on an object and a mapping
  intended for use by the `FormatableMixin.format_as` method.
  The mapping maps field names to values, where the values may be
  callables accepting an object.
  Fetching a value from the mapping will call `value(obj)` if
  the value is callable.
  Some additonal extra field names are provided if not already
  present in the mapping:
  - `self`: the object

*`FormatMapping.__getitem__(self, field_name: str)`*:
Fetch the value for `field_name`.
If the value is callable, call `value(self.obj)` to get the value.

*`FormatMapping.items(self)`*:
Proxy `.items` via `self.mapping`.

*`FormatMapping.keys(self)`*:
Proxy `.keys` via `self.mapping`.
- <a name="FStr"></a>`class FStr(FormatableMixin, builtins.str)`: A `str` subclass with the `FormatableMixin` methods,
  particularly its `__format__` method
  which uses `str` method names as valid formats.

  It also has a bunch of utility methods which are available
  as `:`*method* in format strings.

*`FStr.basename(self)`*:
Treat as a filesystem path and return the basename.

*`FStr.dirname(self)`*:
Treat as a filesystem path and return the dirname.

*`FStr.f(self)`*:
Parse `self` as a `float`.

*`FStr.i(self, base=10)`*:
Parse `self` as an `int`.

*`FStr.lc_(self)`*:
Lowercase using `lc_()`.

*`FStr.path(self)`*:
Convert to a native filesystem `pathlib.Path`.

*`FStr.posix_path(self)`*:
Convert to a Posix filesystem `pathlib.Path`.

*`FStr.windows_path(self)`*:
Convert to a Windows filesystem `pathlib.Path`.
- <a name="get_chars"></a>`get_chars(s: str, offset: int, gochars: str) -> Tuple[str, int]`: Scan the string `s` for characters in `gochars` starting at `offset`.
  Return `(match,new_offset)`.

  `gochars` may also be a callable, in which case a character
  `ch` is accepted if `gochars(ch)` is true.
- <a name="get_decimal"></a>`get_decimal(s: str, offset: int = 0) -> Tuple[str, int]`: Scan the string `s` for decimal characters starting at `offset` (default `0`).
  Return `(dec_string,new_offset)`.
- <a name="get_decimal_or_float_value"></a>`get_decimal_or_float_value(s: int, offset: int = 0) -> Tuple[str, int]`: Fetch a decimal or basic float (nnn.nnn) value
  from the str `s` at `offset` (default `0`).
  Return `(value,new_offset)`.
- <a name="get_decimal_value"></a>`get_decimal_value(s: str, offset: int = 0) -> Tuple[str, int]`: Scan the string `s` for a decimal value starting at `offset` (default `0`).
  Return `(value,new_offset)`.
- <a name="get_delimited"></a>`get_delimited(s, offset, delim)`: Collect text from the string `s` from position `offset` up
  to the first occurence of delimiter `delim`; return the text
  excluding the delimiter and the offset after the delimiter.
- <a name="get_dotted_identifier"></a>`get_dotted_identifier(s: str, offset: int = 0, **kw) -> Tuple[str, int]`: Scan the string `s` for a dotted identifier (by default an
  ASCII letter or underscore followed by letters, digits or
  underscores) with optional trailing dot and another dotted
  identifier, starting at `offset` (default `0`).
  Return `(match,new_offset)`.

  Note: the empty string and an unchanged offset will be returned if
  there is no leading letter/underscore.

  Keyword arguments are passed to `get_identifier`
  (used for each component of the dotted identifier).
- <a name="get_envvar"></a>`get_envvar(s, offset=0, environ=None, default=None, specials=None)`: Parse a simple environment variable reference to $varname or
  $x where "x" is a special character.

  Parameters:
  * `s`: the string with the variable reference
  * `offset`: the starting point for the reference
  * `default`: default value for missing environment variables;
     if `None` (the default) a `ValueError` is raised
  * `environ`: the environment mapping, default `os.environ`
  * `specials`: the mapping of special single character variables
- <a name="get_hexadecimal"></a>`get_hexadecimal(s: str, offset: int = 0) -> Tuple[str, int]`: Scan the string `s` for hexadecimal characters starting at `offset` (default `0`).
  Return `(hex_string,new_offset)`.
- <a name="get_hexadecimal_value"></a>`get_hexadecimal_value(s: str, offset: int = 0) -> Tuple[str, int]`: Scan the string `s` for a hexadecimal value starting at `offset` (default `0`).
  Return `(value,new_offset)`.
- <a name="get_identifier"></a>`get_identifier(s: str, offset: int = 0, *, alpha: str = 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ', number: str = '0123456789', extras: str = '_') -> Tuple[str, int]`: Scan the string `s` for an identifier (by default an ASCII
  letter or underscore followed by letters, digits or underscores)
  starting at `offset` (default 0).
  Return `(match,new_offset)`.

  *Note*: the empty string and an unchanged offset will be returned if
  there is no leading letter/underscore.

  Parameters:
  * `s`: the string to scan
  * `offset`: the starting offset, default `0`.
  * `alpha`: the characters considered alphabetic,
    default `string.ascii_letters`.
  * `number`: the characters considered numeric,
    default `string.digits`.
  * `extras`: extra characters considered part of an identifier,
    default `'_'`.
- <a name="get_ini_clause_entryname"></a>`get_ini_clause_entryname(s: str, offset: int = 0) -> Tuple[str, str, int]`: Parse a `[`*clausename*`]`*entryname* string
  from `s` at `offset` (default `0`).
  Return `(clausename,entryname,new_offset)`.
- <a name="get_ini_clausename"></a>`get_ini_clausename(s: str, offset: int = 0) -> Tuple[str, int]`: Parse a `[`*clausename*`]` string from `s` at `offset` (default `0`).
  Return `(clausename,new_offset)`.
- <a name="get_nonwhite"></a>`get_nonwhite(s: str, offset: int = 0) -> Tuple[str, int]`: Scan the string `s` for characters not in `string.whitespace`
  starting at `offset` (default `0`).
  Return `(match,new_offset)`.
- <a name="get_other_chars"></a>`get_other_chars(s: str, offset: int = 0, stopchars: Optional[str] = None) -> Tuple[str, int]`: Scan the string `s` for characters not in `stopchars` starting
  at `offset` (default `0`).
  Return `(match,new_offset)`.
- <a name="get_prefix_n"></a>`get_prefix_n(s: str, prefix: str, n: Optional[int] = None, *, offset: int = 0) -> str`: Strip a leading `prefix` and numeric value `n` from the string `s`
  starting at `offset` (default `0`).
  Return the matched prefix, the numeric value and the new offset.
  Returns `(None,None,offset)` on no match.

  Parameters:
  * `s`: the string to parse
  * `prefix`: the prefix string which must appear at `offset`
    or an object with a `match(str,offset)` method
    such as an `re.Pattern` regexp instance
  * `n`: optional integer value;
    if omitted any value will be accepted, otherwise the numeric
    part must match `n`

  If `prefix` is a `str`, the "matched prefix" return value is `prefix`.
  Otherwise the "matched prefix" return value is the result of
  the `prefix.match(s,offset)` call. The result must also support
  a `.end()` method returning the offset in `s` beyond the match,
  used to locate the following numeric portion.

  Examples:

     >>> import re
     >>> get_prefix_n('s03e01--', 's')
     ('s', 3, 3)
     >>> get_prefix_n('s03e01--', 's', 3)
     ('s', 3, 3)
     >>> get_prefix_n('s03e01--', 's', 4)
     (None, None, 0)
     >>> get_prefix_n('s03e01--', re.compile('[es]',re.I))
     (<re.Match object; span=(0, 1), match='s'>, 3, 3)
     >>> get_prefix_n('s03e01--', re.compile('[es]',re.I), offset=3)
     (<re.Match object; span=(3, 4), match='e'>, 1, 6)
- <a name="get_qstr"></a>`get_qstr(s, offset=0, q='"', environ=None, default=None, env_specials=None)`: Get quoted text with slosh escapes and optional environment substitution.

  Parameters:
  * `s`: the string containg the quoted text.
  * `offset`: the starting point, default `0`.
  * `q`: the quote character, default `'"'`. If `q` is `None`,
    do not expect the string to be delimited by quote marks.
  * `environ`: if not `None`, also parse and expand `$`*envvar* references.
  * `default`: passed to `get_envvar`
- <a name="get_qstr_or_identifier"></a>`get_qstr_or_identifier(s, offset)`: Parse a double quoted string or an identifier.
- <a name="get_sloshed_text"></a>`get_sloshed_text(s, delim, offset=0, slosh='\\', mapper=<function slosh_mapper at 0x107087740>, specials=None)`: Collect slosh escaped text from the string `s` from position
  `offset` (default `0`) and return the decoded unicode string and
  the offset of the completed parse.

  Parameters:
  * `delim`: end of string delimiter, such as a single or double quote.
  * `offset`: starting offset within `s`, default `0`.
  * `slosh`: escape character, default a slosh ('\').
  * `mapper`: a mapping function which accepts a single character
    and returns a replacement string or `None`; this is used the
    replace things such as '\t' or '\n'. The default is the
    `slosh_mapper` function, whose default mapping is `SLOSH_CHARMAP`.
  * `specials`: a mapping of other special character sequences and parse
    functions for gathering them up. When one of the special
    character sequences is found in the string, the parse
    function is called to parse at that point.
    The parse functions accept
    `s` and the offset of the special character. They return
    the decoded string and the offset past the parse.

  The escape character `slosh` introduces an encoding of some
  replacement text whose value depends on the following character.
  If the following character is:
  * the escape character `slosh`, insert the escape character.
  * the string delimiter `delim`, insert the delimiter.
  * the character 'x', insert the character with code from the following
    2 hexadecimal digits.
  * the character 'u', insert the character with code from the following
    4 hexadecimal digits.
  * the character 'U', insert the character with code from the following
    8 hexadecimal digits.
  * a character from the keys of `mapper`
- <a name="get_suffix_part"></a>`get_suffix_part(s: str, *, keywords: Iterable[str] = ('part',), numeral_map: Optional[Mapping[str, int]] = None) -> Union[Tuple[str, int], Tuple[NoneType, NoneType]]`: Strip a trailing "part N" suffix from the string `s`.
  Return the matched suffix and the number part number.
  Retrn `(None,None)` on no match.

  Parameters:
  * `s`: the string
  * `keywords`: an iterable of `str` to match, or a single `str`;
    default `'part'`
  * `numeral_map`: an optional mapping of numeral names to numeric values;
    default `NUMERAL_NAMES['en']`, the English numerals

  Exanmple:

      >>> get_suffix_part('s09e10 - A New World: Part One')
      (': Part One', 1)
- <a name="get_tokens"></a>`get_tokens(s, offset, getters)`: Parse the string `s` from position `offset` using the supplied
  tokeniser functions `getters`.
  Return the list of tokens matched and the final offset.

  Parameters:
  * `s`: the string to parse.
  * `offset`: the starting position for the parse.
  * `getters`: an iterable of tokeniser specifications.

  Each tokeniser specification `getter` is either:
  * a callable expecting `(s,offset)` and returning `(token,new_offset)`
  * a literal string, to be matched exactly
  * a `tuple` or `list` with values `(func,args,kwargs)`;
    call `func(s,offset,*args,**kwargs)`
  * an object with a `.match` method such as a regex;
    call `getter.match(s,offset)` and return a match object with
    a `.end()` method returning the offset of the end of the match
- <a name="get_uc_identifier"></a>`get_uc_identifier(s: str, offset: int = 0, number: str = '0123456789', extras: str = '_') -> Tuple[str, int]`: Scan the string `s` for an identifier as for `get_identifier`,
  but require the letters to be uppercase.
- <a name="get_white"></a>`get_white(s: str, offset: int = 0) -> Tuple[str, int]`: Scan the string `s` for characters in `string.whitespace`
  starting at `offset` (default `0`).
  Return `(match,new_offset)`.
- <a name="has_format_attributes"></a>`has_format_attributes(*da, **dkw)`: OBSOLETE version of has_format_attributes, suggestion: @has_format_attributes is no longer needed

  A obsolete class decorator formerly used to walk the class for `@formatmethod`
  methods. This is now done by `FormatableMixin.__init_subclass__`.
- <a name="hexify"></a>`hexify(bs: bytes) -> str`: A flavour of `binascii.hexlify` returning a `str`.
- <a name="htmlify"></a>`htmlify(s: str, nbsp: bool = False) -> str`: Convert a string for safe transcription in HTML.

  Parameters:
  * `s`: the string
  * `nbsp`: replaces spaces with `"&nbsp;"` to prevent word folding,
    default `False`.
- <a name="htmlquote"></a>`htmlquote(s: str) -> str`: Quote a string for use in HTML.
- <a name="Identifier"></a>`class Identifier(CoreTokens)`: A dotted identifier.

*`Identifier.parse(text: str, offset: int = 0, *, skip=False) -> Tuple[str, cs.lex.CoreTokens, int]`*:
Parse a dotted identifier from `test`.
- <a name="indent"></a>`indent(paragraph: str, line_indent: str = '  ') -> str`: Return the `paragraph` indented by `line_indent` (default `"  "`).
- <a name="is_dotted_identifier"></a>`is_dotted_identifier(s: str, offset: int = 0, **kw) -> bool`: Test if the string `s` is an identifier from position `offset` onward.
- <a name="is_identifier"></a>`is_identifier(s: str, offset: int = 0, **kw) -> bool`: Test if the string `s` is an identifier
  from position `offset` (default `0`) onward.
- <a name="is_uc_identifier"></a>`is_uc_identifier(s: str, offset: int = 0, **kw) -> bool`: Test if the string `s` is an uppercase identifier
  from position `offset` (default `0`) onward.
- <a name="isUC_"></a>`isUC_(s)`: Check that a string matches the regular expression `^[A-Z][A-Z_0-9]*$`.
- <a name="jsquote"></a>`jsquote(s: str) -> str`: Quote a string for use in JavaScript.
- <a name="lc_"></a>`lc_(value: str) -> str`: Return `value.lower()`
  with `'-'` translated into `'_'` and `' '` translated into `'-'`.

  I use this to construct lowercase filenames containing a
  readable transcription of a title string.

  See also `titleify_lc()`, an imperfect reversal of this.
- <a name="match_tokens"></a>`match_tokens(s, offset, getters)`: Wrapper for `get_tokens` which catches `ValueError` exceptions
  and returns `(None,offset)`.
- <a name="NumericValue"></a>`class NumericValue(_LiteralValue)`: An `int` or `float` literal.

*`NumericValue.parse(text: str, offset: int = 0, *, skip=False) -> 'NumericValue'`*:
Parse a Python style `int` or `float`.
- <a name="parseUC_sAttr"></a>`parseUC_sAttr(attr)`: Take an attribute name `attr` and return `(key,is_plural)`.

  Examples:
  * `'FOO'` returns `('FOO',False)`.
  * `'FOOs'` or `'FOOes'` returns `('FOO',True)`.
  Otherwise return `(None,False)`.
- <a name="phpquote"></a>`phpquote(s: str) -> str`: Quote a string for use in PHP code.
- <a name="printt"></a>`printt(*table, file=None, flush=False, indent='', print_func=None, **tabulate_kw)`: A wrapper for `tabulate()` to print the results.
  Each positional argument is a table row.

  Parameters:
  * `file`: optional output file, passed to `print_func`
  * `flush`: optional flush flag, passed to `print_func`
  * `indent`: optional leadin