cs-tagset 20250528
Tags and sets of tags with __format__ support and optional ontology information.
Keywords: python3

Tags and sets of tags
with __format__ support and optional ontology information.

*Latest release 20250528*:
* Some refactors and small fixes.
* Tag: rename Tag.from_str2 to Tag.parse, drop offset parameter of Tag.from_str.

See `cs.fstags` for support for applying these to filesystem objects
such as directories and files.

See `cs.sqltags` for support for databases of entities with tags,
not directly associated with filesystem objects.
This is suited to both log entries (entities with no "name")
and large collections of named entities;
both accept `Tag`s and can be searched on that basis.

All of the available complexity is optional:
you can use `Tag`s without bothering with `TagSet`s
or `TagsOntology`s.

This module contains the following main classes:
* `Tag`: an object with a `.name` and optional `.value` (default `None`)
  and also an optional reference `.ontology`
  for associating semantics with tag values.
  The `.value`, if not `None`, will often be a string,
  but may be any Python object.
  If you're using these via `cs.fstags`,
  the object will need to be JSON transcribeable.
* `TagSet`: a `dict` subclass representing a set of `Tag`s
  to associate with something;
  it also has setlike `.add` and `.discard` methods.
  As such it only supports a single `Tag` for a given tag name,
  but that tag value can of course be a sequence or mapping
  for more elaborate tag values.
* `TagsOntology`:
  a mapping of type names to `TagSet`s defining the type
  and also to entries for the metadata for specific per-type values.

Here's a simple example with some `Tag`s and a `TagSet`.

    >>> tags = TagSet()
    >>> # add a "bare" Tag named 'blue' with no value
    >>> tags.add('blue')
    >>> # add a "topic=tagging" Tag
    >>> tags.set('topic', 'tagging')
    >>> # make a "subtopic" Tag and add it
    >>> subtopic = Tag('subtopic', 'ontologies')
    >>> tags.add(subtopic)
    >>> # Tags have nice repr() and str()
    >>> subtopic
    Tag(name='subtopic',value='ontologies')
    >>> print(subtopic)
    subtopic=ontologies
    >>> # a TagSet also has a nice repr() and str()
    >>> tags
    TagSet:{'blue': None, 'topic': 'tagging', 'subtopic': 'ontologies'}
    >>> print(tags)
    blue subtopic=ontologies topic=tagging
    >>> tags2 = TagSet({'a': 1}, b=3, c=[1,2,3], d='dee')
    >>> tags2
    TagSet:{'a': 1, 'b': 3, 'c': [1, 2, 3], 'd': 'dee'}
    >>> print(tags2)
    a=1 b=3 c=[1,2,3] d=dee
    >>> # since you can print a TagSet to a file as a line of text
    >>> # you can get it back from a line of text
    >>> TagSet.from_str('a=1 b=3 c=[1,2,3] d=dee')
    TagSet:{'a': 1, 'b': 3, 'c': [1, 2, 3], 'd': 'dee'}
    >>> # because TagSets are dicts you can format strings with them
    >>> print('topic:{topic} subtopic:{subtopic}'.format_map(tags))
    topic:tagging subtopic:ontologies
    >>> # TagSets have convenient membership tests
    >>> # test for blueness
    >>> 'blue' in tags
    True
    >>> # test for redness
    >>> 'red' in tags
    False
    >>> # test for any "subtopic" tag
    >>> 'subtopic' in tags
    True
    >>> # test for subtopic=ontologies
    >>> print(subtopic)
    subtopic=ontologies
    >>> subtopic in tags
    True
    >>> # test for subtopic=libraries
    >>> subtopic2 = Tag('subtopic', 'libraries')
    >>> subtopic2 in tags
    False

## Ontologies

`Tag`s and `TagSet`s suffice to apply simple annotations to things.
However, an ontology brings meaning to those annotations.

See the `TagsOntology` class for implementation details,
access methods and more examples.

Consider a record about a movie, with these tags (a `TagSet`):

    title="Avengers Assemble"
    series="Avengers (Marvel)"
    cast={"Scarlett Johansson":"Black Widow (Marvel)"}

where we have the movie title,
a name for the series in which it resides,
and a cast as an association of actors with roles.

An ontology lets us associate implied types and metadata with these values.

Here's an example ontology supporting the above `TagSet`:

    type.cast type=dict key_type=person member_type=character description="members of a production"
    type.character description="an identified member of a story"
    type.series type=str
    character.marvel.black_widow type=character names=["Natasha Romanov"]
    person.scarlett_johansson fullname="Scarlett Johansson" bio="Known for Black Widow in the Marvel stories."

The type information for a `cast`
is defined by the ontology entry named `type.cast`,
which tells us that a `cast` `Tag` is a `dict`,
whose keys are of type `person`
and whose values are of type `character`.
(The default type is `str`.)

To find out the underlying type for a `character`
we look that up in the ontology in turn;
because it does not have a specified `type` `Tag`, it it taken to be a `str`.

Having the types for a `cast`,
it is now possible to look up the metadata for the described cast members.

The key `"Scarlett Johansson"` is a `person`
(from the type definition of `cast`).
The ontology entry for her is named `person.scarlett_johansson`
which is computed as:
* `person`: the type name
* `scarlett_johansson`: obtained by downcasing `"Scarlett Johansson"`
  and replacing whitespace with an underscore.
  The full conversion process is defined
  by the `TagsOntology.value_to_tag_name` function.

The key `"Black Widow (Marvel)"` is a `character`
(again, from the type definition of `cast`).
The ontology entry for her is named `character.marvel.black_widow`
which is computed as:
* `character`: the type name
* `marvel.black_widow`: obtained by downcasing `"Black Widow (Marvel)"`,
  replacing whitespace with an underscore,
  and moving a bracketed suffix to the front as an unbracketed prefix.
  The full conversion process is defined
  by the `TagsOntology.value_to_tag_name` function.

## Format Strings

You can just use `str.format_map` as shown above
for the direct values in a `TagSet`,
since it subclasses `dict`.

However, `TagSet`s also subclass `cs.lex.FormatableMixin`
and therefore have a richer `format_as` method which has an extended syntax
for the format component.
Command line tools like `fstags` use this for output format specifications.

An example:

    >>> # an ontology specifying the type for a colour
    >>> # and some information about the colour "blue"
    >>> ont = TagsOntology(
    ...   {
    ...       'type.colour':
    ...       TagSet(description="a colour, a hue", type="str"),
    ...       'colour.blue':
    ...       TagSet(
    ...           url='https://en.wikipedia.org/wiki/Blue',
    ...           wavelengths='450nm-495nm'
    ...       ),
    ...   }
    ... )
    >>> # tag set with a "blue" tag, using the ontology above
    >>> tags = TagSet(colour='blue', labels=['a', 'b', 'c'], size=9, _ontology=ont)
    >>> tags.format_as('The colour is {colour}.')
    'The colour is blue.'
    >>> # format a string about the tags showing some metadata about the colour
    >>> tags.format_as('Information about the colour may be found here: {colour:metadata.url}')
    'Information about the colour may be found here: https://en.wikipedia.org/wiki/Blue'

Short summary:
* `as_unixtime`: Convert a tag value to a UNIX timestamp.
* `BaseTagSets`: The base class for collections of `TagSet` instances such as `cs.fstags.FSTags` and `cs.sqltags.SQLTags`.
* `jsonable`: Convert `obj` to a JSON encodable form. This returns `obj` for purely JSONable objects and a JSONable deep copy of `obj` if it or some subcomponent required conversion. `converted` is a dict mapping object ids to their converted forms to prevent loops.
* `MappingTagSets`: A `BaseTagSets` subclass using an arbitrary mapping.
* `RegexpTagRule`: A regular expression based `Tag` rule.
* `selftest`: Run some ad hoc self tests.
* `Tag`: A name/value pair. Each `Tag` has a `.name` (`str`), a `.value` and an optional `.ontology`. The `name` must be a dotted identifier.
* `tag_or_tag_value`: A decorator for functions or methods which may be called as:.
* `TagBasedTest`: A test based on a `Tag`.
* `TagFile`: A reference to a specific file containing tags.
* `TagsCommandMixin`: Utility methods for `cs.cmdutils.BaseCommand` classes working with tags.
* `TagSet`: A setlike class collection of `Tag`s.
* `TagSetCriterion`: A testable criterion for a `TagSet`.
* `TagSetPrefixView`: A view of a `TagSet` via a `prefix`.
* `TagSetsSubdomain`: A view into a `BaseTagSets` for keys commencing with a prefix being the subdomain plus a dot (`'.'`).
* `TagsOntology`: An ontology for tag names. This is based around a mapping of names to ontological information expressed as a `TagSet`.
* `TagsOntologyCommand`: A command line for working with ontology types.

Module contents:
- <a name="as_unixtime"></a>`as_unixtime(tag_value)`: Convert a tag value to a UNIX timestamp.

  This accepts `int`, `float` (already a timestamp)
  and `date` or `datetime`
  (use `datetime.timestamp() for a nonnaive `datetime`,
  otherwise `time.mktime(tag_value.time_tuple())`,
  which assumes the local time zone).
- <a name="BaseTagSets"></a>`Class `BaseTagSets(cs.resources.MultiOpenMixin, collections.abc.MutableMapping)``: The base class for collections of `TagSet` instances
  such as `cs.fstags.FSTags` and `cs.sqltags.SQLTags`.

  Examples of this include:
  * `cs.cdrip.MBSQLTags`: a mapping of MusicbrainsNG entities to their associated `TagSet`
  * `cs.fstags.FSTags`: a mapping of filesystem paths to their associated `TagSet`
  * `cs.sqltags.SQLTags`: a mapping of names to `TagSet`s stored in an SQL database

  Subclasses must implement:
  * `get(name,default=None)`: return the `TagSet` associated
    with `name`, or `default`.
  * `__setitem__(name,tagset)`: associate a `TagSet`with the key `name`;
    this is called by the `__missing__` method with a newly created `TagSet`.
  * `keys(self)`: return an iterable of names

  Subclasses may reasonably want to override the following:
  * `startup_shutdown(self)`: context manager to allocate and release any
    needed resources such as database connections

  Subclasses may implement:
  * `__len__(self)`: return the number of names

  The `TagSet` factory used to fetch or create a `TagSet` is
  named `TagSetClass`. The default implementation honours two
  class attributes:
  * `TAGSETCLASS_DEFAULT`: initially `TagSet`
  * `TAGSETCLASS_PREFIX_MAPPING`: a mapping of type names to `TagSet` subclasses

  The type name of a `TagSet` name is the first dotted component.
  For example, `artist.nick_cave` has the type name `artist`.
  A subclass of `BaseTagSets` could utiliise an `ArtistTagSet` subclass of `TagSet`
  and provide:

      TAGSETCLASS_PREFIX_MAPPING = {
        'artist': ArtistTagSet,
      }

  in its class definition. Accesses to `artist.`* entities would
  result in `ArtistTagSet` instances and access to other entities
  would result in ordinary `TagSet` instances.

*`BaseTagSets.__init__(self, *, ontology=None)`*:
Initialise the collection.

*`BaseTagSets.TAGSETCLASS_DEFAULT`*

*`BaseTagSets.TagSetClass(self, *, name, **kw)`*:
Factory to create a new `TagSet` from `name`.

*`BaseTagSets.__contains__(self, name: str)`*:
Test whether `name` is present in the underlying mapping.

*`BaseTagSets.__getitem__(self, name: str)`*:
Obtain the `TagSet` associated with `name`.

If `name` is not presently mapped,
return `self.__missing__(name)`.

*`BaseTagSets.__iter__(self)`*:
Iteration returns the keys.

*`BaseTagSets.__len__(self)`*:
Return the length of the underlying mapping.

*`BaseTagSets.__missing__(self, name: str, **kw)`*:
Like `dict`, the `__missing__` method may autocreate a new `TagSet`.

This is called from `__getitem__` if `name` is missing
and uses the factory `cls.default_factory`.
If that is `None` raise `KeyError`,
otherwise call `self.default_factory(name,**kw)`.
If that returns `None` raise `KeyError`,
otherwise save the entity under `name` and return the entity.

*`BaseTagSets.__setitem__(self, name, te)`*:
Save `te` in the backend under the key `name`.

*`BaseTagSets.add(self, name: str, **kw)`*:
Return a new `TagSet` associated with `name`,
which should not already be in use.

*`BaseTagSets.default_factory(self, name: str)`*:
Create a new `TagSet` named `name`.

*`BaseTagSets.edit(self, *, select_tagset=None, **kw)`*:
Edit the `TagSet`s.

Parameters:
* `select_tagset`: optional callable accepting a `TagSet`
  which tests whether it should be included in the `TagSet`s
  to be edited
Other keyword arguments are passed to `Tag.edit_tagsets`.

*`BaseTagSets.get(self, name: str, default=None)`*:
Return the `TagSet` associated with `name`,
or `default` if there is no such entity.

*`BaseTagSets.items(self, *, prefix=None)`*:
Generator yielding `(key,value)` pairs,
optionally constrained to keys starting with `prefix+'.'`.

*`BaseTagSets.keys(self, *, prefix=None)`*:
Return the keys starting with `prefix+'.'`
or all keys if `prefix` is `None`.

*`BaseTagSets.subdomain(self, subname: str)`*:
Return a proxy for this `BaseTagSets` for the `name`s
starting with `subname+'.'`.

*`BaseTagSets.values(self, *, prefix=None)`*:
Generator yielding the mapping values (`TagSet`s),
optionally constrained to keys starting with `prefix+'.'`.
- <a name="jsonable"></a>`jsonable(obj, converted: dict)`: Convert `obj` to a JSON encodable form.
  This returns `obj` for purely JSONable objects and a JSONable
  deep copy of `obj` if it or some subcomponent required
  conversion.
  `converted` is a dict mapping object ids to their converted forms
  to prevent loops.
- <a name="MappingTagSets"></a>`Class `MappingTagSets(BaseTagSets)``: A `BaseTagSets` subclass using an arbitrary mapping.

  If no mapping is supplied, a `dict` is created for the purpose.

  Example:

      >>> tagsets = MappingTagSets()
      >>> list(tagsets.keys())
      []
      >>> tagsets.get('foo')
      >>> tagsets['foo'] = TagSet(bah=1, zot=2)
      >>> list(tagsets.keys())
      ['foo']
      >>> tagsets.get('foo')
      TagSet:{'bah': 1, 'zot': 2}
      >>> list(tagsets.keys(prefix='foo'))
      ['foo']
      >>> list(tagsets.keys(prefix='bah'))
      []

*`MappingTagSets.__delitem__(self, name: str)`*:
Delete the `TagSet` named `name`.

*`MappingTagSets.__setitem__(self, name: str, te)`*:
Save `te` in the backend under the key `name`.

*`MappingTagSets.get(self, name: str, default=None)`*:
Get `name` or `default`.

*`MappingTagSets.keys(self, *, prefix: Optional[str] = None) -> Iterable[str]`*:
Return an iterable of the keys commencing with `prefix`
or all keys if `prefix` is `None`.
- <a name="RegexpTagRule"></a>`Class `RegexpTagRule``: A regular expression based `Tag` rule.

  This applies a regular expression to a string
  and returns inferred `Tag`s.

*`RegexpTagRule.infer_tags(self, s)`*:
Apply the rule to the string `s`, return a list of `Tag`s.
- <a name="selftest"></a>`selftest(argv)`: Run some ad hoc self tests.
- <a name="Tag"></a>`Class `Tag(Tag, cs.lex.FormatableMixin)``: A name/value pair.
  Each `Tag` has a `.name` (`str`), a `.value` and an optional `.ontology`.
  The `name` must be a dotted identifier.

  Terminology:
  * A "bare" `Tag` has a `value` of `None`.
  * A "naive" `Tag` has an `ontology` of `None`.

  The constructor for a `Tag` is unusual:
  * both the `value` and `ontology` are optional,
    defaulting to `None`
  * if `name` is a `str` then we always construct a new `Tag`
    with the suppplied values
  * if `name` is not a `str`
    it should be a `Tag`like object to promote;
    it is an error if the `value` parameter is not `None`
    in this case
  * an optional `prefix` may be supplied
    which is prepended to `name` with a dot (`'.'`) if not empty

  The promotion process is as follows:
  * if `name` is a `Tag` subinstance
    then if the supplied `ontology` is not `None`
    and is not the ontology associated with `name`
    then a new `Tag` is made,
    otherwise the original `Tag` is returned unchanged
  * otherwise a new `Tag` is made from `name`
    using its `.value`
    and overriding its `.ontology`
    if the `ontology` parameter is not `None`

  Examples:

      >>> ont = TagsOntology({'colour.blue': TagSet(wavelengths='450nm-495nm')})
      >>> tag0 = Tag('colour', 'blue')
      >>> tag0
      Tag(name='colour',value='blue')
      >>> tag = Tag(tag0)
      >>> tag
      Tag(name='colour',value='blue')
      >>> tag = Tag(tag0, ontology=ont)
      >>> tag # doctest: +ELLIPSIS
      Tag(name='colour',value='blue',ontology=...)
      >>> tag = Tag(tag0, prefix='surface')
      >>> tag
      Tag(name='surface.colour',value='blue')

*`Tag.__init__(self, *a, **kw)`*:
Dummy `__init__` to avoid `FormatableMixin.__init__`
because we subclass `namedtuple` which has no `__init__`.

*`Tag.__hash__`*

*`Tag.__str__(self)`*:
Encode `name` and `value`.
A "bare" `Tag` (`self.value is None`) is just its name.
Otherwise `{self.name}={self.transcribe_value(self.value)}`.

*`Tag.alt_values(self, value_tag_name=None)`*:
Return a list of alternative values for this `Tag`
on the premise that each has a metadata entry.

*`Tag.basetype`*:
The base type name for this tag.
Returns `None` if there is no ontology.

This calls `self.onotology.basetype(self.name)`.
The basetype is the endpoint of a cascade down the defined types.

For example, this might tell us that a `Tag` `role="Fred"`
has a basetype `"str"`
by cascading through a hypothetical chain `role`->`character`->`str`:

    type.role type=character
    type.character type=str

*`Tag.from_arg(arg, offset=0, ontology=None)`*:
Parse a `Tag` from the string `arg` at `offset` (default `0`).
where `arg` is known to be entirely composed of the value,
such as a command line argument.

This calls the `from_str` method with `fallback_parse` set
to gather then entire tail of the supplied string `arg`.

*`Tag.from_str(s, ontology=None, fallback_parse=None)`*:
Parse `s` as a `Tag` definition.
This is the inverse of `Tag.__str__`, and a shim for `Tag.parse`
which checks that the entire string is consumed.

*`Tag.from_str2(*a, **kw)`*:
Obsolete name for `Tag.parse`.

*`Tag.is_valid_name(name)`*:
Test whether a tag name is valid: a dotted identifier.

*`Tag.key_metadata(self, key)`*:
Return the metadata definition for `key`.

The metadata `TagSet` is obtained from the ontology entry
*type*`.`*key_tag_name*
where *type* is the `Tag`'s `key_type`
and *key_tag_name* is the key converted
into a dotted identifier by `TagsOntology.value_to_tag_name`.

*`Tag.key_type`*:
The type name for members of this tag.

This is required if `.value` is a mapping.

*`Tag.key_typedef`*:
The typedata definition for this `Tag`'s keys.

This is for `Tag`s which store mappings,
for example a movie cast, mapping actors to roles.

The name of the member type comes from
the `key_type` entry from `self.typedata`.
That name is then looked up in the ontology's types.

*`Tag.matches(self, tag_name, value)`*:
Test whether this `Tag` matches `(tag_name,value)`.

*`Tag.member_metadata(self, member_key)`*:
Return the metadata definition for self[member_key].

The metadata `TagSet` is obtained from the ontology entry
*type*`.`*member_tag_name*
where *type* is the `Tag`'s `member_type`
and *member_tag_name* is the member value converted
into a dotted identifier by `TagsOntology.value_to_tag_name`.

*`Tag.member_type`*:
The type name for members of this tag.

This is required if `.value` is a sequence or mapping.

*`Tag.member_typedef`*:
The typedata definition for this `Tag`'s members.

This is for `Tag`s which store mappings or sequences,
for example a movie cast, mapping actors to roles,
or a list of scenes.

The name of the member type comes from
the `member_type` entry from `self.typedata`.
That name is then looked up in the ontology's types.

*`Tag.meta`*:
Shortcut property for the metadata `TagSet`.

*`Tag.metadata(self, *, ontology=None, convert=None)`*:
Fetch the metadata information about this specific tag value,
derived through the `ontology` from the tag name and value.
The default `ontology` is `self.ontology`.

For a scalar type (`int`, `float`, `str`) this is the ontology `TagSet`
for `self.value`.

For a sequence (`list`) this is a list of the metadata
for each member.

For a mapping (`dict`) this is mapping of `key->metadata`.

*`Tag.parse(s, offset=0, *, ontology=None, **parse_value_kw)`*:
Parse tag_name[=value] from `s` at `offset`, return `(Tag,post_offset)`.

Parameters:
* `s`: the string to parse
* `offset`: optional offset of the parse start, default `0`
* `ontology`: optional `TagsOntology` to associate with the `Tag`

Other keyword arguments are passed to `Tag.parse_value`.

*`Tag.parse_name(s, offset=0)`*:
Parse a tag name from `s` at `offset`: a dotted identifier.

*`Tag.parse_value(s, offset=0, *, extra_types=None, fallback_parse=None)`*:
Parse a value from `s` at `offset` (default `0`).
Return the value, or `None` on no data.

The optional `extra_types` parameter may be an iterable of
`(type,from_str,to_str)` tuples where `from_str` is a
function which takes a string and returns a Python object
(expected to be an instance of `type`).
The default comes from `cls.EXTRA_TYPES`.
This supports storage of nonJSONable values in text form.

The optional `fallback_parse` parameter
specifies a parse function accepting `(s,offset)`
and returning `(parsed,new_offset)`
where `parsed` is text from `s[offset:]`
and `new_offset` is where the parse stopped.
The default is `cs.lex.get_nonwhite`
to gather nonwhitespace characters,
intended to support *tag_name*`=`*bare_word*
in human edited tag files.

The core syntax for values is JSON;
value text commencing with any of `'"'`, `'['` or `'{'`
is treated as JSON and decoded directly,
leaving the offset at the end of the JSON parse.

Otherwise all the nonwhitespace at this point is collected
as the value text,
leaving the offset at the next whitespace character
or the end of the string.
The text so collected is then tried against the `from_str`
function of each `extra_types`;
the first successful parse is accepted as the value.
If no extra type match,
the text is tried against `int()` and `float()`;
if one of these parses the text and `str()` of the result round trips
to the original text
then that value is used.
Otherwise the text itself is kept as the value.

*`Tag.transcribe_value(value, extra_types=None, json_options=None)`*:
Transcribe `value` for use in `Tag` transcription.

The optional `extra_types` parameter may be an iterable of
`(type,from_str,to_str)` tuples where `to_str` is a
function which takes a string and returns a Python object
(expected to be an instance of `type`).
The default comes from `cls.EXTRA_TYPES`.

If `value` is an instance of `type`
then the `to_str` function is used to transcribe the value
as a `str`, which should not include any whitespace
(because of the implementation of `parse_value`).
If there is no matching `to_str` function,
`cls.JSON_ENCODER.encode` is used to transcribe `value`.

This supports storage of nonJSONable values in text form.

*`Tag.typedef`*:
The defining `TagSet` for this tag's name.

This is how its type is defined,
and is obtained from:
`self.ontology['type.'+self.name]`

Basic `Tag`s often do not need a type definition;
these are only needed for structured tag values
(example: a mapping of cast members)
or when a `Tag` name is an alias for another type
(example: a cast member name might be an `actor`
which in turn might be a `person`).

For example, a `Tag` `colour=blue`
gets its type information from the `type.colour` entry in an ontology;
that entry is just a `TagSet` with relevant information.
- <a name="tag_or_tag_value"></a>`tag_or_tag_value(*da, **dkw)`: A decorator for functions or methods which may be called as:

      func(name[,value])

  or as:

      func(Tag)

  The optional decorator argument `no_self` (default `False`)
  should be supplied for plain functions
  as they have no leading `self` parameter to accomodate.

  Example:

      @tag_or_tag_value
      def add(self, tag_name, value, *, verbose=None):

  This defines a `.add()` method
  which can be called with `name` and `value`
  or with single `Tag`like object
  (something with `.name` and `.value` attributes),
  for example:

      tags = TagSet()
      ....
      tags.add('colour', 'blue')
      ....
      tag = Tag('size', 9)
      tags.add(tag)
- <a name="TagBasedTest"></a>`Class `TagBasedTest(TagBasedTest, TagSetCriterion)``: A test based on a `Tag`.

  Attributes:
  * `spec`: the source text from which this choice was parsed,
    possibly `None`
  * `choice`: the apply/reject flag
  * `tag`: the `Tag` representing the criterion
  * `comparison`: an indication of the test comparison

  The following comparison values are recognised:
  * `None`: test for the presence of the `Tag`
  * `'='`: test that the tag value equals `tag.value`
  * `'<'`: test that the tag value is less than `tag.value`
  * `'<='`: test that the tag value is less than or equal to `tag.value`
  * `'>'`: test that the tag value is greater than `tag.value`
  * `'>='`: test that the tag value is greater than or equal to `tag.value`
  * `'~/'`: test if the tag value as a regexp is present in `tag.value`
  * '~': test if a matching tag value is present in `tag.value`

*`TagBasedTest.by_tag_value(tag_name, tag_value, *, choice=True, comparison='=')`*:
Return a `TagBasedTest` based on a `Tag` or `tag_name,tag_value`.

*`TagBasedTest.match_tagged_entity(self, te: 'TagSet') -> bool`*:
Test against the `Tag`s in `tags`.

*Note*: comparisons when `self.tag.name` is not in `tags`
always return `False` (possibly inverted by `self.choice`).

*`TagBasedTest.parse(s, offset=0, delim=None)`*:
Parse *tag_name*[{`<`|`<=`|'='|'>='|`>`|'~'}*value*]
and return `(dict,offset)`
where the `dict` contains the following keys and values:
* `tag`: a `Tag` embodying the tag name and value
* `comparison`: an indication of the test comparison
- <a name="TagFile"></a>`Class `TagFile(cs.fs.FSPathBasedSingleton, BaseTagSets)``: A reference to a specific file containing tags.

  This manages a mapping of `name` => `TagSet`,
  itself a mapping of tag name => tag value.

*`TagFile.__setitem__(self, name, te)`*:
Set item `name` to `te`.

*`TagFile.get(self, name, default=None)`*:
Get from the tagsets.

*`TagFile.is_modified(self)`*:
Test whether this `TagSet` has been modified.

*`TagFile.keys(self, *, prefix=None)`*:
`tagsets.keys`

If the options `prefix` is supplied,
yield only those keys starting with `prefix`.

*`TagFile.load_tagsets(filepath, ontology, extra_types=None)`*:
Load `filepath` and return `(tagsets,unparsed)`.

The returned `tagsets` are a mapping of `name`=>`tag_name`=>`value`.
The returned `unparsed` is a list of `(lineno,line)`
for lines which failed the parse (excluding the trailing newline).

*`TagFile.names`*:
The names from this `FSTagsTagFile` as a list.

*`TagFile.parse_tags_line(line, ontology=None, verbose=None, extra_types=None) -> Tuple[str, cs.tagset.TagSet]`*:
Parse a "name tags..." line as from a `.fstags` file,
return `(name,TagSet)`.

*`TagFile.save(self, extra_types=None, prune=False)`*:
Save the tag map to the tag file if modified.

*`TagFile.save_tagsets(filepath, tagsets, unparsed, extra_types=None, prune=False)`*:
Save `tagsets` and `unparsed` to `filepath`.

This method will create the required intermediate directories
if missing.

This method *does not* clear the `.modified` attribute of the `TagSet`s
because it does not know it is saving to the `Tagset`'s primary location.

*`TagFile.startup_shutdown(self)`*:
Save the tagsets if modified.

*`TagFile.tags_line(name, tags, extra_types=None, prune=False)`*:
Transcribe a `name` and its `tags` for use as a `.fstags` file line.

*`TagFile.tagsets`*:
The tag map from the tag file,
a mapping of name=>`TagSet`.

This is loaded on demand.

*`TagFile.update(self, name, tags, *, prefix=None, verbose=None)`*:
Update the tags for `name` from the supplied `tags`
as for `Tagset.update`.
- <a name="TagsCommandMixin"></a>`Class `TagsCommandMixin``: Utility methods for `cs.cmdutils.BaseCommand` classes working with tags.

  Optional subclass attributes:
  * `TAGSET_CRITERION_CLASS`: a `TagSetCriterion` duck class,
    default `TagSetCriterion`.
    For example, `cs.sqltags` has a subclass
    with an `.extend_query` method for computing an SQL JOIN
    used in searching for tagged entities.

*`TagsCommandMixin.TagAddRemove`*

*`TagsCommandMixin.parse_tag_addremove(arg, offset=0)`*:
Parse `arg` as an add/remove tag specification
of the form [`-`]*tag_name*[`=`*value*].
Return `(remove,Tag)`.

Examples:

    >>> TagsCommandMixin.parse_tag_addremove('a')
    TagAddRemove(remove=False, tag=Tag(name='a',value=None))
    >>> TagsCommandMixin.parse_tag_addremove('-a')
    TagAddRemove(remove=True, tag=Tag(name='a',value=None))
    >>> TagsCommandMixin.parse_tag_addremove('a=1')
    TagAddRemove(remove=False, tag=Tag(name='a',value=1))
    >>> TagsCommandMixin.parse_tag_addremove('-a=1')
    TagAddRemove(remove=True, tag=Tag(name='a',value=1))
    >>> TagsCommandMixin.parse_tag_addremove('-a="foo bah"')
    TagAddRemove(remove=True, tag=Tag(name='a',value='foo bah'))
    >>> TagsCommandMixin.parse_tag_addremove('-a=foo bah')
    TagAddRemove(remove=True, tag=Tag(name='a',value='foo bah'))

*`TagsCommandMixin.parse_tag_choices(argv)`*:
Parse `argv` as an iterable of [`!`]*tag_name*[`=`*tag_value`] `Tag`
additions/deletions.

*`TagsCommandMixin.parse_tagset_criteria(argv, tag_based_test_class=None)`*:
Parse tag specifications from `argv` until an unparseable item is found.
Return `(criteria,argv)`
where `criteria` is a list of the parsed criteria
and `argv` is the remaining unparsed items.

Each item is parsed via
`cls.parse_tagset_criterion(item,tag_based_test_class)`.

*`TagsCommandMixin.parse_tagset_criterion(arg, tag_based_test_class=None)`*:
Parse `arg` as a tag specification
and return a `tag_based_test_class` instance
via its `.from_str` factory method.
Raises `ValueError` in a misparse.
The default `tag_based_test_class`
comes from `cls.TAGSET_CRITERION_CLASS`,
which itself defaults to class `TagSetCriterion`.

The default `TagSetCriterion.from_str` recognises:
* `-`*tag_name*: a negative requirement for *tag_name*
* *tag_name*[`=`*value*]: a positive requirement for a *tag_name*
  with optional *value*.
- <a name="TagSet"></a>`Class `TagSet(builtins.dict, cs.dateutils.UNIXTimeMixin, cs.lex.FormatableMixin, cs.mappings.AttrableMappingMixin, cs.deco.Promotable)``: A setlike class collection of `Tag`s.

  This actually subclasses `dict`, so a `TagSet` is a direct
  mapping of tag names to values.
  It accepts attribute access to simple tag values when they
  do not conflict with the class methods;
  the reliable method is normal item access.

  *NOTE*: iteration yields `Tag`s, not dict keys.

  Also note that all the `Tags` from a `TagSet`
  share its ontology.

  Subclasses should override the `set` and `discard` methods;
  the `dict` and mapping methods
  are defined in terms of these two basic operations.

  `TagSet`s have a few special properties:
  * `id`: a domain specific identifier;
    this may reasonably be `None` for entities
    not associated with database rows;
    the `cs.sqltags.SQLTags` class associates this
    with the database row id.
  * `name`: the entity's name;
    a read only alias for the `'name'` `Tag`.
    The `cs.sqltags.SQLTags` class defines "log entries"
    as `TagSet`s with no `name`.
  * `unixtime`: a UNIX timestamp,
    a `float` holding seconds since the UNIX epoch
    (midnight, 1 January 1970 UTC).
    This is typically the row creation time
    for entities associated with database rows,
    but usually the event time for `TagSet`s describing an event.

  Because ` TagSet` subclasses `cs.mappings.AttrableMappingMixin`
  you can also access tag values as attributes
  *provided* that they do not conflict with instance attributes
  or class methods or properties.

*`TagSet.__init__(self, *a, _id=None, _ontology=None, **kw)`*:
Initialise the `TagSet`.

Parameters:
* positional parameters initialise the `dict`
  and are passed to `dict.__init__`
* `_id`: optional identity value for databaselike implementations
* `_ontology`: optional `TagsOntology to use for this `TagSet`
* other alphabetic keyword parameters are also used to initialise the
  `dict` and are passed to `dict.__init__`

*`TagSet.__contains__(self, tag)`*:
Test for a tag being in this `TagSet`.

If the supplied `tag` is a `str` then this test
is for the presence of `tag` in the keys.

Otherwise,
for each tag `T` in the tagset
test `T.matches(tag)` and return `True` on success.
The default `Tag.matches` method compares the tag name
and if the same,
returns true if `tag.value` is `None` (basic "is the tag present" test)
and otherwise true if `tag.value==T.value` (basic "tag value equality" test).

Otherwise return `False`.

*`TagSet.__getattr__(self, attr)`*:
Support access to dotted name attributes.

The following attribute accesses are supported:
- an attrbute from a superclass
- a `Tag` whose name is `attr`; return its value
- the value of `self.auto_infer(attr)` if that does not raise `ValueError`
- if `self.ontology`, try {type}_{field} and {type}_{field}s
- otherwise return `self.subtags(attr)` to allow access to dotted tags,
  provided any existing tags start with "attr."

If this `TagSet` has an ontology
and `attr looks like *typename*`_`*fieldname*
and *typename* is a key,
look up the metadata for the `Tag` value
and return the metadata's *fieldname* key.
This also works for plural values.

For example if a `TagSet` has the tag `artists=["fred","joe"]`
and `attr` is `artist_names`
then the metadata entries for `"fred"` and `"joe"` are looked up
and their `artist_name` tags are returned,
perhaps resulting in the list
`["Fred Thing","Joe Thang"]`.

If there are keys commencing with `attr+'.'`
then this returns a view of those keys
so that a subsequent attribute access can access one of those keys.

Otherwise, a superclass attribute access is performed.

Example of dotted access to tags like `c.x`:

    >>> tags=TagSet(a=1,b=2)
    >>> tags.a
    1
    >>> tags.c
    Traceback (most recent call last):
        ...
    AttributeError: TagSet.c
    >>> tags['c.z']=9
    >>> tags['c.x']=8
    >>> tags
    TagSet:{'a': 1, 'b': 2, 'c.z': 9, 'c.x': 8}
    >>> tags.c
    TagSetPrefixView:c.{'z': 9, 'x': 8}
    >>> tags.c.z
    9

However, this is not supported when there is a tag named `'c'`
because `tags.c` has to return the `'c'` tag value:

    >>> tags=TagSet(a=1,b=2,c=3)
    >>> tags.a
    1
    >>> tags.c
    3
    >>> tags['c.z']=9
    >>> tags.c.z
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    AttributeError: 'int' object has no attribute 'z'

*`TagSet.__iter__(self, prefix=None, ontology=None)`*:
Yield the tag data as `Tag`s.

*`TagSet.__setattr__(self, attr, value)`*:
Attribute based `Tag` access.

If `attr` is private or is in `self.__dict__` then that is updated,
supporting "normal" attributes set on the instance.
Otherwise the `Tag` named `attr` is set to `value`.

The `__init__` methods of subclasses should do something like this
(from `TagSet.__init__`)
to set up the ordinary instance attributes
which are not to be treated as `Tag`s:

    self.__dict__.update(id=_id, ontology=_ontology, modified=False)

*`TagSet.__str__(self)`*:
The `TagSet` suitable for writing to a tag file.

*`TagSet.add(self, tag_name, value, **kw)`*:
Adding a `Tag` calls the class `set()` method.

*`TagSet.as_dict(self)`*:
Return a `dict` mapping tag name to value.

*`TagSet.as_tags(self, prefix=None, ontology=None)`*:
Yield the tag data as `Tag`s.

*`TagSet.auto`*:
The automatic namespace.
Here we can refer to dotted tag names directly as attributes.

*`TagSet.auto_infer(self, attr)`*:
The default inference implementation.

This should return a value if `attr` is inferrable
and raise `ValueError` if not.

The default implementation returns the direct tag value for `attr`
if present.

*`TagSet.csvrow`*:
This `TagSet` as a list useful to a `csv.writer`.
The inverse of `from_csvrow`.

*`TagSet.discard(self, tag_name, value, *, verbose=None)`*:
Discard the tag matching `(tag_name,value)`.
Return a `Tag` with the old value,
or `None` if there was no matching tag.

Note that if the tag value is `None`
then the tag is unconditionally discarded.
Otherwise the tag is only discarded
if its value matches.

*`TagSet.dump(self, keys=None, *, preindent=None, file=None, **pf_kwargs)`*:
Dump a `TagSet` in multiline format.

Parameters:
* `keys`: optional iterable of `Tag` names to print
* `file`: optional keyword parameter specifying the output filelike 
  object; the default is `sys.stdout`.
* `preindent`: optional leading indentation for the entire dump,
  either a `str` or an `int` indicating a number of spaces
Other keyword arguments are passed to `pprint.pformat`.

*`TagSet.edit(self, editor=None, verbose=None, comments=())`*:
Edit this `TagSet`.

*`TagSet.edit_tagsets(tes, editor=None, verbose=True)`*:
Edit a collection of `TagSet`s.
Return a list of `(old_name,new_name,TagSet)` for those which were modified.

This function supports modifying both `name` and `Tag`s.
The `Tag`s are updated directly.
The changed names are returning in the `old_name,new_name` above.

The collection `tes` may be either a mapping of name/key
to `TagSet` or an iterable of `TagSets`. If the latter, a
mapping is made based on `te.name or te.id` for each item
`te` in the iterable.

*`TagSet.from_csvrow(csvrow)`*:
Construct a `TagSet` from a CSV row like that from
`TagSet.csvrow`, being `unixtime,id,name,tag[,tag,...]`.

*`TagSet.from_ini(f, section: str, missing_ok=False)`*:
Load a `TagSet` from a section of a `.ini` file.

Parameters:
* `f`: the `.ini` format file to read;
  an iterable of lines (eg a file object)
  or the name of a file to open
* `section`: the name of the config section
  from which to load the `TagSet`
* `missing_ok`: optional flag, default `False`;
  if true a missing file will return an empty `TagSet`
  instead of raising `FileNotFoundError`

*`TagSet.from_line(s, offset: int, **from_str_kw)`*:
Obsolete form of `TagSet.from_str`.

*`TagSet.from_str(tags_s, *, ontology=None, extra_types=None, verbose=None)`*:
Create a new `TagSet` from a line of text.
The line consists of a whitespace separated list of `Tag`s.

This is the inverse of `TagSet.__str__`.

*`TagSet.from_tags(tags, _id=None, _ontology=None)`*:
Make a `TagSet` from an iterable of `Tag`s.

*`TagSet.get_arg_name(self, field_name)`*:
Override for `FormattableMixin.get_arg_name`:
return the leading dotted identifier,
which represents a tag or tag prefix.

*`TagSet.get_value(self, arg_name, a, kw)`*:
Override for `FormattableMixin.get_value`:
look up `arg_name` in `kw`, return a value.

The value is obtained as follows:
* `kw[arg_name]`: the `Tag` named `arg_name` if present
* `kw.get_format_attribute(arg_name)`:
  a formattable attribute named `arg_name`
otherwise raise `KeyError` if `self.format_mode.strict`
otherwise return the placeholder string `'{'+arg_name+'}'`.

*`TagSet.is_stale(self, max_age=None)`*:
Test whether this `TagSet` is stale
i.e. the time since `self.last_updated` UNIX time exceeds `max_age` seconds
(default from `self.STALE_AGE`).

This is a convenience function for `TagSet`s which cache external data.

*`TagSet.name`*:
Read only `name` property, `None` if there is no `'name'` tag.

*`TagSet.save_as_ini(self, f, section: str, config=None)`*:
Save this `TagSet` to the config file `f` as `section`.

If `f` is a string, read an existing config from that file
and update the section.

*`TagSet.set(self, tag_name, value, *, verbose=None)`*:
Set `self[tag_name]=value`.
If `verbose`, emit an info message if this changes the previous value.

*`TagSet.set_from(self, other, verbose=None)`*:
Completely replace the values in `self`
with the values from `other`,
a `TagSet` or any other `name`=>`value` dict.

This has the feature of logging changes
by calling `.set` and `.discard` to effect the changes.

*`TagSet.