Package: cs.mappings
Facilities for mappings and objects associated with mappings.Latest release 20231129:
AttrableMappingMixin: look up ATTRABLE_MAPPING_DEFAULT on the class, not the instance.In particularnamed_column_tuple(column_names),
a function returning a factory
for namedtuples subclasses derived from the supplied column names,
andnamed_column_tuples(rows),
a function returning a namedtuple factory and an iterable of instances
containing the row data.
These are used by thecsv_importandxl_importfunctions
fromcs.csvutils.Functionattrable(o)Likejsonable, returnowithdictsreplaced byAttrableMappings.ClassAttrableMapping(builtins.dict, AttrableMappingMixin)Adictsubclass usingAttrableMappingMixin.ClassAttrableMappingMixinProvides a__getattr__which accesses the mapping value.ClassAttributableList(builtins.list)AnAttributableListmaps unimplemented attributes
onto the list members and returns you a newAttributableListwith the results, ready for a further dereference.Example:>>> class C(object):
...   def __init__(self, i):
...     self.i = i
>>> Cs = [ C(1), C(2), C(3) ]
>>> AL = AttributableList( Cs )
>>> print(AL.i)
[1, 2, 3]MethodAttributableList.__init__(self, initlist=None, strict=False):
Initialise the list.The optional parameterinitlistinitialises the list
as for a normal list.The optional parameterstrict, if true, causes list elements
lacking the attribute to raise an AttributeError. If false,
list elements without the attribute are omitted from the results.Functioncolumn_name_to_identifier(column_name, snake_case=False)The default function used to convert raw column names innamed_row_tuple, for example from a CSV file, into Python
indentifiers.Ifsnake_caseis true (defaultFalse) produce snake cased
identifiers instead of merely lowercased identifiers.
This means that something like 'redLines' will becomered_linesinstead ofredlines.Functiondicts_to_namedtuples(dicts, class_name, keys=None)Scan an iterable ofdicts,
yield a sequence ofnamedtuples derived from them.Parameters:dicts: thedicts to scan and convert, an iterableclass_name: the name for the newnamedtupleclasskeys: optional iterable ofdictkeys of interest;
if omitted then thedictsare scanned in order to learn the keysNote that ifkeysis not specified
this generator prescans thedictsin order to learn their keys.
As a consequence, all thedictswill be kept in memory
and nonamedtuples will be yielded until after that prescan completes.ClassFallbackDict(collections.defaultdict, builtins.dict)A dictlike object that inherits from another dictlike object;
this is a convenience subclass ofdefaultdict.ClassIndexedMapping(IndexedSetMixin)Interface to a mapping withIndexedSetMixinstyle.by_*attributes.MethodIndexedMapping.__init__(self, mapping=None, pk='id'):
Initialise theIndexedMapping.Parameters:mapping: the mapping to wrap; a newdictwill be made if not specifiedpk: the primary key of the mapping, default'id'ClassIndexedSetMixinA base mixin to provide.by_* attributes
which index records from an autoloaded backing store,
which might be a file or might be another related data structure.
The records are themselves key->value mappings, such asdicts.The primary key name is provided by the.IndexedSetMixin__pkclass attribute, to be provided by subclasses.Note that this mixin keeps the entire loadable mapping in memory.Note that this does not see subsequent changes to loaded records
i.e. changing the value of some record[k]
does not update the index associated with the .by_k attribute.Subclasses must provide the following attributes and methods:IndexedSetMixin__pk: the name of the primary key;
it is an error for multiple records to have the same primary keyscan: a generator method to scan the backing store
and yield records, used for the inital load of the mappingadd_backend(record): add a new record to the backing store;
this is called from the.add(record)method
after indexing to persist the record in the backing storeSeeUUIDNDJSONMappingandUUIDedDictfor an example subclass
indexing records from a newline delimited JSON file.ClassJSONableMappingMixinProvide.from_json(),.as_json()and.append_ndjson()methods,
and__str__=as_jsonand a__repr__.ClassMappingChainA mapping interface to a sequence of mappings.It does not support__setitem__at present;
that is expected to be managed via the backing mappings.MethodMappingChain.__init__(self, mappings=None, get_mappings=None):
Initialise the MappingChain.Parameters:mappings: initial sequence of mappings, default None.get_mappings: callable to obtain the initial sequence ofExactly one ofmappingsorget_mappingsmust be provided.ClassMethodicalList(AttributableList, builtins.list)A MethodicalList subclasses a list and maps unimplemented attributes
into a callable which calls the corresponding method on each list members
and returns you a newMethodicalListwith the results, ready for a
further dereference.Example:>>> n = 1
>>> class C(object):
...   def __init__(self):
...     global n
...     self.n = n
...     n += 1
...   def x(self):
...     return self.n
...
>>> Cs=[ C(), C(), C() ]
>>> ML = MethodicalList( Cs )
>>> print(ML.x())
[1, 2, 3]MethodMethodicalList.__init__(self, initlist=None, strict=False):
Initialise the list.The optional parameterinitlistinitialises the list
as for a normal list.The optional parameterstrict, if true, causes list elements
lacking the attribute to raise an AttributeError. If false,
list elements without the attribute are omitted from the results.Functionnamed_column_tuples(rows, class_name=None, column_names=None, computed=None, preprocess=None, mixin=None, snake_case=False)Process an iterable of data rows, usually with the first row being
column names.
Return a generatednamedtuplefactory (the row class)
and an iterable of instances of the namedtuples for each row.Parameters:rows: an iterable of rows, each an iterable of data values.class_name: option class name for the namedtuple classcolumn_names: optional iterable of column names used as the basis for
the namedtuple. If this is not provided then the first row fromrowsis taken to be the column names.computed: optional mapping of str to functions ofselfpreprocess: optional callable to modify CSV rows before
they are converted into the namedtuple.  It receives a context
object an the data row.
It should return the row (possibly modified), orNoneto drop the
row.mixin: an optional mixin class for the generatednamedtuplesubclass
to provide extra methods or propertiesThe context object passed topreprocesshas the following attributes:.cls: the generated namedtuple subclass;
this is useful for obtaining things like the column names
or column indices;
this isNonewhen preprocessing the header row, if any.index: attribute with the row's enumeration, which counts from0.previous: the previously accepted row'snamedtuple,
orNoneif there is no previous row;
this is useful for differencingRows may be flat iterables in the same order as the column names
or mappings keyed on the column names.If the column names contain empty strings they are dropped
and the corresponding data row entries are also dropped. This
is very common with spreadsheet exports with unused padding
columns.Typical human readable column headings, also common in
speadsheet exports, are lowercased and have runs of whitespace
or punctuation turned into single underscores; trailing
underscores then get dropped.Basic example:>>> data1 = [
...   ('a', 'b', 'c'),
...   (1, 11, "one"),
...   (2, 22, "two"),
... ]
>>> rowtype, rows = named_column_tuples(data1)
>>> print(list(rows))
[NamedRow(a=1, b=11, c='one'), NamedRow(a=2, b=22, c='two')]Human readable column headings:>>> data1 = [
...   ('Index', 'Value Found', 'Descriptive Text'),
...   (1, 11, "one"),
...   (2, 22, "two"),
... ]
>>> rowtype, rows = named_column_tuples(data1)
>>> print(list(rows))
[NamedRow(index=1, value_found=11, descriptive_text='one'), NamedRow(index=2, value_found=22, descriptive_text='two')]Rows which are mappings:>>> data1 = [
...   ('a', 'b', 'c'),
...   (1, 11, "one"),
...   {'a': 2, 'c': "two", 'b': 22},
... ]
>>> rowtype, rows = named_column_tuples(data1)
>>> print(list(rows))
[NamedRow(a=1, b=11, c='one'), NamedRow(a=2, b=22, c='two')]CSV export with unused padding columns:>>> data1 = [
...   ('a', 'b', 'c', '', ''),
...   (1, 11, "one"),
...   {'a': 2, 'c': "two", 'b': 22},
...   [3, 11, "three", '', 'dropped'],
... ]
>>> rowtype, rows = named_column_tuples(data1, 'CSV_Row')
>>> print(list(rows))
[CSV_Row(a=1, b=11, c='one'), CSV_Row(a=2, b=22, c='two'), CSV_Row(a=3, b=11, c='three')]A mixin class providing atest1method and atest2property:>>> class Mixin(object):
...   def test1(self):
...     return "test1"
...   @property
...   def test2(self):
...     return "test2"
>>> data1 = [
...   ('a', 'b', 'c'),
...   (1, 11, "one"),
...   {'a': 2, 'c': "two", 'b': 22},
... ]
>>> rowtype, rows = named_column_tuples(data1, mixin=Mixin)
>>> rows = list(rows)
>>> rows[0].test1()
'test1'
>>> rows[0].test2
'test2'Functionnamed_row_tuple(*column_names, class_name=None, computed=None, column_map=None, snake_case=False, mixin=None)Return anamedtuplesubclass factory derived fromcolumn_names.
The primary use case is using the header row of a spreadsheet
to key the data from the subsequent rows.Parameters:column_names: an iterable ofstr, such as the heading columns
of a CSV exportclass_name: optional keyword parameter specifying the class namecomputed: optional keyword parameter providing a mapping
ofstrto functions ofself; these strings are available
via__getitem__mixin: an optional mixin class for the generated namedtuple subclass
to provide extra methods or propertiesThe tuple's attributes are computed by converting all runs
of nonalphanumerics
(as defined by theremodule's "\W" sequence)
to an underscore, lowercasing and then stripping
leading and trailing underscores.In addition to the normal numeric indices, the tuple may
also be indexed by the attribute names or the column names.The new class has the following additional attributes:attributes_: the attribute names of each tuple in ordernames_: the originating name stringsname_attributes_: the computed attribute names corresponding to thenames; there may be empty strings in this listattr_of_: a mapping of column name to attribute namename_of_: a mapping of attribute name to column nameindex_of_: a mapping of column names and attributes their tuple indicesExamples:>>> T = named_row_tuple('Column 1', '', 'Column 3', ' Column 4', 'Column 5 ', '', '', class_name='Example')
>>> T.attributes_
['column_1', 'column_3', 'column_4', 'column_5']
>>> row = T('val1', 'dropped', 'val3', 4, 5, 6, 7)
>>> row
Example(column_1='val1', column_3='val3', column_4=4, column_5=5)ClassPrefixedMappingProxy(RemappedMappingProxy)A proxy for another mapping
operating on keys commencing with a prefix.ClassRemappedMappingProxyA proxy for another mapping
with translation functions between the external keys
and the keys used inside the other mapping.Example:>>> proxy = RemappedMappingProxy(
...   {},
...   lambda key: 'prefix.' + key,
...   lambda subkey: cutprefix('prefix.', subkey))
>>> proxy['key'] = 1
>>> proxy['key']
1
>>> proxy.mapping
{'prefix.key': 1}
>>> list(proxy.keys())
['key']
>>> proxy.subkey('key')
'prefix.key'
>>> proxy.key('prefix.key')
'key'ClassSeenSetA set-like collection with optional backing store file.ClassSeqMapUC_AttrsA wrapper for a mapping from keys
(matching the regular expression^[A-Z][A-Z_0-9]*$)
to tuples.Attributes matching such a key return the first element
of the sequence (and requires the sequence to have exactly on element).
An attributeFOOsorFOOes(ending in a literal 's' or 'es', a plural)
returns the sequence (FOOmust be a key of the mapping).ClassStackableValuesA collection of named stackable values with the latest value
available as an attribute.DEPRECATED: I now recommend mycs.context.stackattrscontext
manager for most uses; it may be applied to any object instead of
requiring use of this class.Note that names conflicting with methods are not available
as attributes and must be accessed via__getitem__.
As a matter of practice, in addition to the mapping methods,
avoid names which are verbs or which begin with an underscore.Example:>>> S = StackableValues()
>>> print(S)
StackableValues()
>>> S.push('x', 1)
>>> print(S)
StackableValues(x=1)
>>> print(S.x)
1
>>> S.push('x', 2)
1
>>> print(S.x)
2
>>> S.x = 3
>>> print(S.x)
3
>>> S.pop('x')
3
>>> print(S.x)
1
>>> with S.stack(x=4):
...   print(S.x)
...
4
>>> print(S.x)
1
>>> S.update(x=5)
{'x': 1}ClassStrKeyedDefaultDict(TypedKeyMixin, collections.defaultdict, builtins.dict)Subclass ofdefaultdictwhich ensures that its
keys are of typestrusingTypedKeyMixin.MethodStrKeyedDefaultDict.__init__(self, *a, **kw):
Initialise theTypedKeyDict. The first positional parameter
is the type for keys.ClassStrKeyedDict(TypedKeyMixin, builtins.dict)Subclass ofdictwhich ensures that its
keys are of typestrusingTypedKeyMixin.MethodStrKeyedDict.__init__(self, *a, **kw):
Initialise theTypedKeyDict. The first positional parameter
is the type for keys.FunctionTypedKeyClass(key_type, superclass, name=None)Factory to create a new mapping class subclassing(TypedKeyMixin,superclass)which checks that keys are of typekey_type.ClassTypedKeyMixinA mixin to check that the keys of a mapping are of a particular type.The triggering use case is the constant UUID vs str(UUID) tension
in a lot of database code.ClassUC_Sequence(builtins.list)A tuple-of-nodes on which.ATTRsindirection can be done,
yielding another tuple-of-nodes or tuple-of-values.MethodUC_Sequence.__init__(self, Ns):
Initialise from an iterable sequence.ClassUUIDedDict(builtins.dict, JSONableMappingMixin, AttrableMappingMixin)A handydictsubtype providing the basis for mapping classes
indexed byUUIDs.The'uuid'attribute is always aUUIDinstance.MethodUUIDedDict.__init__(self, _d=None, **kw):
Initialise theUUIDedDict,
generating a'uuid'key value if omitted.ClassUUIDKeyedDefaultDict(TypedKeyMixin, collections.defaultdict, builtins.dict)Subclass ofdefaultdictwhich ensures that its
keys are of typeUUIDusingTypedKeyMixin.MethodUUIDKeyedDefaultDict.__init__(self, *a, **kw):
Initialise theTypedKeyDict. The first positional parameter
is the type for keys.ClassUUIDKeyedDict(TypedKeyMixin, builtins.dict)Subclass ofdictwhich ensures that its
keys are of typeUUIDusingTypedKeyMixin.MethodUUIDKeyedDict.__init__(self, *a, **kw):
Initialise theTypedKeyDict. The first positional parameter
is the type for keys.Release LogRelease 20231129:
AttrableMappingMixin: look up ATTRABLE_MAPPING_DEFAULT on the class, not the instance.Release 20230612:AttrableMappingMixin.getattr: fast path the check for "ATTRABLE_MAPPING_DEFAULT", fixes unbound recursion.New attrable() function returning an object with dicts transmuted to AttrableMapping instances.Release 20220912.4:TypedKeyMixin: add .get() and .setdefault().Provide names for UUIDKeyedDict, StrKeyedDefaultDict, UUIDKeyedDefaultDict.Release 20220912.3:New TypedKeyClass class factory.Redo StrKeyedDict and UUIDKeyedDict using TypedKeyClass.New StrKeyedDefaultDict and UUIDKeyedDefaultDict convenience classes.Release 20220912.2:
TypedKeyMixin: fix another typo.Release 20220912.1:
TypedKeyMixin: remove debug, bugfix super() calls.Release 20220912:New TypedKeyMixin to check that the keys of a mapping are of a particular type.New TypedKeyDict(TypedKeyMixin,dict) subclass.New StrKeyedDict and UUIDKeyedDict factories.Release 20220626:
Expose the default column name mapping function of named_row_tuple as column_name_to_identifier for reuse.Release 20220605:named_row_tuple et al: plumb a new optional snake_case parameter to snake case mixed case attribute names.Drop Python 2 support and the cs.py3 shims.Release 20220318:
Bump cs.sharedfile requirement to get an import fix.Release 20211208:PrefixedMappingProxy: swap to_subkey/from_subkey prefix/unprefix actions, were backwards.PrefixedMappingProxy: make the key and subkey conversion methods public static methods for reuse.Assorted minor internal changes.Release 20210906:
New RemappedMappingProxy with general subkey(key) and key(subkey) methods.Release 20210717:New IndexedMapping: wrapper for another mapping providing LoadableMappingMixin stype .by_* attributes.Rename LoadableMappingMixin to IndexedSetMixin and make it abstract, rename .scan_mapping to .scan, .add_to_mapping to .add etc.Release 20210306:
StackableValues: fix typo, make deprecation overt.Release 20210123:
AttrableMappingMixin.getattr: some bugfixes.Release 20201228:
New PrefixedMappingProxy presenting the keys of another mapping commencing with a prefix.Release 20201102:StackableValues is obsolete, add recommendation for cs.context.stackattrs to the docstring.New AttrableMappingMixin with agetattrwhich looks up unknown attributes as keys.New JSONableMappingMixin with methods for JSON actions: from_json, as_json, append_ndjson and astrandrepr.New LoadableMappingMixin to load .by_* attributes on demand.New AttrableMapping(dict, AttrableMappingMixin).Release 20200130:
New dicts_to_namedtuples function to yield namedtuples from an iterable of dicts.Release 20191120:
named_row_tuple: support None in a column name, as from Excel unfilled heading row entriesRelease 20190617:StackableValues.push now returns the previous value.StackableValues.update has a signature like dict.update.StackableValues.pop removes entries when their stack becomes empty.StackableValues.stack: clean implementation of save/restore.StackableValues: avoid infinite recursion through ._fallback.StackableValues.keys now returns a list of the nonempty keys.Update doctests.Release 20190103:
Documentation update.Release 20181231:Bugfix for mapping of column names to row indices.New subclass._fallback method for when a stack is empty.Release 20180720:
Initial PyPI release specificly for named_column_tuple and named_column_tuples.