Package: oarepo-taxonomies
oarepo-taxonomiesWrapper that connect Flask-Taxonomies with Invenio.InstallationThe package is installed classically via the PyPi repository:pip install oarepo-taxonomiesThe database is initialized using the management task:invenio taxonomies initConfigSearch serializerTaxonomic facets can only be used if a wrapper (taxonomy_enabled_search) is used to wrap your search serializer in
RECORDS_REST_ENDPOINTS. Wrapper takes two positional argument. First is search_serializer and second is enabled
taxonomy. Now you can usetaxonomy_term_facet.RECORDS_REST_ENDPOINTS={...'search_serializers':{'application/json':taxonomy_enabled_search(json_search,taxonomy_aggs=["degreeGrantor"],fallback_language),},...}UsageAll functionality is provided by flask-taxonomies. For more details see:flask-taxonomies.In addition, this package adds the ability to import and export taxonomies using Excel files (* .xlsx)
and can dereference a reference to a taxonomy in an invenio record.Import from ExcelImporting from Excel is handled by the management task:invenio taxonomies import [OPTIONS] TAXONOMY_FILEOptions:--int TEXT--str TEXT--bool TEXT--drop / --no-drop--helpwhere:TAXONOMY FILEis path to the xlsx file (older xls file is not supported)--int, --str, --booloptions are repeatable options and determine data type--drop/--no-dropSpecifies whether the old taxonomy should be removed from the database when we import a
taxonomy with the sametaxonomy code.Structure of Excel fileBlocksExcel must contain two blocks. The first block contains taxonomy information and must contain one mandatory code column
(taxonomy identifier). Indeed, it can contain other user data (eg. title or description).The second block must be
separated from the first by a blank line and must contain two mandatory columns,levelandslug, in exactly
that order. The other columns are optional.Nested JSONTaxonomies are internally represented as JSON, which can be nested. Excel spreadsheet is inherently linear and can not
store nested data. However, oarepo-taxonomies support nested JSON. Each value in a nested JSON has its own unique
address. Each JSON level is separated by an underscore, so each branched JSON can be transformed to linear as follows.Nested:{"a":1,"b":2,"c":[{"d":[2,3,4],"e":[{"f":1,"g":2}]}]}Linear:{"a":1,"b":2,"c_0_d_0":2,"c_0_d_1":3,"c_0_d_2":4,"c_0_e_0_f":1,"c_0_e_0_g":2}According to the same pattern, headings can be created in Excel and the data is transformed into a nested form.Level orderTaxonomies are tree structures that are also not linear and cannot be transferred to an Excel spreadsheet environment.
Therefore, the sort order goes from root to the lowest child. Root (Taxonomy) -> level 1 first child - ... last
level all children, level 1 second offspring ... etc.Excel examplecodetitle_cstitle_encitiesMěstaCitieslevelslugtitle_cstitle_en1euEvropaEurope2czČeskoCzechia3prgPrahaPrague3brnBrnoBrno2deNěmeckoGermany3berBerlínBerlin3munMnichovMunich2gbVelká BritánieUnited Kingdom3lonLondýnLondon3manManchesterManchesterThe resulting json for the taxonomy will take the following form:{"code":"cities","title":{"cs":"Města","en":"Citites"}}and for individual Taxonomy Term:{"code":"Praha","title":{"cs":"Praha","en":"Prague"}}and tree structure:cities  
└-eu  
  |--cz  
  |  |--prg  
  |  └--brn  
  |--de  
  |  |--ber   
  |  └--mun  
  └--gb  
     |--lon   
     └--manExport to ExcelExcel export is created using a management taskinvenio taxonomies export TAXONOMY_CODE.An xlsx and csv file is created in the current folder where the task was run.MarshmallowThe Marshmallow module serialize Taxonomy and dereference reference from links/self.
The module provides the Marshmallow fieldTaxonomyFieldand schemaTaxonomySchema,
which can be freely used in the user schema.
TaxonomyField/Schema receives any user data and checks if the user data is JSON/dict, string or list.The output format of serialized taxonomies is the Taxonomic List, which contains ancestors in addition to the taxonomy
itself. The order of taxonomy is from the parent term to the finite element of the taxonomy. For taxonomy reason, the
serialization is opinionated. Example of taxonomy serialization is following:[{
        'is_ancestor': true,
        'links': {'self': 'http://127.0.0.1:5000/2.0/taxonomies/test_taxonomy/a'},
        'test': 'extra_data'
    },
        {
            'created_at': '2014-08-11T05:26:03.869245',
            'email': 'ken@yahoo.com',
            'is_ancestor': false,
            'links': {
                'parent': 'http://127.0.0.1:5000/2.0/taxonomies/test_taxonomy/a',
                'self': 'http://127.0.0.1:5000/2.0/taxonomies/test_taxonomy/a/b'
            },
            'name': 'Ken',
            'test': 'extra_data'
        }]Taxonomy representation can be changed in config file (e.g.: invenio.cfg). For more details
please seeFlask-Taxonomies.This library use predefinded config that is located inconfig.py:FLASK_TAXONOMIES_REPRESENTATION={"taxonomy":{'include':[INCLUDE_DATA,INCLUDE_ANCESTORS,INCLUDE_URL,INCLUDE_SELF,INCLUDE_ANCESTOR_LIST,INCLUDE_ANCESTOR_TAG,INCLUDE_PARENT],'exclude':[],'select':None,'options':{}}}There are two ways to use TaxonomyField.The input format is a dictionary or text string containing a link to the taxonomy.dictionary:
The dictionary must contain the nested dictionary with namelinks, which containsself.string: Any text that contains a url to the taxonomy.The input format is list of ancestors, where last is the referenced taxonomy.dictionaryfrommarshmallowimportSchemafromoarepo_taxonomies.marshmallowimportTaxonomyField# custom schemaclassTestSchema(Schema):field=TaxonomyField()# taxonomy dictrandom_user_taxonomy={"created_at":"2014-08-11T05:26:03.869245","email":"ken@yahoo.com","name":"Ken","links":{"self":"http://127.0.0.1:5000/2.0/taxonomies/test_taxonomy/a/b"}}# record dictdata={"field":random_user_taxonomy}schema=TestSchema()result=schema.load(data)assertresult=={'field':[{'is_ancestor':True,'links':{'self':'http://127.0.0.1:5000/2.0/taxonomies/test_taxonomy/a'},'test':'extra_data'},{'created_at':'2014-08-11T05:26:03.869245','email':'ken@yahoo.com','is_ancestor':False,'links':{'parent':'http://127.0.0.1:5000/2.0/taxonomies/test_taxonomy/a','self':'http://127.0.0.1:5000/2.0/taxonomies/test_taxonomy/a/b'},'name':'Ken','test':'extra_data'}]}stringfrommarshmallowimportSchemafromoarepo_taxonomies.marshmallowimportTaxonomyField# custom schemaclassTestSchema(Schema):field=TaxonomyField()# taxonomy reference as any string with urlrandom_user_taxonomy="bla bla http://127.0.0.1:5000/2.0/taxonomies/test_taxonomy/a/b"# record dictdata={"field":random_user_taxonomy}schema=TestSchema()result=schema.load(data)assertresult=={'field':[{'is_ancestor':True,'links':{'self':'http://127.0.0.1:5000/2.0/taxonomies/test_taxonomy/a'},'test':'extra_data'},{'is_ancestor':False,'links':{'parent':'http://127.0.0.1:5000/2.0/taxonomies/test_taxonomy/a','self':'http://127.0.0.1:5000/2.0/taxonomies/test_taxonomy/a/b'},'test':'extra_data'}]}listfrommarshmallowimportSchemafromoarepo_taxonomies.marshmallowimportTaxonomyField# custom schemaclassTestSchema(Schema):field=TaxonomyField()# taxonomy list with ancestor (root ancestor at the first place)random_user_taxonomy=[{'links':{'self':'http://127.0.0.1:5000/2.0/taxonomies/test_taxonomy/a'},},{'links':{'self':'http://127.0.0.1:5000/2.0/taxonomies/test_taxonomy/a/b'},'test':'extra_data','next':'bla','another':'something'}]# record dictdata={"field":random_user_taxonomy}schema=TestSchema()result=schema.load(data)assertresult=={'field':[{'is_ancestor':True,'links':{'self':'http://127.0.0.1:5000/2.0/taxonomies/test_taxonomy/a'},'test':'extra_data'},{'another':'something','is_ancestor':False,'links':{'parent':'http://127.0.0.1:5000/2.0/taxonomies/test_taxonomy/a','self':'http://127.0.0.1:5000/2.0/taxonomies/test_taxonomy/a/b'},'next':'bla','test':'extra_data'}]}TaxonomyField vs. TaxonomySchemaTaxonomySchemais a marshmallow schema, that can be subclassed and used, for example,
insideNested.TaxonomyFieldis a marshmallowFieldthat is used as is. The field also allows extending
taxonomy metadata model with extra properties.Signature of the factory is followingTaxonomyField(*args, extra=None, name=None, many=False, mixins: list = None, **kwargs)args: arbitrary arguments passed to marshmallow.schemaextra: a dictionary of extra marshmallow fields (key: field name, value: instance of Field)name: optional name of the field (it is used as a name of the dynamically created class
on the background)mixins: list of added mixins (class defining extra marshmallow Fields)kwargs: arbitrary named arguments passed to the generated marshmallow schemasclassInstitutionMixin:name=SanitizedUnicode()address=SanitizedUnicode()classTestSchema(Schema):field=TaxonomyField(many=True,mixins=[InstitutionMixin])random_user_taxonomy=[{'links':{'self':'http://127.0.0.1:5000/2.0/taxonomies/test_taxonomy/a'},},{'links':{'self':'http://127.0.0.1:5000/2.0/taxonomies/test_taxonomy/a/b'},'test':'extra_data','next':'bla','another':'something','name':'Hogwarts','address':'Platform nine and three-quarters'}]data={"field":random_user_taxonomy}schema=TestSchema()result=schema.load(data)assertresult=={'field':[{'is_ancestor':True,'links':{'self':'http://127.0.0.1:5000/2.0/taxonomies/test_taxonomy/a'},'test':'extra_data'},{'address':'Platform nine and three-quarters','another':'something','is_ancestor':False,'links':{'parent':'http://127.0.0.1:5000/2.0/taxonomies/test_taxonomy/a','self':'http://127.0.0.1:5000/2.0/taxonomies/test_taxonomy/a/b'},'name':'Hogwarts','next':'bla','test':'extra_data'}]}JSONSchemasThe library offers a predefined JSON schema for taxonomies.
The predefined schema is called with"$ref": "../taxonomy-v2.0.0.json#/definitions/TaxonomyTerm"and is available in Invenio incurrent_jsonschemas.list_schemas().Custom schema example:{"$schema":"http://json-schema.org/draft-07/schema#","id":"https://example.com/schemas/example_json-v1.0.0.json","additionalProperties":false,"title":"My site v1.0.0","type":"object","properties":{"$schema":{"type":"string"},"custom_taxonomy":{"$ref":"../taxonomy-v2.0.0.json#/definitions/TaxonomyTerm"}}}Elasticsearch mappingPredefined mappings can be used for indexing into Elasticsearch. If you want to use this mapping you must use the
libraryOAREPO mapping includes. A reference to
taxonomy mapping is then inserted to custom mapping as either"type": "taxonomy-v2.0.0.json#/TaxonomyTerm"or"type": "taxonomy-term".Custom mapping example:{"mappings":{"date_detection":false,"numeric_detection":false,"dynamic":false,"properties":{"$schema":{"type":"keyword","index":true},"custom_taxonomy":{"type":"taxonomy-v2.0.0.json#/TaxonomyTerm"}}}}SignalsThis module will register the following signal handlers on the Flask Taxonomies signals that handle managing of
reference Taxonomies whenever a Taxonomy or TaxonomyTerm changes:Flask-Taxonomies signalsRegistred signalhandlerDescriptionbefore_taxonomy_deletedtaxonomy_deleteChecks if the changed taxonomy is a reference to any record. If so, they throw an exception.before_taxonomy_term_deletedtaxonomy_term_deleteChecks if the changed TaxonomyTerm is a reference to any record. If so, they throw an exception.after_taxonomy_term_updatedtaxonomy_term_updateReplaces the link in the records to the moved TaxonomyTerm.after_taxonomy_term_movedtaxonomy_term_movedReplaces the contents of the changed taxonomy in the referenced records.