Package: tg-model
TextGrid Import ModellerWhats the aim?This project focuses on attemps for a simple import of text corpora (encoded in XML/TEI) toTextGrid Repositoryby modeling the required metadata file structure.NOTE:
!This is work in progress!
_
Feedback on anything that does not work or needs to be modified is welcome!!!InstallationThe source code is maintained here:https://gitlab.gwdg.de/textplus/textplus-io/textgrid_import_modellingClone the project:gitclonehttps://oauth:{{your.access.token}}@gitlab.gwdg.de/textplus/textplus-io/textgrid_import_modelling.git-o{{your/project/path/name}}It is recommended to install the project in a local virtual python environment and therefore the necessary steps are basically described:Version 1 (recommended)Simply create it within tg_model. Naming invenvwhile setting the prompt to the name of the current directory:cd{{your/project/path/name}}python3-mvenvvenv/--prompt"$(pwd|grep-o"[^/]*$")".venv/bin/activate
pipinstall-e.Version 2Create it at your favored path:# create new virtual environmentpython3-mvenv{path/to/your/virtEnv}# activate virtual environment.{path/to/your/virtEnv}/bin/activate# install this projectpipinstall-e{{your/project/path/name}}What can be done? (so far)Discover SynergiesBuild metadata structure needed for TextGrid importDiscover SynergiesAn initial idea was, to support the process of describing the meta data of the project resp. the corpora by analyzing all XML/TEI-files for synergies.This is simply done by:collect all nodes of first file in listiterate over all other filesiterate over all collected nodescan the node be found in file?yes: keep node in list of collected nodesno: pop node from list of collected nodesrebuild XML structure for all nodes, that remained in the listanalyze parent-node relationsfind out which child-nodes belong to which parent-nodesre-assemble XML-structureAfterwards, an editor can have a look at the synergetic nodes an (hopefully) gets an impression/idea/impulse which synergetic information could be decesive toinit a major config...defining the project path# init a major config...defining the project pathtg_configs-n{projectname}main-s{path/to/base/directory}-t{name/of/directory/containing/tei/files}test synergy analyses as follows# analyze file for synergies...#...print result to consoletg_synergy-n{projectname}run#...write result to XML-file# analyze file for synergies and print result to consoletg_synergy-n{projectname}run-osynergies.xmlsee more details on analysis by printing synergetic nodes or node relations to consoletg_synergy-n{projectname}synergetic-nodes
tg_synergy-n{projectname}node-relationsBuild metadata structure needed for TextGrid import1. init a major config...defining the project and subprojectsYou have different options to set the path(s) to your input data."Manual" optionSimply set the path to the directory of your TEI files. You can also set a list of paths, seperated by comma.single directorytg_configs-n{projectname}main-i{path/to/tei/directory/containing/files}multiple directorytg_configs-n{projectname}main-i{1st/path/to/tei/files},{2nd/path/to/tei/files},{3rd/path/to/tei/files}"Automatic" optionWhen you have many sub-directories or sub-projects you can also let tg_model automatically find the directories containing TEI files by setting the basic path containing all sub-projects + the name of the directory, that contains TEI files. The name of that directory has to be identical for all directories!tg_configs-n{projectname}main-s{path/to/base/directory}-t{name/of/directory/containing/tei/files}2. init a synergy configtg_configs-n{projectname}synergyNow you find 2 files at./projects/{projectname}/{subprojectname}(can be manually defined by seetg_configs --help):synergy.xmlthis file contains nodes, that all totally identic in all XML/TEI files of the corpussynergy.yamlby this file one defines certain attributes, that are need by the integrated modeling-templatesone can define xpaths to nodes inside ofsynergy.xmlmanually...which are used to fill out some mandatory attributes of the following "collection config"3. init a collection configtg_configs-n{projectname}collectionThis creates the final config, which is needed to build the TextGrid metadata structure.What the code does:trying to find proposed xpaths inside of all given XML/TEI filesif it finds a node by a proposed xpath more time than a definedhit_rate(defined inmain.yaml), than this xpath is added to the the "collection config"MandatoryAll attributes for "rights_holder" & "title" have to be filled out, as these attributes get validated (only for existance) before the code models the structure.init a collection configFinally one can build the TextGrid metadata structuretg_model-n{projectname}build-collectionThis puts all the files in./output, but this can be manually definedtg_model build-collection --helpExemplary executionsmkdir/tmp/FluffyModellingcd/tmp/FluffyModellingCoNSSA# get corpusgitclonehttps://github.com/cligs/conssa.gitconssa# initialize all configstg_configs-nCoNSSAall-sconssa-tmasterNow you can find the main config at:/tmp/FluffyModelling/projects/CoNSSAand the related subproject at:/tmp/FluffyModelling/projects/conssa_master_mastercontaining configs for synergy and collection.For CoNSSA, there is no need for manual editing of the configs, so you can go on and create the meta data files:tg_model-nCoNSSAbuild-collectionAfterwards, you can find them at:/tmp/FluffyModelling/projects/CoNSSA/conssa_master_master/resultELTeC-fra# get corpusgitclonehttps://github.com/COST-ELTeC/ELTeC-fraeltec-fra# initialize all configstg_configs-nELTeC-fraall-seltec-fra-tlevel1Now you can find the main config at:/tmp/FluffyModelling/projects/ELTeC-fraELTeC needs modifications at the collection config:nano/tmp/FluffyModelling/projects/ELTeC-fra/FluffyModelling_eltec-fra_level1/collection.yaml# --> set all attributes of 'rights_holder'# create the meta data filestg_model-nELTeC-frabuild-collectionAfterwards, you can find them at:/tmp/FluffyModelling/projects/ELTeC-fra/tgm_outputMulti-project examplestextboxgitclonehttps://github.com/cligs/textbox

tg_configs-ntextboxall-stextbox-ttei

tg_model-ntextboxbuild-collectionELTeCtg_configs-nELTeCall-sELTeC-tlevel1

tg_model-nELTeCbuild-collection4DeveloperThis project is built up in a very simple click-based setup.(see"python click")Allcommandline entry points(e.g.tg_model,tg_configs, ...) are defined within theentry_pointssectionofsetup.py.LicenseWhile the specificimplementationsare located intg_model/cli.py.Copyright [2024] [TU Dresden | CIDS | ZIH]Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License athttp://www.apache.org/licenses/LICENSE-2.0Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.