Package: tabgenie
🧞 TabGenie: A Toolkit for Table-to-Text GenerationDemo 👉️https://quest.ms.mff.cuni.cz/rel2text/tabgenieTabGenie provides tools for working with data-to-text generation datasets in a unified tabular format.TabGenie allows you to:explorethe content of the datasetsinteractwith table-to-text generation modelsload and preprocessthe datasets in a unified formatprepare spreadsheetsfor error analysisexport tablesto various file formatsTabGenie is equipped with user-friendly web interface, Python bindings and command-line processing tools.Frontend PreviewQuickstartpip install tabgenie
tabgenie run --host=127.0.0.1 --port 8890
xdg-open http://127.0.0.1:8890Or try the demo at:👉️https://quest.ms.mff.cuni.cz/rel2text/tabgenieDatasetsThe datasets are loaded from theHuggingFace datasets.Input data in each dataset is preprocessed into a tabular format:each table contains M rows and N columns,cells may span multiple columns or rows,cells may be marked as headings (indicated by bold font),cells may be highlighted (indicated by yellow background).Additionally, each example may contain metadata (such as title, url, etc.) which are displayed next to the main table asproperties.DatasetSourceData type# train# dev# testLicenseCACAPOvan der Lee et al. (2020)Key-value15,2901,8313,028CC BYDARTNan et al. (2021)Graph62,6592,7685,097MITE2EDušek et al. (2019)Key-value33,5251,4841,847CC BY-SAEventNarrativeColas et al. (2021)Graph179,54422,44222,442CC BYHiTabCheng et al. (2021)Table7,4171,6711,584C-UDAChart-to-textKantharaj et al. (2022)Chart24,3685,2215,222GNU GPLLogic2TextChen et al. (2020b)Table  + Logic8,5661,0951,092MITLogicNLGChen et al. (2020a)Table28,4504,2604,305MITNumericNLGSuadaa et al. (2021)Table1,084136135CC BY-SASciGenMoosavi et al. (2021)Table13,6073,452492CC BY-NC-SASportSett:BasketballThomson et al. (2020)Table3,6901,2301,230MITToTToParikh et al. (2020)Table121,1537,7007,700CC BY-SAWebNLGFerreira et al. (2020)Graph35,4251,6661,778CC BY-NCWikiBioLebret et al. (2016)Key-value582,65972,83172,831CC BY-SAWikiSQLZhong et al. (2017)Table + SQL56,3558,42115,878BSDWikiTableTextBao et al. (2018)Key-value10,0001,3182,000CC BYSeeloaders/data.pyfor an up-to-date list of available datasets.RequirementsPython 3FlaskHuggingFace datasetsSeesetup.pyfor the full list of requirements.Installationpip:pip install tabgeniedevelopment:pip install -e .[dev]deployment:pip install -e .[deploy]Web interfacelocal development:tabgenie [app parameters] run [--port=PORT] [--host=HOSTNAME]deployment:gunicorn "tabgenie.cli:create_app([app parameters])"Command-line InterfaceExport dataExports individual tables to file.Usage:tabgenie export \
  --dataset DATASET_NAME \
  --split SPLIT \
  --out_dir OUT_DIR \
  --export_format EXPORT_FORMATSupported formats:json,csv,xlsx,html,txt.Generate a spreadsheet for error analysisGenerates a spreadsheet with system outputs and randomly selected examples for manual error analysis.Usage:tabgenie sheet \
  --dataset DATASET  \
  --split SPLIT \
  --in_file IN_FILE  \
  --out_file OUT_FILE \
  --count EXAMPLE_COUNTShow dataset detailsDisplays information about the dataset in YAML format (or the list of available datasets if no argument is provided).Usage:tabgenie info [-d DATASET]PythonTabGenie can preprocess the datasets without dataset-specific preprocessing methods.See theexamplesdirectory for a tutorial on using TabGenie for finetuning sequence-to-sequence models.HuggingFace IntegrationThe datasets are stored toHF_DATASETS_CACHEdirectory which defaults to~/.cache/huggingface/.Set theHF_DATASETS_CACHEenvironment variable before launchingtabgenieif you want to store the (potentially very large) datasets in a different directory.The datasets are all loaded fromHuggingFace datasetsinstead of their original repositories which allows to use preprocessed datasets and a single unified loader.Note that there may be some minor changes in the data w.r.t. to the original datasets due to unification, such as adding "subject", "predicate" and "object" headings to RDF triple-to-text datasets.Adding datasetsFor adding a new dataset:prepare the datasetadd the dataset to Huggingface DatasetsOR download the dataset locallycreate the dataset loader inloadersa subclass ofHFTabularDatasetfor HF datasetsa subclass ofTabularDatasetfor local datasetscreate a mapping between the dataset name and the class name inloaders/__init__.pyadd the dataset name totabgenie/config.yml.Each dataset should contain theprepare_table(entry)method which instantiates aTableobject from the originalentry.TheTableobject is automatically exported to HTML and other formats (the methods may be overridden).If a dataset is an instance ofHFTabularDataset(i.e. is loaded from Huggingface Datasets), it should contain aself.hf_idattribute. The attribute is used to automatically load the dataset viadatasetspackage.Interactive modePipelines are used for processing the tables and producing outputs.Seeprocessing/processing.pyfor an up-to-date list of available pipelines.Currently integrated:model_api- a pipeline which generates a textual description of a table by calling a table-to-text generation model through API,graph- a pipeline which creates a knowledge graph by extracting RDF triples from a table and visualizes the output using D3.js library,Adding pipelinesFor adding a new pipeline:create a file inprocessing/pipelinescontaining the pipeline class,create file(s) inprocessing/processorswith processors needed for the pipeline,add the mapping between pipeline name and class name toget_pipeline_class_by_name()inprocessing/processing.py.Each pipeline should defineself.processorsin the__init__()method, instantiating the processors needed for the pipeline.The input to each pipeline is acontentobject containing several fields needed for table processing. This interface may be subject to change (see__init.py_:run_pipeline()for more details).The processors serve as modules, i.e. existing processors can be combined to create new pipelines. The interface between the processors may vary, it is however expected that the last processor in the pipeline outputs HTML code which is displayed on the page.Pipeline configThis is an example pipeline configuration intabgenie/config.yml:rdf_triples:
  pipeline: graph
  interactive: true
  datasets:
    - webnlg
    - dart
    - e2eThe keyrdf_triplesis the name of the pipeline which will be displayed in the web interface. It should contain only letters of English alphabet, underscores_or dashes-.Required arguments:pipeline:str- the name of the pipeline as defined inprocessing/processing.py, will be mapped to pipeline classinteractive:bool- whether the pipeline will be displayed in the interactive mode in the web interfaceOptional arguments:datasets:list- the list of datasets for which the pipeline will be active in the web interface (all datasets by default)any other argument, will be passed to the pipeline inpipeline_argsConfigurationThe global configuration is stored in thetabgenie/config.ymlfile.datasets- datasets which will be available in the web interface,default_dataset- the dataset which is loaded by default,host_prefix- subdirectory on which the app is deployed (used for loading static files and sending POST requests),cache_dev_splits- whether to preload all available dev sets after startup,generated_outputs_dir- directory from which the generated outputs are loaded,pipelines- pipelines which will be available in the web interface (see theInteractive modesection for more info).