Package: psqlgml
SummarySample data generation is a common step used for testing and verifying new and existing features that make use of the data commons dictionary. Without validation tools, this step can be super hard and prone to errors. This project aims to provide tooling that helps with generating and visualizing sample data. It is dictionary agnostic, so should work for any given gdc compatible dictionary.Sample data graphs are represented using a customizedGraphMLformat which can be represented in either json or yaml files. This projects provides tools for creating this schema based on selected dictionary and validating data that is targeting this schema.Goalspsqlgml aims to provide the following for projects that makes use ofpsqlgraph:test data validation and visualizationtest data schema that can be integrated with IDE’s for easier test data generationrandomized test data generation based on user requirementsprovide data structures and functions for use in external projectsprovide alternate implementation for loading dictionary with better type checkingRequirementsPython3.6+graphviz(used for visualization)Installationfrom pypi$pipinstallpsqlgmlQuick StartCommand Line# install$pipinstallpsqlgml# validate install$psqlgml--help# generate internal schema to aid validation$psqlgmlgenerate-v2.4.0-ntest_dictionary# validation$psqlgmlvalidate--help# visualize$psqlgmlvisualize--helpAPIimportpsqlgml# load the default dictionarydictionary:psqlgml.Dictionary=psqlgml.load(version="2.3.0")GML SchemaThis is a customizedGraphMLformat based on JSON schema. It allows graphs to be represented as a set of nodes and edges. The schema makes it possible to validate a sample data.unique_field:node_idnodes:-label:programnode_id:p_1name:SM-KD-label:projectnode_id:pr_1edges:-src:p_1dst:pr_1label:programsThis example creats two nodesProgramandProjectthat are linked together using thenode_idproperty. The name of the edge connecting them isprogramsSchema Generationpsqlgml can be used to generate dictionary specific schemas using exposed command line scripts. By default,gdcdictionaryis assumed but parameters can be updated to work with a different project.Generate schema using version 2.4.0 of the gdcdictionarypsqlgml generate -v 2.4.0 -n gdcdictionaryThe generated schema can be used for validating sample data. It can also be added to IDEs like PyCharm for intellisense while creating sample data.Sample Data Validation$ psqlgml validate -f sample.yaml --data-dir <resource dir> -d <dictionary name> -v <dictionary version>The following validations are currently supported:JSON Schema ValidationDuplicate Definition ValidationUndefined Link ValidationAssociation ValidationJSON Schema ValidationChecks the sample data is compliant with the dictionary. It validates things like:
* properties that are not allowed on a node
* property values not allowed on a property
* Invalid enum value
* Invalid/unsupported node typesDuplicate Definition ValidationRaises an error whenever a unique id is used for more than one nodeUndefined Link ValidationThis is raised as a warning, since it is very possible to link to nodes not defined with the sample data. For example, appending data to an existing database.Association ValidationRaises an error whenever an edge exists between nodes that the dictionary does not define an edge for.