KG4CDO 0.2.2
KG4CDO: A Knowledge Based Framework for Objects Models Synthesis
Requires Python: >=3.6
Keywords: knowledge framework object model synthesis

# KG4CDO: A Knowledge Based Framework for Objects Models Synthesis

[KG4CDO website](https://github.com/kulikovia/KG4CDO)

## Contributors

Correspondence to: 
  - [Igor Kulikov](http://) (i.a.kulikov@gmail.com)
  - [Nataly Zhukova](https://) (nazhukova@mail.ru)
  - [Man Tianxing](https://) (mantx@jlu.edu.cn)
  
## Paper

[**Synthesis of multilevel knowledge graphs: Methods and technologies for dynamic networks**](https://www.sciencedirect.com/science/article/pii/S0952197623004281)<br>
Tianxing Man, Alexander Vodyaho, Dmitry I. Ignatov, Igor Kulikov, Nataly Zhukova<br>
Engineering Applications of Artificial Intelligence, 2023. Connected research.

If you find this repository useful, please cite our paper and corresponding framework:
```
@misc{Man_Vodyaho_Ignatov_Kulikov_Zhukova_2023, 
	title={Synthesis of multilevel knowledge graphs: Methods and technologies for dynamic networks}, 
	volume={123}, 
	url={http://dx.doi.org/10.1016/j.engappai.2023.106244}, 
	DOI={10.1016/j.engappai.2023.106244}, 
	journal={Engineering Applications of Artificial Intelligence}, 
	publisher={Elsevier BV}, 
	author={Man, Tianxing and Vodyaho, Alexander and Ignatov, Dmitry I. and Kulikov, Igor and Zhukova, Nataly}, 
	year={2023}, 
	month=aug, 
	language={en} }
```

## Framework files

1. DB_create_script.sql - SQL script for initial DB creation 
2. Deductive_synthesis_(SQL).py - Deductive synthesis program of multilevel KG for the base algorithm
3. Deductive_synthesis_modified(SQL).py - Deductive synthesis program of multilevel KG for the modified algorithm
4. Ind_synthesis_(SQL).py - Inductive synthesis program of multilevel KG 
5. Links_rules-one-level-TEST.csv - Ruls for linking private models elements for one-level model
6. Links_rules.csv - Ruls for linking private models elements for hierarchical model
7. Load_comp_KG_generate_script.py - Script that generates a multi-level knowledge graph consisting
8. Load_comp_KG(linear)_generate_script.py - Script that generates model in the format of one-level knowledge graph
9. Model_creation_hierarchy.py - Script that generates hierarchy private graph models
10. Model_creation_one-level.py - Script that generates one-level private graph models
11. Test_template_1-one-level.csv - Template for the Model_creation_one-level.py script
12. Test_template_1.csv - Template for the Model_creation_hierarchy.py script


## Framework structure
The component diagram of the developed framework is shown beow:
![](/images/2.png)

The framework consists of the following components:
1. A package of multilevel knowledge graph synthesis programs, which includes: 

1.1. Inductive synthesis program of multilevel KG}} вЂ“ Ind_synthesis.py.

**Input data:** Private graph models of SCDO provided by information systems in CSV format (Test_model.csv, columns: MODEL_TYPE (type of private graph model, e.g. access rights system model, network topology model, billing model, etc.), NODE\_TYPE (element type of the private graph model, e.g. network devices and links for the network topology model or user accounts and tariffs for the billing model, etc.), ID (identifier of the element within the private graph model), NAME (name of the element of the private graph model), PARENT_ID (identifier of the parent element of the private graph model), LEVEL_NUM (level number within the private graph model)); rules for linking private graph models in CSV format (Links_rules.csv, columns: SRC_MODEL (name of the type of the first linked model), SRC_ID (element identifier of the first linked model), SRC_NAME (element name of the first linked model), DIST_MODEL (name of the type of the second linked model), DIST_ID (element identifier of the second linked model), DIST_NAME (element name of the second linked model), RULE (name of the linking rule)).

**Output:** The model of the CSDO in the form of a knowledge graph in RDF/XML format and files Model_base.npz, Model_base_links.npz containing the inductive model and Model_base_req.npz, Model_base_links_req.npz containing the inductive model, that takes into account the requirements of users in the format of Numpy data arrays, the runtime of inductive synthesis.

1.2. Deductive synthesis program of multilevel KG for the base algorithm}} вЂ“ Deductive_synthesis_(SQL).py.

**Input data:** Telecommunication network model in the form of knowledge graph in RDF/XML format and Numpy array; set of facts to be processed in CSV format (Facts.csv, columns: MODEL_TYPE (private graph model type), NODE_TYPE (private graph model element type), FACT_ID (identifier of the fact within the private graph model), NAME (fact name), PARENT_ID (identifier of the parent element of the private graph model), LEVEL_NUM (level number within the private graph model)).

**Output:** Level number at which the model is proved, -1 if the model cannot be proved, deductive synthesis runtime.

1.3. Deductive synthesis program of multilevel KG for the modified algorithm}} вЂ“ Deductive_synthesis_modified(SQL).py.

**Input data:** Telecommunication network model in the form of knowledge graph in RDF/XML format and Numpy array; single fact to be processed in CSV format (Facts.csv, columns: MODEL_TYPE (private graph model type), NODE_TYPE (private graph model element type), FACT_ID (identifier of the fact within the private graph model), NAME (fact name), PARENT_ID (identifier of the parent element of the private graph model), LEVEL_NUM (level number within the private graph model)); number of the level at which the synthesis time needs to be evaluated.

**Output:** Time to process a single fact, time to perform deductive synthesis at a given level.

1.4. A model data generator that contains the following procedures:

1.4.1. Private model generator of SCDS component models for building a multilevel model вЂ“ Model_creation_hierarchy.py. Function: create_model(levels_number, elements_number, distribution_type, partial_models_number, linking_level, linking_rules) 

**Input data:** levels_number - number of partial models levels; elements_number - each partial model elements number; distribution_type - the one of distribution type for elements thruth levels ("uniform", "linear", "quadratic", "exponential"); partial_models_number - number of partial models; linking_level - on which level the partial models is linked; linking_rules - string with linking rules separated with ';' e.g. "1-2;2-3" (partial model 1 is linked to partial model 2 and partial model 2 is linked to partial model 3).

**Output:** Private graph models in CSV format (Test_model.csv, columns: MODEL_TYPE (private graph model type), NODE_TYPE (private graph model element type), ID (element identifier within the private graph model), NAME (private g, "quadratic", "exponential"); raph model element name), PARENT_ID (parent element identifier of the private graph model), LEVEL_NUM (level number within the private graph model)); set of facts for processing in CSV format (Facts.csv, columns: MODEL_TYPE (private graph model type), NODE_TYPE (private graph model element type), FACT_ID (identifier of the fact within the private graph model), NAME (fact name), PARENT_ID (identifier of the parent element of the private graph model), LEVEL_NUM (level number within the private graph model)).

1.4.2. Private model generator of SSDS component for building a one-level model}} вЂ“ Model_creation_one-level.py.

**Input data:** Number of elements of model #1 to be linked, number of elements of model #2 to be linked.

**Output:** Private graph models in CSV format (Test_model.csv, columns: \newline MODEL_TYPE (private graph model type), NODE_TYPE (private graph model element type), ID (element identifier within the private graph model), NAME (private graph model element name), PARENT_ID=None and LEVEL_NUM=0 for all the elements of one-level model); set of facts for processing in CSV format (Facts.csv, columns: MODEL_TYPE (private graph model type), NODE_TYPE (private graph model element type), FACT_ID (identifier of the fact within the private graph model), NAME (fact name), PARENT_ID $=$ None and LEVEL_NUM=0 for all the elements of one-level model).

1.4.3. The CSDO test model generators in the format of multilevel knowledge graphs for analyzing SPARQL query execution time: Load_comp_KG_generate_script.py - is a script that generates a multi-level knowledge graph. 

The multi-level knowledge graph structure  is shown beow:
![](/images/3.png)

Function: createRDF(filename, levels_number, elements_number, distribution_type, partial_models_number, linking_level, linking_rules, isDynamic=False, start_date=None, end_date=None, dynamic_levels_elements_number=None, value_min=0, value_max=100)

**Input data:** filename - model file name; levels_number - number of partial models levels; elements_number - each partial model elements number; distribution_type - the one of distribution type for elements thruth levels ("uniform", "linear", "quadratic", "exponential"); partial_models_number - number of partial models; linking_level - on which level the partial models is linked; linking_rules - string with linking rules separated with ';' e.g. "1-2;2-3" (partial model 1 is linked to partial model 2 and partial model 2 is linked to partial model 3); isDynamic - is dynamic data has to be added (True/False, defaulf: isDynamic=False); start_date - start date of dynamic data period e.g. "2025-01-01", default: start_date=None; end_date - end date of dynamic data period e.g. "2025-01-02", default: end_date=None; dynamic_levels_elements_number - string with dynamic data distribution rules separated with ';' e.g. "1-100;2-300" (100 dynamic events on the level 1 and 300 dynamic events on the level 2) default: dynamic_levels_elements_number=None; value_min - minimum value of the dynamic parameter, default: value_min=0; value_max - maximum value of the dynamic parameter, default: value_max=100.

**Output:** The CSDO model in the form of a knowledge graph in RDF/XML format.

1.4.4. Generators of the test CSDO model in the format of one-level knowledge graph for analyzing the speed of SPARQL queries execution вЂ“ Load_comp_KG(linear)_generate_script.py.

**Input data:** The number of elements of the private models to be linked.

**Output:** The CSDO model in the form of a knowledge graph in RDF/XML format. 

2. Knowledge graph component including:

2.1. A graph DBMS with SPARQL 1.1 support;

2.2. Ontology repository;

2.3. Dynamic REST service for organizing interaction with external systems.

3. Relational DBMS PostgreSQL.

4. File system for storing input and output data in file formats. The file system is used in the following cases:

4.1. Inductive and Deductive Synthesis programs perform reading of private graph models and operational data about TNs from files placed in the File System;

4.2. Inductive synthesis program generates TN models in RDF/XML format and places them as files in the File System;

4.3. Inductive and Deductive Synthesis programs save their logs in the File System;

4.4. Inductive and Deductive Synthesis programs use PostgreSQL relational DBMS to implement operations on data tables and store intermediate results;

4.5. Loading models and operational data into the graph data store is performed from RDF/XML files hosted in the File System. 

5. A Python package for evaluating and comparing knowledge graphs wich also contains a set of SPARQL request is built based on model structure and is aimed to test different request types. 


## How to use

Below the main steps that should be executed to evaluate the performance of inductive and deductive synthesis algorithms on the example of building five-levels knowledge graph are enumerated: 
1. Generate private graph models. 
To obtain a multilevel knowledge graph, the script **Model_creation_hierarchy.py** should be used. Input parameters are set as script variables: num_objects1, num_objects2 вЂ“ numbers of elements of linked models at level 6, max_level2 вЂ“ number of elements of linked models at level 2, max_level3 вЂ“ number of elements of linked models at level 3, max_level4 вЂ“ number of elements of linked models at level 4, max_level5 вЂ“ number of elements of linked models at level 5. It should be noted that for the purposes of forming a symmetric structure of the KG at the upper levels, the data generator provides an equal number of elements at levels 2вЂ“5 for linked private graph models. To obtain a one-level knowledge graph it is necessary to use the script **Model_creation_one-level.py**. Input parameters are set as script variables: num_objects1, num_objects2 вЂ“ number of elements of the linked models.
2. Perform inductive synthesis and estimate its execution time. 
To perform inductive synthesis it is required to use the script  **Ind_synthesis_(SQL).py**, Python interpreter version 3 or higher, PostgreSQL version 13 or higher. It is necessary to create a database and to run the SQL script **DB_create_script.sql** in the PostgreSQL console. Input data in the format of CSV files should be located in the same folder with the Python script. Output data is placed in the same folder where the Python script is located; the execution time of synthesis steps is displayed in the console of the Python interpreter.
3. Perform deductive synthesis and estimate its execution time (for basic and modified algorithms). To perform deductive synthesis, it is nesseccary to use the scripts **Deductive_synthesis_(SQL).py** and **Deductive_synthesis_modified(SQL).py**, which require Python interpreter version 3 or higher to run. Input data in CSV file format and arrays in the format provided by the Python module NP, used for working with arrays, must be located in the same folder with the Python script; the number of the level at which the proof occurs is specified via the Python variable of the script max_level. The synthesis time is displayed in the console of the Python interpreter.
4. Analyze the execution time of SPARQL queries to CSDO models.
In order to analyze the execution time of SPARQL queries to the CSDO models in the form of a knowledge graph with different parameters, it is necessary to generate such models using the generators: 

4.1. **Load_comp_KG_generate_script.py**, 

4.2. **Load_comp_KG(linear)_generate_script.py**. 

The Python interpreter version 3 or higher is required to run the scripts. Model parameters are set via the following script variables: max_level_2, max_level_3, max_level_4, max_level_5, max_objects1 and max_objects2. The resulting models in RDF/XML format are loaded into the RDF data repository using standard data storage methods. We used Blazegraph. Queries were executed from the standard Web application Blazegraph which provides a screen display of the execution time of each SPARQL query.

## Experiments

## References

## Patch Note / Major Updates