Package: cognite-neat
kNowlEdge grAph Transformer (NEAT)This is python package handles transformation of domain knowledge graphs (more detailed, overarching) to app specific knowledge graphs.
Documenation available athttps://thisisneat.io.QuickstartOption 1 - Python Packageneatcan be installed as a Python package, and a docker image for more robust execution and support such
asgraphdbfor caching. This quickstart guide uses the Python package.Option 2 - Docker containerRunning NEAT as docker container :docker run -p 8000:8000 --name neat cognite/neat:latestRunnin NEAT as docker container with local data folder mounted :docker run -p 8000:8000 --name neat -v $(shell pwd)/docker/vol_data:/app/data  cognite/neat:latestInstallation (Development setup)Prerequisites:poetryinstalled on your system,see installation.npminstalled on your system,see installationreact-scriptsinstalled on your system,npm i react-scriptsClone this repo:git clone git@github.com:cognitedata/neat.gitCreate wheel:make build-python. This will create a wheel in thedistfolder.Install wheel in your preferred environment,pip install dist/neat-[NEAT_VERSTION]-py3-none-any.whlRun Neat (Python Package)Once installed you can run neat with the commandneatThis will open your browser and launch the neat application.Note this will automatically create adatafolder in your current working directory, which is populated
with default workflows, rule sheets, and source graphs. In addition, there is aconfig.yamlyou can add
your CDF credentials to allowneataccess CDF.DocumentationDocumentationDocker container build processmake build-dockerStarting the application locally using GraphDB as external rdf storeRun the command :make compose-upThe command starts NEAT container and GraphDB container and createsvol_dataandvol_sharedlocal folders that are mounted as volumes.vol_data- is mounted only in NEAT container. It containsconfig.yaml,workflows,rules. You can setload_examplestotrueto populate it with examples.vol_shared- is mounted to both containers and files placed here become automatically visible for GraphDB import process. Main intension is to write files from NEAT container and make them available for GraphDB for import.Web UIhttp://localhost:8000Open API docs :http://localhost:8000/docsProm metrics :http://localhost:8000/metricsWorkflows and configurationsAll configurations are divided into 2 categories :global configurations - global for entire service , loaded at startup (config.yaml)workflow configurations - local to workflow , loaded from workflow manifest (.yaml)Global configurationsGlobal configurations loaded by the application during startup process, either from ENV variables or file.The application tries to load the configuration as follows.Try ENV variables. This is done by checking for the ENV variableNEAT_CDF_PROJECT, and if it exists load the rest
as they are listed below.Try from file. This is done by checking the ENV variableNEAT_CONFIG_PATH, which defaults to$(cwd)/config.yaml,
for a path. The config file es expected to have the structure as shown below.Exit. It the ENV variableNEAT_CDF_PROJECTand the config file does not exist, the applications exists.Env VariablesNEAT_CDF_PROJECTNEAT_CDF_CLIENT_NAMENEAT_CDF_CLIENT_IDNEAT_CDF_CLIENT_SECRETNEAT_CDF_BASE_URLNEAT_CDF_TOKEN_URLNEAT_CDF_SCOPESNEAT_WORKFLOW_DOWNLOADER_FILTER - list of workflows or tags that must be loaded from CDF automatically during startupNEAT_DATA_PATH - local directory for workflow and rules storageNEAT_CDF_DEFAULT_DATASET_ID - default dataset id that is used for sharing workflows, rules fies and execution tracesNEAT_LOG_LEVEL - supported logging levelsDEBUG,INFO(default)Config fileworkflows_store_type:filedata_store_path:/app/datacdf_client:project:get-power-gridclient_id:"623c2450-cfc2-43d6-9036-10e14dad8ccf"client_secret:"my-super-secret"base_url:https://az-power-no-northeurope.cognitedata.comscopes:-https://az-power-no-northeurope.cognitedata.com/.defaulttoken_url:https://login.microsoftonline.com/e55e1701-82f8-4c52-af10-28e4d942c589/oauth2/v2.0/tokencdf_default_dataset_id:2626756768281823workflow_downloader_filter:-tag:gridlog_level:DEBUGAutomatic workflow loading from CDF during application startupIfworkflow_downloader_filteris set, the app will try to download workflows from CDF based on provided filtering conditions .
Filtering format :name:<workflows_name>=<workflow_version>- loading workflow by name and version or latest by name if version is not set.tag:<tag_name>- loading all workflows that tagged with specific tag.