Package: siwi
Siwi the voice assistantSiwi (/ˈsɪwi/) is a PoC of Dialog System With Graph Database Backed Knowledge Graph.For now, it's a demo for task-driven(not general purpose) dialog bots with KG(Knowledge Graph) leveraging Nebula Graph with the minimal/sample dataset fromNebula Graph Manual/NG中文手册.Tips: Now you can play with the graph online without installing yourself!Nebula Playground|Nebula Playground - China MainlandSupported queries:relation:What is the relationship between Yao Ming and Lakers?How does Yao Ming and Lakers connected?serving:Which team had Yao Ming served?friendship:Whom does Tim Duncan follow?Who are Yao Ming's friends?Deploy and TryYou can try with it from scratch here:https://katacoda.com/wey/scenarios/siwi-kgqaHow does it work?This is one of the most naive pipeline for a specific domain/ single purpose chat bot built on a Knowledge Graph.BackendThe Backend(Siwi API) is a Flask based API server:Flask API server takes questions in HTTP POST, and calls the bot API.In bot API part there are classfier(Symentic Parsing, Intent Matching, Slot Filling), and question actors(Call corresponding actions to query Knowledge Graph with intents and slots).Knowledge Graph is built on an Open-Source Graph Database:Nebula GraphFrontendThe Frontend is a VueJS Single Page Applicaiton(SPA):I reused a Vue Bot UI to showcase a chat window in this human-agent interaction, typing is supported.In addtion, leverating Chrome'sWeb Speech API, a button to listen to human voice is introducedA Query Flow┌────────────────┬──────────────────────────────────────┐
│                │                                      │
│                │  Speech                              │
│     ┌──────────▼──────────┐                           │
│     │            Frontend │   Siwi, /ˈsɪwi/           │
│     │ Web_Speech_API      │   A PoC of                │
│     │                     │   Dialog System           │
│     │ Vue.JS              │   With Graph Database     │
│     │                     │   Backed Knowledge Graph  │
│     └──────────┬──────────┘                           │
│                │  Sentence                            │
│                │                                      │
│   ┌────────────┼──────────────────────────────┐       │
│   │            │                              │       │
│   │            │              Backend         │       │
│   │ ┌──────────▼──────────┐                   │       │
│   │ │ Web API, Flask      │   ./app/          │       │
│   │ └──────────┬──────────┘                   │       │
│   │            │  Sentence    ./bot/          │       │
│   │ ┌──────────▼──────────┐                   │       │
│   │ │                     │                   │       │
│   │ │ Intent matching,    │   ./bot/classifier│       │
│   │ │ Symentic Processing │                   │       │
│   │ │                     │                   │       │
│   │ └──────────┬──────────┘                   │       │
│   │            │  Intent, Entities            │       │
│   │ ┌──────────▼──────────┐                   │       │
│   │ │                     │                   │       │
│   │ │ Intent Actor        │   ./bot/actions   │       │
│   │ │                     │                   │       │
│   └─┴──────────┬──────────┴───────────────────┘       │
│                │  Graph Query                         │
│     ┌──────────▼──────────┐                           │
│     │                     │                           │
│     │ Graph Database      │    Nebula Graph           │
│     │                     │                           │
│     └─────────────────────┘                           │
│                                                       │
│                                                       │
│                                                       │
└───────────────────────────────────────────────────────┘Source Code Tree.
├──README.md
├──src
│├──siwi# Siwi-API Backend││├──app# Web Server, take HTTP requests and calls Bot API││└──bot# Bot API││├──actions# Take Intent, Slots, Query Knowledge Graph here││├──bot# Entrypoint of the Bot API││├──classifier# Symentic Parsing, Intent Matching, Slot Filling││└──test# Example Data Source as equivalent/mocked module│└──siwi_frontend# Browser End│├──README.md
│├──package.json
│└──src
│├──App.vue# Listening to user and pass Questions to Siwi-API│└──main.js
└──wsgi.pyManually Run ComponentsGraph DatabaseThe backend relis on theNebula Graph, an Open Source Distributed Graph Database.Install Nebula Graph in oneliner:curl-fsSLnebula-up.siwei.io/install.sh|bashLoad thebasketballplayer dataset.~/.nebula-up/console.sh
nebula-console-addrgraphd-port9669-userroot-pnebula-e":play basketballplayer"BackendInstall and run.# Install siwi backendpython3-mbuild# Configure Nebula Graph EndpointexportNG_ENDPOINTS=127.0.0.1:9669# Run Backend API servergunicorn--bind:5000wsgi--workers1--threads1--timeout60For OpenFunction/ KNativedockerbuild-tweygu/siwi-api.
dockerrun--rm--namesiwi-api\--env=PORT=5000\--env=NG_ENDPOINTS=127.0.0.1:9669\--net=host\weygu/siwi-apiTry it out Web API:$curl-s--header"Content-Type: application/json"\--requestPOST\--data'{"question": "What is the relationship between Yao Ming and Lakers?"}'\http://192.168.8.128:5000/query|jq{"answer":"There are at least 23 relations between Yao Ming and Lakers, one relation path is: Yao Ming follows Shaquille O'Neal serves Lakers."}Call Bot Python API:fromnebula3.gclient.netimportConnectionPoolfromnebula3.ConfigimportConfig# define a configconfig=Config()config.max_connection_pool_size=10# init connection poolconnection_pool=ConnectionPool()# if the given servers are ok, return true, else return falseok=connection_pool.init([('127.0.0.1',9669)],config)# import siwi botfromsiwi.botimportbot# instantiate a botb=bot.SiwiBot(connection_pool)# make the question queryb.query("Which team had Jonathon Simmons served?")Then a response will be like this:In[4]:b.query("Which team had Jonathon Simmons serv...:ed?")[DEBUG]ServeActionintent:{'entities':{'Jonathon Simmons':'player'},'intents':('serve',)}[DEBUG]queryforRelationshipAction:USEbasketballplayer;MATCHp=(v)-[e:serve*1]->(v1)WHEREid(v)=="player112"RETURNpLIMIT100;[2021-07-0202:59:36,392]:Getconnectionto('127.0.0.1',9669)Out[4]:'Jonathon Simmons had served 3 teams. Spurs from 2015 to 2015; 76ers from 2019 to 2019; Magic from 2017 to 2017; 'FrontendReferring tosiwi_frontendDeploy with K8s + OpenFunction┌─────────────────────────────┐
 │ kind: Ingress               │     ┌───────────────────┐
 │   path: /                   │     │ Pod               │
 │    -> siwi-frontend     ────┼─────┤  siwi-frontend    │
 │                             │     │                   │
 │                             │     └───────────────────┘
 │                             │
 │   path: /query              │     ┌───────────────────────────────────┐
 │    -> siwi-api          ────┼─────┤ KNative Service                   │
 │       KNative Serving       │     │  serving-xxxx                     │
 │                             │     │                                   │
 │                             │     │ apiVersion: serving.knative.dev/v1│
 │                             │     │ kind: Service                     │
 └─────────────────────────────┘     └─────────┬─────────────────────────┘
                                               │
                                               └────────────┐
                                                            │
 ┌───────────────────────────────────────────────────────┐  │
 │apiVersion: core.openfunction.io/v1alpha1              │  │
 │kind: Function                                         │  │
 │spec:                                                  │  │
 │  version: "v1.0.0"                                    │  │
 │  image: "weygu/siwi-api:latest"                       │  │
 │  imageCredentials:                                    │  │
 │    name: push-secret                                  │  │
 │  port: 8080                                           │  │
 │  build:                                               │  │
 │    builder: openfunction/builder:v1                   │  │
 │    env:                                               │  │
 │      FUNC_NAME: "siwi_api"                            │  │
 │      FUNC_TYPE: "http"                                │  │
 │      FUNC_SRC: "main.py"                              │  │
 │    srcRepo:                                           │  │
 │      url: "https://github.com/wey-gu/nebula-siwi.git" │  │
 │      sourceSubPath: "src"                             │  │
 │  serving:                                             │  │
 │    runtime: Knative  ─────────────────────────────────┼──┘
 │    params:                                            │
 │      NG_ENDPOINTS: "NEBULA_GRAPH_ENDPOINT"            │
 │    template:                          │               │
 │      containers:                      │               │
 │        - name: function               │               │
 │          imagePullPolicy: Always      │               │
 └───────────────────────────────────────┼───────────────┘
                                         │
                              ┌──────────┘
                              │
 ┌────────────────────────────┴───────────────────────────┐
 │apiVersion:lapps.nebula-graph.io/v1alpha1               │
 │kind: NebulaCluster                                     │
 │spec:                                                   │
 │  graphd:                                               │
 │    config:                                             │
 │      system_memory_high_watermark_ratio: "1.0"         │
 │    image: vesoft/nebula-graphd                         │
 │    replicas: 1                                         │
 │...                                                     │
 └────────────────────────────────────────────────────────┘Assumed we have a k8s with OpenFunctions installedRun it!Install a Nebula Graph withkubesphere-all-in-onenebula installer on KubeSphere:curl-sLnebula-kind.siwei.io/install-ks-1.sh|bashGet Nebula Graph NodePort:NEBULA_GRAPH_ENDPOINT=$(kubectlgetsvcnebula-graphd-svc-nodeport-oyaml-ojsonpath='{.spec.clusterIP}:{.spec.ports[0].port}')echo$NEBULA_GRAPH_ENDPOINTLoad Dataset into the nebula cluster:wgethttps://docs.nebula-graph.io/2.0/basketballplayer-2.X.ngql

~/.nebula-kind/bin/console-uroot-ppassword--address=<nebula-graphd-svc-nodeport>--port=32669-fbasketballplayer-2.X.ngqlCreate the siwi-api powered by Openfunction:catsiwi-api-function.yaml|sed"s/NEBULA_GRAPH_ENDPOINT/$NEBULA_GRAPH_ENDPOINT/g"|kubectlapply-f-Get the function nebula-siwi and the KNative Service:kubectlgetfunctionsnebula-siwiFUNCTION=$(kubectlgetfunctionsnebula-siwi-ogo-template='{{.status.serving.resourceRef}}')kubectlgetksvc-lopenfunction.io/serving=$FUNCTIONKSVC=$(kubectlgetksvc-lopenfunction.io/serving=$FUNCTION-o=jsonpath='{.items[0].metadata.name}')kubectlgetrevision-lserving.knative.dev/service=$KSVCREVISION=$(kubectlgetrevision-lserving.knative.dev/service=$KSVC-o=jsonpath='{.items[0].metadata.name}')echo$REVISIONVerify the function worked fine:curl-s--header"Content-Type: application/json"\--requestPOST\--data'{"question": "What is the relationship between Yao Ming and Lakers ?"}'\$(kubectlgetksvc-lopenfunction.io/serving=$FUNCTION-o=jsonpath='{.items[0].status.url}')/queryCreate the siwi-app resources on K8s:catsiwi-app.yaml|sed"s/REVISION/$REVISION/g"|kubectlapply-f-Verify the function worked fine through the ingress:Here nodeport with http port 31059 was used as ingress controller endpoint.curl-s--header"Content-Type: application/json"\--requestPOST\--data'{"question": "how does Tim Duncan and Lakers connected?"}'\demo-siwi.local:31059/queryVerify the frontend:curl$(kubectlgetsvc-lapp=siwi-o=jsonpath='{.items[0].spec.clusterIP}')Verify the frontend beind the ingress:curldemo-siwi.local:31059Get all resources in siwi-app:kubectlgetservice,pod,ingress,function-lapp=siwiAnd it should be something like this:[root@weynebula-siwi]# kubectl get service,pod,ingress,function -l app=siwiNAMETYPECLUSTER-IPEXTERNAL-IPPORT(S)AGE
service/siwi-frontend-fileClusterIP10.233.60.81<none>80/TCP64m

NAMEREADYSTATUSRESTARTSAGE
pod/siwi-frontend-file1/1Running064m

NAMECLASSHOSTSADDRESSPORTSAGE
ingress.networking.k8s.io/siwi-service<none>demo-siwi.local8059m

NAMEBUILDSTATESERVINGSTATEBUILDERSERVINGAGEfunction.core.openfunction.io/nebula-siwiSucceededRunningbuilder-sbfz6serving-vvjvl26h[root@weynebula-siwi]# kubectl get service,pod,ingress,function -l app=siwiNAMETYPECLUSTER-IPEXTERNAL-IPPORT(S)AGE
service/siwi-frontend-fileClusterIP10.233.60.81<none>80/TCP65m

NAMEREADYSTATUSRESTARTSAGE
pod/siwi-frontend-file1/1Running065m

NAMECLASSHOSTSADDRESSPORTSAGE
ingress.networking.k8s.io/siwi-service<none>demo-siwi.local8059m

NAMEBUILDSTATESERVINGSTATEBUILDERSERVINGAGEfunction.core.openfunction.io/nebula-siwiSucceededRunningbuilder-sbfz6serving-vvjvl26hHow to Build the imagedockerbuild-tweygu/siwi-frontend.-fDockerfile.froentend
dockerpushweygu/siwi-frontendFurther workUseNBA-APIto fallback undefined pattern questionsWrap and manage sessions instead of get and release session per request, this is somehow costly actually.Use NLP methods to implement proper Symentic Parsing, Intent Matching, Slot FillingBuild Graph to help with Intent Matching, especially for a general purpose botUse larger Dataset i.e. fromwyattowalsh/basketballThanks to Upstream Projects ❤️BackendI learnt a lot from theKGQA on MedicalKGcreated byHuanyong LiuFlaskpyahocorasickcreated byWojciech MułaPyYamlFrontendVueJSfor frontend frameworkVue Bot UI, as a lovely bot UI in vueVue Web Speech, for speech API vue wrapperAxiosfor browser http clientSolarizedfor color schemeVitesomefor landing page design