# Robust Text-to-Cypher Using Combination of BERT, GraphSAGE, and Transformer (CoBGT) Model
## 3 Proposed Model
### Relation-Properties Prediction Module  
This module predicts the matching between the questions and relevant relations or properties in the schema. We used the GraphSAGE [29] neural network; this model represents the connections between tokens in the question and the relevant relationship properties in the Cypher query. The reason we are choosing GraphSAGE is that, unlike traditional graph embedding methods that require a complete retraining on the entire graph for any new node, GraphSAGE leverages node features and a neighborhood sampling strategy to generate embeddings in an inductive manner. This makes it highly scalable and suitable for dynamic graphs where nodes and edges can frequently change. Question-schema relationship graph modeling  
Before using the GraphSAGE neural network model, we need to construct a graph that illustrates the relationship between the question and the schema's internal structure. The Graph G = (V, E) consists of nodes V and edges E inside the graph; V represents the node of the graph. In our Question-Schema Relationship Graph, we define two types of  
Figure 5: Key value extraction module process. nodes: V\({}_{1}\) and V\({}_{2}\). V\({}_{1}\) represents the word terms appearing in the input question, while V\({}_{2}\) represent the relation and properties appearing in the schema. Word term node (V\({}_{1}\)) building: First, we preprocess the input question by removing unnecessary words, such as stop words. The remaining text is then chunked and Part-of-Speech (POS) tagged. These chunks and their corresponding POS tags are added as V\({}_{1}\) node in the graph. Relations and properties node (V\({}_{2}\)) building: Next, for each Cypher query corresponding to the question used for word term node (V\({}_{1}\)) building, we extract the relations and properties from the query. Each extracted relation or property is added as a separate V\({}_{2}\) node. So, each V\({}_{2}\) node will contain only one relation or property. Establishing connections between V\({}_{1}\) and V\({}_{2}\): Finally, we establish connections between the V\({}_{1}\) and V\({}_{2}\) nodes. Let Q = {\(q_{1}\), \(q_{2}\), \(q_{3}\), \(\ldots\), \(q_{n}\)} represent the set of input questions and C = {\(c_{1}\), \(c_{2}\), \(c_{3}\), \(\ldots\), \(c_{n}\)} represent the corresponding set of Cypher queries. For each question \(q_{n}\), we have a list of word term nodes T = {\(t_{1}\), \(t_{2}\), \(\ldots\), \(t_{n}\)}. Similarly, from each Cypher query \(c_{n}\), we have a list of relation-property nodes P = {\(p_{1}\), \(p_{2}\), \(\ldots\), \(p_{n}\)}. Connections are established between each term node in T from \(q_{n}\) and corresponding relation-property node in P from \(c_{n}\). After that, a comprehensive representation of the relationships between the question and the schema is established. The question-schema relationship graph building example is also described in Figure 6.  
After creating the connection between V\({}_{1}\) and V\({}_{2}\) nodes, we also established a connection between V\({}_{1}\) nodes. The condition for the connections of two nodes is that those nodes belong to a question asking about the same schema and template question and two nodes have a text similarity  
After building the question-schema relationship graph and establishing the connection between V\({}_{1}\) nodes, we used GraphSAGE to learn the node representation (node embedding). The matching score between two nodes V\({}_{1}\) and V\({}_{2}\) was calculated by executing dot product \(p(v_{1}\), \(v_{2})=v_{1}\)-\(v_{2}\) (\(v_{1}\), \(v_{2}\) is the embedded vector of V\({}_{1}\) and V\({}_{2}\) node). We used the binary cross-entropy loss function:  
\[Loss_{bc}=-\frac{1}{N}\sum_{i=1}^{N}y_{i}.\text{log}\left(p(u_{i},v_{i})\right) +(1-y_{i}).\text{log}(1-p(u_{i},v_{i})) \tag{1}\]  
Relation-properties prediction process:  
The pseudo-code:  
Let us call the input question Q and schema S with relation-properties R = {\(r_{1}\), \(r_{2}\)... \(r_{m}\)}  
1.