# Robust Text-to-Cypher Using Combination of BERT, GraphSAGE, and Transformer (CoBGT) Model
## 5 Discussion  
From Table 2, we observe that both the relation-properties and key-value features enhance the model's performance. However, the relation-properties feature plays a more significant role in this improvement. The key-value extraction module allows the model to focus on the most relevant parts of the question. It also helps reduce the noise in the input data, allowing the Transformers module to process the core components more effectively. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Method** & **Training Time (s/epoch)** & **Prediction Time (s)** \\ \hline Proposed model & 59.1921 & 0.7194 \\ T5 (base) & 158.2713 & 1.16944 \\ GPT2 (standard) & 223.2184 & 1.7131 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Execution time on training and prediction process. \begin{table}
\begin{tabular}{c c} \hline \hline
**Method** & **EM** \\ \hline Proposed model & 87.1\% \\ Without key value extraction & 72.74\% \\ Without relation–properties extraction & 52.26\% \\ Only transformers & 47.43\% \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Comparison of our method with and without key value extraction and relation–properties extraction. \begin{table}
\begin{tabular}{c l l} \hline \hline
**Question** & **Model Output** & **Ground Truth** \\ \hline \multirow{4}{*}{\begin{tabular}{c} Can you provide a list of actors who \\ performed in films released in 1992? \\ \end{tabular} } & MATCH & MATCH \\  & (a:Person)-[:ACTED\_IN]->(m:Movie) & (a:Person)-[:ACTED\_IN]->(m:Movie) \\  & WHERE m.released = 1992 & WHERE m.released = 1992 \\  & RETURN DISTINCT a.name & RETURN DISTINCT a.name \\ \hline \multirow{4}{*}{\begin{tabular}{c} Who are the employees responsible for \\ handling orders placed by customers \\ from Venezuela? \\ \end{tabular} } & MATCH & MATCH \\  & (c:Customer)-[:PLACED]->(:Order), & (c:Customer)-[:PLACED]->(:Order), \\  & (:Order)<[-SOLD]-(e:Employee) & (:Order)<[-SOLD]-(e:Employee) \\  & WHERE c.Country = “Venezuela” & WHERE c.Country = “Venezuela” \\  & RETURN e.FirstName & RETURN distinct e.FirstName \\ \hline \multirow{4}{*}{\begin{tabular}{c} Who are the actors who worked in films \\ directed by [PERSON]? \\ \end{tabular} } & MATCH \\  & (a:Actor)-[:ACTED\_IN]->(m:Movie), & (a:Actor)-[:ACTED\_IN]->(:Movie), \\  & (m:Movie)-<[:DIRECTED]-(d:Director) & (:Movie)-<[:DIRECTED]-(d:Director) \\  & WHERE d.name = “[PERSON]” & WHERE d.name = “[PERSON]” \\  & RETURN DISTINCT a.name & RETURN DISTINCT a.name \\ \hline \multirow{4}{*}{\
\begin{tabular}{c} Which movies include [PERSON] as \\ part of the cast? \\ \end{tabular} } & MATCH \\  & (m:Movie)-<[:ACTED\_IN]-(a:Actor) & (m:Movie)-<[:ACTED\_IN]-(a:Actor) \\  & WHERE a.name = “[PERSON]” & WHERE a.name = “[PERSON]” \\  & RETURN m.title & RETURN m.title \\ \hline \multirow{4}{*}{
\begin{tabular}{c} What are some films helmed by [PERSON] that premiered before 1993? \\ \end{tabular} } & MATCH \\  & (:Movie)-<[:DIRECTED]-(d:Director) & (m:Movie)-<[:DIRECTED]-(d:Director) \\ \cline{1-1}  & WHERE d.name = “[PERSON]” & WHERE d.name = “[PERSON]” \\ \cline{1-1}  & RETURN m.released \textless{}= 1993 & AND m.released \textless{} 1993 \\ \cline{1-1}  & RETURN m.title & RETURN m.title \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Some output prediction from our model. Similarly, relation-properties extraction also helps focus on the important relationship and properties in the schema. However, while the key-value feature only provides information with one or few entities, the relation-properties extraction focuses on information between many entities and relationships, so it provides more useful information. Besides using exact match calculation for evaluation, we could also apply execution accuracy, which compares the outputs of the generated queries. However, with our dataset, there are cases where executing the ground truth Cypher query does not return a value, even though the query is entirely correct. This occurs when the information being queried does not exist in the database.