# Experimental Evaluation of Graph Databases: JanusGraph, Nebula Graph, Neo4j, and TigerGraph
## 5 Experimental Evaluation
### Experimental Results  
The results are grouped by average node loading time, average query execution time, and average CPU and RAM consumption for query execution. We first compared the node loading results between the scale factors in each graph database when loading nodes. The percentage differences in node loading time between the scale factors for JanusGraph, Nebula Graph, Neo4j, and TigerGraph are shown in Table 1. It can be seen that, for all graph databases, there is a more significant percentage difference when going from scale factor 0.3 to 1 and from scale factor 3 to 10. This significant difference is due to the increase in the size of the scale factor datasets. This means that the size of the datasets more than triples when moving from one scale factor to the other. It should also be noted that as the scale factor increases, the number of nodes and relationships increases, so the average time for node loading increases for prominent scale factors. The total average time for each graph database at the different scaling factors is compared in Table 2.  
Neo4j had the lowest average node load time, along with higher scalability than the other graph databases. Although TigerGraph has very similar characteristics to Neo4j, it had the second-best average node load time, with lower scalability. The worst-performing databases were JanusGraph and Nebula Graph, with the worst average node load time, and therefore much lower scalability than Neo4j. According to the data presented in Table 2, TigerGraph had a 13% increase in average node load time compared to Neo4j, JanusGraph had a 28% increase, and Nebula Graph had a 42% increase. For each scale factor, 29 queries were executed in each graph database, grouped in three ways: fourteen complex queries, seven more specific and tapered queries called'short,' and eight queries that performed updates. Figure 7 shows the average execution time for each graph database (ms) for a scale factor (SF) of 0.1. Of all the executions performed, Neo4j was the database with the shortest execution time, followed by TigerGraph, then JanusGraph, and finally Nebula Graph. The query consisted of finding the shortest path between two people, and can take some time, because all paths have to be compared. All the short queries are also notable for their high execution time, as these types assume a tapered search and involve many nodes with many properties. \begin{table}
\begin{tabular}{c c c c c} \hline \hline  & **JanusGraph** & **Nebula Graph** & **Neo4j** & **TigerGraph** \\ \hline SF 0.1 \(\rightarrow\) 0.3 & 18\% & 17\% & 8\% & 11\% \\ SF 0.3 \(\rightarrow\) 1 & 54\% & 48\% & 70\% & 59\% \\ SF 1 \(\rightarrow\) 3 & 21\% & 28\% & 7\% & 12\% \\ SF 3 \(\rightarrow\) 10 & 100\% & 100\% & 100\% & 100\% \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Percentage increase in node load time between scaling factors for each graph database. \begin{table}
\begin{tabular}{c c c c c} \hline \hline  & **JanusGraph** & **Nebula Graph** & **Neo4j** & **TigerGraph** \\ \hline SF = 0.1 & 7044 & 9477 & 4168 & 6049 \\ SF = 0.3 & 8608 & 11,468 & 4552 & 6762 \\ SF = 1 & 18,682 & 22,181 & 15,029 & 16,494 \\ SF = 3 & 23,542 & 30,985 & 16,136 & 18,690 \\ SF = 10 & 547,642 & 609,012 & 429,412 & 486,344 \\ _Total_ & 152.14 \(h\) & _169.19 \(h\) & _119.29 \(h\) & _135.11 \(h\) \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Average node loading time (ms) in each graph database. Figure 7: Query runtime in milliseconds for SF = 0.1. Figure 8 shows the average query execution time (ms) in the graph databases for SF 0.3. The higher the scale factor, the larger the data size. However, the queries remain, and we again see that query 13 and the short queries have a longer average execution time for the same reasons as at a scale factor of 0.1. Nevertheless, the execution times were longer than the scale factor might indicate, because the data volume increased. The database with the shortest average execution time was Neo4j, followed by TigerGraph, JanusGraph, and Nebula Graph.