Preventing and Tackling Outbreaks
By: Luís Rita
Tags: Outbreak, Data Visualisation, Unsupervised Learning, Phylogenetics, Web Applications

Results

Web Application

In order to facilitate the analysis and visualization of the results after the execution of the previous algorithms, a web application was implemented (mscthesis.herokuapp.com).

Its backend was conceived using Node.js and it runs in a Heroku server which is linked to a GitHub repository (github.com/warcraft12321/Thesis) containing all the implementations and documents related to the project. A Digital Object Identifier (DOI) was attributed to this repository using Zenodo (zenodo.org/badge/latestdoi/162063699). An image of the app is available at Docker Hub (cloud.docker.com/u/warcraft12321/repository/docker/warcraft12321/thesis). The following actions are strictly executed in the cloud:

Generation of GN and LFR benchmark networks. The latter are generated using a binary from a C++ implementation [22]. This was possible after importing a package from NPM which simulates the command line in the server;

The real test networks had their data properly processed in the cloud, so that it could be sent and displayed in the frontend;

Once the execution of Louvain, Infomap and LLP algorithms is not performed in the user’s browser, this may raise some privacy issues. Although, it was not feasible to run Infomap in the user side and still keep the web application functional in all devices;

The communication between the app and NPM was established in the server, so that it is possible to import and plot the statistics from each package uploaded to NPM in the application;

Finally, all benchmark data was obtained after running the algorithms in the cloud.

The frontend is both static and dynamic. The static counterpart uses HTML, CSS and JavaScript. The dynamic execution is managed by Node.js, which runs in the server-side (Figure 8).

Figure 8 Web application.

Community Finding Algorithms

In terms of accuracy, Louvain outperformed all others for weakly mixed networks (μ < 0.4). It is followed by Infomap, LP and LLP (γ = 0.5) for μ < 0.3 or Infomap, LLP and LP (γ = 0.5) for 0.3 ≤ μ ≤ 0.4. By increasing μ, LLP, which is able to detect communities based on the panorama of the whole network, consistently becomes the most effective for μ ≥ 0.5. Comparing Louvain and Infomap when μ ≥ 0.5, their performance is similar. Based on the CI 95%, it is not possible to state one performs better than the other. Another point to highlight is that Louvain and Infomap are very sensitive, consequently, unreliable detecting communities in networks with μ ≈ 0.5. LLP and LP do not present any specific value in which community detection is steeply affected. These conclusions are all valid to GN and LFR networks (Figure 9 and Figure 10).

Figure 9 Clustering quality of each algorithm: Louvain, Infomap, LP and LLP in terms of the mixing parameter in GN networks, with the remaining properties as in Figure 2.

Figure 10 Clustering quality of each algorithm: Louvain, Infomap, LP and LLP in terms of the mixing parameter in LFR networks, with the remaining properties as in Figure 3.

Having considered only two gamma values in LLP, a more precise analysis was needed to check whether there could be others achieving better results.

In the case of GN network (Figure 11), it is possible to detect communities with higher accuracy in networks less mixed. The performance decreases whenever we increase γ for μ < 0.5. When μ ≥ 0.5, the NMI between the detected partition and the one in the original network is higher as long as we keep increasing γ. Another point to highlight is the possibility of detecting communities with higher precision in networks with higher μ using appropriate γ than in others with lower μ but with a non-optimal γ. This is valid for networks with μ > 0.5(Figure 12).

Figure 11 GN network (Figure 2) after running LLP. When γ is closer to 0, the coarse structure of the network with few, big and sparse communities is highlighted. As γ grows, fine-grained structure is unveiled. Communities become smaller and denser. Represented using D3.js and SVG.

Figure 12 LLP algorithm accuracy in terms of the resolution parameter. Analysis performed for mixing parameters 0–1 (blue — brown) in GN network, with the remaining properties as in Figure 2.

In LFR networks, increasing gamma always leads to a decrease in the NMI (Figure 13).

Figure 13 LLP algorithm accuracy in terms of the resolution parameter. Analysis performed for mixing parameters 0–1 (blue — brown) in LFR network, with the remaining properties as in Figure 3.

The influence of the average node degree was tested to check whether it affects the capacity of each algorithm to identify communities.

In the case of GN networks, for μ ≤ 0.5, an increased average degree enhances the capacity of Louvain, Infomap, LP and LLP to differentiate communities. For μ > 0.5, NMI clustering quality is lower as higher is the average node degree. Assuming it is not expected the algorithms to identify the original communities for μ ≈ 1, this suggests that higher average degrees tends to decrease detection by chance. This observation can be applied to GN and LFR networks (Figure 14).

Figure 14 LLP algorithm accuracy in terms of the mixing parameter. GN network with average node degrees: 15, 20 and 25. The remaining properties are similar to the ones in Figure 2.

In terms of speed, Louvain algorithm performed significantly better than any other. One important property that contributes to its efficiency is that it only calculates modularity variation at each iteration of Modularity Optimization phase (1). In Infomap, minimum description length is recalculated at the end of each iteration of the optimization step (2). Due to this computationally intensive step, Infomap presented the poorest performance in the benchmark tests. Based on Table 1, it was expected the difference of execution time between Infomap and all others to be logarithmic. Due to inefficient steps in the implementation, this was not achieved. Comparing LLP and LP, the fastest was the one that needed to optimize the simplest equation — LP (Figure 15). Although, time complexity appears to be the same, as predicted in Table 1. Again, using the same table, it was expected LLP and LP to execute about as fast as Louvain. The small difference observed in Figure 15 might be explained by some non-optimized step in the implementation of LLP/LP.

Figure 15 Time that Louvain, Infomap, LLP and LP took to finalize the analysis in terms of the size of the LFR networks. The remaining properties are similar to the ones in Figure 3.

Benchmark Networks Algorithms

Previous GN and LFR networks were generated so the previous tests could be performed. The algorithm to create the former was implemented in the project. The time it takes to generate such networks increases exponentially with the average node degree and the mixing parameter (Figure 17). Becoming unfeasible to execute it for average degrees higher than 20, in an ordinary computer. Its performance was compared to the Fortunato’s implementation which is highly scalable and allows the generation of networks with much higher number of links in shorter times (Figure 16). It was verified an approximately linear increase in the time of generation of networks with progressively higher mixing parameters, contrarily to the one implemented in the project. Just like before, generating networks with more edges, requires additional computational power.

Figure 16 Time needed to generate GN networks (using LFR implementation) in terms of their mixing parameter. Analysis performed in networks with average node degrees: 15, 20 and 25. The remaining properties are similar to the ones in Figure 2.

Figure 17 Time needed to generate GN networks (using project implementation) in terms of their mixing parameter. Analysis performed in networks with average node degrees: 15 and 20. The remaining properties are similar to the ones in Figure 2.

Additional tests were performed in LFR networks. The execution time was tested against the number of nodes and the mixing parameter of the network. In both cases, it was verified an approximately linear relation (Figure 18 and Figure 19). The number of nodes was not considered in GN networks, once they have a fixed number of elements per community and of communities. In spite only the mixing parameter and the number of nodes were analyzed, the user can still choose to adjust nodes maximum degree, the exponents of the nodes degree distribution and communities size distribution, the maximum/minimum number of nodes per community, the number of overlapping nodes per community (covers were not considered in the project) and the number of memberships for the overlapping nodes [22].

Figure 18 Time needed to generate LFR networks in terms of their number of nodes. Analysis performed in networks with average node degrees: 15, 20 and 25. The remaining properties are similar to the ones in Figure 3.

Figure 19 Time needed to generate LFR networks in terms of their mixing parameter. Analysis performed in networks with average node degrees: 15, 20 and 25. The remaining properties are similar to the ones in Figure 3.

Every community finding, benchmark network generator, accuracy measure and related algorithms were made available in NPM (Table 2). This way, they can be used in any user-specific application.

Table 2 Algorithms implemented in the project were made available in NPM.

Visualization Frameworks

D3.js (SVG) allowed the user to visualize all networks used in the web application, as well as to represent the benchmark plots with the highest resolution possible (limited by density of pixels in the user’s device). In terms of performance, it is the second fastest.

D3.js (Canvas) is the fastest among the analyzed frameworks. Nevertheless, when compared to the previous, the nodes and links from the network lose resolution.

Cytoscape.js is advisable when network analysis will be performed along with its representation. There is a considerable number of algorithms, including community finding ones, that can facilitate the study of the graph. The lack of working examples and the slow processing of network data made Cytoscape.js the less efficient. Representation of S. aureus SLV network and others with similar or higher number of nodes/edges overflows the web application.

Staphylococcus aureus

Louvain, Infomap and LLP algorithms were executed in the biggest component of the SLV graph obtained from the MLST dataset of S. aureus.

Louvain algorithm was executed using three modularity variation thresholds: 0.1, 0.02 and 0.01. It is possible to verify that high modularity partitions are rapidly achieved (2 passes needed to obtain final result), as predicted in Introduction. This characteristic makes modularity-based algorithms fast.

Using Infomap algorithm, several minimum description length thresholds were considered: 0.1, 0.02 and 0.01. For all values, every small dense group of nodes descending from the same ancestor was separated in different communities. Thus, it was not further considered.

LLP algorithm considered three values for the resolution parameter: 0, 0.5 and 1. Similarly to Infomap, it partitioned the network in several small and dense communities.

From the benchmark analyses of Louvain, Infomap and LLP against GN network (Figure 9 and Figure 10), it was predictable Louvain would perform better than the others, once it run in a network with well-defined communities. After adjusting the stopping parameter of Louvain, such that the communities identified were the most pertinent — nodes from densely connected regions descendent from the same ancestor belonging to the same community, a final partition was obtained (Figure 20). It is important to note that some clusters analogous to the one highlighted with a square were subdivided by the algorithm. One explanation is the presence of high recombination rates between strains of different sub-groups.

By uploading to PHYLOViZ Online the generated metadata file with the nodes labeled accordingly to the inferred communities, one may observe the original phylogenetic tree with the same communities identified.