## 3 Problem Description and Proposed Solution
### Method of the Proposed Approach
#### 3.2.3 Optimized PC Algorithm Based on Knowledge Graph
Figure 6 is an example MSA; there is a time-out exception at microservice 3, which is the source vertex in the algorithm, and there are four possible causal paths:

* Path 1: CPU usage of container 3 \(\rightarrow\) Response time of microservice 3
* Path 2: Memory usage of container 3 \(\rightarrow\) Response time of microservice 3
* Path 3: CPU usage of container 1\(\rightarrow\) Response time of micro-service 1 \(\rightarrow\) Response time of microservice 3
* Path 4: Memory usage of container 1 \(\rightarrow\) Response time of microservice1 \(\rightarrow\) Response time of microservice 3

According to the analysis results, the time-out problem of microservice 3 might have been caused by either the high CPU or memory usage of container 3 or the high CPU or memory usage of container 1. O&M personnel can further determine the root cause of the exception by auditing the logs of container 3 and container 1. By combining with the weight assignment method described above, the path can be sorted by rules. In this study, we sorted the paths by two rules:

* Rule 1: The sum of the weights of all edges on the path: the larger the sum, the higher the priority. * Rule 2: The length of the path: the shorter the length, the higher the priority. As shown in Figure 7, the highest priority of sorting is most probably the root cause path. ## 4 Empirical Study

This section introduces how we validate the proposed method on a classic microservice framework application named Sock Shop ([[https://microservices-demo.github.io](https://microservices-demo.github.io)]([https://microservices-demo.github.io](https://microservices-demo.github.io))). We developed a prototype that could help explore the O&M knowledge graph of the Sock Shop infrastructure, as shown in Figure 8. We also implemented the root cause search module shown in Figure 9. Finally, we did further data experimental verification for the causal search part. Figure 6: A root cause tracing example for performance anomaly. Figure 7: Root cause identification with weight priority. ### Test-Bed and Experiment Environment Setup

Sock Shop is a classic microservice application system based on Kubernetes. As Figure 10 shows, the entire Sock Shop system consists of User, Front-end, Order, Payment, Catalogue, Cart, and Shipping microservices. Each microservice can run independently and has a separate database. The implementation languages and databases of the different microservices in Sock Shop are different. The communication between microservices in Sock Shop is mainly HTTP. All service interfaces conform to the RESTful interface design style, and all services are hosted based on containers. Figure 8: Explore function O&M knowledge graph in the prototype. Figure 5.0 Causality search function in the prototype. As shown in Figure 11, the entire deployment environment is divided mainly into the controller server and the cloud platform target test environment. We simulated the abnormal behaviors by injecting and disturbing the target cloud environment through the controller server applications. We used the chaos-testing tool Chaos Toolkit ([[https://chaostoolkit.org](https://chaostoolkit.org)]([https://chaostoolkit.org](https://chaostoolkit.org))) for system disturbance, and a stress test framework Locust ([[https://locust.io](https://locust.io)]([https://locust.io](https://locust.io))) for the stress testing scenario. Ansible ([[https://www.ansible.com/](https://www.ansible.com/)]([https://www.ansible.com/](https://www.ansible.com/))) was used to do the automation task in the experiment. At the same time, various data collection tools were used in the experiment.