Project: ShardingSphere (App Definition and Development / Database)
File: Jan_28_Full_Link_Online_Stress_Testing_for_Production_Database_Apache_ShardingSphere_Shadow_Database_Feature_Upgrade.en.md

+++ title = "Full Link Online Stress Testing for Production Database: Apache ShardingSphere Shadow Database Feature Upgrade" weight = 34 chapter = true +++ As the Internet industry is growing rapidly, businesses that operate with large amounts of data are seeing a rapid expansion. Predictably, ever changing customer demand is having a significant impact on the stability of their whole systems. For example, online food delivery platforms receive most of their customer orders at noon and in the evening. Online shopping sprees and time-limited sales promotions are also good examples. All businesses are served by a series of business systems which are distributedly deployed in different machines. Data Plannig cannot only ensure systems stability but also save costs, which are some of the major problems for technology teams. To precisely get the right service from a specific machine, stress testing should be conducted in the production environment. This can ensure the authenticity of the environment and the data, significantly improving the precision of data planning. Performing a stress test on an online business system is evidently risky. For example, data corruption or performance problems may arise. Just imagine how it would compromise customer experience, if customers found their order has been lost or an unpaid order popped up? Full-link online stress testing implies a significant amount of complicated work which requires cooperation between microservices and middlewares. focuses on database-level solutions in full-link stress testing. With ShardingSpheres powerful SQL parsing capability, Apache ShardingSphere released the shadow database stress testing feature to determine shadow databases by executing SQL, and to meet the online stress testing needs in complicated business scenarios with the flexible configuration of shadow algorithms. By routing stress testing traffic to shadow databases, and normal online traffic to production database, stress testing data will be isolated and the data corruption problem is solved. Shadow database function was initially realized in version 4.1.0 by adding a logic shadow column. By parsing, executing, routing and rewriting SQL, Apache ShardingSphere deleted the shadow column and column value. Users do not need to setup or do anything during this process. They only need to modify SQL accordingly, add shadow fields and corresponding configurations. Before performing stress testings, users need to modify test-related SQL according to actual business needs. SQL modification will increase implementation damage and reduce the accuracy of stress testing results. After discussion in the ShardingSphere community, we decided to upgrade shadow database functions. Apache ShardingSphere 4.1.1 GA shadow database API had relatively simple

Whether to open shadow database is determined by corresponding values of the logicColumn. The upgraded shadow database API of 5.0.0 GA is more powerful. Users can determine whether the shadow database function is enabled through the enable attribute. Configurable shadow table can determine what needs to be stress tested by means of a table, and supports multiple shadow algorithms. For example, column value matching algorithm, regular expression matching and SQL comment matching algorithm. Online full-link stress testing diagram: Suppose an e-commerce website needs to perform online stress testing for an order (demonstrating how to use stand-alone deployment). Suppose stress testing table `t_order` is an order table, and the ID of the test user is 0. The data generated by test user order is executed on ds_shadow shadow database, and the production data is executed on ds production database. Prepare testing environment: Download ShardingSphere-Proxy 5.0.0 GA from the , and for installation configuration details, please refer to . Configure ShardingSphere-Proxy in the hypothetical stress testing scenario mentioned above: server.yaml config-shadow.yaml Order service Order-related businesses are not discussed here. Considering the simplest request taking and order table inserting as an example, the order table structure is as follows: Order Table Structure Use `postman` to simulate the order created by test user, which is as follows: SQL executor being routed to and executed in shadow database can be seen in ShardingSphere-Proxy execution log: Shadow database `dsshadow` executes query sentence `SELECT * FROM torder;` Query result: Production database ds executes query sentence `SELECT * FROM t_order;` Query result: The data generated from test user order creation will be routed to shadow database. For more complicated configurations, please refer to Shadow Database Stress Testing in the . As mentioned in the introduction, full-link online stress testing is a complicated task that requires collaboration between microservices and middlewares to meet the needs of different traffic and stress testing tag transmissions. Additonally, the testing service should be stateless and immediately available. , which is jointly maintained by Apache ShardingSphere, Apache APISIX and Apache SkyWalking provides out-of-the-box (OoTB) solution to run load test in your online system. is responsible for making tags on testing data at the gateway layer, while is responsible for transmission through the whole scheduling link, and finally, Apache ShardingSphere-Proxy will isolate data and route testing data to the shadow database. The 0.1.0 version of CyborgFlow has been . HouYang SphereEx Middleware Developer, Apache ShardingSphere Contributor. Currently he focuses on the design and development of ShadowDB and full-link stress testing.