In the previous article we went over a load test whose data could be used to choose a load-appropriate server.  In the course of the testing, we would publish a stream on one WCS, and we would pick up that stream several times using a second WCS. The acquired results could be used as a basis for decisions on server operability.  Some would (justly) have concerns regarding the possible biases in such a test — after all, one of our servers was used to test another one of our servers. Could it be that we were using a specially optimized code that skewed the results in our favor?  It is true that the end user will not use a second server to watch streams. The end user will watch them in browser.  This is why, seemingly the simplest and most logical way would involve manual testing: open a browser, open a tab with the player, specify the stream name and click "Play;" and then repeat 1000 times. All that's left is to find a guinea pig a volunteer tester and a PC that could handle 1000 tabs with video. Alternatively, and more realistically, the same test could involve a group of people and multiple PCs. Or we could just use scripts. In this article we are going to discuss another testing method, one that uses a headless browser. We will compare the test results with the results from the stream capture-based testing.   Нeadless-browser is something without a head. In the context of the frontend, it is an indispensable developer tool with which you can test code, check the quality and consistency of the layout, programmatically create scenarios of user interaction with the site, followed by fixing the results of these scenarios for use in tests.  Headless Chrome) is a full-featured browser without a graphical interface, which means it draws everything in memory. Headless Chrome is faster and uses less memory than a regular browser.   The last statement seems like a contradiction. But the reduction in memory usage is achieved by the absence of a graphical component. The headless browser has no real content rendering, which means that it does not need to render illustrations weighing several gigabytes, with which modern websites are often flooded. At the same time, the headless browser will download all the content from the web page, just like a regular browser. Headless Chrome is software-driven using an API and can be installed on "pure" Linux. You just need to install the package, and the browser will work out of the box, just like its brother with a head – Google Chrome. Load testing using Headless Chrome will be closest to reality because it simulates real users connecting to WCS using a browser. So here we go. We will install on the server with the following characteristics 2x Intel(R) Xeon(R) Silver 4214 CPU @ 2.20GHz (a total of 24 cores, 48 streams); 192GB RAM; 2x 10Gbps a WCS and perform work to prepare for operation in production. Read more in the article and in documentation.   In the standard Two-way Streaming example, we published the stream from the virtual camera titled "stream1".  That's all. Now leave the server to wait for the load test.  A server with Ubuntu 20.04 OS was used in testing  Characteristics:  2x Intel(R) Xeon(R) Silver 4214 CPU @ 2.20GHz (a total of 24 cores, 48 streams); 192GB RAM; 2x 10Gbps Setup and testing process:  1. Install Xvfb, Xorg for working with a virtual output device: 2. Configure the virtual monitor – edit the configuration file /usr/share/X11/xorg.conf.d/xorg.conf: 3. Install Headless Chrome 4. Download and unpack the load testing scripts. An archive with load testing scripts is available for downloading here. 5. Go to the directory in which the archive was extracted The directory contains: player with Web-player scripts  start_xorg.sh - script for running Xorg  loadtest.sh - script for running tests The following parameters can be passed to the loadtest.sh script: required: ​url - URL of the player page with the necessary parameters ​ or  ​urlsfile - the path to the file containing several such URLs, which will be traversed in order optional: stressrate - interval of adding a new subscriber in milliseconds; ttl - subscriber lifetime in seconds; maxsubscribers - maximum number of subscribers. 6. Run testing The screenshot below shows the test result:   Unfortunately, because of the heavy load on the testing server it was not possible to connect to the test 1000 subscribers. Chrome, although headless, but took up all available CPU and RAM resources, with only 50 subscribers, as you can see in the CPU Load Averrage and memory consumption graphs. A long-lasting CPU Load Averrage value of more than 100 points indicates a high load on the processor. The RAM of the testing server also turned out to be occupied by more than 50%.  And if the RAM resource was still available, the CPU load did not allow to reach the planned number of spectators.  In this test it was possible to connect a little more than 50 subscribers. In this case, there is no heavy load on the WCS server.  CPU Load Averrage value does not exceed 1 point. Pauses in the operation of Z Garbage Collector did not exceed 3.5 milliseconds. There were no degraded streams.  The testing methodology is discussed in detail in the article and in documentation We will use two servers with the following specifications to test using stream capture: 2x Intel(R) Xeon(R) Silver 4214 CPU @ 2.20GHz (a total of 24 cores, 48 streams); 192GB RAM; 2x 10Gbps Let's launch load testing using the Console web app: 1. On the first server, open the Console app via HTTP http://your.WCS.server.name:9091/client2/examples/demo/streaming/console/console.html 2. Specify the domain name or IP address of the first server and click "Add node." This will be the server under test, which will be the source of streams. Then similarly connect the second server, which will simulate subscribers and capture streams. 3. For the first server, run the standard Two-way Streaming example and publish the stream from the web camera. The stream name can be anything. 4. In the Console app, select the second server, click 'Pull streams' button, and set the test parameters: Choose node — choose the first server; Local stream name — specify the name of the stream on the testing server in which the stream from the one being tested will be captured. (index corresponding to the number of the captured stream will be added to the stream name) Remote stream name — specify the name of the stream published on the server under test; Qty — specify the number of spectators (for our test — 1000). 5. Then press the "Pull" button to start the test:  The test result is on the screenshot below:  When testing using stream capturing, we managed to reach the planned number of viewers (1000 subscribers). The graphs show the increased load on the WCS server being tested.  The test using the second WCS server cannot be considered completely independent. We test our server using the second server of the same type. And, as it was already mentioned above, this variant is very rare in practice.  Despite the fact that we were not able to reach the estimated load of 1000 viewers when testing with Headless Chrome, this test clearly shows that with a small number of real subscribers there is no need to use powerful hardware to host WCS, and you can save some money when buying or renting a server.  Happy streaming!  Demo WCS on Amazon EC2 WCS on DigitalOcean WCS in Docker Load testing using Headless Chrome Settings file flashphoner.properties Settings file wcs-core.properties Server tuning recommendations Memory management in Java Before moving to production Load testing using WebRTC/RTMP pulling Quick deployment and testing of the server Load and resource usage information Monitoring with Prometheus Streaming video engineer