Repository: wix/Detox

Issue #4075: [SG4] Implement 2 hops with a wstunnel-like solution❗
Created: 2023-05-22T11:25:29Z
Object of the Research
The research aimed to evaluate the effect of network tunneling (via WebSocket) techniques on the performance of Detox testing in debug and release configurations of React Native applications.
The objective was to understand the latency and potential limitations when running Detox tests remotely, focusing specifically on the implications for large RN bundles in debug configurations and the impact on smaller network requests in release configurations.
Research Methodology
The experiment was conducted using a local Mac Mini serving both as the user machine and the simulator host, and a cloud server on DigitalOcean acting as the reverse proxy. For simplicity, and to simulate the latency involved in real-world scenarios, the communication loop was set from the local Mac Mini to the DigitalOcean server and back to the same local Mac Mini.
User Computer and Mac Mini (Reverse Proxy Client & Socat Forwarding):
Script 1:
const wstun = await import('@mdslab/wstun');
const reverse_client = new wstun.default.client_reverse();
// ask the reverse proxy server to open 10001 port, so that:
// server-name:10001 <--> with localhost:8082
reverse_client.start(10001, 'ws://server-name:5555', 'localhost:8082');
Script 2:
# forward requests on :8081 to the real React Native packager on :8082
socat TCP-LISTEN:8081,fork TCP:localhost:8082
Script 3:
npx react-native start --port 8082
Reverse Proxy Server (DigitalOcean):
const wstun = await import('@mdslab/wstun');
const reverse_server = new wstun.default.server_reverse();
reverse_server.start(Number(process.env.PORT || '5555'));
This setup simulated an environment where a remote iOS simulator communicates with the React Native packager on the user’s machine through the WebSocket-operated network tunnel:
npx detox test -c ios.debug
Results
The experiment resulted in varying latency based on the size of the RN bundles and the configuration:

In debug configurations:

for a large bundle (25MB), the tunneling technique added a substantial overhead: the latency increased threefold, from 11 seconds to 33 seconds.
for a small bundle (2.5MB), the added latency was minimal: from 11 seconds to 15 seconds.


In release configurations, where an application typically makes only small network requests to the mock server are made, with a total roundtrip on average taking around 0.3s (time to connect was 0.07s). On the contrary, direct localhost:* requests took about 0.03 seconds (x10 faster).

Throughout the experiment, the setup showed promising stability with no observable glitches over an hour of operation.
Conclusion
The results suggest that advanced tunneling techniques can influence the performance of Detox tests running remotely in both debug and release configurations of React Native apps. While larger bundles (debug configurations) may experience a significant latency, the overhead for smaller bundles and requests (release configurations) seems to be quite negligible.
Making more than a few sequential calls to a mock server might introduce a noticeable delay (more than 1 second) in tests, so we'd recommend batching requests for optimal performance when needed.
The short-term stability of the solution appears promising, although long-term stability and reliability are yet to be fully established.
To replicate this experiment, the described setup can be followed, simulating similar network conditions for consistency of results, i.e., loop from the local machine to a cloud server (e.g., DigitalOcean) and back to the same local machine.

Duplicate issue #4073: [SG] wstunnel PoC
Created: 2023-05-22T11:15:18Z

Closing comment (2023-07-13T08:45:31Z): Duplicate of #4075.