## 3 Results
### Scaling and Reliability
#### 3.4.3 Analysis of Stress Testing Results
\begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Model Name** & **Type** & **10 Users** & **20 Users** & **30 Users** & **40 Users** & **50 Users** \\ \hline \multicolumn{7}{c}{Single-Instance Models} \\ \hline
**Qwen2 2b** & VLM & \(5.559\pm 2.696\) & \(10.435\pm 5.554\) & \(15.412\pm 9.321\) & \(22.001\pm 12.731\) & \(24.624\pm 14.281\) \\
**Qwen2 7b** & VLM & \(5.328\pm 2.281\) & \(14.707\pm 6.930\) & \(16.487\pm 8.810\) & \(22.261\pm 11.897\) & \(38.554\pm 21.313\) \\
**Llava 13b** & VLM & \(8.936\pm 4.228\) & \(15.412\pm 7.147\) & \(21.148\pm 12.046\) & \(29.623\pm 17.567\) & \(29.830\pm 18.053\) \\
**Llama 1b** & LLM & \(7.295\pm 4.926\) & \(12.455\pm 7.518\) & \(23.649\pm 11.872\) & \(32.659\pm 17.825\) & \(46.858\pm 24.627\) \\
**Llama 3b** & LLM & \(14.966\pm 7.089\) & \(23.895\pm 11.596\) & \(34.184\pm 18.992\) & \(50.214\pm 28.914\) & \(55.470\pm 31.084\) \\
**Llama 8b** & LLM & \(20.624\pm 11.779\) & \(37.698\pm 21.906\) & \(56.025\pm 34.384\) & \(65.835\pm 33.842\) & \(102.509\pm 60.606\) \\ \hline \multicolumn{7}{c}{Multiple-Instance Models (Ray)} \\ \hline
**Qwen2 2b** & VLM & \(\mathbf{4.667\pm 1.193}\) & \(\mathbf{3.734\pm 1.297}\) & \(\mathbf{4.635\pm 1.823}\) & \(\mathbf{5.919\pm 2.563}\) & \(\mathbf{6.673\pm 4.006}\) \\
**Qwen2 7b** & VLM & \(\mathbf{3.122\pm 1.379}\) & \(\mathbf{4.478\pm 2.458}\) & \(\mathbf{6.667\pm 3.532}\) & \(\mathbf{10.054\pm 6.426}\) & \(\mathbf{8.461\pm 3.814}\) \\
**Llava 13b** & VLM & \(\mathbf{3.342\pm 1.093}\) & \(\mathbf{5.512\pm 1.950}\) & \(\mathbf{7.949\pm 4.255}\) & \(\mathbf{8.024\pm 3.564}\) & \(\mathbf{10.024\pm 4.667}\) \\
**Llama 1b** & LLM & \(\mathbf{5.092\pm 3.495}\) & \(\mathbf{4.664\pm 3.667}\) & \(\mathbf{6.172\pm 3.961}\) & \(\mathbf{9.492\pm 5.306}\) & \(\mathbf{9.169\pm 7.175}\) \\
**Llama 3b** & LLM & \(\mathbf{5.208\pm 3.357}\) & \(\mathbf{6.093\pm 4.569}\) & \(\mathbf{9.046\pm 4.595}\) & \(\mathbf{8.723\pm 5.127}\) & \(\mathbf{15.909\pm 7.837}\) \\
**Llama 8b** & LLM & \(\mathbf{5.889\pm 3.477}\) & \(\mathbf{12.563\pm 6.528}\) & \(\mathbf{16.241\pm 7.456}\) & \(\mathbf{14.167\pm 8.645}\) & \(\mathbf{16.938\pm 11.953}\) \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Performance comparison of single-instance and replica models. Single instance refers to a single replica of each Llama model, whereas replica Models consist of 20 replicas for the Llama-1B and Llama-3B models and 12 replicas for the Llama-8B model. The table presents the mean response time (standard deviation) in seconds for each user group (10, 20, 30, 40, and 50 concurrent users). These results demonstrate the robustness of the scaling mode container, illustrating that as the number of concurrent users increases, the response time per query becomes more pronounced, particularly for single-instance models. These results underscore JAVIS's capacity to handle substantial user loads, even on a relatively modest four-GPU A6000 cluster. In a real-world scenario, it is rare that all users query the system simultaneously; there is usually \"think time\", reading time, or idle periods between queries. As such, the effective number of users that can be supported in production scenarios is even higher. For instance, in our lab setup, with a single GPU card and just two instances, supporting around 40 researchers with acceptable latency is achievable. Stress tests, which were deliberately designed to push the system with simultaneous, synchronized queries, indicate that JAVIS can scale reliably. The back-end infrastructure has been validated to operate consistently under heavy loads, maintaining communication with the containers without failing or degrading catastrophically. In summary, these experiments show that with a four-A6000 GPU cluster, even models at 8B parameters (**Llama 8B**) can serve up to **50 simultaneous users** in approximately **15 s** (multiple instances) versus over 100 s (single instance).