## 3 Multilayer Measurement Methodology
### Measurement Tools, Performance Indicators, and Measurement Methodology
#### 3.2.2 Measurement Methodology  
These measurement tools can evaluate network performance from different perspectives, as shown in Table 2. In our concepts, the measurements are divided into two layers: the IP network layer and the wireless network layer. The IP network layer measurement evaluates end-to-end communication performance, and the wireless network layer measurement evaluates wireless link performance. Since the communication performance of the IP network layer includes that of the wireless network layer, it is unnecessary to investigate all indicators in every evaluation. The measurement methodology is as follows: The communication performance of the IP network layer is evaluated with _iperf3_ and _ping_ first. Throughput is the most straightforward indicator that judges whether the communication performance meets the application's requirements. If the result is satisfactory, no further consideration would be necessary. Alternatively, if a user is concerned about the communication performance for real-time applications such as network games, then packet loss, RTT, and jitter should also be examined. If the communication performance of the IP network layer is unsatisfactory, further investigation of the wireless condition is required. The measurement tool _iw_ provides RSSI and Tx bitrates as the wireless link information. RSSI is a well-known indicator of a wireless link. However, it is insufficient for properly understanding the wireless condition because the RSSI value cannot detect interference. Hence, the communication performance may be poor even if the RSSI value is high. Additionally, it would be difficult for non-specialists to understand such a condition. Our methodology introduces Tx bitrates. Tx bitrates would be a helpful indicator for understanding a wireless link performance because the logical transmission speed is directly related to the throughput. Since IEEE 802.11 ad employs a rate adaptation mechanism in that the bitrate is adjusted according to the wireless network conditions, checking the bitrate is effective for judging whether the wireless network environment is proper. Figure 1 illustrates concrete measurement methods. Our methodology introduces measurement nodes, i.e., sender and receiver, and an operational PC. For instance, in Figure 1a, the AP is a measurement node for STA1. However, it is difficult for users to run the tools on separate nodes simultaneously. Of course, users can operate both nodes via either of the nodes, but the operation may cause unnecessary traffic within the target communication flow. In addition, it is also difficult to operate two devices via the IEEE 802.11 ad link if either device is not within the same communication range or if both are at a distance where the communication is unstable. Therefore, our methodology employs an operation PC to manage the measurement nodes. As shown in Figure 1, the operation PC communicates with the measurement nodes via _ssh_ over an ad hoc mode and wired line. That is, in addition to an IEEE 802.11 ad \begin{table}
\begin{tabular}{c c c c c c} \hline \hline \multirow{2}{*}{**Measurement Tools**} & \multirow{2}{*}{**Indictors**} & \multicolumn{3}{c}{**Sender**} & \multicolumn{3}{c}{**Receiver**} \\ \cline{3-6}  & & **TCP** & **UDP** & **TCP** & **UDP** \\ \hline \multirow{3}{*}{_iperf3_} & Throughput & ✔ & ✔ & ✔ & ✔ \\  & Retry & ✔ & & & \\  & Jitter & & & & \\ \hline \multirow{3}{*}{_ping_} & RTT & ✔ & ✔ & ✔ & ✔ \\  & Packet loss & ✔ & ✔ & ✔ & ✔ \\ \hline \multirow{3}{*}{_iw_} & RSSI & ✔ & ✔ & ✔ & ✔ \\  & Tx bitrate & ✔ & ✔ & ✔ & ✔ \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Measurement tool and performance indicators employed for multilayer measurement. interface, the STA has a Wi-Fi interface, i.e., 2.4/5 GHz, to establish the ad hoc connection. It means that as the operation PC controls the nodes via an ad hoc connection, unnecessary traffic does not generate in the IEEE 802.11 ad communication. In Figure 1a, since we use a configurable AP running hostapd [37], the AP can execute the measurement tools.