# Wi-Fi Performance Testing

## Summary


Often customers complain to their ISPs about low data rates on their broadband or fibre connections, when the actual source of the problem is the variable performance of Wi-Fi. Wi-Fi performance is influenced by many factors including the quality of devices on the network, other networks in the area that are causing interference, or obstructions such as walls. 

When testing Wi-Fi for performance, it's important to measure beyond just the throughput involving only a single active client. Testing should mimic real-world scenarios as much as possible. Performance testing for Wi-Fi clients is usually simpler than that of access points.

The purpose of such testing is to validate a deployment or identify performance issues so that corrective actions can be taken.

## Discussion

### What are the main metrics to measure during Wi-Fi performance testing?

A good reference for Wi-Fi performance testing is TR-398, *Indoor Wi-Fi Performance Test Standard*, defined by the Broadband Forum. This looks at six metrics: RF performance, bandwidth, stability, interference, capacity, and coverage. Bandwidth deals with throughput and fairness of resource allocation across stations. 

We should check for seamless connectivity as clients move across access points. To identify bottlenecks, we need to look at CPU utilization, memory usage, and buffer status.

Application-level metrics matter as well. Packet loss, latency and jitter are important metrics for audio and video streaming applications. These can be translated into an objective Mean Opinion Score (MOS) to determine how the user perceives quality. 


### In a real-world Wi-Fi deployment, what performance tests should I run?

In a typical deployment, engineers check for signal levels, coverage and throughput to ensure basic performance requirements are met. Usually a single client is used but this is far from real-world scenarios. It's therefore important to check performance when the network is loaded with multiple busy clients. Since it's hard to use hundred of real clients, an emulation-based approach can be adopted. 

Wi-Fi environment is too dynamic to reproduce performance issues. In fact, performance testing doesn't help with troubleshooting problems. A better approach is to collect data from live networks and analyse this data. With modern Wi-Fi networks being controlled from the cloud, it's become possible to adjust a configuration, collect performance data, send it to the cloud and then evaluate if the adjustment had a positive effect. 

Some performance problems could be due to misconfiguration such as access point connected to the wrong switchport, poor quality cables, wrong channel assignments, or bandwidth limits unknowingly imposed. 


### What traffic models should I use for Wi-Fi performance testing?

To mimic real-world scenarios, a variety of traffic types coming from multiple devices must be used. This may include a mix of TCP and UDP data, streaming video or audio, voice calls, and a mix of mission-critical and recreational apps. 

The idea of mixing different traffic types is that pattern of packet arrivals is different. Voice is periodic while video starts with a surge and then a constant rate. HTTP traffic comes in bursts as pages are loaded. We should thus monitor buffers and CPU utilization for bursty traffic as well.

A classroom scenario could include multicast video streaming, parallel file downloads, and background social media traffic of small packets. A dormitory scenario could include parallel video streaming, two-way video chats, online gaming, web browsing, and social media traffic. A stadium scenario would be heavy on video streaming or audio commentary involving thousands of clients. Performance testing should consider these diverse real-world deployment scenarios.


### Could you share some insights discovered by others during Wi-Fi performance testing?

Before starting any tests, check all cabling and configuration. Devices should be configured to exercise their maximum capability. Turn off other devices that are not part of the tests. Avoid a VPN while testing. Restart devices, stop background tasks or updates, disable antivirus, or even do a clean OS installation before starting the tests. Update devices to latest drivers. 

One suggestion is to establish a baseline application throughput. This can be done by copying a large file from a server connected via Ethernet to the wireless router, to a client device connected via Wi-Fi to that router. No other clients should be on the network. Then you can add more clients to the network or try streaming traffic to see how these affect the baseline. 

It's been noted that CPU utilization is higher than usual and varies the most when packet sizes are small, such as for VoIP traffic. Another finding is that device OS plays a role as well. For example, Windows clients have a longer inter-packet idle periods, thus resulting in lower throughput compared to Linux clients.


### What tools are available for doing Wi-Fi performance testing?

While there are many online sites to help you measure your Internet speed, this is really an end-to-end performance test. To test the performance of only the Wi-Fi network, you should do an internal test. Likewise, the speed shown in Windows OS is the transfer speed between the PC's adapter and the router, not the throughput seen by applications. 

For network deployment, there are tools for site survey and generation of heat maps of signal strengths: Ekahau Site Survey, iBwave, NetScout AirMagnet, Tamograph Site Survey, SolarWinds, VisiWave Site Survey, AirMagnet Survey, Acrylic Wi-Fi, and NetSpot. 

For throughput testing, iPerf3 (and its variants), Tamosoft, HE.NET Network Tools, and Ixia's IxChariot are examples. iPerf3 is open source and many tools such as WiFiPerf Professional are built on top of iPerf3. 

Spirent offers products that can emulate hundreds of Wi-Fi clients with different traffic types and also emulate movement across access points. Similar emulation is offered by Alethea and LANforge WiFIRE.

## Milestones

2010

ESNet begins work on **iperf3** since iperf2 was not updated for a while. The goal is to keep the tool simple and maintainable. Hence it's designed as a single thread as opposed to the multithreaded nature of iperf2. 

2014

Bob McMahon of Broadcom revives **iperf2**, fixing many of its problems and adding new features. Version 2.0.8 is released in 2015. Since iperf2 is multithreaded, it may be a better choice for generating parallel traffic streams. 

Feb  
2019

Broadband Forum releases TR-398, Indoor Wi-Fi Performance Test Standard. Prior to this standard, it's been generally difficult to compare performance of different devices. TR-398 enables systematic quantitative testing and performance comparisons. In it's first release, TR-398 doesn't cover 802.11ax.