## 4 Analysis
### Baseline Payload Information Entropy  
The results of our analysis are shown in Figure 6. This table shows average entropy values for common network services, together with their overall mean, directional mean, and standard deviations In a small number of cases, only one flow direction is recorded, typically because such protocols are unidirectional or broadcast in nature. Note that the service list has been derived dynamically from the datasets cited in Section 3.2. For further information on specific port allocations and service definitions, see [37]. The table forms a consistent view of the expected 'ground truth' across a range of deployment contexts. As might be expected, services that are encrypted (such as SSH) tend to exhibit high entropy (close to 8.0), whereas plaintext services (such as DNS) tend to have low to mid-range entropy values. It is worth noting that entropy values close to zero are unlikely to be observed in real-world network traffic, since this would equate to embedding symbol sequences with very little variety (for example, by sending a block of text containing only repetitions of the symbol 'x'), and even plaintext messages are likely to have entropy values in the range 3-4. This may not be immediately obvious and so in Figure 7, we show how low entropy values (close to zero) might be achieved by severely restricting the symbolic content artificially. Figure 6: Mean and standard deviation for payload entropy values averaged over multiple traffic sources, by flow direction (outbound and inbound, with respect to session initiation). Note that encrypted services such as SSH, SSL, and HTTPS have average entropy values closer to 8.0, whereas unencrypted services such as Telnet, LDAP, and NetBios have low entropy values, indicating that the payload has a larger proportion of plaintext data. These data were aggregated across multiple deployment contexts (enterprise, network backbone, industrial, etc.). To account for the wide variations in sample sizes for specific protocols between packet traces, we weighed the means by sample size, so that potential outliers in small packet traces do not influence the overall mean results disproportionately. As noted earlier, the sample count indicates the number of packet level observations found in the data, and here we see a wide variation in the distribution frequency across the services represented. For example, web based flows (HTTPS and HTTP) dominate the dataset composition, whereas older protocols such as TFTP are less well represented. While sample size can be used as a rough analogue for confidence in these baseline estimates, we have excluded services that had extremely low representation. We observed a strong consistency across many of the packet traces; however, it is worth noting that in practice, some services may exhibit deviation in entropy from baselines during normal activities, and this may depend on the context. For example, some services are specifically designed to encapsulate different types of file and media content that could vary markedly in composition and encoding (e.g., compressed video and audio content will tend to exhibit high entropy, whereas uncompressed files may exhibit medium-range entropy). Also note that peer-to-peer protocols may also be encapsulated within protocols such as HTTP and HTTPS, and this can make it harder to characterise the true underlying properties of the content (without deeper payload inspection) as highlighted in [10]. t is also worth noting that system administrators sometimes change the port allocations to mask service usage or conform to strict firewall rules (this is common practice with protocols such as FTP and OpenVPN). It is, therefore, important to use appropriate domain expertise when performing analysis, with an understanding of the communication context. Note that the standard deviation metrics are also presented in Figure 6, on a per service, per flow direction basis. For most of the services, we analysed the standard deviation sits typically below 2.0.