# Water-Tight IoT-Just Add Security
## 4 Case Study of a Typical IoT Solution
### Black-Box Testing Methodology  
In a black-box testing scenario, the security expert has very little or no previous knowledge of the targeted system or device. At a high level, the methodology consists of sending inputs to the system, the \"black box\", and analyzing the obtained outputs to deduce the internals of the target. Having made some guesses, the attacker can adjust her inputs to confirm her thoughts or to exploit the target. This is presented in Figure 4.  
Figure 4: High-level diagram of the black-box testing methodology. Figure 3: Typical architecture of an IoT solution (here, in the water sector). The black-box testing methodology has several advantages over the white and grey ones. Its primary objective is to test a system under real conditions to emulate a real attack scenario. This means that such a test might catch errors made during the deployment of the system, such as default passwords, misconfigurations in general or even the lack of security training for operators (weak passwords). This methodology also presents a low false-positive ratio as the security expert can assess the risks associated with a vulnerability directly, i.e., if the vulnerability can be exploited or not. While black-box testing can miss some vulnerabilities and should likely not be the first test performed, it is an excellent way to assess how a system stands against attacks and to get an idea of the path an attacker would take to compromise the solution, and thus gives indications on how to tackle those potential low-hanging fruits. It can later be completed by a deeper assessment following a grey- or white-box approach. Hardware Security Testing Methodology for IoT  
Our process can be split into five different tasks (see Figure 5). The very first one is the hardware analysis. Once the device is acquired, we started analyzing its components to know what the exposed interfaces and debug interfaces are, but also the chips that are on the board. To know this, we had to open the device to access the Printed Circuit Board (PCB) to analyze it (see example in Figure 6a). Knowing the components and available interfaces, we then started looking for documentation such as datasheets, Request for Comments (RFC) or any other relevant information about the device. The goal of that second step is to understand the overall system and come up with some first hypotheses about it. Figure 5: Hardware black-box testing methodology iterative cycles. From those hypotheses, we then came up with testing scenarios to be performed on the device. Those scenarios have two possible outcomes: either a success or a failure. However, the success or the failure of a testing scenario is determined by the expected result, which in reality means that even failure brings us new information about the system. The results of a specific scenario need to be interpreted. This interpretation is called a finding. Those findings are then used to look for new documentation and/or infer new hypotheses about the device. For example, if the hypothesis is \"the device has debug ports exposed on the Universal Asynchronous Receiver-Transmitter (UART) pins and is providing the attacker with a shell when connected to it\", then the testing of that particular scenario will lead to either a confirmation or a rejection of the hypothesis. The interpretation here is quite easy as it is the hypothesis itself. This finding can then be reinjected in the documentation phase to infer new testing hypotheses such as \"the attacker is given a root shell when connecting to the UART console\" or \"the attacker can access the filesystem when connecting to the UART console\". Those findings are finally gathered to be reported. The reporting step is the one where the device is considered back in its whole ecosystem. That means the findings are interpreted again, but this time with regards to different metrics. In the case of the above example, one can wonder what is the impact of the attacker having access to the filesystem on an IoT device?