# Defacement Detection with Passive Adversaries
## 4 System Architecture and Implementation
### Overall System Architecture  
The overall system architecture is described in Figure 5, where all activity is triggered by two separate keyed schedulers: (1) a scheduler for the URL tester, on the top right side of the figure, and (2) a scheduler for the keyed learning component, at the bottom of the figure. The input to the scheduler for the URL tester is a comma separated list of values (CSV), where each value consists of a \(<\)URL, weight\(>\) pair. We generate the URLs in the CSV by crawling the application, while corresponding weights decrease as we run further from the site's document root, unless our application-specific component risk labeling states otherwise. Thus, in general, the application's home page will be checked more frequently than content leaves, that may have a lower reputational impact. The scheduler will spawn a process in due time, and a corresponding URL testing procedure is followed (see Figure 6). As a consequence, also within the same client, multiple \(<\)URL, weight\(>\) pairs are consumed by independent processes running in parallel. First, the URL content is downloaded, together with its inline external components (scripts, images, etc.). In our experimentation, we targeted a large enterprise site where a CDN was in place. In this case, it is important to download content from the original content source, and not from the CDN or intermediate proxies. Second, two basic tests are performed, availability and equality (see again Figure 6):  
* **Availability**: if the URL cannot be downloaded, because an error is returned (e.g., http 4 xx or 5 xx), or a timeout occurs, the availability test fails and the spawned process instance stops. A corresponding log line is stored and a possible alarm is generated in the alerting subsystem. * **Equality**: if a hash of the downloaded content is equal to the one computed for previous downloads, the detection procedure was already run with the same data, and again the process  
Figure 5: System architecture. stops. This is very important in practice, because the equality check will cause a stop most of the time, as the application does not change continuously, resulting in significant computational savings. Equality is actually verified in two steps: - domHash: compares the hash (sha256) of the most recently saved DOM, with the DOM that was currently downloaded; - contentHash: extracts the \"pure text\" from the current DOM (including the _alt_ tag values), and from the previously saved DOM, and checks the corresponding hash values for equality. If domHash succeeds (the two hash values are equal), then equality succeeds and the process stops. It may, however, happen that domHash fails only because some script names differ or other content-unrelated differences apply--in this case contentHash succeeds and the defacement detection process stops, without generating any alarm. We do the check in those two phases for efficiency reasons, as domHash is more easily computed and detects most situations that do not require further processing. Third, a number of fixed custom tests, and one adaptive test are fired, yielding an overall defacement score. This will cause possible logs, alarms and SOC-alerting procedures to be generated. The overall system is then composed of three functional components: (1) keyed schedulers, (2) keyed learning, and (3) URL tester. The components are interconnected as shown in Figure 5, and the motivation of each of them can be summarized as follows:  
* There are two keyed schedulers: a \"testing\" scheduler (top and right in Figure 5) and a \"learning\" scheduler (bottom of Figure 5). The schedulers work as shown in Figure 4 and have the purpose of making system actions start at times that are unpredictable for our adversary. The use of a testing scheduler is a common requirement of all complex alarm systems: if the adversary knows when the system checks for anomalies, the time of attack will be planned so as to avoid the check.