# Defacement Detection with Passive Adversaries
## 3 Keyed Learning of a Defacement Detector
### Methodology Motivations and Limitations  
In this subsection we discuss why the keyed learning methodology is needed to improve classifier performance in an adversarial setting. We then list some limitations of the approach. The use of a secret key as an input to the learning algorithm amounts to _hiding_ information from our adversary. Previous work on keyed intrusion detection [21; 22; 23; 24; 25] and on adversarial learning  
Figure 1: Keyed Learning, producing a learned hypothesis H, e.g., a classifier or a cluster. Figure 2: Computing the learning system’s parameters from the secret key k.  
with randomization [18; 19; 20] is based on similar considerations. The principle has also been informally suggested in the context of defacement detection [6]. In this paper, we take a security perspective, following [PERSON] principle and putting all hidden information into one secret key \(k\). Moreover, we extend this information hiding principle to the whole system behaviour, including aspects that were not previously considered, such as the learning and application time of the anomaly detector. The requirement for \(k\) is not _randomness_, but rather _practical unpredictability_. For example \(k\) could not be physically random, because it is output by a pseudo-random number generator, but, if our adversary cannot replicate the process, it will be perfectly adequate for our purposes. In the context of anomaly detection, one may argue that use of a key or other more informal ways of hiding information from our adversary is not only a sound approach to learning with passive adversaries, but even a mandatory requirement when all input data is public. In fact, if the learning system is completely known and the input data is available, an adversary could simply replicate the learning process, obtaining the very same classifier we will use. This is the case for defacement detection, because the input examples are taken from the target Web site, and from known defacement examples. In other applications, such as anti-spam filters, the situation can be different because the examples of non-spam emails for a specific user are normally not available to the adversary. When the anomaly detector is applied in the production environment, the adversary will look for an attack pattern that is classified as normal, and use it to defeat the alarm system. As a consequence, evasion is simple and straightforward when no information is hidden from an adversary, and keyed learning is a natural solution that improves classifier performance in this context. However, a number of limitations of the approach should be considered. First, we must consider consequences on the prediction performance of the learning system. As stated by [PERSON] et al. in [18]:  
_*an excessive randomisation could lead to selecting a classifier with a poor performance, to the extent that the advantage attained by hiding information to the adversary is lost._*  
Similarly, it should be noted that the key is random and unpredictable, and could lead to the selection of a poor set of features, from a prediction perspective. The feature subset selection process induced by the key should not be confused with \"feature selection\" as addressed in Pattern Recognition and Machine Learning, where a set of features is selected at training time, with the goal of achieving better prediction. Here we only mean that some features are used and others are not, based on the value of the secret key--the selected set of features may not be optimally performing, and could actually be redundant and include useless features from the point of view of prediction. Nothing prevents the training phase from doing some further feature selection, in its classical meaning, after the keyed feature selection has been done. Traditional feature selection, aimed at better accuracy, is normally predictable, and thus a possible vulnerability [43] from an adversarial point of view.