Paper:
Deep artificial neural networks can achieve an extremely small difference between training and test accuracies on identically distributed training and test sets, which is a standard measure of generalization. However, the training and test sets may not be sufficiently representative of the empirical sample set, which consists of real-world input samples. When samples are drawn from an underrepresented or unrepresented subset during inference, the gap between the training and inference accuracies can be significant. To address this problem, we first reformulate a learning algorithm as a procedure for searching for a source code that maps input features to classes. We then derive a necessary and sufficient condition for generalization using a universal cognitive similarity metric, namely information distance, based on Kolmogorov complexity. Using this condition, we formulate an optimization problem to learn a more general classification function. To achieve this end, we extend the input features by concatenating encodings of them, and then train the classifier on the extended features. As an illustration of this idea, we focus on image classification, where we use channel codes on the input features as a systematic way to improve the degree to which the training and test sets are representative of the empirical sample set. To showcase our theoretical findings, considering that corrupted or perturbed input features belong to the empirical sample set, but typically not to the training and test sets, we demonstrate through extensive systematic experiments that, as a result of learning a more general classification function, a model trained on encoded input features is significantly more robust to common corruptions, e.g., Gaussian and shot noise, as well as adversarial perturbations, e.g., those found via projected gradient descent, than the model trained on uncoded input features.

Review:
This paper proposes a novel approach to training classifiers inspired by concepts from algorithmic information theory. Specifically, the inputs are augmented with additional features formed by encoding the image (using a source code). In addition, the paper argues to use the normalized compression distance as a loss criterion when training; this is a computationally tractable proxy for the normalized information distance, which the paper argues is what one really should use to generalize well, but which is not computationally tractable. (In particular, evaluating the Kolmogorov complexity, which is required to compute the normalized information distance, is not tractable.) The idea is demonstrated via experiments training common image classifiers (VGG-11 and ResNet-18) on clean and corrupted versions of the CIFAR-10 dataset, and comparing how they fare against adversarially trained methods on noisy and adversarial examples.
I believe that this paper contains interesting and novel ideas, but there are also some inconsistencies and some points that are vague. I would consider raising my score if the authors address or clarify these issues, which are detailed below.
The introduction discusses generalization very broadly, suggesting that the usual definition of generalization (e.g., accuracy on a hold-out set, drawn from the same distribution as the training set) is not the version of generalization that one should focus on. However, the precise definition of generalization is never provided. The intro also mentions briefly about adversarial examples, and this is the focus of the experiments. It would help to clarify whether the paper focuses on being robust to adversarial examples as a form of generalization, or more precisely, what how is the concept of generalization defined/measured in this paper. (E.g., the first sentence of Sec 2.1 mentions "our running definition of generalization," but no precise definition has yet been provided.
To make the paper self-contained, it would help to recall the formal definition of Kolmogorov complexity; this could be done in the appendix.
Sec 2 begins to make the connection between a classifier and a source code. However, these definitions are not precise. For example, in the usual PAC learning setting one has a distribution D over the training data (x,y). After specifying a particular loss function, the Bayes-optimal classifier is well defined. Does the source code C correspond this classifier? I would guess not, since the paper seems to be introducing a specific loss criterion, but it would be good to clarify this in the paper. Since you are assuming there is a "true output function" f, does this correspond to what is usually referred to as the "realizable" setting (i.e., where y = f(x) is a deterministic function of x)?
Notation is sometimes used without having been defined. For example, what is $\vec{x}_{S}$?
It isn't clear how one can train using the criterion (4), or equivalently, solve the problem set forth in Prop 2, in practice, since the true source code C is not known. Please elaborate on this.
In the experiments, how precisely is the VGG network modified to handle the encoded inputs? Are these just passed as additional input channels? Is the network architecture need to be modified in any other way to account for these additional inputs? Also, what loss criterion is used for training? Cross-entropy? Or something related to normalized information distance? This isn't clear from the discussion in Sec 3.
In Sec 3.2, is it really fair for the PGD attacks to only use the gradient of the loss wrt the uncoded input? Is this still really "white box"? How does the proposed approach perform if the attack also has access to the encoded input? It would be good to report both settings in the paper.
The introduction made connections to other forms of generalization, including domain generalization and domain adaptation. Based on this, I had expected to experiments illustrating the utility of the proposed for these problems.
The existing experiments do make clear that the encoded inputs are more robust to perturbations and aversarial attacks than uncoded inputs. Is the encoding assumed to be performed on the perturbed input, or on an unperturbed input (i.e., before it is perturbed)?
I don't find the results in Table 1 fully convincing that the proposed approach should be preferred over previous approaches. For instance, the approach of Madry et al. (2018) achieves slightly lower test accuracy, but substantially higher inference accuracy. In general, one expects to see a tradeoff here. If one were to plot these points, would the proposed method clearly fall above the Pareto frontier? Minor: Please clarify the definition of "inference accuracy" in the paper.