Paper:
Parametric texture models have been applied successfully to synthesize artificial images. Psychophysical studies show that under defined conditions observers are unable to differentiate between model-generated and original natural textures. In industrial applications the reverse case is of interest: a texture analysis system should decide if human observers are able to discriminate between a reference and a novel texture. For example, in case of inspecting decorative surfaces the detection of visible texture anomalies without any prior knowledge is required. Here, we implemented a human-vision-inspired novelty detection approach. Assuming that the features used for texture synthesis are important for human texture perception, we compare psychophysical as well as learnt texture representations based on activations of a pretrained CNN in a novelty detection scenario. Additionally, we introduce a novel objective function to train one-class neural networks for novelty detection and compare the results to standard one-class SVM approaches. Our experiments clearly show the differences between human-vision-inspired texture representations and learnt features in detecting visual anomalies. Based on a digital print inspection scenario we show that psychophysical texture representations are able to outperform CNN-encoded features.

Review:
The submission investigates the problem of detecting perceptual anomalies in visual textures.
It proposes features from three different models, the Portilla & Simoncelli texture model (PS), the Spatial Vision model by Schuett and Wichmann (SW) and CNN features from the VGG network. From these features it trains two anomaly detectors: one out of the box one-class SVM and a 3 layer neural network. The network is optimised with a loss function that encourages output values for the original texture to be larger than for a white-noise image obtained by shuffling its pixels. At the same time the range of output values for the original texture is encouraged to be small.
 
The performance of the different approaches is evaluated using synthetic anomalies. However no distinction between perceptually striking and perceptually negligible anomalies is made and quantitative results are only reported for all synthetically generated anomalies.
Two attempts are made to control if an approach specifically picks up on perceptually striking anomalies.
a) detection rate on gaussian noise as a proxy for perceptually negligible anomalies
b) anecdotal evidence from visual inspection.
I do not think that either of the two controls is sufficient to make a clear statement about which method is best in detecting perceptually striking anomalies. Therefore my main concern is that the performance evaluation is not suitable to achieve meaningful results.
Furthermore the technical depth of the submission appears fairly limited. The main original contribution is the CNN loss that is introduced. However, the loss does not strike me as particularly compelling. It resembles a classifier between textures and white noise samples with the same pixel-wise statistics. I am not sure why this should be particularly suited to detect perceptual anomalies.
Finally, showing quantitative results from only two textures does not feel like a very comprehensive analysis.
In general  the submission tackles an interesting research topic. However, to show meaningful results I believe that one has to collect psychophysical data for the anomalies of interest to distinguish between anomalies that are perceptually detectable and those that are not.
With such a test set one could then start testing hypothesis on which feature representation is most appropriate to model the perceptual results or optimise features directly to match human psychophysical results (similar to the study by Berardino et al. 2017 [1]). In its current form I am not sure what I can learn from the submission both in terms of anomaly detection and feature spaces particularly suited to detect perceptual anomalies in visual textures. 
[1] Eigen-distortions of hierarchical representations
A Berardino, V Laparra, J Ballé, E Simoncelli
Advances in neural information processing systems, 3530-3539