Paper:
We introduce a method to create Universal Adversarial Perturbations (UAP) for a given CNN in a data-free manner. Data-free approaches suite scenarios where the original training data is unavailable for crafting adversaries. We show that the adversary generation with full training data can be approximated to a formulation without data. This is realized through a sequential optimization of the adversarial perturbation with the proposed dilate loss. Dilate loss basically maximizes the Euclidean norm of the output before nonlinearity at any layer. By doing so, the perturbation constrains the ReLU activation function at every layer to act roughly linear for data points and thus eliminate the dependency on data for crafting UAPs. Extensive experiments demonstrate that our method not only has theoretical support, but achieves higher fooling rate than the existing data-free work. Furthermore, we evidence improvement in limited data cases as well.

Review:
This paper proposed a white-box (known network architecture, known network weight) data free (without need to access the data) adversarial attacking method. The main idea is to find a perturbation that maximizes the activations at different layers jointly. But the optimization is done sequentially, treating each layer’s activation (before ReLU) as a linear transformation output.
The method is compared with existing methods (only one existing approach for the problem, GDUAP by Mopuri et al. 2018) in terms of the fool rate. It shows significant improvement. Ablation study is carried out to compare with baselines like perturbation maximizing only first layer activation, only last layer activation, etc. Also on some other settings (black-box testing, less data) the proposed method outperforms GDUAP. 
The problem of data-free white-box attack is very interesting and does make sense. The proposed method achieve significant improvement over the previous one (GDUAP). I do have the following concerns though.
1), the novelty of the proposed idea seems relatively limited. The proposed idea seeks perturbation maximizing activations over all layers. It incur perturbation before ReLU. But overall, the flavor of the idea is not significantly different from GDUAP, despite the significant performance boost. 
2), it was mentioned that compare with GDUAP, this paper has more theoretical analysis. But this is not very convincing to me. There are many steps of approximation/relaxation from the original problem (Equation (1)) to the final formula (Equation (10)). Many assumptions are made over the steps. It is OK to use these steps to derive a heuristic. But these steps can hardly be called "theoretical analysis".
I am particularly uncomfortable with Equation (5), which is the basis of the main idea. It assumes that all data in $W_1X$ are in the same orthant as $W_1p$. But this is unrealistic as different data in X will for sure incur different activation patterns. Did I misunderstand anything?
3) I do like the experimental results. It looks impressive. But the baselines are really limited (granted, there are not many existing approaches). There is only one task (image classification). How about other tasks like segmentation etc shown in Mopuri et al. 2018? Also it would be nice to also show the results of other UAP methods, as it gives us a better sense of the gap between with and without data.
4) I wonder how will the attack affect some model which has been trained with some defense mechanism, e.g., adversarial training.
Typo:
Equation (5), RHS missing a max