Task: Language Identification
Input (Spanish, English): This paper provides a new perspective to understanding the ResNet and Highway net. The new perspective assumes that the blocks inside the networks with residual or skip-connection are groups of successive layers with the same hidden size, which performs to iteratively refine their estimates of the same feature instead of generate new representations. Under this perspective, some contradictories with the traditional representation view induced by ResNet and Highway network and other paper can be well explained.  The pros of the paper are: 1. A novel perspective to understand the recent progress of neural network is proposed. 2. The paper provides a quantitatively experimentals to compare ResNet and Highway net, and shows contradict results with several claims from previous work. The authors also give discussions and explanations about the contradictories, which provides a good insight of the disadvantages and advantages between these two kind of networks.  The main cons of the paper is that the experiments are not sufficient. For example, since the main contribution of the paper is to propose the “unrolled iterative estimation" and the stage 4 of Figure 3 seems not follow the assumption of "unrolled iterative estimation" and the authors says: "We note that stage four (with three blocks) appears to be underestimating the representation values, indicating a probable weak link in the architecture.". Thus, it would be much better to do experiments to show that under some condition, the performance of stage 4 can follow the assumption.   Moreover, the paper should provide more experiments to show the evidence of "unrolled iterative estimation", not comparing ResNet with Highway Net. The lack of experiments on this point is the main concern from myself.
Output (English): en