UPDATE:  I have read the authors' responses.  I did not read the social media comments about this paper prior to reviewing it.    I appreciate the authors' updates in response to the reviewer comments.  Overall, however, my review stands.  The authors have taken a task that had not yet been addressed with a straightforward modern deep learning approach, and addressed it with such an approach.  I assume that if we pick up any task that hasn't been worked on for a while, and give it a solid deep learning treatment, we will do well.  I do not see such papers as a contribution to ICLR, unless they also provide new insights, analysis, or surprising results (which, to my mind, this paper does not).  This is a general point and the program chairs may disagree with it, of course.  I have removed my recommendation that this be accepted as a workshop paper, as I have since noticed that the workshop track this year has a different focus.    ************************  ORIGINAL REVIEW:  The authors show that an appropriately engineered LSTM+CNN+CTC network does an excellent job of lipreading on the GRID corpus.  This is a nice result to know about--yet another example of a really nice result that one can get the first time one applies such methods to an old task--and all of the work that went into getting it looks solid (and likely involved some significant engineering effort).  However, this in itself is not sufficiently novel for publication at ICLR.  The paper also needs to be revised to better represent prior work, and ideally remove some of the vague motivational language.  Some specifics on what I think needs to be revised:  - First, the claim of being the first to do sentence-level lipreading.  As mentioned in a pre-review comment, this is not true.  The paper should be revised to discuss the prior work on this task (even though much of it used data that is not public).  Ideally the title should also be changed in light of this.  - The comparison with human lipreaders needs to be qualified a bit.  This task is presumably very unnatural for humans because of the unusual grammar, so perhaps what you are showing is that a machine can better take into account the strong contraints.  This is great, but not a general statement about LipNet vs. humans.  - The paper contains some unnecessary motivational platitudes.  We do not need to invoke Easton and Basala 1982 to motivate modeling context in a linguistic sequence prediction task, and prior work using older sequence models (e.g. HMMs) for lipreading has modeled context as well.  The McGurk effect does not show that lipreading plays a crucial role in human communication.  - It is worth noting that even without the spatial convolution, your Baseline-2D already does extremely well.  So I am not sure about the "importance of spatiotemporal feature extraction" as stated in the conclusion.  Some more minor comments, typos, etc.:  - citations for LSTMs, CTC, etc. should be provided the first time they are mentioned. - I did not quite follow the justification for upsampling. - what is meant by "lip-rounding vowels"?  They seem to include almost all English vowels. - Did you consider keeping the vowel visemes V1-V4 separate rather than collapsing them into one?  Since you list Neti et al.'s full viseme set, it is worth mentioning why you modified it. - "Given that the speakers are British, the confusion between /aa/ and /ay/..." -- I am not sure what this has to do with British speakers, as the relationship between these vowels exists in other English dialects as well (e.g. American). - The discussion about confusions within bilabial stops and within alveolar stops is a bit mismatched with the actual confusion data in Fig. 3(b,c).  For example, there does not seem to be any confusion between /m/ and /b/ or between /m/ and /p/. - "lipreading actuations":  I am not sure what "actuations" means in this context - "palato-alvealoar" --> "palato-alveolar" - "Articulatorily alveolar" --> "Alveolar"?
Output: en