Paper:
Existing attention mechanisms, are mostly item-based in that a model is trained to attend to individual items in a collection (the memory) where each item has a predefined, fixed granularity, e.g., a character or a word. Intuitively, an area in the memory consisting of multiple items can be worth attending to as a whole. We propose area attention: a way to attend to an area of the memory, where each area contains a group of items that are either spatially adjacent when the memory has a 2-dimensional structure, such as images, or temporally adjacent for 1-dimensional memory, such as natural language sentences. Importantly, the size of an area, i.e., the number of items in an area or the level of aggregation, is dynamically determined via learning, which can vary depending on the learned coherence of the adjacent items. By giving the model the option to attend to an area of items, instead of only individual items, a model can attend to information with varying granularity. Area attention can work along multi-head attention for attending to multiple areas in the memory. We evaluate area attention on two tasks: neural machine translation (both character and token-level) and image captioning, and improve upon strong (state-of-the-art) baselines in all the cases. These improvements are obtainable with a basic form of area attention that is parameter free. In addition to proposing the novel concept of area attention, we contribute an efficient way for computing it by leveraging the technique of summed area tables.

Review:
I have several concerns about this paper.
[originality]
Some important related studies are missing.
# Related studies about the perspective of “area”.
The consecutive position in sequence is often referred to as “span” in NLP filed, which is identical to what the authors call “area” in this paper.
Then, the idea of utilizing spans currently becomes a very popular in NLP field. We can find several papers, 
e.g.,
Wenhui Wang, Baobao Chang, “Graph-based dependency parsing with bidirectional lstm”, ACL-2016.
Mitchell Stern, Jacob Andreas, Dan Klein, “A Minimal Span-Based Neural Constituency Parser”, ACL-2017.
Kenton Lee, Luheng He, Mike Lewis, Luke Zettlemoyer, “End-to-end Neural Coreference Resolution”, EMNLP-2017.
Nikita Kitaev, Dan Klein, “Constituency Parsing with a Self-Attentive Encoder”, ACL-2018.
Similarly, there are several related studies in image processing field,
e,g.,
Marco Pedersoli, Thomas Lucas, Cordelia Schmid, Jakob Verbeek, “Areas of Attention for image captioning”, ICCV-2017
Quanzeng You, Hailin Jin, Zhaowen Wang, Chen Fang, and Jiebo Luo, “Image Captioning with Semantic Attention”, CVPR-2016.
# Related studies about the perspective of “structured attention”. 
Several papers about structured attention have already been proposed, 
e.g.,
Yoon Kim, Carl Denton, Luong Hoang, Alexander M. Rush. “Structured Attention Networks”, ICLR-2017.
Vlad Niculae, Mathieu Blondel. “A Regularized Framework for Sparse and Structured Neural Attention”, NIPS-2017.
I think the authors should explain the relations between their method and the methods proposed in the above listed papers.
[significance]
# Concern about experimental settings
The experimental setting for NMT looks unnormal in the community.
Currently, most of papers use sentences split in subword units rather than character units. I cannot find a reason to select the character units. I think the authors should report the effectiveness of the proposed method on the widely-used settings.
# computational cost
The authors should report the actual calculation speed by comparing with the baseline method and the proposed method.
In Sec. 2.2, the authors provided the computational cost. 
I feel that the cost of O(|M|A) is still enough large and that can unacceptably damage the actual calculation speed of the proposed method.
Overall, the proposed method itself seems to be novel and interesting.
However, in my opinion, writing and organization of this paper should be much improved as a conference paper. I feel like the current status of this paper is still ongoing to write.
Thus, it is a bit hard for me to strongly recommend this paper to be accepted.