Venue: ICLR 2022 Conference

Reviews:

final_score: rating: null
review_content: I haven't been able to join the discussion actively (due to health reasons), but after reading all the discussion so far, I am happy to bump my score to a 6: I still am not convinced that the paper/experiments show us something substantially new, but many other issues are addressed carefully by the authors, and the other reviewers seem to find the findings significant.
review_title: Changes based on the discussion

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The paper asks an interesting question, one that appeared eternally relevant for NLP: Are languages harder or easier to contextually language-model by Transformer in light of their varying morphological complexity? The claimed main finding is provocative, indicating that the debate morphological variation is not relevant in the context of the given computational machinery.
main_review: Strengths:
- Very nice writeup and structure, the paper is for the most part a very enjoyable read.
- Bold in tackling important questions from a fresh angle.

Weaknesses (may be obsolete, see discussion):
- The contributions are overstated in light of the experiment setup.
- Only six resource-rich languages are used and only one respective dataset, the UN parallel corpus.
summary_of_the_review: I recommend that the paper be rejected so that it may further develop its empirical setup & verify (or reject) the findings on a larger dataset with more substantial linguistic depth (more resource variation) and breadth (more languages). In its current form, the paper's larger-than-life claims are simply not sufficiently supported with the choice of one dataset and six resource-rich languages, even if the hypothesis and findings are indeed interesting.

Edit after first round of author discussion, 2021-11-16:

I am concerned that there is a disconnect between the stated contributions of the paper (which are expressed as rather broad and general in its narrative) on one side, and the setup of the empirical investigation on the other side. While I may view the insight as to the (ease of) availability of broad-coverage linguistic resources sympathetically, the fact still remains that the findings are closely tied with the UN parallel corpus and its six languages. This in itself would not disqualify the paper at face value. However, text e.g. under "Summary of findings" 1-5 and later in "Conclusion" comes across as masking this fact and painting a much broader picture.

I would be much more inclined to accept this (undoubtedly insightful) paper should it insert a clear Limitations section which would reflect on the preliminary nature of these findings given the dataset constraints.

As for the language resources, I would very much love to hear which of those e.g. in OPUS (https://opus.nlpl.eu/index.php) were not fitted to the experiment while only the UN corpus passed the filter. This would be an insightful addition to the limitations section, as the author response now seems to indicate massive limitations in all the other available datasets. Do share with the community!

Edit after 2021-11-18:

Upvoting the overall score after insight re: multi-parallel datasets available (see discussion).
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: In this paper, authors aim to untangle the disparities in performance between morphologically rich languages (e.g. RU) and morphologically poor languages (e.g. ZH). In order to untangle such disparities, the authors propose a highly controlled experiment setup, where they control for a number of potentially confounding factors (e.g. language mappings, data size, text encoding, etc.). After the controlled experiments, the main take-away of the paper is that vocab size seems to be the biggest indicator of how well or poorly a CLM will perform — since there can be several different granularities for the concept of a “word” for morphologically rich languages, and this can explain why modelling morphologcially rich languages can be more difficult.  They perform additional experiments with different representations of ZH, and were able to mimic the struggles of morphologically rich languages with ZH, and use this as further evidence to support the claim that the debate within NLP about the best segmentation is irrelevant.

Overall, this work is exciting, has has a lot of potential to help inform future language modelling for morphologically rich languages.  The very controlled nature of the experiment in the paper is what makes it most convincing, and this is the paper’s largest contribution. 
main_review: Pros: 
— Very interesting and engaging work, with 
--- Experiments are very controlled, from data, number of lines, language mappings, and segmentations. 
— Experiments seem to be reproducible just from the description in the Experimental Setup. Very clear. 
— Thoughtful connections from human language processing and linguistics to try to explain the Char/Byte/Word performance disparities. 
--- Really interesting follow up experiments on ZH!

Cons: 
— The “A0, B0, C0, A1 & A2, A3-7”, etc. nomenclature for experiments is unhelpful to readers. Can the experiment names be changed to be something more intuitive, so that readers don’t need to memorize “oh, A0 is one of the controlled experiments for data size”?
-- Figures in this paper are largely unreadable and do not help as visual aids. I have to zoom in to 310% (at minimum!!!), and even then my eyes struggle to distinguish the colours and line styles from each other. Also, the subfigures within subfigures are not readable. This is a big problem, because the figures are used to support claims that the authors are trying to make, and I think many readers would be unable to draw the same conclusions from these figures themselves. 
 

Formatting, grammar, etc.:
1. Abstract: “Performance disparity is not a necessary condition” is worded strangely to me. 
2. Introduction: Formatting at bottom of 1.1 is weird. Use of bullet points makes no sense here, they just split up the sentence. 
3. S2, “Experimental setup“: There’s a double parenthesis )) with the (Jurafsky & Martin, 2009) citation
4. S5, your tables do not follow the same typesetting conventions as your Figures (i.e., the description is on top)

summary_of_the_review: Reasons for score:  
Frankly, if the figures and tables in this paper were significantly improved, this would be an 8 or higher. The main motivation of the paper is exciting, the methods are well controlled, and the findings are interesting. The core of the problem (as I discuss in "Cons") is that the claims made in the paper rely on the figures/tables for illustration as evidence, and the figures/tables are entirely unhelpful. In otherwords, the tables and figures detract from your arguments because they are unreadable. Once this can be fixed, I think its great work, and it would be interesting and directly useful to many people in NLP. 

EDIT - Wednesday, 17 November, 2021: In light of the authors rebuttal to my review, and their changes to v2 of the document, I am bumping my score up from 5 (marginally below threshold) to 8 (good paper, accept). My biggest problem with the paper before, were the figures, but I think that the authors addressed my comments to the best of their abilities by adding a lot more clarity to the textual descriptions of the images, in the paper and in the captions. The figures are still not great, but with their new textual descriptions, you can look at the general shapes of things, and visually understand the point, and before v2 I did not have this same level of visual understanding.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: At its core, this paper focuses on testing whether languages with different morphological complexity equally challenging to language-model in a bilingual / MT-like setting using a 6 layer Transformer architecture.  Specifically, it tests the assumption whether the previous observations about languages with higher morphological complexity are difficult to language-model, using a controlled experiment with a n-way parallel corpus across 6 languages of interest. The key finding of this paper is a simple one : that the choice of input representation is highly important, given everything else is kept constant, and that differences in language-model perplexities can be normalized and made to disappear by choosing the same representation (word/character/byte)
main_review: This paper contains a series of sound, controlled experiments to study the role of input representations in language-modeling. I think the experiments are well designed, and with a clear focus to answer the research questions laid out in 1.2. Overall, I do think that this paper will benefit the multilingual NLP community in thinking carefully about the choice of input representations, rather than advocating for a one-choice fits all solution. However, I do have a few concerns that prevent me from giving a higher score. 

1. The claim that this paper "renders a decade-long debate on morphology irrelevant" is made multiple times in this paper, and in my opinion, is too strong a claim to be made on the basis of experiments on a single task (conditional LM), on a handful of languages (however typologically diverse they may be), and using a single architecture (6-layer Transformer). I would recommend scoping this claim down to something that can be clearly supported from the results of this paper.

2. Second, how does the practical recommendation in Section 3 ("A practical takeaway from this set of experiments: in order to obtain more robust training results, use bytes for ZH and characters for AR and RU") square with the multilingual nature of recent models which typically have a shared input representation for many languages? I think more generally this point is one of several that seems slightly disconnected from downstream applications (the claim about morphological complexity being irrelevant would also fall in this bucket.).

3. There are a few citations from prior work in NLP that are worth contextualizing here. Specifically, the work of Gillick at al. on byte level processing for sequence tagging [1], that of Wang et al. on byte-level Neural MT, and more recently, the work by Clark et al. on building a tokenization-free encoder seem particularly relevant [3]. In fact, the whole body of literature on character-level neural models are particularly relevant and the omission of these is critical, especially given the stance of the paper in advocating moving beyond "word" level models (e.g. Footnote 8)

[1] - https://arxiv.org/pdf/1512.00103
[2] - https://arxiv.org/abs/1909.03341
[3] - https://arxiv.org/abs/2103.06874
summary_of_the_review: This paper contains sound and controlled experiments that test the role of input representation in bringing disparity to language modeling results, which I believe will be useful for the larger community. At the same time, several claims in the paper are too strong or not grounded in practical use-cases, and some relevant literature is not referred to, leading me to be marginally inclined to accept this paper.

### Edit - 22 November, 2021
I spent some time reading the discussion between the authors and the other reviewers, as well as revisiting v0.2 of the paper. In light of these discussions, I think I am fairly comfortable accepting this paper as my main concerns have been addressed by the review and the comments of the authors. Hence, I revised my overall recommendation to 8, while adjusting a few other scoring parameters too.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper asks: Are languages which have been traditionally considered
morphologically rich (Arabic and Russian) and poor (Chinese) equally
hard to learn by transformer-based conditional language model (CLM).

To quantify how hard it is for CLM to model a language $l_1$, this paper
proposes to train a transformer-based encoder-decoder model to translate
$l_1$ to another language $l_2$, and uses the \"unnormalized perplexity
score\" (the number of bits needed to encode the same development set
for the target language) as an indicator for the \"hardness\" for CLM to
learn the $l_2$. For two languages $l_1$ and $l_2$, there exists
**disparity** if their perplexity distributions are significantly
different. This paper includes extensive experiments across 30 language
pairs using three different representation level (including character,
byte, and word) for different dataset sizes.

They have the following findings:

1.  They find that the performance disparity between morphologically
    rich and poor languages cannot be justified due to morphological
    complexity. Representation level matters more.

2.  They find that the hardness for CLM to model a language is related
    to the level of representation used to encode the language, the
    vocabulary size, and/or the sequence length.

3.  They find that it is equally hard to learn a certain target language
    (that are tested in their experiment) using transformer-based
    conditional language models, independent of the source language.
main_review: I want to thank the authors for the interesting paper. Overall, the
paper contains a substantial amount of experiment results with some
interesting findings, but the paper is not easy to read and not easy to
follow. I am also not sure if this paper reveals substantially new
things to the readers.

The strength of the paper is as follows:

-   This paper proposes to use conditional language modeling (machine
    translation) to evaluate how hard it is for transformer models to
    model a language. This is a clever setting that allows one to fix a
    source language and compare the difficulty of modeling different
    target languages.

-   The paper uses statistical tests to verify if there exists
    performance disparity between languages of different morphological
    complexity.

-   This paper shows that transformer conditional language models are
    equally capable of modeling the six languages evaluated in the paper
    when the suitable representation level is chosen. This indicates
    that transformer models are **fair** and are not biased toward any
    the language they tested, and morphological complexity is not indicative
    of how hard it is for transformer-based conditional language models
    to learn.

The weakness of this paper and questions about the paper is listed as
follows:

### Not enough novelty compared to prior works

An important prior work of this paper is Mielke et al. (2019). I find
this paper highly resembles the work of Mielke’s from some aspects. I
will summarize the work of Mielke et al. (2019) as follows first:

1.  Research Question: Do LSTM-based language models serve all
    languages? What are the characteristics of languages that are easier
    for LSTM-based language models to model?

2.  Experimental Setup: Use parallel corpora containing 21 languages.
    Train a mono-lingual character/BPE LSTM language model on each
    language, and evaluate the development set perplexity for each
    language.

3.  Results and Findings: Language difficulties are not significantly
    predicted by linguistic factors (including morphological
    complexity), no matter the representation is character or BPE. For
    LSTM language model with character-based representation, the
    difficulty for LSTM to model correlates with the raw length in
    characters. For LSTM language model with BPE-based representation,
    the difficulty for LSTM to model correlates with the vocabulary
    size.

The main difference between this work and Mielke et al. (2019) are as
follows:
1.  The models used in this work are transformer-based conditional
    language models, while Mielke et al. (2019) used RNN-based
    (unconditioned) language models.

2.  The indicator of performance in this paper is unnormalized
    perplexity score, and Mielke et al. (2019) used the normalized
    version.

3.  This paper uses statistical tests to compare if there exists
    performance *disparity* between language pairs, which was not
    presented in Mielke et al. (2019) (since Mielke et al. (2019) did
    not use conditional language modeling).

The main findings of Mielke et al. (2019) that I listed above are also
observed in this work. And this work also has the following new findings
that are not presented Mielke et al. (2019):

1.  They find that for transformer conditional language models, the
    difficulty of modeling a certain target language is independent of
    the source language.

While the experiment settings are slightly different, from the aspects
of the model architecture and evaluation metrics, the core of the experiment
and the main takeaway does not make me learn substantially new things. I
do recognize that it is important to empirically verify the results of
Mielke et al. (2019) on transformer-based conditional language models,
but I have reservations about whether the contribution of this work is
ample enough.

Additionally, I think this paper will definitely benefit from a more
thorough discussion of their difference with the previous works.

### Experiment settings are not clear enough to follow

-   In the "Disparity/Inequality" part in Section 2, the paper states
    "Each $l_{src}$ or each $l_{tgt}$
    consists of scores from all models trained across various sizes and
    directions." I have difficulty understanding how the comparison
    tests can be done. In my opinion, this is the most novel setting in
    this work, and I would like to see more elaboration! Since for each
    source/target language, there are scores for different directions,
    dataset sizes, random seeds, and datasets, how is the comparison
    between the "distribution of the scores" done? As far as I
    understand, the "distribution of the scores" of a language will
    consist of scores obtained under different settings, and I am not
    sure if comparing the distribution as a whole is rational. Since the
    score distribution under certain factor fixed (for example, when the
    dataset size is fixed) may be statistically significantly different,
    but when all the factors change together, we cannot tell the
    influence of each factor.

-   Is there any reason why not running the $10^6$ for setting
    A3-7, B1 & C1.

-   I don’t understand how the random seeds are controlled in their
    experiments. Is the random seed only related to how the model is
    initialized? Or the random seed will also affect how dataset B and C
    are sampled? Why is there not a C(0) model (in appendix A). I think
    for different data sizes and datasets, they should have a common
    random seed, and the experiments should be run using multiple
    different common random seeds.

-   I have difficulty understanding what "transformer-preprocess d;
    transformer-prostprocess drn" in appendix B is. I have never seen
    this parameter in the context of transformer-based language models.

### Some words or statements are ambiguous

-   In Section 1.1, "favoring of certain outcomes". I do not understand
    what "certain outcomes" are in this paper after I read the whole
    paper. Do the authors mean "better at modeling certain languages"?

-   In Section 1.2, "bounded by representation level". I am not sure
    what "bounded by" means here , and I am skeptical about the phrase
    "bounded by". Since the experiments only use a set of
    hyperparameter, the claim that "representation bounds hardness" may
    be too strong. May be the hyperparameter just don’t work for certain
    representation.

-   Section 2, in **Conditional language modeling**: "without performing
    any downstream translation". What is "downstream translation"? This
    seems not to be a common phrase in the context of machine
    translation, and may need some explanation. Do the authors try to
    refer to something like beam search?

-   Section 2, in **Conditional language modeling**: "To explicitly
    focus on modeling the complexities that may or may not be intrinsic
    to the languages". I don’t understand why this paper is able to
    explicitly focus on "modeling the complexities" under their setting.
    I also don’t quite get what is intrinsic to the languages and what
    is not.

-   Section 2, in **Controlled experiments as basic research for
    scientific understandings**: "that make language data different from
    other data types". I fail to understand what "other data types" this
    paper is trying to refer to. I also don’t see the connection between
    the relationship between the quoted line above and "data-centric
    approach".

-   Section 5, "disparity is not a necessary condition". What is the
    necessary condition for?

### Some claims in the paper are not well supported by the experiments results

For example, the paper writes in page 6 that \"on the byte level,
$AR_{trg}$ and $RU_{trg}$ display non-monotonic and unstable
behaviors\". I am not sure what the \"non-monotonic behaviors\" the
paper is referring to, but I think in Figure 1(e), not only AR and RU
are non-monotonic, all results in Figure 1(e) are not monotonic. All of
them decay from $1e-02$, reach the lowest perplexity at $1e-04$, and
rise afterward.

### Miscellaneous concerns

-   I am concern about the title \"Fairness in Representation\". First,
    since the word fairness is mostly used to refer to societal biases
    in NLP community, I am not sure if the word 'fairness' may be
    misleading at the first sight. Maybe statistical bias will be a more
    better choice. Second, I am not sure what \"Fairness in
    Representation\" means; it seems to refer to the performance
    disparity between different representation levels. But since the core
    problem in the paper is the disparity between different languages of
    different morphological complexity, I guess the whole paper is more
    like \"Transformer's Fairness for Different Languages\" instead of
    \"Fairness in Representation\".

-   Some passages seem to be not highly related to the experiment
    results. For example, in page 6, I don't see what the passages \"But
    this is often \... and accepted in NLP.\" has to do with the
    experiment result in this section.

-   I am concerned about using Pinyin in Chinese. Will single Pinyin
    correspond to multiple characters in Chinese? If so, then changing
    the target from character to Pinyin does not merely change the
    vocabulary size and makes the whole comparison unclear. Whether
    Pinyin can be an alternative for character needs to be carefully
    discussed.

-   I am concerned about the paper's only using a set of hyperparameter. I
    understand that using the same set of hyperparamater is for
    \"controlled experiments as basic research for scientific
    understanding\", but sometimes the paper attributes the performance
    fluctuation to the fact that they did not search for better
    hyperparameter. This seems contradicting, and one can question that
    the results in the paper will not hold if we had searched the
    optimal parameters for different language pairs, dataset, data
    sizes.

-   If the goal is to compare morphologically rich languages (AR, RU)
    and frugal language (ZH), then why does the paper also compare the
    other three languages in United Nations Parallel Corpus? What do
    their results tell us? What is the morphological complexity of the
    remaining three languages?

-   The languages used in this paper is not too many. The conclusion of
    Mielke et al. (2019) are based on 21 languages, I am not sure
    whether the conclusion of this paper will still hold if more
    languages are included.

-   If in 2(b,d), only ZH is changed, I think it is better to only show
    the result of ZH as a target since all remaining 5 subsubfigures are
    the same as in 1(d).

### Presentation problems

The figures are not easy to read. Figure 1, 2, 3 are too small and are
unreadable on A4 paper. The problem of the figure hinders my
understanding to some extent. For example, in the second last line in
page 7, it is said that \"AR/RU \> ES/FR \> EN/ZH\", I cannot arrive at
the conclusion since Figure 1(c) is too small to read. While the enlarged
versions of figures are provided in the appendices, I still find them not
so useful for some cases; for example, the standard error bars in Figure
4 EN are overlapped with the lines.
summary_of_the_review: This paper studies the fairness of transformer models to model languages
of different morphological complexity. This work extends the previous
observation from Mielke et al. (2019) and show that similar results hold
for transformer-based conditional language models: morphological
complexity is not indicative of the difficulty for transformer to conditional
language model, and representation level matters more. This indicates
that transformers are **fair** towards the languages tested in this
work, in terms of conditional language modeling. They propose to use
conditional language models to make it easier to compare the difficulty
of modeling a specific target language, and the usage of statistical
tests for testing performance disparity among languages is a novel act.
However, experiment settings and findings/results resemble the work of
Mielke et al. (2019) to some degree. The part on statistical testings is
somewhat baffling and some of the statements are ambiguous. The figures
for the main experiments are not very easy to read. The title is also
somewhat not quite indicative of the content (while it does seem
relevant), mainly because of the term \"fairness in representation\".

## Update on Nov 22
After a thorough and in-depth discussion with the authors, I upvote the score from 3 (reject, not good enough) to 5 (marginally below the acceptance threshold), meaning that I am still inclined to reject.
I change my score mainly because the authors have addressed many of my initial concerns, including the experiment settings, some odd wordings or claims, and the presentation problems.
But I still do not find the paper is good enough to be accepted. 
I am still not convinced by the authors' claim of their novelty; the novelties listed by the authors seem to be more of "something that was not being done in prior works", but I am not sure how those "something" can much advance our understanding on this topic.
Overall, I think this paper presents some nice and well-controlled experiments, but the interpretation of the results tends to be too high-level and might be overstating; I often have difficulty linking the quantitative results and qualitative interpretations.
I am not sure what I have learned from this paper.


## Edit again on Nov 22
I will upvote my score to 6. 
After going over the paper several times, I think there indeed are certain empirical contributions of the paper. 
I think varying the data size is an important experiment that has yet to be presented in any prior works and is valuable to be empirically verified, I do agree with the authors on this.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper explores the performance of conditional language models for
different languages with the aim of establishing "fairness" in NLP
systems across languages. In particular, the paper focuses on
perceived difficulty of "morphologically rich" languages for NLP.  The
paper presents a series of experiments with 6 languages in the UN
parallel corpus, using different units (words, sub-words, characters,
bytes) for training the conditional language models. The paper
concludes with the general observation that the use of characters or
bytes as units instead of words removes (or reduces) the performance
disparity between different languages.

main_review: Existence of "inherent" difference in difficulty of NLP for different
languages is interesting for theoretical reasons as well as the
applied NLP. The paper also presents a large number of experiments in
a controlled way for this purpose.

Although I agree that the topic is important, and the study covers a
fair amount of experiments with sound methodology, I do not think we
learn a lot from the present paper: (1) the effectiveness of sub-word
units for morphologically rich languages is expected, and (2) the
domain of application (conditional LMs) is rather narrow. Given the
issue is studies using (non-conditional) LMs earlier (as the paper
cites), the benefits of narrowing it further down to conditional LMs
is not clear to me (after reading the paper). The issue with word
segmentation in Chinese is also studied before, and I do not see any
additional information the present paper provides.
Besides the main above point, which is the basis of my main evaluation 
of the paper, there are a few minor issues I list below.

- The paper is written in a mainly error-free English, but it is
  rather difficult to read due to the organization and the flow of the
  concepts/text. Although I do not have many concrete suggestions, the
  this prevented me from understanding some simpler issues at first
  reading. Examples include:
    - There are some unclear statements like "To which extent is
      morphological complexity necessary in computing?" (p2): this
      probably means "handling/modeling morphological complexity" but
      literally, "need for complexity" is not something anyone would
      claim to be necessary.
    - There are divergences from the main topic of discussion, for 
      example "Our conventional way of referring to “language” (as a
      socio-cultural product or with traditional word-based
      approaches, or even for most multilingual tasks and
      competitions) is too coarse-grained ..." (p5) is just to general
      to substantiate here while defining the training set size in the
      previous context. The same goes for the discussion of ZH as
      "high-resource" language on page 6: it is not clear to me why
      this is relevant here, and what is the relation to the overall
      aim of the paper.

- Overall, the paper feels too dense. In my opinion, paper could
  benefit from presenting less, explaining the main matters more
  clearly. For example, I do not fully see the utility of the
  byte-based models, especially that the paper goes into resolving
  practical issues due to code-set differences among languages. The
  additional data may be of interest for a small subset of the
  audience, but for the most, this is not relevant for the main
  question tackled. Leaving some of these data/exposition to appendix
  may allow allow clearer exposition of the main points.

- The figures, even the enlarged ones in the appendix, are rather
  difficult to read: too many lines/patterns and too small fonts. No
  concrete suggestions, but a clearer display of the results would
  help the reader a lot.

- "Summary of findings and insights" is too detailed at this point. A
  more concise, general version would be much better. Many of the
  references/notation here is not possible digest before reading the
  rest of paper.

- Although the data used is a standard data set, it would be useful
  for most readers to have more detailed description of the data
  (e.g., the type of documents, size of the overall data that the
  parts used are sampled, ...). It would also be good to indicate what
  a "line" corresponds to (is it a sentence, a paragraph, or some
  other unit?).

- Footnote marks should be place after punctuation.


summary_of_the_review: Although I agree that the topic is important, and the study covers a
fair amount of experiments with sound methodology, I do not think we
learn a lot from the present paper: (1) the effectiveness of sub-word
units for morphologically rich languages is expected, and (2) the
domain of application (conditional LMs) is rather narrow. Given the
issue is studies using (non-conditional) LMs earlier (as the paper
cites), the benefits of narrowing it further down to conditional LMs
is not clear to me (after reading the paper). The issue with word
segmentation in Chinese is also studied before, and I do not see any
additional information the present paper provides.
review_title: null

Meta-review: The authors address a very important question pertaining to the relevance of morphological complexity in the ability of transformer based conditional language models. Through extensive (controlled) experiments using 6 languages they answer as well as raise very interesting questions about the role of morphology/segmentation/vocab size which mat spawn more work in this area.

All the reviewers were positive about the paper and agreed that the paper made significant contributions which would be useful to the community. More importantly, the authors and reviewers engaged in meaningful and insightful discussions through the discussion phase. The authors did a thorough job of addressing all reviewer concerns and changing the draft of the paper accordingly. 

I have no hesitation in recommending that this paper should be accepted.
Decision: Accept (Spotlight)