Venue: EMNLP 2020 Workshop NLP-COVID

Reviews:

final_score: rating: 3: Clear rejection
review_content: In this work the author presents a novel BERT architecture with the goal of provide a summarization of lengthy papers, applied to the CORD-19 dataset. The author does a great job at framing the problem and providing relevant related work. The proposed model description is intuitive and seems reasonable, however, figure 1 is a bit difficult to read as it compressed for space constraints. The dataset description is not clear at all, only discussing about the abstracts used from the CORD dataset, but then table 1 mentions 3 other datasets. The 'experiment' section is quite brief and generic, things that should have been discussed are: was this setup for all dataset? just one dataset? and more details are needed. On the results section the author only presents results on one dataset, with the proposed model not being the best performant. The whole evaluation of the proposed model hinges on the manual review of ONE, summarized article, making this quite lax and not sufficient to show what the author claims in this paper. Without a proper and rigorous evaluation, the discussion section seems a bit mute. While the ideas in this paper of the novel model are interesting, the work seems rushed and not at the publication stage yet. 
review_title: Promising idea with a weak evaluation

final_score: rating: 5: Marginally below acceptance threshold
review_content: Summary

The authors present Continual BERT, a model designed for being able to continuously update and summarize COVID literature. It is trained using CORD-19 to predict abstracts from document text.


Reasons to accept:
- the idea of online learning is interesting and has interesting properties
- the model is interesting

Reasons to reject:
- The exposition of model is inconsistent in its detail - going into depth about relatively trivial modifications to a recurrent layer but glossing over important details in overall model structure. Section 3.1 should be revisited and expanded in depth. Additionally, the model figure 1 is hard to decipher.
- in terms of ROUGE scores, the continual BERT lags significantly behind standard methods (BERT Sum)
- insufficient exploration of continual bert's inability to summarize literature - if old literature is the issue then it could be excluded or this impact directly measured.
- predicting abstracts is perhaps a poor choice for extractive models. They frequently contain positioning information and words not found elsewhere in the document. An upper bound for ROUGE scores from extractive methods would be informative.

etc:
- There are some broken citations
review_title: Interesting work, with unimpressive analysis or results

final_score: rating: 5: Marginally below acceptance threshold
review_content: The author combines two bert models and adds a transformer on top to perform extractive summarization. The model architecture is optimized to avoid catastrophic forgetting

Reasons to accept:
* The author creates a hybrid architecture based on a state of the art language model.
* The example shows that the model satisfactorily generates extractive summarizations.

Reasons to reject:
*  The author does not provide evidence why their model is either better, or faster, or more computationally cheap than other alternatives. It shows that the proposed technique is slightly worse than BertSum.


Comment: 
I  cannot assess if the paper provides a methodological contribution, given my limited level of expertise in language models.
review_title: A hybrid BERT architecture for covid-19 paper extractive summarization