Show and Write: Entity-aware Article Generation with Image Information
Authors: Zhongping Zhang
Yiwen Gu
Bryan A. Plummer
Venue: Submitted to ICLR 2023 (2023)
Keywords: image-to-text generation
language modeling
named entity recognition

Abstract: Prior work for article generation has primarily focused on generating articles using a human-written prompt to provide topical context and metadata about the article. However, for many applications, such as generating news stories, these articles are also often paired with images and their captions or alt-text, which in turn are based on real-world events and may reference many different named entities that are difficult to be correctly recognized and predicted by language models. To address this shortcoming, this paper introduces an ENtity-aware article Generation method with Image iNformation, ENGIN, to incorporate an article's image information into language models. ENGIN represents articles that can be conditioned on metadata used by prior work and information such as captions and named entities extracted from images. Our key contribution is a novel Entity-aware mechanism to help our model recognize and predict the entity names in articles. We perform experiments on three public datasets, GoodNews, VisualNews, and WikiText. Quantitative results show that our approach improves generated article perplexity by 4-5 points over the base models. Qualitative results demonstrate the text generated by ENGIN is more consistent with embedded article images. We also perform article quality annotation experiments on the generated articles to validate that our model produces higher-quality articles. Finally, we investigate the effect ENGIN has on methods that automatically detect machine-generated articles.

Under review as a conference paper at ICLR 2023
SHOW AND WRITE: ENTITY-AWARE ARTICLE GENERATION WITH IMAGE INFORMATION
Anonymous authors Paper under double-blind review
ABSTRACT
Prior work for article generation has primarily focused on generating articles using human-written prompt to provide topical context and metadata about the article. However, for many applications, such as generating news stories, these articles are also often paired with images and their captions or alt-text, which in turn are based real-world events and may reference many named entities that are difficult to be correctly recognized and predicted by language models. To address this, we introduce ENtity-awan article Generation with Image iNformation, ENGIN, to in- corporate an article's 's image information into language models. ENGIN represents articles that can be conditioned on metadata used by prior work and information such as captions and named entities extracted from images. Our key contribution is a novel Entity-aware mechanism help model recognize and predict the entity names in articles, improving article generation. We perform experiments on three public datasets, GoodNews, VisualNews, and WikiText. Quantitative results show that our approach improves generated article perplexity by 4-5 points over the base models. Qualitative results demonstrate the text generated by ENGIN is more consistent with embedded article images. We also perform article quality an- notation experiments on the generated articles to validate that our model produces higher-quality articles. Finally, we investigate the effect ENGIN has on methods that automatically detect machine- machine-generated articles.
INTRODUCTION
Automatically writing articles is complex and challenging language generation task. Developing a
reliable article generation method enables wide range of applications such as story generation (Fan)
ct al. 2018 Peng et al. 2018), automated journalism (Leppänen et al., 2017 Brown et al. 2020 de-
Tending against misinf formattoon (Zellers et al. 2020 Tan et al. (020).. writing Wiki articles (Banerjee
& Mitra 2016 Merity et al. 2016). among other applications. In early work (Lake et al.) 2017 Jia &
Liang] 2017: Alcorn et al. 2019). language models were trained using domain-specift data. These specialized methods worked well for in-domain data, but did not generalize to out-of-distribution inputs. To address this, language generators finetune large-scale pretrained language models (Rad-
ford et al. 2018 2019. Brown et al.) 2020) on domain-specific data such as news (Zellers et al., 2020) and Wikipedia (Merity et 2016). These methods can generate articles with unconditional sampling given the first few sentences of articles (Radford et al. 2018 2019) or with conditional sampling given metadata such as title and author (Zellers et al. 2020 Brown et al. 2020p.
There are two important challenges not explored in prior work for article generation. First, they only model text (Brown al. 2020 Zellers al.) 2020) (Figure a)), ignoring images embedded in the articles that may provide additional insights. Second, these methods only implicitly model named entities that commonly appear in long articles like organizations, places, and dates to provide context (Radford et al., 2019; Brown et al. (020). These named entities are critical to accurately modeling a long article. but often is not known what named entities may appear at test time.
To address these challenges, propose ENtity-aware article Generation framework with Image iNformation (ENGIN), which leverages image information and a novel entity- mechanism for article generation. Named entities indicate important contextual information about the events related to the news report in Figure bb). However, in prior work (Radford et al., 2019: Brown et al.
2020 Zellers al. 2020 named entities are modeled together with the other text, and the language

---

Under review as a conference paper at ICLR 2023
(e) Text-only Articl Generation
Artice Geveration with Image
Thit Angdis jolk KLIK Staday Protoor Londie School
Ntmved hutty Secogetttins
Years
2015 Dicats ansaciaaaooo
coup ta rie haad sas refgge gear simon
Bods Aspllaa Jelke 1it, Dvanssiniing
qudd
Eantor Storet Rosoal
Sodr Angiln Jal nts to
the Unctde Biuddd hitennttne agand tite will the lender Steed Screanis Coras
Schoo of feoe oele Coem Seree, hear Socurty
Sox radig peatusan
caura'ss propls Dut
sncurd Monday in the infectty Bease's
Neus Irag Captios
Pas naw caurar prigin oue mut silag slladd Honday movored
leckit coheriggnee andi
"casaggee
whece tee Foodel the Promesse Soual Yeckee Goodal
Nocion
Mr Hige progran No. ole the treaddi hemsonss
heterro
meeign
Prior work (Brown al., 2020 Zellers et al., 2020), shown in (a), produces an article (black text) conditioned on article metadata (gray text), ignoring image information. This paper, shown in (b), also conditions on image information like extracted named entities, which may provide important context (e.g., knowing the woman in the image is an actress), when generating articles.
Figure 1:
model may find it difficult to distinguish entity names from the other text in articles. To solve this issue, we propose an entity-aware mechanism to help ENGIN recognize and predict named entities. Specifically, we insert special tokens after each entity name to indicate its entity category. ENGIN models the named entity with its entity category jointly. An additional benefit brought by our Entity- aware mechanism is the named-entity recognition (NER) ability, i.e., our model not only recognizes and predicts the entity names but also predicts the entity category simultancously.
Prior work has proposed entity-aware mechanisms for related tasks like news image captioning (e.g.. (Biten et al., 2019; Tran et al.) 2020, Liu et al., 2020 However, these methods do not generalize to article generation as they rely on having a lot of contextual information (an article) as well as direct indication of what entity they need to generate a caption for (from the image). In contrast, when generating an article conditioned on collection of images and captions, the model has to select when to use each entity in the metadata. In addition, some key entities may not be present in the metadata. For example, in Figure a[) the article mentioned Angelina Jolie Pitt is an Oscar winner, but this entity doesn't 't appear in the image or caption shown in Figure [b). In news image captioning, entities used in the caption almost always appear in the body of the article (Liu lt al.] 20200 Tan et al.] 2020)) Thus, as we will show, adapting entity-aware mechanisms used in prior work (e.g.. (Liu al. 2020 Dong al. (02Ib) results in poor performance on our task.
While we show that providing a list of named entities for generating an article boosts performance, it does require small overhead cost. To address this, we also show gains without manual input. Specifically, as shown in Figure [[b),, images and captions can contain important events or key figures associated with an article. Since large vision-language models have seen various named entities during training, we use CLIP (Radford et al. 2021) to automatically select a set of likely named entities from an image (see Section Figure presents the overall pipeline of ENGIN.
In summary, the contributions of this paper are:
We propose an entity- language model called ENGIN for article generation. Compared to existing models focusing on text-only context (Zellers et al./ 2020 Brown et al. 2020), ENGIN effectively leverages information from images and captions to generate high-quality articles.
We propose Entity- -ware mechanism to help language models better recognize and predict named entities by also modeling entity categories, boosting performance.
Experiments on GoodNews (Biten et al.] 2019) and VisualNews (Liu et al. 2020) report 1.5B params ENGIN-XL boosts perplexity by 2.5pts over 6B param GPT-J (Wang & Komatsuzaki
202I). We also show ENGIN generalize via zero-shot transfer WikiText (Merity et al.) 2016). We perform a user study to verify that ENGIN produces more realistic news compared to prior work (Radford et al. 2019 Zellers al., 2020). This suggests that our model may help provide additional training data for learning more powerful machine- text detectors.
RELATED WORK
Article Generation in recent research produces text using large-scale pretrained transformer mod- els, which can be divided into two categories: unconditional text generation (Radford et al., 2018
2

---

Under review as a conference paper at ICLR 2023
Impur (Neea fied) denads
Terqet (Dody field)
date
Vasal
like Hepp Spari Flce
It seedustce for wered Morcs 5. 1841 -oattiy when he blals and though
conpact
(14777-- In ante that the Photes
atdgle
o0 staycitintiinp les sooon Mil Lighe when
Nanol Entrics
brugh Captionra
Naddew Chriker sirkir
Andzo Mclanciinn 25, afttc Suadegg Russs wasnnng le bels chiss hachal groms
Nos tu Pura lealing to lh fieudh apái by ineme chateantic sitrect placr the
aigh Tinnor PoT
tinkey Crikee 41rppoing- dic heethhegge
The Hurld (10000))
-fattteet Wiell persotp.
Volles s2300
Soom canaeryy sag
unowly hoe use de soiking lane. 'dta more staro sead cal enc juer barrheann Jant Unwker (uttip.
Figure 2: ENGIN overview. ENGIN generates articles conditioned on (1) meta field of articles including domain, date, title, summary, and so on (Section 3.1p: and (2) vision field that consists of captions and named entities extracted from the embedded imgges (Section 3.2). Empty fields are skipped (like summary in this example) by using the start token of the next field. In addition, we propose an entity-aware mechanism (Section 3.3) to better represent named entities in articles.
2019) and conditional text generation (Brown et al. 2020 Zellers et al. 2020). Generating articles via unconditional samples has been shown to be less effective because the models may interpret the first sentence of articles as tweet and start posting responses (Brown et al. 2020). To en- able controllable generation, GPT3 (Brown et al.] 2020) generates articles conditioned on titles, Grover (Zellrs et al.) 2020) decomposes news articles into separate parts and generates articles conditioned on the metadata like the author or organization. In this paper, we further explore the effect of visual information and named entities. Specifically, ENGIN produces articles conditioned on both the metadata like prior work and the visual information extracted from embedded images, with special care for explicitly extracting and modeling named entities.
Article-basee Image Captioning such as news image captioning is designed to caption images based articles and images. [Ramisa et al. (2017) proposed end-to-end framework that takes the concatenation of article and image features as input and outputs captions by an LSTM decoder. However, they often failed to predict named entities that were not seen during training. Thus, more recent work has included entity- mechanisms (Biten et al. 2019 Tran et al. [2200] Liu et al.) 2020) to more accurately model these elements. In this paper, we effectively reverse the inputs and outputs of these papers, iee. we generate an article based on images and captions rather than generating captions based on images and articles. As discussed in the introduction, this shift breaks the assumptions used by entity-aware mechanisms in image captioning, so they do not generalize.
ENGIN: ENTITY- ARTICLE GENERATION WITH IMAGE INFORMATION
The goal of our task is generate articles via conditional samples. In this paper, we propose ENGIN, which leverages image information and entity-aware mechanism to generate higher-quality articles. The input to ENGIN consists of both metadata and image information that is composed of named entities and captions (summarized in Figure Section introduces the article generation problem in detail. Section 3.2 discusses how leverage image information. Section 3.3 presents our novel entity-awand mechanism. Finally, Section 3.4 summarizes the overall training strategy of ENGIN.
3.1 ARTICLE GENERATION
Given a set of documents {21, {21, D2. 2n) each consists of variable length sequences of symbols {81, ..., 9m }. the statistical language model of a text document x can be represented by the prod- uct of the conditional probability of next symbol given all the previous ones (Bengio et al. 2003):
pxx)= = p(s 81 8:-1)
I
(1)
3

---

Under review as a conference paper at ICLR 2023
where each symbol is processed uniformly and the document is viewed as unstructured text field (also referred as body field later). Language models based only on Eq. produce articles via unconditional samples. As a result, these models cannot output controllable generation (Hu et al, 2017). For controllable generation, the language model can be formulated by the joint distribution of separate fields decomposed from the article (Zellers et al. 2020
p(æ)
pl meta
body
where meta field is data-dependent term consisting of a set of subfields. For instance, meta includes date, title, summarry in GoodNews (Biten et al. 2019) and domain, date, topic, title in VisualNews (Liu et al. 2020). Thus, we model by:
p(x) = p( body meta ) meta
(3)
According to Eq. | we introduce special tokens <start->>> and <end-r> to indicate the boundaries of field т. The content of a target field T is sampled from the model starting with <start->> and ending with <end-r>.
3.2 EXTRACTING INFORMATION FROM IMAGES
One of the advantages of ENGIN over prior work (e.g., (Radford et al. 2019 Zellers et al. 2020)) is
that we can take advantage of the information provided by embedded images articles. The current state-of-the-ar that combines the image and language descriptions adopts the Transformer (Vaswani 2017) architecture and models the text and image tokens as single stream of data (Ramesh et al. 2021 D.
al.
However, we found using images directly is less effective in the article generation setting. Unlike typical image captioning systems, articles such as news or Wikipedia report events happening in the real world and tend not to describe objects in embedded images directlyA In addition, methods like (Ramesh et al. 202I) take 1024 tokens to encode the image. If we adopt this image encoding, the sequence length of our language models will be doubled and the model weights cannot be initialized from the standard pretrained language models. However, this is not to say the images contain useful information. Indeed, as illustrated in Figure [ many images contain information about named entities. Thus, as illustrated in Figure 2 we use named-entities extracted from images in our model, along with the ground truth image captions directly.
We provide information from images to our model via a combination of two fields, caption field and named-entity field. We ground truth image captions directly the content of caption field. Below we discuss two ways to build named-entin field.
Oracle named-entities. This approach assumes we are provided with all the named entities that would appear in articles. To accomplish this, we extract named entities from news articles using SpaCy (Honnibal & Montani 2017). which is input directly to our model.
CLiP-based NER. Most existing methods identify a predefined list of named entities (Yadav & Bethard 2019 Li et al. (020). However, we note that CLIP (Radford et al. 2021) was trained on 400 million image-text pairs collected from the internet, many of which likely contained named entities. Thus, we use CLIP build an open-ended visual NER framework. First, we construct the NER candidate list for each image by extracting named entities from the dataset articles using SpaCy (Honnibal & Montani] 2017). Then we use CLIP to predict the similarity between the article image(s) and the candidate entities. The top A entities are then provided as input to our model.
3.3 ENTITY-AWARE MECHANISM
As discussed in the Introduction, accurately generating named entities can help avoid inconsistencies between the images associated with an article and the article text itself. For example, in NBA news, an entity-awan model should be able to predict "Curry" given the previous word "Stephen" while
For example, the language description of the image in Figure 2 is baseball player swing bat at ball" (by NIC (Vinyals et al. 2015). However, the ground truth caption is "AAnrrew McCutchen, 27, of the Pitthurgh Pirates is among the heirs apparent to hecome the next links a long chain baschall greats" and the article is discussing Andrew McCutchen's contract and his future career.
We set & 10 in this paper.
4

---

Under review as a conference paper at ICLR 2023
Help Worned:
Foce
Help Warred: Seeking Sport's New Face
cenciaa sscer Stark 5 hott, nightt ao coueect
kepp goud 2314
choordll Fins
beille
silandd plaur who the Higour aue io city where Sklayy Goblyy
Rocthd dorger and The Feitmall work Josr, thought hue the Woeld Sidke oags sabed
porsuasee aalene Ohallaad caddiide Juaee
for the puact Deic porglic cusiiiliiiiiie beights T.Mid Tour vile Roei th 'adeot Mart Tibeit..
H.d. Vied becoona
sodfen Hightet
Figure 3: Example of the entity-aware mechanism in ENGIN. Each named entity is attached by its corresponding entity category. ENGIN models the entity name and category jointly avoid the inconsistency between named entities of images and article text.
the traditional language models might fail. Existing methods model named entities uniformly with the other text, making the leverage of named entities less effective. To help our language model be aware of named entities, we insert the entity category predicted by SpaCy (Honnibal Montani 2017) after each entity name. We use special tokens as the indicator of these entity types. Then the entity name and its corresponding category are modeled jointly by ENGIN. We visualize our entity-aware mechanism by an example in Figure
3.4 OVERALL LEARNING STRATEGY
Architecture. We build ENGIN using the same architecture of GPT2 (Radford et al. 2019). Fol- lowing GROVER (Zellers al. 2020), we propose three model sizes: (1) ENGIN Base has layers and 124 million parameters, on par with GPT2-1 1PMM 224 and GROVER-Base (2) ENGIN-Mediun has 24 layers and 355 million parameters, on par with GPT2-355M and GRovER-Large; (3) ENGIN-XL has 48 layers and 1.5 billion parameters, on par with GPT2-1.5B and GROVER-Mega.
Encoding Strategy. Given the image information extracted from the embedded image, Equation is re- re-formulated as:
p(x)= p( body meta vision
p meta
vision
where vision field consists of caption field and named-entity field (from Section 3.2p. To sam-
ple from Equation F
we define a canonical order among the fields (or subfields) of articles fir)) and model the articles left-to-right in the order using Equation
(fi
< f2
JIFI If specific field f. is missing, our model will automatically skip that field by of 5/ "'/s""''
introducing the start token of next field fi+1. We illustrate such an example in Figure where ENGIN starts generating the body field after title because the summary field is empty.
Decoding Strategy. dikelihooddmaxxiizzaioo decoding strategies like greedy search or beam search work well in close-ended generation such as image captions, machine translation, or summariza- tion. However, these methods suffer from the repetitive text problem in open-ended generations like dialog or story generation (Hashimoto et al.) 2019 Holtzman et al. 2019). Sampling meth- ods (Fan al. 2018 Holtzman al. 2019) are therefore proposed to introduce more random- ness and surprise to text generation. In our work, we adopt the top-p sampling (nucleus sampling) method (Holtzman et al. 2019) as our decoding strategy for article generation.
EXPERIMENTS
4.1 DATASETS AND EXPERIMENT SETTINGS
Datasets. We evaluate ENGIN on three public datasets: GoodNews (Biten et al.) 2019), Visual- News (Lu et al. 2020), and WikiText (Merity et al. 2016). The GoodNews dataset provides the
We define canonical order in Goodnews Biten et al. 2019) as: domain, date, named-entity, title, caption, summary, hody; and Visualnews (Liu al. 2020 as: domain, date, topie, named-entity, title, caption, body.
5

---

Under review as a conference paper at ICLR 2023
Table 1: Comparison of different generation methods and model sizes using perplexity (PPL) to measure performance on the GoodNews and VisualNews datasets. ClipNE denotes that we select CLIP-based named entities in named-entie field (described in Section 3.2) NE denotes that apply oracle named entities in named-entity field. PPL calculated only the article body
GoodNews PPL
VisualNews
Model Name
72 paraams
Mayyrs
moodel
Thheada
PPL
GPT2-124M (Radford et 2019
124M
768
23.6
27.5
GROVER-Base (Zellers ct a
2020
124M
12
768
12
23.8
21.9
GPT-Nco- 125M (Gao et al.
2020
125M
12
768
12
27.1
29.3
GPT2-124M (Finetuned)
124M
12
768
12
17.3
18.3
EXGIN-Base (ClipNE)
124M
12
768
12
14.8
16.1
ENGIN-Base (NE)
124M
12
768
12
12.0
13.1
GPT2-355M Radford
2019
355M
24
1024
16
17.8
20.1
GROVER-Large (Zellers et a 2020)
355M
24
1024
16
18.5
16.4
GPT-Neo-1.3B (Gao et al. 2020)
1.3B
24
2048
15.3
15.9
GPT2-355M (Finetuned)
355M
24
1024
16
13.5
14.0
ENGIN-Mcdium (it-MMddduu((Cpppe) ENGIN-Mcdium(NE)
355M
24
1024
11.6
12.5
355M
24
1024
16
9.5
10.2
GPT2-1.5B (Radford al. 2019
1.5B
48
1600
25
13.9
15.7
GROVER-Mega (ellers el al.
2020)
1.5B
48
1600
25
14.5
12.6
GPT-Neo-2.7B (Gao et 2020)
2.7B
32
2560
20
13.5
14.0
GPT-J-6B (Wang & Komatsuzaki 2021
6B
28
4096
16
11.3
11.6
GPT2-1.5B (Fiettued)
1.5B
48
1600
25
12.6
12.4
(CCiNN)))(ClipNE) ENGIN-XI-(NE)
1.5B
48
1600
25
10.8
11.1
1.5B
48
1600
8.7
9.0
URLs of news from New York Times ranging from 2010 to 2018. We were able to download 307,286 news articles from the provided URLs. The remaining articles are either broken links or non-English articles. Following the split ratios of (Biten et al. 2019), randomly split 15,365 arti- cles validation set, 30,728 articles in test set, and the rest articles in training set. The VisualNews dataset contains news articles from four news sources: Guardian, BBC, USA Today, and Washington Post. We obtain 582,194 news articles in total after we removed broken links and articles without metadata. Similarly, WikiText dataset contains 600 Wikipedia articles in training set, 60 articles in validation set, and 60 articles in test set. We also performed zero-shot article generation the test of WikiTexx.
get 491,796 training 28,932 validation set, and 57,889 test set. The
Metrics. We adopt Perplexity (PPL.) to quantitatively evaluate our language models. Perplexity is defined as the exponentiated average negative log-likelihood of sequence. Given Equation | the perplexity of r is calculated by:
m
(P) exp
log pog 1ogp(s)/11,,...8--)))
(5)
where $1,..., 8; are ground truth tokens in x and p(-) is the probability predicted by the model.
4.2 LANGUAGE MODELING RESULTS
Perplexity. Table presents sizes, architectures, and perplexity results of different models on Good- News (Biten et al.) 2019) and VisualNews (Liu et al. 2020). We see that ENGINS of all three model sizes significantly outperform the baselines. On the base size, our model ENGIN- (BNNN) improves over the original GPT2- 124M model by factor of (23.6~ 27.5 + 13.1). 1). We draw three major conclusions from Table I First, the data distribution still plays an important role. Finetuned GPT2s improve PPL over the original GPT2s. The improvements become less obvious with greater model size (VisualNews: 27.5 18.3 base size; 15.7-+1244 of XL size). Second, ENGIN notice- ably improves the performance over finetuned GPTs (4-5 perplexity points both datasets), which

---

Under review as a conference paper at ICLR 2023
18:3
Table 2: Comparison of baselines adapted from close- related tasks using PPL. (a) leverages visual features directly extracted from images, (b) contrasts entity- aware mechanisms.
Gecefoon
Figure 4: Ablation results of ENGIN-Base GoodNews and VisualNews. Text-only denotes the model focuses only on text information, which is same to finetuned GPT2 model. Cap de- notes caption field, EA denotes the Entity-aware mechanism. CapNE denotes named entities ex- tracted from captions.
Good- Visual- News
Model Name
News
BU (Anderson al.
2018)
26.6
24.3
BDA (Post & Vilar 2018
51.2
InfoSurgeon (Fung al.
2021
41.8
InjType (Dong et al.
2021
18.2
19.0
VNC (Liu et al. 2020)
16.7
17.8
(NGIN-Base(N
12.0
13.1
demonstrates the effectiveness of our approach. Third, PPL improves with increased model sizes, indicating that more powerful generator could be trained with even greater model sizes.
Parameter Efficiency. Table that ENGIN can achieve a comparable performance with alter- native models using much fewer parameters. For instance, (NGIN-Base((N)) only has 124M param- eters but outperforms the GPT-NEO-2.7B and achieves comparable performance with finetuned GPT2-1.5B (12.0 vs. 12.6 PPL on GoodNews, 13.1 vs. 12.4 PPL VisualNews). ENGIN-Medium (NE) model with 355M parameters already outperforms all the basclines including GPT-J-6B.
Additional baselines adapted from closely-related tasks are presented in Table 2 image embeddings extracted by the bottom-up features (Anderson et al. 2018) with meta informa- tion as input. InjType (Dong et al. (021) applies classification decoder predict entity names from a list of candidate names. VNC (Liu et al.) 2020) adds a named entity set to input as the entity- aware mechanism. BDA (Post & Vilar 2018 and InfoSurgeon (Fung al. 2021) apply copy mechanisms to sesuuence-t--equeence translation frameworks. In Table we see that BU struggles to boost performance, likely due the loose correlation between image content and their articles as discussed in Section [ Table 2 demonstrates that the antity-aware mechanisms proposed for generating short text do not generalize to generating longer articles, where our approach obtains a 4-5 PPL. improvement on both datasets. Besides, we find that BDA and InfoSurgeon do not perform well on article generation task. may because the sequence-to-sequene to-eueence translation frameworks of these models find it challenging to effectively leverage prior knowledge from pretrained language models (e.g., (Radford et al. 2019)).
B concatenates
Ablation Study. Figure 4 shows ablations of ENGIN-Base to supplement our analysis. We observe that both the caption and named-entity fields boost performance, revealing that cues from news im- ages can help produce sigher-quality articles. Comparing using only captions (Cap) vs. combining them with our Entity-aware mechanism we get a minimum gain of 0.6 PPL, demonstrating its effec- tiveness. In addition, observe that ClipNE outperforms CapNE, validating that deteeette named entities are more effective than those extracted from captions.
Article Quality User Study. Following ZZelerss al. 2020 Kreps al. 2020 Brown et al. 2020), we ask annotators to distinguish machine-generated articles from human-written articles. We randomly selected 50 news stories from GoodNews and VisualNews test sets (100 total). Given the metadata and news images. we generated news articles using three different language models: GRovER-Mega, GPT2-1.5B (finetuned), and ENGIN-XL. This results in total of 200 articles per dataset. We recruited 200 Qualified Amazon Mechanical Turk (AMT) workers per news dataset. Each article was annotated times by AMT workers, where each worker was presented with the
'InJType (Dong et al. 2021 BU (Anderson et al. 2018), and VNC (Liu et al, 2020) are adapted from closc-ended paragraph generation, image captioning, and news image captioning, respectively. BDA (Post & Vilar 2018 and InfoSurgeon (Fung et al. 202T) are adapted from sequenceeto-ssqqueec models.

---

Under review as a conference paper at ICLR 2023
Title A Deaigner Unites the Basqua Country
Domal:: mwcondimes.comm
Date: 20-10:206
Jon Losas irannii chacking dee 3D sodl modd of that
Haman Nore Armole
ENGEN- oe nezard Newo Amii
2007, de Freadh indan lal designe Jax Lask lazadl opuooda rene madion in waed cilie in the fidillls of the Pymeess Iis secioulis "sulag" has seuatly bucoss usilady cealne epiceier indurrial production butk Pay Seupe.
Fellowing in the foottes of ti sude cnsoreppat 2807 Frusd. Spaak auditet Juse Lask listicks whose anciires becare the subject of led yea's Barpe/ Trascoplle Radeo Divre edailtiion Anda Eluss, deiggnr the ancouneed
Spuckls
Basque
dinetaie his Dodgs Augsls siglle-- in. Merken, the lagan deige far for beash of dhe Blaga Fruroophone world
As has yoa's Malic & cligi desigs dak, Mr. huraok won plandla for deigging one the world fire looplane cherr sade of leedegrdhile plan hard part of bong bunddit
pxlpwer
Roope
HRB,
Hes
know evyywhens Lives who of
mater regiin (Mr brstok wis boon Frence St jonvdellng He his alo eccestly oourepel the leand aleetity of Caeessee (a Reayee chood conpury) and ldpall dodgg the inseiees lacursio cao lodge Sudden,, aont
desgee: is the city of RMl roring the fire boor contrnation taae them
willeg crseetukeens is part coueusiity' i growing besions And the sane Franes."
Gatig back Biighan viurt the buttuge of Makeen Osger frendes ssil
This year, be fiende a now practic Blsqu dodgnor Spuzkia ourrectues Ander Lixas, with at craating a erchsparpon dodge madio inspaeed by
lox inbaing ha same with io coldeatzd ligh- cth--chharr awedittonn be ceplazed. the hous was fuily avar of Buuque rects"
GPTE peressted News Anici
GRDANi grasrand New Artkl
Thur sladed Checs Blach and his will, Sauna, whea the ust syu on cadh wetlee be
buth cailing dething poliiis Mis Mad estiios to candiie mraitiin designa
they
int toreriie somuner owine batk ofer Their finesliip would
produces
reflevials
ferelles
cest yours woeld
Evrryshnng in beok sands as her slestity desgger "Tes therent paoineaae
fy away froms deer eatee Baage repion live and work and play oigeler in deeealoss to seddmiy see happll
persoe dei bagg cencckolees locisse I'm foreaass Mr. Midie snll inve very itrong
wnall
Rushh desdad
Thene
wass Cara, the artists dracar of the Grppy flose Los bousigee in thas Villge and Saura, Soanker and crattre dinster hot owe company Fara, which spocaalens foral sarkks people
brashzn will
The 4b-yuur-lll menker of twoe mustag snce bratnonn It's stoeng comben
traditical falkon and actriirs She hoptag to raki eveny and ofter corvica tie pulitical actiiim Bunqae sounty- and bquand.
cuplata. Soa lanc vith fuom fow woch what yee cas sey With mensthing you ware do fildd-dedd West impage I huse propoaa you non Gooe. The caople rountly caardd Misss, Below dacas undecamman tf
In phopur four detalor Mr. Musied has bect tandred in socil and pobtica
actinies She wire on henger wthe dier the was studedd witk to pd. Her pailas to bolocconn eedloeee
Basqu causary
bus coo Blage gestnssst
altiling brinnd.
Figure 5: Qualitative examples comparing ENGIN-XL -L with GPT2-1.5B (Finetuned) and GROVER- Mega on GoodNews. We cut the articles to fit the figure size. The entity names from image infor- mation are highlighted in light purple (PERSON tag) and light blue (ORG tag) colors. We can see that the named entities in captions also appear in the human-writter and ENGIN-generae articles. In contrast, the GPT2-generated and GROVER-generaae articles do not contain the correct entity names corresponding to the image and caption.
article titles, images and captions, and was asked to indicate if the article was human or machine generated. If they thought the article was machine-generaten they were asked to indicate a reason for it following the same option format as Tan et al. (2020).
Table a) reports annotation accuracy in identifying articles from VisualNews as machine or human- generated. We see ENGIN-XI -XL is able to generate hard-to-deted news articles (we see 2% boost over GPT2-1.5B, and a 5% gain over GROVER -Mgga), validating the effectiveness of our approach.
Machine Discriminator. We also apply machine discriminator to detect the generated articles. Specifically, we apply a RoBERTa (Liu et al. 2019) detector finetuned with the outputs of GPT2- 1.5B (Radford et al.) 2019) to detect generated articles. The maximum article length is cut to 512 to fit the model input size. For comparison, we use the same article set from our user study. The RoBERTa accuracy is shown in 3b). We observe that the machine discriminator is much better machine-generated news. We also see that articles produced by ENGIN-XL can be reliably detected by RoBERTa though it gets the lowest accuracy on human evaluation. This can be due to the fact that GROVER- -Mega, GPT2-1.5B, and ENGIN-XL all share a similar underlying model architecture. Thus, they may contain enough similarities in the distributional features that are recognized by the machine discriminator.
at identifying the machine-ee generated
Zero-shot Article Generation We perform zero-shot experiments on Wikipedia articles demon- strate the generalization of ENGIN. Table reports the perplexity points on WikiText. From the table, we see that our oracle named entities and entity-aware mechanism still can improve the per- formance over several baselines, even though the data distribution between Wikipedia and news is significantly different. For example, the GPT2 models finetuned on Visualnews get worse perfor- mance than the original GPT2 models on WikiText. However, our ENGIN models get comparable or better results than the original GPT2 across different model sizes.
Qualitative Results. We provide a qualitative comparison of GoodNews articles in Figure 5 Con- sistent with our annotation experiment, we compare the muman-writted article with three machine- generated articles. From the results, can see that ENGIN-XL model can effectively produce
8

---

Under review as a conference paper at ICLR 2023
Table 3: (a) reports the performance of AMT workers at correctly identifying article as human or machine generated over different generation methods while (b) uses OpenAl's machine generated text detector based on RoBERTa (Liu et al. 2019) to perform the same task.
(a) Human-based detector
(b) RoBERTa detector (Liu et al. 2019)
GRoVER-Mega
72.8%
GROVER-Mega
90%
GPT2-1.5B (Finetuned)
69.6%
GPT2-1.5B (Finetuned)
84%
ENGIN-XI.
67.6%
ENGIN-XI.
84%
Table 4: Zero-shot results WikiText (Merity et al.
2016).
Both finetuned GPT2 and ENGIN
models are finetuned based on VisualNews (Liu et al. articles is set to 1024.
2020
The maximum length of Wikipedia
Method
Parrams
PPL
Πparams
PPL
Ppraams
PPL
GPT2 (Radford et al.
2019)
124M
26.1
355M
19.1
1.5B
14.8
GPT-Neo (Gao et al.
2020)
125M
24.9
1.3B
13.1
2.7B
11.5
GPT-J-6B (Wang & Komatsuzaki
2021
6B
9.0
GPT2 (Finetuned)
124M
33.8
355M
25.2
1.5B
25,9
ENGIN- (NE)
124M
20.7
355M
15.4
1.5B
16.3
articles with the named entities learned from image information. contrast, finetuned GPT2-1.5B and GROVER-Mi EM-Maga failed to generate correct named entities in articles. For example, both ENGIN- generated article and the human-written article mentioned "Hean Louis Tratzoki" and "Alki", which are appeared in the caption. In contrast, articles generated by GPT2 or GROVER are discussing some other entities such as "Chris Burch" and "Ms. Madrid."
4.3 DISCUSSION
Defending against machine-generated misinformation. In our paper, we mainly investigate mod- eling machine-ggnnratee articles, which can be used directly for generation, but also can provide strong language features to support applications like article retrieval. However, actors can also use the same technology to generate articles with misinformation such as fake news by modifying in- formation of specific fields to realize two purposes: monetization (ad revenue through clicks) or propaganda (communicating targeted information) (Zellers et al. 2020). Therefore, the develop- ment of better article generator potentially help train a more powerful discriminator. When comparing the results the RoBERTa detector for GPT2-1.5B and ENGIN-XL. in Table 3 we find that only 25% of the articles that were predicted as human written came from the same generation prompts. Thus, the two methods can provide different views given the same prompt, which can provide additional information for train- ing an even more powerful machine generated text detector. We note that our contributions are largely architecture agnostic, so they could be provided as input to ONN-based generators, which may provide a larger distribution shift in the generated articles that may fool a discriminator trained only on Transformer- based outputs.
not only help humans write high-quality articles but also
5 CONCLUSION
In this paper, we proposed an antity- article generation method called ENGIN to address two factors that are unexplored by prior work: image information and named entities. Concretely, EN- GIN produces articles conditioned on both metadata and visual information extracted from embedded images in articles. Moreover, we introduced an entity-aware mechanism to help ENGIN recognize and predict named entities more effectively. ENGIN outperforms current popular language models using much fewer parameters in quantitative and qualitative experiments on GoodNews and Visu- alNews. For example, ENGIN-XL outperforms GPT-J by roughly 2.5 perplexity points using only quarter parameters of GPT-J. The noticeable improvements demonstrate that ENGIN generate articles more accurately and efficiently. The article quality annotation experiment further validates that ENGiN-generated articles have higher quality compared to existing methods.

---

Under review as a conference paper at ICLR 2023
6 REPRODUCIBILITY STATEMENT
We implemented our models mainly based on Pytorch (Paszke et al. 2019) and Transformer (Wolf et al. 2020) libraries. The maximum sequence length of language models set 1024. For ENGIN- Base and ENGIN-Medium, we used a batch size of and a maximum learning rate of 1 x 10-4 For ENGIN-XL, we used a batch size of 4 to fit the GPU memory. Correspondingly, the maximum learning rate is set to 20.5 x 10-4. We trained our models around 3 epochs with 0.06 epoch for linear warm-up on both datasets. We parallelized ENGIN-XL on NVIDIA RTX-A6000s and ENGIN- Medium on 2 NVIDIA RTX-A6000s. The longest training time, ENGIN-X0 XL on VisualNews, takes approximately two weeks on our system. We will also release our code ensure reproducibility after the paper is accepted.
7 ETHICS STATEMENT
ENGIN is a model for article generation. can either help automated journalism or defending against machine-generation articles. However, there is perfect system which can generate 100% accurate articles. Therefore, it is critical for practitioners to check the fact mentioned in articles and avoid the misinformation brought by failure generation cases. Additionally, someone could use our approach to generate misinformation. However, ENGIN applies the network structure that is same to GPT2, which means the discriminators trained for GPT2 articles (e.g., RoBERTa detector (Liu al. 2019) are also effective to discriminate ENGIN -eeneratd articles. Our paper helps to highlight the need for building tools like the RoBERTa detector.
REFERENCES
Michael A Alcorn, Qi Li, Zhitao Gong, Chengfei Wang, Long Mai, Wei-Shinn Ku, and Anh Nguyen. Strike (with) a pose: Neural networks are easily fooled by strange poses of familiar objects. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp-
4845-4854, 2019.
Peter Anderson, Xiaodong He, Chris Buehler, Damien Teney, Mark Johnson, Stephen Gould, and Lei Zhang. Bottom-up and top-down attention for image captioning and visual question answer- ing. In Proceedings of the IEEE Conference on Computer Vision and Puttern Recognition (CVPR), June 2018.
Siddhartha Banerjee and Prasenjit Mitra. Wikiwrite: Generating wikipedia articles automatically.
In IJCAI, pp. 2740-2746 2016.
Yoshua Bengio, Réjean Ducharme, Pascal Vincent, and Christian Janvin. A neural probabilistic language model. The journal of machine learning research, 3:1 3:1137-1155, 2003.
Ali Furkan Biten, Lluis Gomez, Marçal Rusinol, and Dimosthenis Karatzas. Good news, every- one! context driven entity-aware captioning for news images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pp. 12466-12475, 2019.
Tom B Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. arXiv preprint arXiv:2005.1411 2020.
Xiangyu Dong, Wenhao Yu, Chenguang Zhu, and Meng Jiang. Injecting entity types into entity- guided text generation. In Conference on Empirical Methods in Natural Language Processing (EMNLP), 2021.
Angela Fan, Mike Lewis, and Yann Dauphin. Hierarchical neural story generation. arXiv preprint arXiv: 1oXiv:8055004 04433, 2018.
Yi Fung, Christopher Thomas, Revanth Gangi Reddy, Sandeep Polisetty, Heng Ji, Shih-Fu Chang, Kathleen McKeown, Mohit Bansal, and Avirup Sil. Infosurgeon: Cross-mest media fine- grained infor- mation consistency checking for fake news detection. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural I Language Processing (Volume I: Long Papers), pp. 1683-1698, 2021.
10

---

Under review as a conference paper at ICLR 2023
Leo Gao, Stella Biderman, Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang, Horace He, Anish Thite, Noa Nabeshima, et al. The pile: An 800gb dataset of diverse text for language modeling. arXiv preprint arXiv:2 2020.
Tatsunori B Hashimoto, Hugh Zhang, and Percy Liang. Unifying human and statistical evaluation for natural language generation. arXiv preprint arXiv:1904.02792 2019.
Felix Hieber, Tobias Domhan, Michael Denkowski, David Vilar, Artem Sokolov, Ann Clifton, and Matt Post. Sockeye: A toolkit for neural machine translation. arXiv preprint 2017.
Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. The curious case of neural text degeneration. arXiv preprint arXiv: 2019.
Matthew Honnibal and Ines Montani. spacy 2: Natural language understanding with bloom embed- dings, convolutional neural networks and incremental parsing. To appear, 7(1):4114444 2017.
Zhiting Hu, Zichao Yang, Xiaodan Liang, Ruslan Salakhutdinov, and Eric Xing. Toward controlled generation of text. In International Conference Machine Learning, pp. 1587-1596. PMLR, 2017.
Robin Jia and Percy Liang. Adversarial examples for evaluating reading comprehension systems. arXiv preprint arXiv: 170..07723 2017.
Sarah Kreps, R Miles McCain, and Miles Brundage. All the news that's fit to fabricate: Ai-generated text as a tool of media misinformation. Journal of Experimental Political Science, pp. 1-14, 2020.
Brenden M Lake, Tomer D Ullman, Joshua Tenenbaum, and Samuel Gershman. Building machines that learn and think like people. Behavioral and brain sciences, 40, 2017.
Leo Leppänen, Myriam Munezero, Mark Granroth-Wai Wilding, and Hannu Toivonen. Data-driven news generation for automated journalism. In Proceedings of the 10th international conference natural language generation, pp.
2017.
Jing Li, Aixin Sun, Jianglei Han, and Chenliang Li. A survey deep learning for named entity recognition. IEEE Transactions on Knowledge and Data Engineering. 2020.
Fuxiao Liu, Yinghan Wang, Tianlu Wang, and Vicente Ordonez. Visualnews: Benchmark and challenges in entity-aware image captioning. arXiv preprint arXiv:2010.03743, 2020.
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach. arXiv preprint 1907. 1692, 2019.
Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. Pointer sentinel mixture models, 2016.
Antoine Miech, Baptise Alayrac, Lucas Smaira, Ivan Laptev, Josef Sivic, and Andrew Zisser- man. End-to-emt learning of visual representations from uncurated instructional videos. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9879- 9889, 2020.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga. et al. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32: 8026-8037, 2019.
Nanyun Peng, Marjan Ghazvininejad, Jonathan May, and Kevin Knight. Towards controllable story generation. In Proceedings of the First Workshop Storytelling, pp. 43-49, 2018
Matt Post and David Vilar. Fast lexically constrained decoding with dynamic beam allocation for neural machine translation. arXiv preprint 1804 06609,, 2018.
Alec Radford, Kanthik Narasimhan, Tim Salimans, and Ilya Sutskever. Improving language under- standing by generative pre-training. 2018.
11

---

Under review as a conference paper at ICLR 2023
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAl blog, 1(8):9, 2019.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. arXiv preprint 1tt::2203300002 2021.
Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation. arXiv preprint arXiv:2102.12092, 2021.
Arnau Ramisa, Fei Yan, Francesc Moreno-Noguer, and Krystian Mikolajezyk. Breakingnews: Arti- cle annotation by image and text processing. IEEE transactions on pultern analysis and machine intelligence, 40(5):1072-1085, 2017.
Reuben Tan, Bryan A. Plummer, and Kate Saenko. Detecting cross-modal inconsistency to defend against neural fake news. Empirical Methods in Natural Language Prucessing (EMNLP), 2020.
Reuben Tan, Bryan A Plummer, Kate Saenko, JP Lewis, Avneesh Sud, and Thomas Leung. Newssto- ries: Illustrating articles with visual summaries. European Conference on Computer Vision, pp. 644-661. Springer, 2022.
Alasdair Tran, Alexander Mathews, and Lexing Xie. Transform and tell: Entity-aware news im- age captioning. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 13035-13045, 2020.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Fukasz Kaiser, and Illia Polosukhin. Attention is you need. Advances in neural information processing systems, pp. 5998-6008, 2017.
Oriol Vinyals, Alexander Toshev, Samy Bengio, and Dumitru Erhan. Show and tell: A neural image caption generator. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 3156-3164, 2015.
Ben Wang and Aran Komatsuzaki. GPT-1-6B: A 6 Billion Parameter Autoregressive Language Model. htttps://gituu.. com k kingof ol http://eshtttassfoooooo jax May 2021.
Thomas Wolf, Julien Chaumond, Lysandre Debut, Victor Sanh, Clement Delangue, Anthony Moi, Pierric Cistac, Morgan Funtowicz, Joe Davison, Sam Shleifer, et al. Transformers: State-of-the- art natural language processing, In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pp. 38 38-45, 2020.
Vikas Yadav and Steven Bethard. A survey on recent advances in named entity recognition from deep learning models. arXiv preprint arXiv:/9100 / 2019.
Rowan Zellers, Ari Holtzman, Hannah Rashkin, Yonatan Bisk, Ali Farhadi, Franziska Roesner, and Yejin Choi. Defending against neural fake news. Neurips, 2020.
A ADDITIONAL EXPERIMENT RESULTS
A.1 ARTICLE GENERATION
Additional qualitative results on article generation provided in Figure g as the supplement our main paper, demonstrating that ENGIN is able to generate articles given different news sources.
10
I3
A.2 ARTICLE QUALITY ANNOTATION
Table reports human accuracy in identifying articles machine human camman-generated to supplement the main paper. We see that ENGIN-XL is able to generate more realistic-lookin news articles on Goodnews, consistent with the conclusion in our main paper.
12

---

Under review as a conference paper at ICLR 2023
Table 5: The performance of AMT workers at correctly identifying article as human machine generated on GoodNews.
Human-based detector
GROVER-Mega
76.4%
GPT2-1.5B (Finetuned)
73.6%
ENGIN-XL
70.4%
Table 6: Comparison to basclines with copy mechanism on GoodNews.
Method
PPL +
BDA (Post & Vilar 2018)
51.2
InfoSurgeon (Fung al. 2021
41.8
ENGIN -Base
12.0
A.3 COMPARISON TO BASELINES WITH COPY MECHANISM
In our paper, we mainly explore the antity-aware mechanism in language modeling. Another way to include named entities in articles is applying copy mechanism (Post & Vilar 2018) to the decoding process. In 6 we compare our method with BDA (Post & Vilar [ilar]22018 and InfoSurgeon (Fung al. 2021). We reproduce both baselines based SOCKEYE (Hieber et al. 2017). In Table We see that our model outperforms these anselines(5122 .22 vs. 41.8 vs. 12.0). There are two major reason that may drop the performance of these models, First, since BDA and InfoSurgeon are built based on grquence-to-sequenn framework, these methods may not effectively leverage the prior knowledge from pretrained generative language models such as GPT2 (Radford al. 2019). Second, though named entities are enforced to appear in the output, the models may not put all named entities in correct positions.
A.4 ABLATION STUDY ON TOP-K NAMED ENTITIES
We provide the ablation study on top-k named entities in Table 7 We see that the model achieves the best performance when & is set to 15. When greater than 10, the improvement is limited.
A.5 PARAMETER EFFICIENCY
Figure 6 plots the perplexity of language models GoodNews and VisualNews function of the number of parameters, further validating that ENGIN models get comparable or better results to alternative methods using far fewer parameters.
A.6 ARTICLE QUALITY ANNOTATION TEMPLATES
Following (Tan et al. 2020), annotators are asked to indicate a reason for whether the articles are numan-writeen machine maniillaaa,p We provide a view of the AMT worker interface in Figure
14
A.7 LANGUAGE FEATURES FOR ARTICLE-TO-IMANN RETRIEVAL
To validate that the language features of our model can also be applied to downstream tasks, we provide additional experiments on article-to-mmage retrieval in Table g We train linear probe between features computing using ENGIN-base and Finetuned GPT2 from Table of our paper and image features. Specifically, image features are extracted by CLIP and language features are extracted from the last hidden layer language models. Then, given an article, the goal is to retrieve the ground truth image from set of 1K images. We that ENGIN outperforms both the CLIP and MIL-NCE baselines used in Tan et al., 2022, as well as our own linear probe over finetuned GPT2. This helps demonstrate the ability of approach generalize other downstream tasks.
13

---

Under review as a conference paper at ICLR 2023
Table 7: Ablation study on the number of named entities detected by CLIP (GoodNews).
top-k named entities
5
10
15
20
PPL +
15.5
14.8
14.5
14.6
27.5
23.0
27.5
GPT]
25.0
GPT]
22.5
ENGIN
perplexity
20.0
perplexity
20.0
27.5
17.5
15.0
12.5
12.5
10.0
x10"
X1UP
X1.0
x18P
#parameters
#parameters
Figure 6: Comparison of the perplexity of different language models on GoodNews (left) and Visu- alNews (right) as a function learned parameters.
Article-to-image retrieval GoodNews. We follow the experiment settings in NewsSto- Table 8:
ries (Tn et al. 2022) for evaluation.
Method
R@1
R@5
R@ 10
Pretrained CLIP (Radford et al. 2021)
18.43
36.59
46.92
Single Image (Radford et al. 202T)
17.14
33.77
43.56
MIL-NCE (Miech et al. 2020)
15.50
28.96
37.60
Finetuned GPT2
16.08
31.99
41.45
ENGIN
18.24
35.62
45.36
B LIMITATIONS AND FUTURE WORK
We discuss potential improvements to work in this section. First, though our method ef- fectively predict the correct entity names in articles. their corresponding entity categories might be mistakenly predicted. For example, in Figure [ the brand name "Alki" is recognized by ENGIN. Therefore, more accurate entity-aware mechanism could be developed in future work. Second, the image information can further explored. In this paper, we mainly investigate the captions and named entities of news images. However, other information such the locations of images within articles may also prove useful for article generation. addition, our current methods detect named entities from images considering each entity independently using text-image match- ing framework. However, since the relationships between entities also affect the probability that entities appear in images, the incorporation of entity relationships can also be considered further improve the entity detection module.
city name
14

---

Under review as a conference paper at ICLR 2023
Tite: Daniel Agger leaves Liverpool to return to former club Brondby Domain: www.thepaadian.com 38-30-2014
Daniel Agger rejoined Brandby for an undisclosed fee.
merman-written News Article
ENGIN exnnrrrgee News Article
Liverpool have announced the sale of the defender Daniel Agger to the Danish club Brondby for an undisclosed fee. The 29-year-old Denmark international joined the Anfield from Brondby 2006 went make than 200 appearances for the club but had fallen down the pecking order under Brendan Rodgers.
The Liverpool defender Daniel Agger has left Anfield his previous club Brondby. The Danish international midfielder was reported to have agreed personal terms with club. The 29 year-old left Liverpoul the Premier League and has featured for the national team in Denmark's recent seasons.
following their relegation from
"Liverpool have been such big purt of my life and my family's lives for long, that leaving is extremely difficult," Agger said. "The opportunity arose for me to return home to Brondby and stage of my career wouldn't leave here go anywhere else and that has been proven by my actions in recent seasons I have turned down many offers to more to other Premier League and European clubs, would like
Broudby said be cash plus deal. We're very pleased Daniel Agger has chosen to continuc his development at Brondby, read statement on the club website. He's a young man who shown great potential look forward working with him again. Brondby, who Danish Superliga, have squad of more than 200 players. The Liverpool manager Brendan Rodgers said As foothaller you dream of playing in the Premier League Daniel Agger has chesen come back to Brondby. He's good defender and I'm sure be will work very hard to prove to the club that he good player. We like think him as very important player for his nationality. You might think he's bit of character but you 'll see he's strong character as any of the other players here. Agger said : "I am extremely proud play Brondby. of the happlest days of my and very memorable time, and the supporters have been fantastic. My future lies with the club, and hopefully, it will be here for a long time.
The
the right decision for
Premier League, and the
incredible backing they have given Liverpool supporters for the warmth and generosity of spirit displayee displayed has been our home and somewhere we hav been very happy and settled. However, more recently have started feel effects of playing regularly in league that is physically demanding as the Premier Leugue is, and I would not want to stay not think that was able to meet challenge weak in, week out.''
very pleased thac
time here and the family. "This
has great character.
always consider it far, It has been great
want thank the Liverpool fans support. Without them wouldn't have been able carry fighting for this shirt, thank
Figure 7: Example of human-written and ENGIN-generated news articles from Guardian.
15

---

Under review as a conference paper at ICLR 2023
Title: State Britain's wildlife "increasingly fragile'
Domain: www. com.com Date: 12-08-2015
Dark green fétullaary butterfly
Human Human-wzitten written News Article
ENGIN generated Artile
Article
Britain's wildlife in incressingly fragile state, with animals carrying out vital jobs for farmers being lost more rapidly than others, say sclentians, Insects that pollirate crops fight pesta risk of disappuaring putting food production in juqurryy according the the research brings together millions of wildlife records spanning 40 years. It suggetss areas, they ard conservation efforts should focus on the Univenitty of Reading, who led the research, said it was the biggest and most comprehensiive report ever assembled world. "y standardising records from army of amatuur biologists across the country, we have amassed an impressive array of data, giving our most complete picture of the state of Britain's wildlife," he told BBC News. 'The picture that emerges is an incressingly fragile syseem, particularly in species that do vital jobs for bumans. "Unless efforts are made to reverse some of there declines, face funure where will be less confident that effectively grow our food. Benefits from nsture.
The of Britain's wildlife has hecome increasingly fragile due loss of hablut and changes in the climate a study las suggested. Mass bird and butterfly losses have incressed along with climste change, scientists found, which means many specios are not as well equipped to cupe, And while changes food distribution, such floods, may cause short- term problems, they can make long term problems worse. Drought and flood events could put millions birds and insects risk, concluded. Monitoring of more than 4,000 species of birds, butterllies and moths across England, Wales that the UK's biodiversity was still very good- only amaller than it was 1975, But biological communitics in which they live were now less diverse than when these earlier surveys took place, which meant their ability to recover was less certain, they added. "We have lost several spocies of already codangered species." explained lead researcher Dr Tom Oliver, a naturalist at the University of Reading. "And we are maving into many these habitats that are becoming drter and thus less suitable for them. "The results of the study were" far more alarming" "than expected. "The situation looks prenty dire, "said Dr James Bullock from the Centre for Evology and Hydrology at Onfordshire, who was not invelved with the research. He told BBC News Britain's wildlife was already extremely stressed. Our natural environment is suffering from climate change, which melting ice sheets expanding the occans, human generated pollution, so we are going to have much mare polluted environment the future. And the species we are losing are particulatly important bocause when you lose species, you do numbers, they cesse to exist numbers. "The loss a single species Britain alone has the potential to affect millions other organisms. "The impact of the findings likely to be felt most in many communities across the British Isles. "We have lost Oliver added. It's going to have big impact on the scology of Britain." He explained that lightly : We are more concerned about short - term impacta, "The rewarchers believe that much work moeds be done understand how, in time, this situation might change. Research in Nature Communications suggets that climate change means sharp decline in the number of butterfly species spotted in northern England down by a third berween 1970 and 2009css such as the large blue have struggled for many generations as the cold weather has made it too cold to overwinter in FolFollow Helen Twitter.
scientists
any counery in
Scotland for 40 years concluded
The resturchers looked at records of the changing fortunes of more 4,000 types plant and animal living in England, Wales Sootland between 1970 and Species such as bess, dragonfies, grasshoppers ladybirds, plants moses were put into groups based on the role they play in Groure providing pollination while those
and pest control benefits had undergone involved in functions such decay mopping up carbon emissions were more Plants and animals regarded as cultural importance to humans, such birds, butterflies and hrdgehogs, also fazed badly. Prof James Ballock, of the Centre for Ecology and Hydrology in Oxfordabire, conservation efforts should focus on the areas where there was strong evidence of declining fortunes, "Consevvaion actions, such wildiife friendly farming. can avoid the less of bindiversity and the rocultnn crosion of the pollination, pest control and other benefits we derive from nature," he sai Dr Oliver said there were number of measures that individuals could take help improve the future for wildlifc in the British ThlysThey include: Making gardens wildlif: frieadly Purchasing way to wildlife Helping to collect data wildlife. The snady is publishad in the journal Nature Communications.
co researcherr on the study; said
namber of species, Dr
of long- term damage
unlikely to be taken
been
grown sustainable
Figure 8: Example of human-written and ENGIN- news articles from BBC.
16

---

Under review as a conference paper at ICLR 2023
Titl: Robberies plague Tenleytown over the past few weeks
Domain: www.wsshingtooppes.om Date; 05-23-2016
Video shows suspects robbing attacking Georgetown Day student
News Article
ENGiN-generated News Article
student at Georgetown Day School robbed month in the same of the District where student another school was stabbed during robbery attempt, and near where two other robberies also occurred monthTTe student Georgetown Day robbed about 1:30 p.m. April in the 4200 block of Davenport Street near the private school. Three robbers approached, and began beating the victim, police said, although they did not identify him studencThhy went through his pants pocket and took his glasses, wallet and phone before fleeing foot, police A spokeswoman for Georgetown Day issued statement saying that a student there was robbed along 42ed Street Tenleytown April 27, and the school provided immeddate medical and security support.The statement said the school has held an assembly devoted to neighborhood safety training and it has adjusted the patrols school security That robbery occurred two days after April 25 incident in which authorities said student at Woodrow Wilson High School was stabbed in the leg during an attempted robbery near the school.In the Wilson incident, the robber described
group that robbed Georgetown Day School of valuables including cash last month and purs while it was locked up dormitory has also robbed spate of Georgetown Day students, police said. The initial theffs occurred April 27 and April Two men entered the school, stealing the wallets and purses of two female students, policc said. A total of four men were released from custody. The school said the toral was 170.The robbery accurred about 1:30 p.m. April near the at 4200 Davenport Street. Theee male suspects fled the school on foot, and a local resident engaged in an effort to help the robbers, police said. It took about 15 minutes before the able locate them. The three defendants his 20s, one in his late 20s and one in his late reens fied along Davenport Street NW and into residential neighhorhnod, police said. That residenc robbers and assisted in helping apprehend them. The robbery at Georgetown Day came two days after robbery at Woodrow Wilson High School in Tenkytown. That April 25 robbery took place 42od Street and Davoporr Street NW, major thoroughfare in Tenleytown, According police, two men in their 20s robbed a woman in her 30s. The robbers fied with purse containing cash that was a pink leather wallet. On Monday, the Wilson rabbery went unsolved for nearly week. Staff at the school said no incidents bad been reported. "We do have a staff member watching the campus. You go through the normal procedures of settin up alarms, said Tankytowo police spokeswoman. We were n't able to ger that information from the yer. The person you are talking to is someone within security.
local
identified as friend the
bis 30s. No description provided of the robbers Davenport Street NW, Police released video Monday of three people described as "persons of interest." 'The video showed the victim and the robbers approaching each other on sidewalk, with school buses lined up student is knocked down and one or two of the robbers appear to go through his pockets. But video does not appear to make possible detailed description of the robbers.he stabbing comes nearly week after two robberies, one of them involving student vierim, were reported on streets near Wilson and near Georgetown Day. Bath accurred within
curb. Suddenly the
15 minutes of other April each instance, three people, of whom who armed with knife, appreached female vietims and demanded their cellphones and debit cards. A juvenile male has been arrested in those cuses, police said,
to keep eye on him. "Correction This story has been updated reflect that the Davenport Street robbery occurred April 19, not April 27.
Figure 9: Example of Human-writter and ENGIN- -enne-ggeeertted news articles from Washington Post.
17

---

Under review as a conference paper at ICLR 2023
Title: Courts again rule against streaming upstart Aereo Domain: www. www.oooodd..comm
Date: 08-21-2014
this photo illustration Acreacom Web service that provides television shows online is shown on an iPhone 4S on April 22 2014 in New York City
Human-written News
ENGIN generated News Article
A federal court will nor hear Aerea's argument that the streaming media company should be allowed to operate similarly cable compuny,The Sccond Circuit Court Appeals ruled Thursday that Acreo will have take its argument to the district court, accrrding to court documents reported by Washingron Posr.This decision continues broadcasters' winning streak against the upstart service. Launched two years ago, Acreo delivered over-the-uir channels via online stresming to compurers
The high court delivered another blow to Aeren in pair of unanimous court opinions Thursday declaring the popular streaming service illegally alters the contents of customers cable subscriptions and guilty of violating copyright law. The Second Circuit Court of Appeals, which first rukal against Acrvo two years agu, urged the jusrices to reject claim that simply a device consumers live TV without paying for it. Acreo lets consumers digital antenna to reaive live broudcasts online and streams those broadeasts as they being transmitted over Internet, just like cable TVTToooggh those transmissions the consumer also becomes subscriber to the over - the air service. Bur even if they keep Aereo's service on, only portion of the amount paid cable service remains. For example, for 16.95 per nearly all the TV shows they request, regardless of what shows they watch. For about $8 each the consumer loses the signals they might already
tablets about 58 Tht Supreme Court ruled two months ago that Aereo was violating copyright law by not paying the networks for transmitting their signuls After that, the strcuming media company voluntarily shut down and told the U.S. Districr Courr that it wanted to be treated as the equivalent to cable TV system and apply for a compulsory license to stream the Copyright Office's general counsel said that the office did not think that Acrco would qualify, but fell short of denying Acreo's filing because issue remained Follow Mike Snider Twitter: @iileeSnider
month and had brought the service
Acrco customers rective
broadeasts Subsqquerly
over copyright law by redistributing the service company violating already pay for without paying for the service they are recuiving from cable operators, "The Court finds that Acreo's basiness model violates section the Copyright Act," 11 judges Washington Post, which first reported on the decision, called it blow to digital media and to consumers. "'This unanimous decision means lot," Acrco chief exccutive Mike Snider said in statement. Consumers no longer have pay receive the tradirional way they watch TV. They now have the option to pay only for a service that channels, all for "Aereo has petitioned court which is
Interner.
in the
in the majority opinion. The
change the district court's ruling the full appeals likely to do so. The Supreme Court upheld the same ruling from two months ago. Aereo said will appeal the Distritt of Columbia. Experts tell the Post that its previous shows that the Copyright Office's determination to give consumers access to streaming services without paying retailed companics likely reversed by the appeals court. Follow afjwartz Twitter
District
Figure 10: Example of human-written and ENGIN -eeer-geee news articles from USA Today.
18

---

Under review as a conference paper at ICLR 2023
Title: James F. Holland, Trailblazing Cancer Researcher, Dies at 92 Domain: www.nytimee.com Date: 03 27-2772011
wife, Dr. Jimmic Holland, who pioneered the field of psycho oncology. While she treated the emotional distress of cancer patients, be focused on discovering universal remedics. She died in Devember.
Dr. Holland
James Holland undated photograph. In the 1950s, he and his colleagress considered cowboys by some colleagues, he said.
Hunta--witten News Articks
ENGIN-gurcttted News Artick
James F. Helland, an innovative oncologist whose scientific insights made him, as once said, Google for cancer research, died on Thursday his home in Scarsdale, N.Y. He
James Holland, a founding father of chemotherapy who for podiatric
helped pioncer lifesaving drug leukemia patients, died on Thursday at his home in Scarsdale, N.Y. He was 92.
The cause was widespread mycloid leukemia subtype, a cancer the that
blood colls in bone parients,
fewer than three
The cause respiratory fallure, his son Steven said.
Dr. Holland and his medical collaboratoes the were scorned by some colleagues as research "oowboys," as he put it, because they audaciously tackled what was considered an incurable discase tried simultancously, instead of sequentially.
Holland one of the encologists propese, the 1950s, the chemical treatment known stereosporin hence the name. He was also among the first to suggest though not actually carry out became known as precancer therapy turning nonmedical patients to their regular doctors, while suppkmenting their therapics with medications to kill cancer cells, The mere suggestion of precancer therapy. seemed some,
with chemicals
their research began, fewer than who were found to have acute lymphoblastic leukernia lived long as year. Today, as many as nine in 10 survive the discase, which the bone marrow blood cells.
three children
rendered toxic.
At the time, though, Mr. Holland thought should not, He and bis colleagues been working for long year on drugs to prevent acute lymphoblastic leukemia which cells on the surface of the blood can proliferate out of control being overexposed by chemotherapy. Doctors usually only after the patient's is health deteriorated. Today, many hospitals will not perform precancer therapy, either.
too many immature white
patients with advanced discase,
Another clinical trial Dr. directed 1970s, contain mycloid leukemia, somewhat similar malfunction in blood cell production, resulted in what became the standard treatment.
But the 1970s, he and his colleagues cowbays by their colleagues. Known for treating many nine of 10 cancer cases, they had boen derided even by their own specialty, nuclear medicine, for treating these patients when the science does not justify it, Dr. Steven Holland, his brother and his only sibling, szid.
regarded research
Figure 11: Example of human-written and ENGINN-eneraatd news articles from New York Times.
19

---

Under review as a conference paper at ICLR 2023
Robert Boulter
Human-writen Wikipedia
ENGIN ENGIN-generated Wikipedia
Robert Boulter is an English film, television and theatre actor. He had a guest @-@ starring role on the television series The Bill 2000. was followed by starring role in the pla Herons written by Simon Stophens, which was performed in 2001 the Royal Court Theatre. He had guest role in the television series Judge John Deed 2002. In 2004 Boulter landed role as V Craig" V in the spisode y 'Tedyyss s Story 1" of the television series The Long Hem
Robert Boulter is an English film, television and theatre actor. He had a guest a-@ starring role on the television was followed by starring role in the pla Herons production from 2004 at the Drum Theatre, adapaed from John Tiffayy and Ben Whishaw play Tyger: family lost in time the Menier Chocolate Factory in 2005, won many critical accolades. In 2006, Boulter starred Constellations,
The 2000. This
starred alongeido actors
production directed by Harry Kent at the Bush Theate, where the actors included Fraser Dominic Hall. dementia and was typical Boulter- light fare, but also showcased
with Shane Zaza
Mark Strong and Derek Jacobi, He was cast in the 2005 theatre productions of Philip Ridky play Mercury performed at the Drum Thearre in Plymouth and the Menier Chocolate Factory in London, He was directed by John Tiffany and starred alongside Ben Whishaw, Shane Zaza, Harry Kent, Fraser Ayres, Sophie Stanton and Dominic Hall. Vnin In 2006, Boulter starred alongside Whishaw in the play Citizenship written by Mark Ravenhill, He appeared on 2006 episode of the television series, Doctors, followed by directed by Josie Rourke. How to Curse was performed at Bash Theatre in the Landon Borough of Hammersmith and Fulham. Boulter starred in two films in 2008, Daylight Robbery by filmmaker Paris cones, and Donkey Punch directed by Olly Blackbarn, In May 2008, Boulter made a guest appearance of the television series Waking the Dead, followed by an appearance on the television series Survivors in November 2008. He a recurring role in ten episodes of the relevision series Camulty in 2010, as \ Kieron Hetther V. Boulter starred in the 2011 film Mercenaries directed by Paris Leonti. 'nnn In 2000 Boulter had guest e-@ starring role on the television series The Bill; he portrayed ! Scort qpisode
which
Ayres, Sophie Stanton
play was about
Boulter's fine facial features his exceptional ability to stage deeply moring and emotive role. After starring another Whishaw production, Tyrant, in 2006, Boalter appeared agsin in the same at Hammersmith's Bush Theatre, directed by Mark Ravenhill, Zaza became his colleague in the 2008 production of The Long Firm, which also starred Josie Rourke as Prime Minister Boulter as scorpion that resembles the prime minister, and their take on the writing of The Sting brought Boulter further recognition. After another of Boulcer's roles, 2002's Devil Doll, this time he was also plying Toro the dog. and played the tirle role in an adsptation of The Ducllssts at the Theatre Royal, Plymouth. Boulter continued reseive plaudits for his performances in the Devil Doll and Curse, two productions at Landon's Bush Theatre artistic directos, Jonarhan Kent, starring children's entertainers such as Simon Stephens and Simon Callow, London's Bush Theatre then produced Boulter's production of the play for international theatre festival, Romco & Juller, in May 2008. It ran at the London Lesbian, Gay Bisexual festival and featured the titles Donkey Punch and Think Heaven For Me. The April Production at the ExCel. Centre in November 2008 was double bill written Boulter and starring actor Paris Leonti theatre actor, Toby Steele. The production of Boulter's 's play Waking the Dead, to be produced by Paris Leonti at the Royal Court Theatre in 2010, is co production with Kamila Shamsie. In 2011 Boulter's play Sentences, with the writer Kieron Flercher, opened stage at the Drum Theatre in Loodon with two plays from Boulter's personal collection. His production of Independent on Sugar Lauf, which was top ten hit for the Menier Chocolate Factory Dominic Hall had brief role in Boulter play, The Long Firm, in 2000, directed by Fraser Ayres The Lyric. Southampton 2001, about mental health and dementis.
2007 theatre production How Curse
two e e part spisode are
directed by the theatre's
Safe Hands Boulter starred \" in the play Herons written by Simon Stephens, which was performed in at the Royal Court Theatre. review of Boulter's performance in The Independent on Sunday described him 1*' borriblly menacing V in the role, and he received critical reviews in The Herald, and Evening Standard. He appeared the television scries Judge John Deed in 2002 as \ Addem Armitage \" the eplsode \* Political Expediency V, and had role Steele 12 on The Bill, Vain He had recurring role in 2003 on two episodes of The Bill, character V Connor Price V". 2004 Boulter landed role
Parry
Scott
the main lead and the undervalued resident
character Y Tabyy
Craig V in the qpisude V Tiddy's Story V of the television series The Long Firm he starred alongside actors Mark Scrong and Derek Jacobi. Boulter starred as \ Darren V", in the 2005 theatre productions of the Philip Ridley play Mercury Fur.
Example of human-written and ENGIN Wikipedia articles (from WikiText dataset (Merity et al., (2066)). We generate articles conditioned on both the title Robert Boulter, and the first 50 tokens highlighted by light yellow. The articles are cut to fit the figure size.
Figure 12:
20

---

Under review as a conference paper at ICLR 2023
1933 Treasure Coast hurricane
Homan-written Wikipediz
ENGIN-generated Wikipedia
The 1933 Treasure Coast hurricane was the sccond @*5 most intense tropical gyclone to strike the United States during the active 1933 Atlantic hurricanc
The 1933 Treasure Coast hurricane was the secand 69-6 most intense tropical cyclone to strike the United State during the active 1933 Atlantic burricanc scason. The deventh tropical storm, fifth hurricane. A Caregory 4 eyclone with winds of 100 mph generated winds of 161 km per hour and 6 people, flouded an arta of 868 sq km, and caused $ 3 million in damage. Although missed the Florida coast, the system swept across the rest of the United States less than 24 hours. The heat from the system crused widespread fires and continued to bring heavy rain and wind throughout the south and central regions, In south Florida alone. the fires killed two men and destroyed 98 homes in the Lester Antilles and southwest Horida, The I lurks and Caicos islands
The clevench tropical storm, fitth hurricane, and the third major hurricane of the sesson, it formed east G--r northeast of the Leeward Islands August The tropical moved rapidly west @-0 northwestward, stcadily incemsifying to hurricane. acquired peak winds and passed over portions of the Bahamas on September Including Eleutbera and Harbour Island, causing severe damage to crops, buildings, and infrustructure. Winds over 100 mph ( 1661 km h affected many islands in its path, especially those that encountered Its center, and many wharees and made landfäll at Jupiter, Florida, early on September with winds mph 201 km/ ). hurricane moved across state, passing near Tampu before moving into Goorgia and dissipating. In Foridd,, the strong winds the cyclone blew buildings off their prostrated citrus groves. The Treasure Coast regian received the mose extensive destruction, and Stuart, Jupiter, and Fort Pierce were heavily damaged. Inland, the cyclone weakened rapidly but produced prodigious amounts of rain, causing dam to collapse near Tampa. The storm caused $ million in damage ( 1933 USD) after damaging or destroying 6 exB 848 homes. Antn Unusually, the storm hit Horrida less than 24 hours before another major hurricane bearing 125 @ mile 2 per @ hour ( 201 km/ h winds struck South Texas; ; never have two major cyclones hit the United States such clos succesion.
temperature of 49 degrees. The storm killed
miles per bour 225 /
rained. 'oln Subsequently. weakersed
particularly
badly hic because of storm surge that produced high seas and flooding, especially along the coast Rico. The Turks and Caicos islands experienced over 125 mpp winds, winds of 125 @-ø mile per hour, and only I em to 2 inches of rain in the 30 minure interval starting around 1400 UTC. The Turks and Caicos islands experienced an over 140 mph wind shear at 5875 kp m. south Georgia, home the Simpson and Penally countics, worst of the damage was felt in the Savannah area, where 800 + homes were damaged or destroyed
Tampa, Georgla, Puerto
foundations, and numerous
he demolished. The most intense tropical storm to strike the United States during the 1933 Atlantic hurricane sesson accurred when Tropical Depression 6 formed over the Bahamas during the winter of 1913. The storm resulted in catastrophic loss of life and destruction. As of 24 km west of the Bahamas, the lowest pressure was maximum sustained winds of 200 km/h, with gusts to over 200 km/h. south Florida, the intense storm produced winds south western sourhwest Florids and a south- western coast generally between the vdges of the Low Range and the extreme southeast of the In western Cuba, the storms produced winds up what the Weather Buresu descrihed as 1 abbuu 65 mph ! in some spots, and all respecivvly. were downgraded
over 1000
The origins of the hurricane were from tropical wave that possibly spawned tropical depression on Auguse 27, although there minimal data over the noxt few days as it tracked to the west g-$ northwest. On August 31, marby ship reported gale force winds, which indicated that cropical stoem had developed to the Based on continuity, It is estimated the storm attained hurricane status that day. Moving quickly to the west 8-a northwest, the storm passed north of the Lesser Antilles and Puerto Rico. Early on September 2, ship called the Gulfwing reported barometric presoure of 978 mbar ( 28 s. 88 inHg }. which confirmed that the storm attained hurricane status After passing north of the Turks and Caicos slands, the hurricane struck Eleuthera and Harbour Island the Bohamas on September 3, the latter at 1100 UTC, A station on the latter island reported pressure of e.e inHig 9945 mb) during the 30 minure passage of the
and the
had
mph over
northease the Lesser Antilles
tropical storms Category 4, 5 to tropical storm or depression. Texas though was hit by Hurricane Norhert in early Octoher 1913. At time, Weather Bareau reported that the storm produced winds of 800 900 km/h, and storm tide of 848 m. The storm passed over northeast Toxas between midnight and dawn, and was the first major tropical cyclons impact the United States during the winter of 1913. Norbert resulted in 848 lives being lost and 98 in damages, amongst other catastrophic damage, well as numerous faraliries.
Figure 13: Example of human-written and ENGIN-generated Wikipedia articles about 1993 Trea- sure Coast hurricane.
21

---

Under review as a conference paper at ICLR 2023
Requester: Mtaen Group
$1
Dumiio:: Minitee
Cualiicatora Require:: Appooval flw (uttrr ietauuneerr 14'h 56, Number Hifl Aggrooee ter
Instructions: News Articles Evaluation
Hello, are trying to determine how susceptible people are to maachnn--generatee fake news.
In this study, you are asked to discriminata between human and machine-gonarated news articles.
You will be assignd 5 article samples. Each article sample contains the headline, article body and image/caption pairs. The imagee and captions are located after the articie body where each caption situated below coresponding image. You should spend more than to 1.5 minutes on each articie.
Please be sure click the submit buttom for each article.
Please nota that we ra-rendieaad all the article samplea. As rasut, the primitive display is NOT indicator of whether the artiila human machina peneraled.
Instruatios
Shartouts
Is tho news ontole human or mochine generated? Roaso choose option that yourhhin the medt culsable for givon rewe artole.
Select option
Human-Generrate
Duterte, Philippine President, Boasts He Killed Someone as a Teenager
Machine- Generated because one or more imaga ls not very relevant to the article body
Machine-Gennerttee
one or captions very relevent the article body
HANILA The desth Chief Justice Ser Nan III, front gate when was 13, was the most recent what mounting taly police killings the Philopincs. nos.
Generatae because
somehing that had harely up beforw
tranogrcsatte,
the caption
Since at leest the stort of cominittration June. Duterte has been everyene iengnitying spconents of colicies.
Machine- Machine-Generatod Generated because the image/caption pairs are not rolavent the article body
adinion at joumnllstss
disbilicess
joumalltss and judpes whom
Machine Generated because something about the article appeers off
browicaea
and print futertt projected gang
Charles Leffo the grounds o provircal court 2016 The nextt year, Mr. Duter beat man to death with woecen rod crowded Roniia public mai fet during on ant operotias.
drig
Machine Generated
the article
Submit
Figure 14: The interface used by AMT workers in our article quality annotation experiment.
22


Reviews:
[{"review_id": "iVzUysws1J", "reviewer": "Reviewer_5iPM", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This work is in the domain of text generation for long-form articles. The key contribution of this work is to provide additional metadata in the form of named entities to the generative model. Such entities are either provided in advance (oracle) or can also be derived using entity retrieval based on image associated with the article. The empirical results show the benefits of the approach.", "summary_of_the_review": "Overall, while the method is quite simple and elegant and will propel more future work, this work has limited novelty in its current form.", "strength_and_weaknesses": "**Strengths**\n\n-- The proposed method is quite elegant and simple and is easy to extend by following works.\n\n-- The entity retrieval method (using entity retrieval from a pretrained image-text model) makes the approach more generalizable and not dependent on a provided list of entities.\n\n-- The empirical results show effectiveness of the model as it outperforms all the baselines.\n\n-- The paper is clearly written and easy to follow\n\n**Weaknesses**\n\n-- While the application of named entities as additional metadata is novel to this domain, the same has been studied extensively in NLP literature. It is a good observation that existing methods do not explicitly model named entities, but the technical novelty of the proposed solution is limited.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "i6F9W1onYIj", "reviewer": "Reviewer_ewLW", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The authors propose a method for generating news articles conditioned on entities to appear in an article. The authors claim that existing methods for generating news articles only model text and ignore visual content and secondly that these methods do not explicitly account for named entities to be mentioned in generated articles. The authors argue that this makes the machine generated articles less natural and prone to failing to mention entities that should be mentioned and which would make the article more realistic.\nThe authors propose a method, called Engin, for generating news articles conditioned on entity information. Engin takes as input metadata and a list of entities to appear in the article and is trained to generate news articles.\nEngin also desires to take \"image information\" into account in generating the article. However, Engin is *not* conditioned on visual features from the image. Instead, the model uses the *ground truth\" captions as the representation of the image. \nFor providing named entities to the model, the authors consider two techniques. One is providing the ground truth entities to appear (oracle) and another is using a CLIP retrieval technique to discover these entities automatically.\n\nAuthors experimentally evaluate their method at generating news articles conditioned on entities and \"image information\". They compare their method against a number of recent techniques (GPT2, Grover, etc.) and show improved perplexity on their generated articles on two datasets (VisualNews and GoodNews). \nThe authors also provide a number of supplemental experiments. They performed a user study to assess how well humans could detect the generated articles and found that their articles were harder to detect than the baseline articles (hence higher quality). They also performed other ablations, testing the impact of different metadata fields on generation, the ability for machine discriminators to detect their articles, and for zero-shot article generation.\nThe authors also provide qualitative examples of the articles generated by their method. ", "summary_of_the_review": "The authors present a method for generating news articles that mention entities. As explained, the paper is well put together and the appropriate experiments are performed. However, the method is a straightforward extension of existing work and offers little technical novelty - it is highly similar to GROVER just adding an additional conditioning metadata field essentially.\nObvious avenues for improvement, such as improving CLIP-based NER were not performed.\nSimilarly, relevant work (e.g. InfoSurgeon) was not compared against or cited. \nVisual features are not used by the model.\n\nAt this time, I would argue the paper does not offer a sufficient contribution for publication as a conference paper.", "strength_and_weaknesses": "[Strengths]\nThe idea of incorporating entity information into the generator is an important one. Existing methods for generating news articles, like GROVER, usually take in a title and generate the article based on that (and maybe the source it is supposed to be from). The article, however, fails to mention critical entities that would be mentioned in an article about that subject, making the generated article less realistic. By incorporating entity information into the generator, the model is able to generate more realistic articles.\n\nSimilarly, the authors attempt to integrate \"image information\". The authors argue that news image captions are different from typical image captioning in that the captions are much less visually grounded. The authors argue that directly integrating visual features is not idea and instead also condition their article generator on the captions. In this way, the generated article is much more likely to be related to the visual content paired with it.\n\nExperimentally, the authors demonstrate that their method is better able to fool humans and the machine discriminator, suggesting that the articles it generates are more naturally. The authors also demonstrate impressive gains in perplexity in their experiments and the qualitative results bear out the quality of the generation.\n\n[Weaknesses]\nThere are a number of weaknesses of the proposed approach.\n\nThe model is in some sense an extension to methods like GROVER. GROVER is quite similar to the proposed method in that GROVER trains a news article generator conditioned on metadata. In this paper, the authors essentially just re-train GROVER, but now add two additional metadata fields to the conditioning - one is the list of entities to be mentioned and the other is the *ground truth* image caption. From a technical perspectivie, there is little novelty to this approach. All the authors are doing is retraining GROVER using additional conditioning. Their is no novelty in the learning scheme, despite their being room for it.\n\nFor example, in the learning process used by ENGIN, no loss constraints enforce that ENGIN actually mentions all the named entities in the generated article. People have typically tackled this problem by using pointer-networks or similar strategies to perform a copying of named entities from the input into the output. The authors do not do this and it is unclear why not. Moreover, there is a lack of explicit loss to enforce named entities to appear and to penalize the model for their not appearing, save the standard text generation loss. Why don't authors enforce the model to actually use these entities and not ignore them?\n\nMost importantly from my perspective is that critical related work is not cited. The authors are not the first to condition news article generation on entities or visual information. InfoSurgeon (Fung et al, ACL 2021, oral) presents a method for detecting machine generated news articles. As part of the paper, the authors train a news article generation method. The generator takes in a knowledge graph consisting of entities, relations between entities, events, as well as purely visual entities detected using image features as well as image events detected using imSitu. In sum, the input to the model and its reliance on visual features is much richer than ENGIN. \n\nAs it stands, ENGIN relies on ground truth image captions to generate the article and incorporate visual information. However, as it stands, the model is never able to mention facts that are purely observable visually - for example, maybe an image shows three people cheering but the caption and entity list don't mention this. ENGIN is unable to make any such claim in the article text because it doesn't ingest any visual features. I also am somewhat concerned at the way the paper claims to take in \"Image Information\" but doesn't actually make use of any purely visual features (objects, image events, etc.) and instead relies on captions. \n\nThe CLIP-based NER mechanism is very simple. The authors just use CLIP to take an image's embedding and retrieve a set of candidate entities. Given that entities are a key component of your model, I would have expected a more rigorous or trained model to improve the entity selection mechanism.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "eKI94S7rovg", "reviewer": "Reviewer_LeSY", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "Summary:\n\nThis paper proposed to care about article generation with image information where the image info is used in an entity-aware way instead of directly using the visual features. That is, the embedded image in the associated article is firstly transformed into the textual space in the form of textual captions and textual named entities. After the image transformation, the textual contents extracted from the image act like normal body texts of an article, and are then integrated into standard controllable text generation models. To differentiate the entity texts from other texts, the entity categories are appended to the entity texts so the model can learn to predict the labels of named entities as well. Extensive experiments on three datasets are conducted with different model sizes.\n", "summary_of_the_review": "Comments:\n\nC1: The ENGIE model keeps the top $k=10$ candidate entities according to the similarity scores computed by the CLIP, any parameter cures varying with this $k$ to show its impact? \n\nC2: Furthermore, it seems (not clearly said in the paper) that the similarity score is independently computed between each individual entity and the image, and does not consider the relationships among different entities. (This is motivated by the CRF decoding strategy in named entity recognition where the labels/categories of the entities are related in a sequence.) For example, the top $k$ entities, say E1,E2,…, have the maximum similarity scores independently, but can we find a semantic-dense (defined in a proper way) subgraph G’, say it contains E1’,E2’,…, which has larger sum similarity score SUM(G’{E1’,E2’,…}) than that of SUM(G{E1,E2,…}). Hope some discussions are given in this direction.\n", "strength_and_weaknesses": "Strong & Weak:\n\nS1: It is novel to exploit image info in the form of textual entities extracted from it. In this way, the info contained in image and article body is now in the same/homogeneous feature space to be input to the generation model. The side effect is to explicitly alleviate the heterogeneity across the original text space and visual space in the input layer instead of the hidden layers. This may make the learning of modality fusion easier. Hope the authors can discuss furthermore from this perspective.\n\nW1: One possible weakness in my own opinion: While the proposed ENGIE is a new method for article generation with image recommender, each of the individual piece of ENGIE (entities extraction by SpaCy Python library, ranking candidate entities by CLIP, standard controllable text generation model) is not new in itself and has been widely adopted in existing works as pointed out in the paper. This may be not a big issue in practice, since ENGIE demonstrated a strong empirical result on three datasets.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "OXaI4MDZBTg", "reviewer": "Reviewer_g2Tq", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a method for article generation given visual information. Visual features from the embedded images are used to extract name-entity based on the CLIP model. Language model conditioned on the extracted entities along with other meta information is utilized to generate the body of the text. Experiments on three datasets including two news datasets show the efficacy of the model compared to GPT baselines.\n", "summary_of_the_review": "I think the paper overclaims that it uses image information and the method is not novel enough for publication.", "strength_and_weaknesses": "Strength\n1. The writing is clear and the proposed method seems straightforward and easy to implement.\n\nWeaknesses\n1. I’m strongly skeptical about the prospect of using this technology to generate news articles, as the authors seem to aim at. Any words (especially nouns and verbs) hallucinated from the model instantly make the intended news article “fake news”. See the example in Figure 5. How much of the generated news article is true? Did Lizaso actually say that? This is also frightening as it sounds plausible (in a news article tone).\n\n2. The method lacks novelty. The image information essentially is only used to extract name entities (the caption is provided).\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "ElOfiiRKuX", "reviewer": "Reviewer_2PvE", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper designs an entity-aware and image-information-integrated article generation method. The entities in the article are recognized with the help of image information and labeled with entity types. The paper claims that by annotating these entities on a target sequence, it enables the model to perceive entity information. The proposed model is evaluated on two news datasets and zere-shot Wikitxt and achieves excellent performance compared to the baselines.", "summary_of_the_review": "Overall, I don't think the contribution of this paper is sufficient to be accepted as a conference paper.", "strength_and_weaknesses": "+ Strength\n  - The motivation of this paper is clear enough and it is interesting to introduce image and entity information in the article generation task. Image information augmented NLU and NLG has been explored in many works [1,2,3].\n  -  The model proposed in this paper is simple and effective and achieves good results compared to the baselines.\n+ Weaknesses\n  - Although the authors claim that image information is introduced, the model actually makes use of ground-truth captions and entities extracted from the article. The image information is only used for CLIP to retrieve the relevant entities. Thus, in my opinion, the utilization of images in this paper is too shallow and the ENGIN is more like a named entity-conditioned article generation work. Also, how would it work if use textual information (metadata) instead of image information to retrieve related entities from the candidate entities?\n  - There are some details of the method that I don't understand well enough. For example, \n     1. In inference, the generated text may contain annotations of entity types, right? then what do you do with these entity type annotations? \n     2. Is the utilization of named entities reflected in two aspects? The first is the generation conditioned on named entities. The second is the annotation of named entities and their categories in the target sequence during training. \n     3. When using image information for named entity retrieval, do you use entity names directly or construct specific prompts? Can you further analyze and evaluate the performance of named entity retrieval based on oracle NEs.\n  - I think the statements are inaccurate in many places, e.g.\n    1. 'our key contribution is a novel Entity-aware mechanism to help our model recognize and predict the entity names in articles'. The purpose of this paper is not to recognize entities. This statement would be confusing.\n    2. 'captions and named entities extracted from images'. I can't agree with the statement that the entities are extracted from the images, because the image information is only used to filter the entities recognized by Spacy.\n    3. 'existing methods model named entities uniformly with the other text, making the leverage of named entities less effective.' Can you explain the reasons for this? Or provide some experimental results to support it.\n\n\n[1] Visualize Before You Write: Imagination-Guided Open-Ended Text Generation\n\n[2] Imagination-Augmented Natural Language Understanding\n\n[3] Visually-Augmented Language Modeling", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "2mEM4iunhrp", "reviewer": "Reviewer_HgLz", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposed an entity-award article generation framework that incoperates image information. By using external tools such SpaCy and CLIP, this framework can extract named entity candidates to supervise text generation.", "summary_of_the_review": "Novel and interesting work, with good paper organization. But the structure of the framework is not fully explored. ", "strength_and_weaknesses": "This paper is well-written and easy to follow. The idea is simple but intuitive. Incorporating image information is very useful in article generation. The authors conducted extensive experiments and ablation tests to demonstrate the effectiveness of the proposed framework. \n\nWeaknesses: the proposed model only extracted the entity information from the image, while images contain various information out of entities. I would suggest to add a discussion section to outlook how to fully use the image information in this task. In addition, the way of entity-aware article generation is simple (this is not a weakness), but other structures to restrict/prioritize identified entities during the generation should also be tried.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject