SlowGuess/NAACL_2022_OCR huggingface dataset:
To further analyze the capability of our proposed model, we also apply REFLECT on a domain-specific corpus, WikiCatSum (Perez-Beltrachini et al., 2019) as shown in Table 6. WikiCatSum is a multi-document summarization dataset derived from WikiSum (Liu et al., 2018) and represents three distinct domains (Animal, Company, and Film). We compare our model with several baselines. TF-S2S is a Transformer sequence-to-sequence model of Liu et al. (2018) and CV-S2D+T is a variant of CV-S2S (Gehring et al., 2017) with a single sequence encoder and a structure decoder. Both CV-S2D+T (Perez-Beltrachini et al., 2019) and TWAG (Zhu et al., 2021) introduce the topic detection model to guide the generation. The decoder of CV-S2D+T is trained to predict the topics as an auxiliary task, while TWAG uses the topic information to group the input paragraphs and encodes them separately. Liu et al. (2021b) apply knowledge distillation to alleviate the problem of single reference in maximum likelihood training, while our model leverages reinforcement learning to the train-test mismatch issue. The results of BARTlarge models are fine-tuned on WikiCatSum  $^{4}$ . REFLECT outperforms all baselines in all three domains, especially in the ROUGE-1 score, showing that our generated summaries carry more information. The results between REFLECT (MLE) and REFLECT (CASC) also manifest the effectiveness of Credit-Aware Self-Critic learning for bridging the gap between training and testing.
<sup>3</sup>We can not tell the version of score metric ROUGE-L used in Multi-XScience, and the evaluation code is also unavailable. Therefore, we do not report the ROUGE-L score from Lu et al. (2020)
4The models of Liu et al. (2021b) and REFLECT are with pretraining. We exploit the pretrained model from https://huggingface.co/facebook/bart-large-cnn.
Animal
Model	R-1	R-2	R-L	R-LSum
TF-S2S	44.0	28.8	40.0	-
CV-S2D+T	42.7	27.9	37.9	-
TWAG	43.1	24.4	40.9	-
Liu et al. (2021b)	45.9	32.2	41.4	-
BART-large	46.3	29.2	39.6	44.1
REFLECT (MLE)	46.5	27.1	38.2	43.6
REFLECT (CASC)	48.6	30.2	41.5	46.0
Company
Model	R-1	R-2	R-L	R-LSum
TF-S2S	26.0	9.5	20.4	-
CV-S2D+T	27.5	10.6	21.4	-
TWAG	34.1	11.9	31.6	-
Liu et al. (2021b)	33.5	15.0	25.9	-
BART-large	36.8	15.1	25.6	33.6
REFLECT (MLE)	40.3	15.5	27.0	36.2
REFLECT (CASC)	40.8	15.8	27.5	36.6
Film
Model	R-1	R-2	R-L	R-LSum
TF-S2S	36.5	18.8	31.0	-
CV-S2D+T	38.0	21.2	32.3	-
TWAG	40.8	21.2	34.3	-
Liu et al. (2021b)	42.7	26.1	36.8	-
BART-large	44.3	25.5	35.9	41.7
REFLECT (MLE)	46.7	25.6	36.5	43.2
REFLECT (CASC)	47.6	26.8	37.9	44.1
Table 6: Performance of REFLECT with various baselines on WikiCatSum corpus. The results of Transformer sequence-to-sequence (TF-S2S) and CV-S2D+T are referenced from Perez-Beltrachini et al. (2019), and TWAG is from Zhu et al. (2021). REFLECT outperforms all baseline models in a large margin for all domains, especially for Company and Film in the ROUGE-1 score.
B Additional Ablations & Analyses
B.1 Effects of Hierarchical Architecture
In this subsection, we study the effect of hierarchical architecture in the extractor. As described in Section 4.1, the token- and sentence-level encoder jointly contain 12 layers in our settings. Thus, we experiment with different numbers of layers distributed for the token-level encoder. Also, we consider a case for no (0) layer, where we directly aggregate the word embeddings from the token-level encoder as sentence features. Table 7 demonstrates that hierarchical architecture improves the resulted abstraction performance through better sentence representations. The performance is enhanced with the increasing number of layers, while more than 3 layers only makes marginal improvements. Thus, we use 3 token-level layers in this paper.
Layer Number	R-1	R-2	R-L	R-LSum	Average
0	47.94	18.68	23.46	43.87	33.49
1	48.01	18.63	23.53	43.95	33.53
2	48.11	18.87	23.62	44.03	33.66
3	48.16	18.87	23.61	44.06	33.68
Table 7: Performance of different extractor configurations with MLE learning. Note that all configurations share the same amount of learnable parameters. ROUGE score is abbreviated as  $R$ .
B.2 The Choice of Summary Reference at Training Stage
In this subsection, we study the effect of the summary reference at training stage. As mention in the subsection 3.3, directly using human-written summaries as reference may cause the severe train-test mismatch. To verify the idea, we take the human-written summaries as the summary referencing for the extractor during training, and perform the standard test process, that is utilizing generated summaries as the references. The results shown in Table 8 verify our assumption that training with such explicit signals reduces the ability of the model to generalize. Therefore, in our experiments, we take the generation results of BART-large as the summary referencing of the extractor.
MLE	R-1	R-2	R-L	R-LSum	Average
Ground-truth	47.54	18.60	23.40	43.63	33.29
Generated	48.16	18.87	23.61	44.06	33.68
Table 8: Performance of the MLE training results when taking the ground-truth summary as the summary reference (SR) during training. ROUGE score is abbreviated as  $R$ .
B.3 Effect of Input Settings for Abstractor Fine-tuning
In REFLECT, we decouple the learning for the extractor and the abstractor. Therefore, we study the effect of the input settings for the fine-tuning of the abstractor. Table 9 shows the performance of BART under different model configurations with either using truncated articles or pseudo oracle sentences as inputs. We find that using pseudo oracles for BART-large fine-tuning can effectively improve the performance even when the testing input is truncated articles. In addition, the results of using pseudo oracles for both training and testing demonstrate the upper bound performance of the extractor
under MLE training, which suggests a promising development of our framework as the extractor can make more precise predictions.
BART	Train/Test	R-1	R-2	R-L	R-LSum
base	A/A	45.71	17.12	23.82	41.54
base	O/A	45.62	16.58	22.99	41.59
base	A/O	49.53	22.02	26.78	45.01
base	O/O	51.93	23.89	27.78	47.42
large	A/A	46.80	18.01	23.80	42.57
large	O/A	47.79	18.37	23.78	43.57
large	A/O	51.01	22.77	27.00	46.53
large	O/O	52.98	24.28	28.02	48.34
Table 9: Performance of BART with different configurations and train/test input settings. The input settings include truncated articles (A) and concatenated sentences with pseudo oracles (O). ROUGE score is abbreviated as  $R$ .
B.4 Generation Examples
We demonstrate two examples of generation in Table 10 and Table 11. The results demonstrate that generations from REFLECT-(CASC) could provide more faithful information from multiple documents, which mainly resulted from the better sentence extraction strategy learned through CASC.
All of our experiments are conducted on a single NVIDIA Tesla V100 32GB GPU with PyTorch. The hierarchical extractor is initialized by RoBERTa-base<sup>5</sup>, and the first three layers are exploited as token-level encoder and the rest layers are sentence-level encoder. The BART-base<sup>6</sup> is used as the abstractor to provide the rewards during extractor training, while BART-large<sup>7</sup> is used for generating the final results. We generate the initial summary reference (SR) by the BART-large model. The hyper-parameter  $\gamma$  in POR is set to 10. We use Adam with constant learning 3e-5 for optimization, and select the model with highest ROUGE-1 F1 score on validation set. For the pseudo extraction oracle, we greedily select at least 30 sentences from article as the pseudo oracle for the extractor during MLE learning. The selection criteria is based on the average of ROUGE-1 recall and ROUGE-2 recall.
For evaluations, we report ROUGE, BERTScore and factual consistency derived from FactCC frame
work. For ROUGE, we use rouge_score package  $^{8}$  to report ROUGE-1, 2, L, and LSum scores. For BERTScore, we use official implementation  $^{9}$  and report the F1 scores. For factual consistency, we use official implementation of FactCC  $^{10}$  and follow previous work (Perez-Beltrachini and Lapata, 2021) to calculate the factual consistency for multi-document summarization.
$^{5}$ https://huggingface.co/deepset/roberta-base-squad2  
 $^{6}$ https://huggingface.co/facebook/bart-base  
 $^{7}$ https://huggingface.co/facebook/bart-large
<sup>8</sup>https://pypi.org/project/rouge-score  
<sup>9</sup>https://github.com/Tiiger/bert_score  
<sup>10</sup>https://github.com/salesforce/factCC
Article: Wesley Snipes Begins Serving 3-Year Prison Stint. This Wesley Snipes began serving a three-year sentence at a federal prison in Pennsylvania on Thursday for failure to file income tax returns. Snipes, 48, arrived shortly before noon at the Federal Correctional Institution McKean in the tiny northwestern Pennsylvania town of Lewis Run, federal prisons spokesman Ed Ross said. He had been ordered to surrender by noon. The minimum security prison camp is worlds away from the harsh prison fortresses depicted in the Snipes' films 'Undisputed' and 'Brooklyn's Finest.' The minimum-security camp doesn't have fences around its perimeter. The 300 nonviolent inmates live in barracks that feature two-man rooms, daily showers and double-feature movie showings Friday through Sunday. ...
Gold Summary: – Wesley Snipes enters prison in Pennsylvania today to serve a three-year sentence for tax evasion—and though the minimum-security McKean prison camp isn't exactly a five-star resort, it's not as bad as it could be. The AP gives details on what his life will be like: No fences: But he will submit to five daily head counts, three of which are overnight. Living quarters: He'll have a two-man room in the barracks, where the other 300 nonviolent inmates live. Daily schedule: Wake-up time is 6:35am, and jobs are performed for seven hours per day. Conjugal visits: He'll have to limit himself to just a kiss in the visitors room. Money: He can earn pennies an hour by doing laundry or other chores, and is allowed to spend $290 a month at the commissary. Entertainment: Double-feature movies are shown Friday through Sunday, but no R-, NC-17-, or X-rated films are screened. Exercise: Sand volleyball, indoor basketball, exercise machines, badminton, bocci, and bridge will be available to him. Medical facilities: The copay at the infirmary is only $2. Church: Almost any group you can think of, including Wiccans, hold weekly meetings. Click for more, including why Snipes failed to pay taxes—or watch his Tuesday night appearance on Larry King Live.
REFLECT (MLE): - Wesley Snipes has begun his three-year prison sentence for failure to file income tax returns. The Blade: Trinity star arrived at the Federal Correctional Institution McKean in Lewis Run in Pennsylvania today, reports the AP. The minimum security prison camp is worlds away from the harsh prison fortresses depicted in Snipes' films Undisputed and Brooklyn's Finest, notes the AP, but the 48-year-old actor will have to adjust to earning just pennies an hour handling kitchen, laundry, and other campus chores. "We recognize that he is high profile, but we treat all our inmates the same," says a spokeswoman. Snipes, a dues-paying member of a tax-protest group that challenges the government's right to collect taxes, has been free on bail since his 2008 conviction, but he made a last-minute request for a new trial this week, arguing that the judge erred by not allowing defense attorneys to interview jurors about misconduct allegations. He also told Larry King last night that he wasn't nervous about reporting to prison. "Don't send me up the river, yet," he said.
REFLECT (CASC): - Wesley Snipes has begun serving a three-year sentence in a minimum-security prison in Pennsylvania for failure to file income tax returns. The Blade: Trinity star arrived at the Federal Correctional Institution McKean yesterday, just before noon, reports the AP. He had been ordered to surrender by noon. The minimum security prison camp is worlds away from the harsh prison fortresses depicted in the Snipes' films Undisputed and Brooklyn's Finest. The 300 nonviolent inmates live in barracks that feature two-man rooms, daily showers, and double-feature movie showings Friday through Sunday. The martial-arts enthusiast can get his exercise playing sand volleyball or indoor basketball, or work out on an elliptical machine or stair climber. Alas, no NC-17, R or X ratings allowed, which knocks out much of Snipes' action-heavy repertoire. The most jarring aspect of the celebrity's stay might be the five daily head counts, three during the overnight hours. And Snipes, who earned a reported  $13 million for the Blade: Trinity sequel, will have to adjust to earning just pennies an hour handling kitchen, laundry, or other campus chores. He can spend just$ 290 a month at the prison commissary. "We recognize that he is high profile, but we treat all our inmates the same," says a spokeswoman. Snipes made a last-minute request for a new trial on Wednesday, arguing that the judge erred by not allowing defense attorneys to interview jurors about misconduct allegations.
Table 10: A generation example for REFLECT on Multi-News corpus. Key overlaps between generations and the gold summary are bolded.
Documents: Vacant lot at 53 New York Avenue NE in Washington, where a government permit has been granted for work connected with Elon Musk's Hyperloop project. (Michael Laris/TWP) It's not much now, just a parking lot with a discarded gin bottle and an old exterminator receipt. But the slice of pavement near the Bureau of Alcohol, Tobacco, Firearms and Explosives in the District could be the gritty precursor to a tunnel network that could propel pods filled with people and speeding platforms topped with Teslas and Toyotas between the nation's capital and New York in 29 minutes. Or it could be just be a parking lot littered with dashed transportation dreams. Electric-car pioneer and space entrepreneur Elon Musk has been touting his vision for a high-speed transportation system since his tweeted announcement last summer that he had received "verbal govt approval" for his tunnel-digging firm, the Boring Company, to build a "NY-Phil-Balt-DC Hyperloop." The Boring Company team has received an early, and vague, building permit from the D.C. government...
Gold Summary: -- Elon Musk's dream of building a hyperloop that can move people between Washington, DC, and New York City in 29 minutes may be a small step closer to becoming a distant reality. A Nov. 29 permit issued by DC's Department of Transportation allows Musk's Boring Company to dig at an abandoned lot beside a McDonald's in northeast Washington that the company says might eventually become a station on a hyperloop connecting NYC, Philadelphia, Baltimore, and Washington. The system would speed up travel times, using electricity and electromagnetic levitation to send people and cars whipping through tunnels in pods. But before you ditch your car, the Washington Post points out the permit is only for "some preparatory and excavation work" at the site, and more permits are needed before any real construction begins. DC Mayor Muriel Bowser has visited a test tunnel Musk is constructing under Los Angeles. But though "we're open to the concept of moving people around the region more efficiently ... we're just beginning, in the mayor's office, our conversation to get an understanding of what the general vision is for Hyperloop," says Bowser's chief of staff, John Falcicchio. Still, the permit "is comparable to an athlete stretching before a race" and shows Musk is serious about moving forward with his plans for high-speed travel since claiming "verbal" government support for an East Coast hyperloop last July, reports TechCrunch. He's also received a conditional permit to build a tunnel beneath Maryland Route 295 in Hawthorne, per the Post, with Maryland officials saying a leg linking Washington and Baltimore will come first. (Read about the hyperloop pod's first test.)
REFLECT (MLE): - Elon Musk's Boring Company has received a permit from the DC government to begin work on a tunneling project in the nation's capital, the Washington Post reports. The permit, issued Nov. 29, is a "very small, very vague step forward" for Musk's plan to build a "NY-Phil-Balt-DC Hyperloop" tunnel network that would run from New York to Philadelphia to Baltimore to DC in 29 minutes, according to the New York Times. But it could also be a "gravel precursor to a tunnel network" that could propel pods filled with people and speeding platforms topped with Teslas and Toyotas between the nation's capital and New York in just 29 minutes. Musk's company has received backing from the White House Office of American Innovation, led by Jared Kushner, as well as Maryland Gov. Martin O'Malley and DC Mayor Muriel Bowser, who visited a tunnel in California earlier this month to learn more about the technology the company says will make tunneling faster and cheaper. The DC Department of Transportation is figuring out what other permits the company will need to cut under city roads and other public spaces. Asked whether the Bowser administration supports the project, he was somewhat upbeat but noncommittal, adding: "We're open to the concept of moving people around the region more efficiently."
REFLECT (CASC): - Elon Musk's plan to build a high-speed transportation system that would run from New York to Philadelphia to Baltimore to DC in 29 minutes has gotten a "very small, very vague step forward," the Washington Post reports. Musk's Boring Company has received a permit for preparation and preliminary excavation of a site in Washington, DC, next to a McDonald's and near the Bureau of Alcohol, Tobacco, Firearms, and Explosives. It's not much now, just a parking lot with a discarded gin bottle and an old exterminator receipt, but it could be "the gritty precursor to a tunnel network that could propel pods filled with people and speeding platforms topped with Teslas and Toyotas between the nation's capital and New York in 29 minutes," per the Post. Or it could "be just be a Parking Lot littered with dashed transportation dreams." Musk has been touting his vision for a Hyperloop system since last summer, when he tweeted that he had received "verbal government approval" to build the "NY-Phil-Balt-DC Hyperloop." Maryland officials told the Post that the tunnel would run under Maryland Route 295, with the DC-Baltimore leg being built first. DC Mayor Muriel E. Bowser visited the company in California this month, walking in a tunnel to learn more about the technology the company says will make tunneling faster and cheaper. The District's Department of Transportation is figuring out what other permits the company would need to cut under city roads and other public spaces, according to Bowser's chief of staff. "We're just beginning, in the mayor's office, our conversation to get an understanding of what the general vision is for Hyperloop," he says. Asked whether the Bowser administration supports the project, he says, "We're open to the concept of moving people around the region more efficiently."
Table 11: A generation example for REFLECT on Multi-News corpus. Key overlaps between generations and the gold summary are bolded.