Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper investigates the presence bias introduced by text summarization models. The authors propose to measure bias in two dimensions: content and structure. They define content bias as tendency to mention a specific group (e.g., gender, religion, etc) in a text, whereas structural bias refers to bias as a result of structural features of the text (position, sentiment, and style). In this paper, they limit the scope of investigation to one type of content bias (underrepresentation) and two types of structure bias (position and style). To measure content bias, a representation score R(T, g) is proposed to gauge the proximity of a group g to a text T. For position bias, the content of model-generated summaries and reference summaries is compared according to its position in the input documents. Finally, for style bias, the authors measure the impact of paraphrasing sentences in the original document, under the assumption that the same content should be selected regardless of the style of writing. Those criteria were applied to a synthetic dataset generated by GPT-2 and the CNN/DM news summarization dataset. Based on the results, the authors conclude that summarization models exhibit preference for certain groups over others, amplify patterns of bias, and demonstrate sensitivity to the structure of articles.
strength_and_weaknesses: Strengths:
1. The paper addresses an important and underexplored topic in the automatic summarization.
2. The experiments show that under similar conditions of positioning in the source documents, summarization systems demonstrate preference for groups, for instance, by amplifying content related to men (versus women).

Weaknesses:
1. The content bias analysis is based on a synthetic dataset generated by GPT-2, so that the summary inference was performed on out-of-domain data. If summarization models are sensitive to paraphrasing and structural features, then synthetic documents may also exhibit artifacts that influence the results. It would be important to perform a similar analysis on selected documents from CNN/DM. Alternatively, a human evaluation experiment using the synthetic data could provide more evidence that the bias is really introduced by the summarization models.
2. The position bias analysis, the authors state that "clearly amplified by MatchSum, PreSumm, and Azure, as shown in Fig. 4b." However, it is not clear if the observed pattern is caused by model preference or simply by truncation of inputs. If we observe the figure in Appendix E, the distribution for PEGASUS and BART are more similar to the reference summaries distribution. In contrast, MatchSum, Presumm, and Azure decrease the frequency sharply between 0.4 and 0.6. Interestingly, Presumm and MatchSum use the same backbone model with a maximum input length of 512 tokens, whereas PEGASUS and BART support inputs of 1024 tokens, which might explain their capacity to select more content from the later positions of the articles. For reference, CNN/DM input documents have 766 words on average ("Extractive Summarization as Text Matching", Zhong et al., 2020). Do the quantile calculation in the figures take into account this truncation effect?
3. No details or citation is provided for the paraphrasing model. Just the link for its checkpoint at https://huggingface.co/tuner007/pegasus_paraphrase.
4. Authors mention that they "conduct manual analysis on a randomly selected subset of the article" but no further details are provided.
5. Experiments are based only on CNN/DM, which is well known for its lead bias for important content. Experiments on additional datasets could make the claims stronger.

Minor comments:
1. Typo in page 6: "Overall, t hese results show a pattern of unpredictability..."
clarity,_quality,_novelty_and_reproducibility: The work well-motivated and addresses a relevant problem in language generation. However, the methods lack clarity in some respects (like the paraphrasing approach) and some conclusions are not strongly supported by empirical evidence (see weaknesses 1 and 2 above). The authors mention the release of dataset and code, which helps in the reproducibility.
summary_of_the_review: The claims that summarization models exhibit content bias are weakly supported by the experiments. In terms of structural bias, this paper finds that summarization models trained on news articles have lead bias, which is a well-known fact in the literature. For the lack of novelty and solid evidence for the claims, the recommendation is for the paper rejection.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: The paper draws attention to the problem of Bias Analysis in text summarization tasks which has not been addressed before. It defines the different types of biases present in summarisation task. It performs Bias Analysis using both abstractive and extractive automatic summarising models. The experiments suggest biasness depicted by these models
strength_and_weaknesses: Strengths
It addresses a significant problem largely ignored by the Automatic Text Summarisation Community of Bias Analysis. 
It points out potential risks due to the ignorance of Bias analysis in Text Summarisation.
Defines different types of biases in Automatic Text Summarisation.
Good initial exploration to push the research in this direction.
Weaknesses
The paper does not compare the results of automatic summarizers with human curated summarization.
Only used single embedding word2vec (word embeddings can themselves be baised) and only considered one dataset CNN/Dailymail, without explaining any rationale behind the selection. To add value, authors should have explored other text summarization datasets with different writing styles to really bring out the different style biases rather than just paraphrasing.
The authors only considered a few models, ignoring sota models on CNN/Dailymail dataset. It may have been possible that the SOTA model behaves differently.
The paper lacks any association/description between the metrics used for bias in this paper v/s different text summarisation metrics. It would have been nice to consider why and how these metrics are different to give a better understanding.
Paper lacks any results about biases based on religion and race even though they claim to have done so in the initial section of the paper. Only one incomplete figure about religion is present which is hard to comprehend as it lacks any legend.
The results and figures need to be explained further. Explaining the ideal score for an automatic summarizer in the figures would have given a better idea. I did not really understand the dashed lines in the figure given as Baselines. An explanation with an example will go a long way in making things much more clear and bringing out the results effectively.
First it claims no work has been done on bias-ness in text summarization then it points out a previous work on structure biasness (Jung et al, 2019).
In the paraphrasing of articles, the authors do not show how good the paraphrased sentences are. This raises the concern that if the paraphrased version of sentences itself were curated in a manner that they did not include proper information. A human evaluation of a few paraphrased sentences would have cleared this issue. Also there are no examples from the synthetic dataset curated by the authors.
In the Fig.3, two of the extractive summarisers are not mentioned namely presumm and matchsum

clarity,_quality,_novelty_and_reproducibility: Clarity
The paper is not very clear, there are a lot of loopholes that need to be fixed mainly concerning the explanation of the author’s ideas and implementation.
Quality
The quality needs improvement in terms of considering more experiments.
Novelty
The work is novel.
Reproducibility
Yes it seems reproducible as they have provided the codes and datasets.

summary_of_the_review: The paper lacks comprehensive evaluations using different embeddings, models and datasets. Further, the paper can be hard to comprehend due to lack of explanations of figures and curated examples.

============After rebuttal============================

The authors acknowledge the limitations pointed out, however the paper is very rushed and incomplete in terms of experimentation and their results. Adding the suggested results would definitely improve the quality of the paper, therefore I encourage the authors to do. However, it is still a reject from me with no changes in the previous score.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper looks at the problem of bias in automatic summarization algorithms. The authors design a method to quantitatively measure the extent to which an article's attributes, such as structure, paraphrasing, and article content influence _bias_ in generated summaries. They also study the causes of _bias_ by varying the influential factors, including the summarization parameters and distribution of input documents. The findings indicate that machine learning based summarization algorithms can introduce position or content bias, which can lead to misrepresentation of the original text and/or biasing the reader. The authors recommend that more bias evaluation methods be used when evaluating new summarizers to avoid creating unintentionally biased summarizers in the future.
strength_and_weaknesses: ## Strengths:
- Paper is well motivated. The scope of text summarization methods in practice points to a need to pay much more attention to this question.
- Some of the results offer interesting insights. Such as results in Figure 2, as it shows model reliance on what comes early in an article. But there are also weaknesses in this analysis as I share below.

## Weaknesses:
- The exposition is confusing, several details are missing and assumptions are made without declaration. There is no qualitative analysis to help the reader better follow how the authors are changing structure or content, or what a synthetic summary looks like, or what does it mean to be an article about _men only_ but still have sentences about _women_ in them (Figure 3) or whether the authors' several perturbation strategies aren't altering valuable information?
- The paper relies on the premise that a certain way of choosing a summary (say weighing what comes earlier in an article more), given an article is wrong. But it does not show what the alternative would be and whether that alternative is indeed more preferable to an end user. It is hard to say that given two different texts to an end user (since these are different as they've been reordered), they would somehow still write the same summary for both articles. There is also no end user analysis showing that these _biased_ summaries from a model that rely more on what comes earlier in the article are offering less value to the end users than what a counterfactual _unbiased_ summary would offer. Experiments in Figure 2 come close to this point but it does not come clearly. Summarization is not an objective task, so the gold standard is to show resulting summaries to humans and see whether they prefer one model or the other. But that's hard as the authors don't characterize what such an _unbiased_ summary would be?
- The paper appears rushed. Some examples: Figure 7 has no legend, Table 3's caption says some text is in red and other in blue, but the only color there is yellow. On Page 3, its said that the generation details for the synthetic corpus are in Section 5, on Page 4 its said that they are in Appendix B, but it seems they are in neither. Appendix B comes close to sharing some information but after reading it, I could still not tell how someone could replicate this synthetic data generation because details are sparse. Also, a footnote in this paragraph says that more details are present in the Appendix?
- Several claims have been presented as facts but no citations or data are offered to support them. Some examples: 
  - Introduction: "Readers, however, expect that summaries faithfully represent articles." First, what do you mean by faithful in this context? Second, if this is the case, please add at least one behavioral science citation to this. 
  - Footnote 3: What notion of fairness does this relate to? 
  - Section 3.3: "While we could expect a human to generate a summary..." What is the evidence to support this expectation? Is there any data, any pilot study you did, any previous work that looks at this?
  - Section 5: "We conduct manual analysis..." No details are shared on this analysis and no results are presented.
- I have read the experiments and results section several times, yet I can't tell whether I would be able to replicate the setup and reproduce these results given how sparse the details are.
  - There are error bars in the plots. What do these represent?
clarity,_quality,_novelty_and_reproducibility: The paper is not easy to understand, details are sparse and it is hard to judge the validity of the experiments without having a complete view of the setup. Some of the insights are novel but I can't validate their correctness as there are not enough details in the paper to help me understand what is going on in each experiment. Reproducibility is also difficult for the same reason. The Supplementary material is a zip file of zero bytes that does not expand so I can't run any code either.
summary_of_the_review: The paper is well motivated, but the exposition is confusing and several details are missing. The authors rely on the premise that a certain way of choosing a summary is wrong, but they do not show what the alternative would be practically and whether that is indeed more preferable to an end user. The paper appears rushed, with some claims presented as facts without any supporting data or citations. It is also difficult to tell how one could replicate the experimental setup and reproduce the results given the sparse details.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper describes the typeAs of biases that can appear in automatic
text summarization and presents an initial study of such biases using
two corpuses: a synthetically generated one and the CNN/Daily mail
corpus.

The paper selects a few types of biases that can be analyzed
automatically and proposes metrics (or proxies) for assessing biases,
such as content bias wrt representation and structure biases.

Across several models analyzed (both extractive and abstractive),
there seem to be a general tendency to minimize the representation of
a group over the other. Similarly, most summarizers (with the
exception of GPT-3) seem to prefer sentences appearing at the
beginning of the text over sentences at the end of the text.
strength_and_weaknesses: Strengths:
- first study that I am aware of that looks at bias in text summarization
- discussion on types of biases possible in text summarization
- an intial study of bias in text summarizers

Weaknesses:
- I'm not convinced whether the proxies/metrics for measuring content bias are accurate
- Lack of some human validation
clarity,_quality,_novelty_and_reproducibility: Overall, the paper is straightfoward to follow, the novelty is great and the analysis is interesting and reproducible with reasonable \
effort.

I think the paper would be much stronger if the metrics were validated using some human analysis. For example, can you select some gro\
ups and show that humans find the representation score appropriate? Moreover, can you select 10-100 samples that are both analyzed by \
humans and the automated scores and show that the scores match the human evaluation?

There have been quite a bit of evidence that bias metrics at the embedding level do not follow extrinsic measures of bias and such hum\
an validation would make the results more convincing.
On the Intrinsic and Extrinsic Fairness Evaluation Metrics for Contextualized Language Representations
Intrinsic Bias Metrics Do Not Correlate with Application Bias

Is there a reason why word2vec was used to create the representation scores (and not something newer, some LM-embeddings)?

I have a few suggestions for improvement:

* In the intro, when describing types of biases, show examples, much easier to follow with examples (and the reader is motivated to st\
ay engaged). Expand Table 1 with examples.

* List challenges for bias estimation in text summaries.

* Include a validation for metrics used as explained above

* Fig 1: Do a and b differ in the order in which the groups appear in the text (i.e., a men first, women second and b women first, ben\
 second)?

* I don't seem to find Footnote 4.

* Fig 7, no legend so not sure what the bar/colors/lines mean


UPDATE: Thank you for your response, I will maintain my scores. 
summary_of_the_review: The paper is a first to analyze bias in automatic text summaries and
it would improve in quality if a human validation of metrics used and
results obtained would be included. I strongly encourate the authors
to include such a validation.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: The reviewers, quite in consensus, felt this paper is not ready yet for publication. There were concerns about correctness and the evaluation as compared to previous work, and the contribution itself, even if correct, being sufficiently significant. The authors did not respond to all reviewer comments in the second round.
justification_for_why_not_higher_score: n/a
justification_for_why_not_lower_score: n/a
Decision: Reject