Summarizing Text on Any Aspects: A Knowledge-Informed Weakly-Supervised Approach (2020)

Abstract: Given a document and a target aspect (e.g., a topic of interest), aspect-based abstractive summarization attempts to generate a summary with respect to the aspect. Previous studies usually assume a small pre-defined set of aspects and fall short of summarizing on other diverse topics. In this work, we study summarizing on arbitrary aspects relevant to the document, which significantly expands the application of the task in practice. Due to the lack of supervision data, we develop a new weak supervision construction method and an aspect modeling scheme, both of which integrate rich external knowledge sources such as ConceptNet and Wikipedia. Experiments show our approach achieves performance boosts on summarizing both real and synthetic documents given pre-defined or arbitrary aspects.

## introduction remarkable progresses have been made in generating generic summaries of documents @xcite @xcite , partially due to the large amount of supervision data available. in practice, a document, such as a news article or a medical report, can span multiple topics or aspects. to meet more specific information need in applications such as personalized intelligent assistants, it is often useful to summarize a document with regard to a given aspect, i.e., aspect-based summarization. recent research has explored the problem of aspect-based abstractive summarization @xcite . a key challenge of the task is the lack of direct supervision data containing documents paired with multiple aspect-based summaries. previous studies have created synthetic data from generic news summarization corpora which have a small set of aspects (e.g., "sports", "health" and other 4 aspects in @xcite ). as a result, models trained on these data tend to be restricted to the pre-defined set and fall short of summarizing on other diverse aspects. this paper aims to go beyond pre-defined aspects and enable summarization on arbitrary aspects relevant to the document. the arbitrary aspect may not be explicitly mentioned but only implicitly related to portions of the document, and it can be a new aspect not seen during training. to this end, we develop a new approach that integrates rich external knowledge in both aspect modeling and weak supervision construction. specifically, we derive weak supervisions from a generic summarization corpus, where the conceptnet knowledge graph @xcite is used to substantially expand the aspect scope and enrich the supervisions. to assist summarization model to better understand an aspect, especially a previously unseen one, we augment the model inputs with rich aspect-related information extracted from wikipedia. our approach is compatible with any neural encoder-decoder architectures. in this work, we use the large pre-trained bart model @xcite and fine-tune with the proposed method. experiments on real news articles show our approach achieves performance boosts over existing methods. when adapting to the previous synthetic domain, the bart model after fine-tuning with our weak supervisions becomes substantially more data efficient, and outperforms previous best-performing systems greatly using only 0.4% training examples. ## related work aspect-based summarization as an instance of controllable text generation @xcite offers extra controllability compared to generic summarization to ensure concise ner colony collapse disorder has killed millions of bees. scientists suspect a virus may combine with other factors to collapse colonies. disorder first cropped up in 2004, as bees were imported from australia. @xmath015 billion in u.s. crops each year dependent on bees for pollination. extracted aspects aspect: u.s. {…, dollar, technology, …} tf-idf ranking {…, dollar, texas, technology, agriculture,…} document document [u.s.]:[…, dollar, …]<@xmath1.s. crops each year dependent on bees for pollination. (3) identifying aspect related words in the document using wikipedia and (4) feeding both aspect and related words into summarization model. ## approach given a document and an aspect which can be a word or a phrase, the task aims to generate a summary that concisely describes information in the document that is relevant to the aspect. we present our approach that enables a neural summarization model to summarize on any aspects. the aspect can be any words relevant to (but not necessarily occurring in) the document. our approach incorporates rich external knowledge sources, including con-ceptnet for enriching weak supervisions in training (sec 3.1) and wikipedia for advising the documentaspect relation to improve comprehension (sec 3.2). figure 1 shows an overview of our approach. an advantage of our approach is that it is compatible with any neural summarization architectures, such as the popular encoder-decoders. this enables us to make use of the large pre-trained network bart @xcite , on which we apply our approach for fine-tuning and improved inference. ## conclusions this paper studies the new problem of summarizing a document on arbitrary relevant aspects. to tackle the challenge of lacking supervised data, we have developed a new knowledge-informed weakly supervised method that leverages external knowledge bases. the promising empirical results motivate us to explore further the integration of more external knowledge and other rich forms of supervisions (e.g., constraints, interactions, auxiliary models, adversaries) @xcite in learning. we are also interested in extending the aspect-based summarization in more application scenarios (e.g., summarizing a document corpus). document the palestinian authority officially became the 123rd member of the international criminal court on wednesday, a step that gives the court jurisdiction over alleged crimes in palestinian territories. the formal accession was marked with a ceremony at the hague, in the netherlands, where the court is based. the palestinians signed the icc's founding rome statute in january, when they also accepted its jurisdiction over alleged crimes committed "in the occupied palestinian territory, including east jerusalem, since june 13, 2014." later that month, the icc opened a preliminary examination into the situation in palestinian territories, paving the way for possible war crimes investigations against israelis. as members of the court, palestinians may be subject to counter-charges as well. israel and the united states, neither of which is an icc member, opposed the palestinians' efforts to join the body. but palestinian foreign minister riad al-malki, speaking at wednesday's ceremony, said it was a move toward greater justice. "as palestine formally becomes a state party to the rome statute today, the world is also a step closer to ending a long era of impunity and injustice," he said, according to an icc news release. " indeed, today brings us closer to our shared goals of justice and peace." judge kuniko ozaki, a vice president of the icc, said acceding to the treaty was just the first step for the palestinians. "as the rome statute today enters into force for the state of palestine, palestine acquires all the rights as well as responsibilities that come with being a state party to the statute. these are substantive commitments, which cannot be taken lightly," she said. rights group human rights watch welcomed the development. "governments seeking to penalize palestine for joining the icc should immediately end their pressure, and countries that support universal acceptance of the court's treaty should speak out to welcome its membership," said balkees jarrah, international justice counsel for the group. "what's objectionable is the attempts to undermine international justice, not palestine's decision to join a treaty to which over 100 countries around the world are members." in january, when the preliminary icc examination was opened, israeli prime minister benjamin netanyahu described it as an outrage, saying the court was overstepping its boundaries. the united states also said it "strongly" disagreed with the court's decision. "as we have said repeatedly, we do not believe that palestine is a state and therefore we do not believe that it is eligible to join the icc," the state department said in a statement. it urged the warring sides to resolve their differences through direct negotiations. "we will continue to oppose actions against israel at the icc as counterproductive to the cause of peace," it said. but the icc begs to differ with the definition of a state for its purposes and refers to the territories as "palestine." while a preliminary examination is not a formal investigation, it allows the court to review evidence and determine whether to investigate suspects on both sides. prosecutor fatou bensouda said her office would "conduct its analysis in full independence and impartiality." the war between israel and hamas militants in gaza last summer left more than 2,000 people dead. the inquiry will include alleged war crimes committed since june. the international criminal court was set up in 2002 to prosecute genocide, crimes against humanity and war crimes. cnn's vasco cotovio, kareem khadder and faith karimi contributed to this report. aspect: palestinian summary: palestinian foreign minister: "the world is also a step closer to ending a long era of impunity and injustice". the palestinian authority becomes the 123rd member of the international criminal court. the move gives the court jurisdiction over alleged crimes committed in palestine since june. aspect: israel summary: israel and the u.s. opposed the palestinians' efforts to join the court, which is based in the netherlands. palestinian foreign minister: "today brings us closer to our shared goals of justice and peace" the court has jurisdiction over alleged crimes committed in the occupied territories since june. document around 1980, bill gates gave microsoft, the company he founded, a : "a computer on every desk and in every home. "' "but microsoft ceo satya nadella, who in 1992 and rose to the top job in 2014, thinks gates' famous mission had a big flaw." '"when i joined the company in 1992, we used to talk about our mission as putting a pc in every home, and by the end of the decade we have done that, at least in the developed world," nadella told published on monday. "it always bothered me that we confused an enduring mission with a temporal goal. "' "in other words, nadella is saying that gates' vision for the future of microsoft had a logical stopping point and did not consider what the company's direction would be once the goal was achieved. in the 2000s, under former ceo steve ballmer, microsoft became better known for its efforts to than for innovating." 'nadella believes in making microsoft more driven by a sense of purpose -in 2015, he said was "to empower every person and every organization on the planet to achieve more." and he has encouraged the company .' "under nadella, microsoft has taken the focus off windows and the pc and pinned its hopes to the rise of its azure and office 365 products, as the company's older businesses stagnate. while this approach hasn't translated to huge revenue growth, it has revitalized the company's image." "gates told usa today that he enjoys working with nadella, serving his successor as a special adviser and helping guide the company's investments in technology as it competes with apple, google, and amazon." aspect: innovation summary: in the 2000s, under former ceo steve ballmer, microsoft became better known for its efforts to than for innovating. under nadella, microsoft has taken the focus off windows and the pc and pinned its hopes to the rise of its azure and office 365 products. aspect: apple summary: gates says he enjoys working with nadella, serving as a special adviser and helping guide the company's investments in technology as it competes with apple, google, and amazon. 'it always bothered me that we confused an enduring mission with a temporal goal,' he said.