CTRL SUM : T OWARDS G ENERIC C ONTROLLABLE T EXT S UMMARIZATION
Anonymous authors
Paper under double-blind review
A BSTRACT
Current summarization systems yield generic summaries that are disconnected from users’ preferences and expectations.
To address this limitation, we present
CTRLsum, a novel framework for controllable summarization.
Our approach enables users to control multiple aspects of generated summaries by interacting with the summarization system through textual input in the form of a set of keywords or descriptive prompts.
Using a single uniﬁed model, CTRLsum is able to achieve a broad scope of summary manipulation at inference time without requiring additional human annotations or pre-deﬁning a set of control aspects during training.
We quantitatively demonstrate the effectiveness of our approach on three domains of summarization datasets and ﬁve control aspects: 1) entity-centric and 2) length-controllable summarization, 3) contribution summarization on scientiﬁc papers, 4) invention purpose summarization on patent ﬁlings, and 5) question-guided summarization on news articles in a reading comprehension setting.
Moreover, when used in a standard, uncontrolled summarization setting, CTRLsum achieves state-of-the-art results on the CNN/DailyMail dataset.
I NTRODUCTION
Neural summarization systems aim to compress a document into a short paragraph or sentence while preserving key information.
There are largely two categories of summarization systems: extractive summarization that extracts important portions of a document (Cheng & Lapata, 2016;
Nallapati et al., 2017; Narayan et al., 2018), and abstractive summarization that freely generates novel sentences (Rush et al., 2015; See et al., 2017; Paulus et al., 2018) which can produce coherent and ﬂuent summaries more ﬂexibly.
In this paper we focus on abstractive summarization.
Typically abstractive summarization methods take a document as input and yield a generic summary to cover certain information identiﬁed by the model.
However, content of interest is user-dependent.
Summaries should select information with respect to preferences of a user.
For example, Figureshows an NBA basketball news article, and the reference summary describes several match results.
However, fans of certain basketball stars in these teams such as Lebron James or Stephen Curry might only be interested in the matches they played and would like to know the player’s scores as well.
Motivated by this, we focus on controllable summarization which allows the users to manipulate the summaries from the model.
We propose CTRLsum, a framework to control summaries through control tokens in the form of a set of keywords or descriptive prompts.
At training time, the model learns to predict summaries conditioned on both the source document and keywords that serve as external guidance.
During inference, keywords and optional prompts, which are the target preﬁx to constrain decoding, are combined as control tokens to convey user preferences as shown in Figure 1.
Keywords and prompts are complementary.
Prompts do not perform well in many cases such as entity or length controlled summarization as our preliminary experiments imply, but keywords can achieve those goals in a ﬂexible way, for example, by using entity as keywords or varying the number of keywords to control entities and length respectively.
However, keywords struggle in more open-ended scenarios like summarizing a list of contributions of scientiﬁc papers, while constraining the decoding with prompt “ the main contributions of this paper are:(1) ” is possibly sufﬁcient to achieve the goal.
[Figure]
Reference Summary
Control Tokens
Keywords-based
Model
[Figure]
CTRLsum is trained using only keywords as additional input which can be easily identiﬁed from training summaries.
It requires neither extra human annotations nor pre-deﬁning control aspects for training, yet is quite ﬂexible to achieve a broad scope of text manipulation as we will show in this paper.
In contrast, prior work primarily rely on pre-deﬁned “control codes” (Fan et al., 2018; Liu et al., 2018; Keskar et al., 2019), thus need to collect annotations for training and cannot generalize to unseen control aspects easily at test time.
We use pretrained BART (Lewis et al., 2019) as the underlying architecture and perform experiments on three datasets in three distinct domains: CNN/Dailymail news articles (Hermann et al., 2015), arXiv scientiﬁc papers (Cohan et al., 2018), and BIGPATENT patent documents (Sharma et al., 2019).
We quantitatively evaluate CTRLsum on ﬁve control aspects: (1) entity-centric (§4.2) and (2) length-controllable summarization (§4.3), (3) summarizing the contributions of scientiﬁc papers, (4) summarizing the purpose of an invention (§4.4), and (5) summarizing answers to given questions in a zero-shot reading comprehension setting (§4.5).
Notably, our approach also achieves comparable or superior performance to the strong BART summarization model on all datasets in a standard, uncontrolled setting (§4.6), leading to state-of-the-art results on the CNN/Dailymail dataset.
CTRL SUM 2. O VERVIEW
Unconstrained neural summarization methods are trained to learn the conditional distribution p ( y | x ) , where x and y represent the source document and summary respectively.
The generated summaries depend solely on the document x without human involvement.
To control the output summaries, we propose using additional control tokens z to represent user preferences and training a summarization model that predicts the conditional distribution p ( y | x , z ) .
The control tokens z include keywords as extra inputs during training and inference.
They can also optionally include prompts at test time to further constrain the decoding process.
As shown in
Figure 1, control tokens – in the form of keywords, prompts, or a combination of both – act as an interface between users and an otherwise black-box neural model, providing a ﬂexible way for users to explicitly control automatic summarization.
Next we describe how to obtain automatic keywords for training as well as potential applications at test time.
2. A UTOMATIC K EYWORD E XTRACTION
In addition to extracting keywords from training data to train the model, CTRLsum also features an automatic keywords extraction mechanism at test time, which can be used to suggest automatic keywords according to user preferences, or perform uncontrolled summarization without user signals.
Next we describe the keywords extraction methods at training and inference time respectively.
Training.
For training, we use the ground-truth summary to identify keywords in the source document.
Speciﬁcally, we ﬁrst greedily select sentences from the document that maximize
the
ROUGE scores (Lin, 2004) with the reference summary.
This step constrains keywords to those found in important sentences.
Then, we identify all the longest sub-sequences in the extracted sentences that have matched sub-sequences in the ground-truth summary, similar to the copying word recognition method in (Gehrmann et al., 2018).
Finally, we remove duplicate words and stop words and keep the remaining tokens as keywords.
Compared to other keywords extraction methods (Riloff &
Lehnert, 1994; Mihalcea & Tarau, 2004) which output only a few salient words, our extraction retains most content words found in the summary.
This encourages dependence on the given keywords by building a reliable correlation between their presence in the input and the target.
It in turn ensures that user-provided keywords are not ignored by the model at test time, which is catastrophic for a controllable summarization system.
Inference.
We formulate the keyword extraction problem at test time as a sequence labeling task.
Concretely, we train a BERT-based sequence tagger (Devlin et al., 2018) on the keywords and documents from training dataset.
This tagger then computes the selection probability q j for each token in the test document.
Similar to training time extraction, we ﬁrst select n s sentences with the highest average token selection probability.
Within these sentences words with q j > (cid:15) are selected as keywords up to a maximum number of m max .
The three hyperparameters n s , (cid:15), m max are selected based on the uncontrolled summarization performance on validation datasets.
The results are reasonably robust to different settings (see Appendix D for details).
2. S UMMARIZATION : T RAINING D ETAILS
Format.
At training time we prepend the keyword sequence to the source document separated with a special token.
The summarization model is then trained to maximize p ( y | x , z ) in an end-to-end fashion.
The keyword sequence maintains the order of the keywords as they were in the source document, but we observe that the model often ignores this ordering as it frequently differs between source and target summary.
We also separate keywords from different source sentences with the special token (“ | ”).
In applications where the sentence boundary is unknown, as when users propose their own keywords, the “ | ” token can be ignored as in some of our experiments.
Keyword Dropout.
As mentioned in §2.2, our keyword extraction strategy retains most words from the summary found in the source document.
Without regularization, the dependence on such keywords is strong enough that the model rarely generates novel words in the summary.
To remedy this, we randomly drop keywords at training time so that the model learns to rely on keywords that are present in the input, while also learning to still carry over key information from the source document that is not present in the keywords.
Note that keywords dropout is applied at training time only.
Next we are going to introduce the ﬁve control aspects that we study in this paper as example use cases of CTRLsum.
Qualitative examples of them are shown in Table 1.
2. S UMMARIZATION : I NFERENCE WITH K EYWORDS .
The keywords provide a generic interface to control multiple aspects of summaries, which allows the user to optionally rely on automatically extracted keywords, user provided keywords, or a combination of both.
This method provides clean separation of test-time user control and the training process, including pretraining.
Consequently, CTRLsum can be adapted to new use cases without changing model parameters.
For example, though nothing during training speciﬁcally focuses on controlling entities or length, examples below demonstrate the general applicability of keyword control to entity and length manipulation.
Entity Control.
The goal of entity control is to produce summaries that focus on entities of interest.
Figure 1 exempliﬁes summarization with respect to different players when those player names are included as keywords directly inﬂuencing the summary.
Length Control.
Users may have different preferences as to the length of summaries.
We allow such manipulation of the summary length through a user-speciﬁed length parameter.
Speciﬁcally, we
[Table] [Table]
this paper
[Table]
ﬁrst separate the training data into 5 buckets by summary length so that each bucket has the same number of examples.
Then we compute the average number of keywords K l for each bucket on the training data.
At test time, a user can specify length parameter l ∈ { 0 , 1 , 2 , 3 , 4 } to include the K l keywords with the highest selection probability computed by the sequence tagger.
This is similar to (Saito et al., 2020a), which uses the number of “guiding words” to control summary length.
2. S UMMARIZATION : I NFERENCE WITH K EYWORDS AND P ROMPTS
Prompts are pre-deﬁned text sequences used as the target preﬁx to constrain decoding.
They have been utilized to perform multi-purpose text generation with a single uniﬁed model (Radford et al., 2019; Brown et al., 2020).
In the CTRLsum framework, prompts are a kind of control token sequence, and we always use such tokens as both the target preﬁx and keywords (ablation results on using prompts as keywords or preﬁx alone can be found in Appendix C).
We ﬁnd that using prompts as keywords besides preﬁx helps focus on prompt-related content and mitigate the over-generation issue of vanilla summarization models, as we will show in §4.4.
To the best of our knowledge, we are the ﬁrst to evaluate such a prompt-based control method for summarization systems.
Summarizing Contributions.
Existing datasets about scientiﬁc papers such as arXiv (Cohan et al., 2018) collect paper abstracts as the summaries, which often include extra background context and lack detailed contribution descriptions for the associated paper.
In many cases, readers would beneﬁt from an explicit list of contributions in order to understand the novelty and value of the paper.
For these cases, we propose using control tokens – “ the main contributions of this paper are:(1) ”.
This prompt then triggers generation of a summary focused on contributions.
Summarizing Invention Purpose.
Patent article summaries in existing datasets such as BIGPATENT (Sharma et al., 2019) can be over-complicated, often covering core method details.
Yet for a non-technical reader it would be preferred to provide a one-sentence summary that states the purpose of the invention while ignoring technical details.
To apply CTRLsum in this scenario, we use the control tokens, “ the purpose of the present invention is ”.
This triggers a concise summary focused on patent purpose.
Question-guided summarization.
Human summarization can be constrained by questions (Kry´sci´nski et al., 2019) that require answers to be found in the summary.
This points to an important connection between summarization and reading comprehension that we further explore.
We hypothesize that a summarization model can directly answer some questions about the article if guided properly.
This suggests the possibility of subsuming reading comprehension as a form of summarization.
To verify this hypothesis, we use the control tokens “ Q: question text?
A: ” to trigger reading comprehension behaviour.
We note that prompts- and keywords-based control are complementary in practice – while prompts could theoretically achieve any type of control, empirically they often do not work well for many aspects and the model is very sensitive to the precise wording of the prompt.
For example, we found that using prompts such as “ a summary focused on [entity] is: ” or “ a short summary is: ” does not work as well as explicitly using keywords for entity or length control (details can be found in Appendix C).
R ELATED W ORK
Previous work on controllable summarization often collects control codes such as entity or length as supervision to train the model conditioned on both the code and article together (Fan et al., 2018; Liu et al., 2018).
These methods do not generalize for controlling aspects of the summarization that were not seen during training.
Recently
Saito et al.
(2020a) use the number of word prototypes to control summary length in a similar way to how we use keywords.
Interactive summarization provides a way for users to continuously control the information that is included in the summary (Bornstein et al., 1999; Leuski et al., 2003).
More broadly, controllable text generation has been studied for styles (Hu et al., 2017; Fu et al., 2018;
He et al., 2020b), topics (Tang et al., 2019; Huang et al., 2019), and templates (Guu et al., 2018; Wiseman et al., 2018;
He et al., 2020a).
Keyword-guided text generation has been applied in other contexts with different motivations.
Gehrmann et al.
(2018) utilize copying words at test time to mask copying operations in a summarization task.
Li et al.
(2018) and Saito et al.
(2020b) use keywords as extra input to improve the uncontrolled summarization performance.
Wang et al.
(2016), Mou et al.
(2016), and Yao et al.
(2019) use textual input to plan poetry, dialogue, and stories respectively.
Lexically-constrained decoding speciﬁes certain lexicons as hard constraints in the target text (Hokamp & Liu, 2017; Post & Vilar, 2018).
Preﬁx-constrained decoding was used in machine translation (Knowles & Koehn, 2016;
Wuebker et al., 2016) and also to demonstrate the multi-task ability present in large pretrained models (McCann et al., 2018;
Radford et al., 2019; Keskar et al., 2019; Brown et al., 2020).
E XPERIMENTS
Our experiments below are designed to (1) test the control efﬁcacy of CTRLsum on ﬁve different aspects, and (2) examine the performance of CTRLsum in a traditional summarization setting without external control signals.
Also, extensive model output examples can be found in Appendix E.
4. E XPERIMENTAL D ETAILS
We perform experiments on three distinct-domain summarization datasets: CNN/Dailymail (CNNDM) news articles (Hermann et al., 2015), arXiv scientiﬁc papers (Cohan et al., 2018), and
BIGPATENT patent articles (Sharma et al., 2019).
For all datasets the source documents are truncated to 1024 tokens and the target summaries are truncated to 256 tokens following (Zhang et al., 2019).
The conditional distribution p ( y | x , z ) in CTRLsum is our ﬁne-tuned version of the pretrained
BART LARGE model (Lewis et al., 2019), which achieves state-of-the-art performance on several summarization benchmarks.
The automatic keyword tagger at test time is based on the pretrained
BERT LARGE model (Devlin et al., 2018) ﬁne-tuned as described in §2.2.
Our summarization model implementation is based on the fairseq toolkit (Ott et al., 2019) and the automatic keyword extraction model is based on the HuggingFace
Transformers library (Wolf et al., 2019).
Complete setup and training details can be found in Appendix A.1.
[Table] [Table]
For evaluation, we measure commonly used ROUGE scores (Lin, 2004) and the recently proposed
BERTScore (Zhang et al., 2020) when ground-truth is available.
For control-related evaluation where we often do not have reference summaries, we (1) collect ground-truth summaries when possible, (2) examine whether summaries respect the control signal, or (3) resort to human evaluation.
4. E NTITY CONTROL
Setup.
We ﬁrst simulate user preference by providing the model with oracle entities extracted from the ground-truth target.
Then we compare it to the model using automatic keywords in a uncontrolled setting to show the effect of oracle entities.
To examine whether the decoded summaries respect entity change, we sample 100 documents and repeatedly acquire every entity in the document to generate summaries, following Fan et al.
Then we compute Success Rate , the fraction of requested entity actually occurring in the output summaries.
The results are reported in separation of whether the entity is from leading 3 sentences or from the full article.
To test if the summaries from different entity input are factually consistent with the document, we sample another 100 documents, and for each we randomly sample one “important” entity that appears in the reference, and one “unimportant” entity that occurs neither in the reference nor the leading three source sentences to produce summaries.
For each (article, summary) pair we ask 3 annotators from Amazon Mechanical
Turk to make a binary decision as to whether the summary can be entailed from the article.
We then take the majority vote as the result and report the fraction of factually correct summaries.
We evaluate on CNNDM only since many examples in arXiv and BIGPATENT do not have identiﬁable entities.
Results.
In Table 2 we observe that the use of oracle entities helps boost the ROUGE-2 score by 3. points compared with using automatic keywords, which means CTRLsum is able to take advantage of the given entities.
Table 3 shows the Success Rate and factual correctness evaluations.
We include the numbers from Fan et al.
(2018) (EntityCode) for reference point.
We note that their numbers come from a convolutional seq2seq architecture (see Appendix B for ablation analysis on this) and their method utilizes entity annotations during training time, thus is not very comparable to CTRLsum.
Remarkably, our model achieves a high success rate for both lead-3 and full-article entities reaching around 95% .
Yet other systems struggle to include the given entities especially for the ones that do not occur in the beginning of the article.
Factual correctness scores from human annotators suggest that CTRLsum is able to generate factually consistent summaries no matter whether the entity of interest is important or not, comparable to the unconstrained BART baseline.
[Table]
[Table]
4. L ENGTH CONTROL
Setup.
Similar to entity control, we ﬁrst examine the effect of oracle length signal from the reference to simulate user preference.
In addition to ROUGE and BERTScore, we measure the length distance between the decoded summary and the reference following (Liu et al., 2018).
Speciﬁcally, we compute the mean of absolute deviation (MAD) of the actual length bucket code l sys of the decoded summary from the ground-truth control code l ref , as 1 N (cid:80) Nn | l ( n ) sys − l ( n ) ref | .
To assess the summary variations as length signals change, we further sample 1000 documents and decode 5 different-length summaries for each document.
Then we report the Pearson Correlation Coefﬁcient (PCC) between the input bucket code and actual bucket code.
Experiments are conducted on CNNDM and arXiv.
Results.
In Table 2 CTRLsum with oracle length signals only presents relatively small gains over the automatic CTRLsum baseline.
This implies that oracle lengths only convey limited additional information to help generate the reference summary.
We also run the LengthCode baseline (Fan et al., 2018) based on BART, where the ground-truth length bucket code is prepended to the article at both training at test time.
However, LengthCode fails to consistently improve over BART with oracle length signals.
Moreover, we ﬁnd that the BART model ﬁne-tuned with LengthCode method almost ignores the length signal with PCC close to 0, as shown in Table 4.
This is not very surprising since length code would be less useful when the summarizers grow stronger, which can already learn a good length predictor implicitly.
In contrast, CTRLsum with length-guided keywords achieves high positive PCC between control signal and actual output length, and is able to reduce the length deviation MAD compared to automatic baselines.
4. C ONTRIBUTION AND P URPOSE SUMMARIZATION
Contribution Summarization Setup.
There is no existing dataset to evaluate contribution summarization of scientiﬁc papers, bringing challenges to our evaluation.
However, researchers often summarize the bullet contributions of their paper in the Introduction section, which inspire us to extract such contribution claims as the reference summary.
Therefore, we resort to the entire arXiv database, 2 and download all the papers whose ﬁrst submission time is within the ﬁrst six months of 2019 3 that gives us 67K papers.
We extract the Introduction section and bullet contributions with regular expression and ﬁlter out the ones that fail.
The contributions are used as the reference and the
Introduction section after removing the contribution claims is used as the source article – we aim to predict contributions from the rest of the introduction section.
This procedure leads to 1018 test examples.
We test the model trained on arXiv.
Purpose Summarization Setup.
To collect a test dataset that features one-sentence invention purpose summaries, we sample 1000 test examples from BIGPATENT and present their reference summaries to human annotators from Amazon Mechanical Turk.
For each example we ask one annotator to select the sentence that convey the purpose of the invention.
We also provide the option for annotators that the invention purpose cannot be identiﬁed.
After ﬁltering out the invalid examples, we collect 763 examples as our test data.
[Table]
[Table] [Table]
Results.
Table 6 shows results of contribution summarization on scientiﬁc papers and invention purpose summarization on patent ﬁlings.
Through using the prompt text as both the decoder preﬁx and keywords, CTRLsum outperforms the BART baseline in most cases.
We further report the precision (P) and recall (R) scores in BERTScore besides F1.
We observe that the BART baseline tends to over-generate a full summary with low precision scores while CTRLsum is able to focus on keywords-related content.
4. Q UESTION - GUIDED SUMMARIZATION
Setup.
We directly test question-guided summarization on reading comprehension benchmarks in a zero-shot setting.
Speciﬁcally, we evaluate the CNNDM summarization models on in-domain
NewsQA (Trischler et al., 2017) and out-of-domain SQuAD 1.1 (Rajpurkar et al., 2016) respectively.
We note that some NewsQA test articles are present in the CNNDM summarization training dataset, yet we think it is still a reasonable unsupervised setting since our model never sees questions or answers during training.
In addition to comparing with the vanilla BART model, we also include the zero-shot performance from GPT2 language models (Radford et al., 2019) (without ﬁne-tuning) as a reference point.
We omit the largest GPT2 model with 1.5B parameters since it cannot be evaluated in our single GPU device due to memory limits.
We report F1 scores on the two benchmarks.
Results.
BART is pretrained with a denoising task to predict the denoised version of the source, and performs poorly on zero-shot reading comprehension out of box, as shown in Table 5.
Interestingly, however, BART ﬁne-tuned on a summarization task – without seeing any question-answer pairs in the training data – is able to improve the F1 scores by 24.4 and 25.9 points on NewsQA and SQuAD respectively.
Moreover, CTRLsum equipped with question keywords is able to further boost the performance by 15.6 and 17.9 points, approaching the supervised MatchLSTM (Wang & Jiang, 2017) score on NewsQA.
Such results suggest that summarization might be a suitable transfer task for abstractive reading comprehension, which we leave for future work to explore.
4. A UTOMATIC S UMMARIZATION
Table 7 shows the uncontrolled summarization performance without any user input, where our method uses the automatically extracted keywords as described in §2.2.
On CNNDM and arXiv datasets
CTRLsum outperforms the strong BART and PEGASUS baselines by a large margin, leading to new state-of-the-art performance on CNNDM.
It also performs comparably to the BART baseline on
BIGPATENT in terms of BERTScore, though with an inferior ROUGE-2 score.
Yet there is a big performance gap between BART-based models and PEGASUS on BIGPATENT.
The reasons might be different dataset processing, 4 sub-optimal learning schedule, or inherent difference between BART and PEGASUS.
[Table] [Table]
[Table]
H UMAN E VALUATION
In this section we present human evaluation results for both controlled and uncontrolled summarization.
Full experiment details can be found in Appendix A.2.
Controlled Summarization.
We present further human evaluation results to evaluate “control” directly by informing annotators the intended control signal.
We conduct experiments on entity and purpose control.
Speciﬁcally, we inform the annotators our intent (to obtain summaries focused on a speciﬁc entity or purpose of patent), then we ask them to provide scores in scale 1-5 over two dimensions: (1) Control Accuracy (CA): whether the summary contains accurate main information with respect to the intent, and (2) Control Relevance (CR): how the summary is relevant to the control intent overall – a summary that contains redundant contents that are unrelated to the intent will be penalized.
Results including signiﬁcance tests are shown in Table 8.
The control accuracy for important entity control and purpose control are comparable between BART and CTRLsum without signiﬁcant difference (p-value > 0.05), while CTRLsum shows signiﬁcantly better control relevance overall by focusing on the desired information.
Also, the unconstrained BART are unable to generate unimportant-entity-related summaries and thus suffers from poor scores on both dimensions.
Uncontrolled Summarization.
We follow (Grusky et al., 2018; Fabbri et al., 2020) to ask human annotators from Amazon Mechanical Turk to score summaries (scale 1-5) over four dimensions: (1)
Factual Consistency (FAC): the summary should only contain statements that can be entailed by the source document, (2) Relevance (REL): the summary should only contain important information of the source document, (3) Fluency (FLU): each sentence in the summary should be ﬂuent, and (4)
Coherence (COH): the summary should be well-structured and well-organized.
Results including signiﬁcance tests are present in Table 9.
The quality of summaries from all systems on all dimensions is generally good with a score mostly higher than 4.0.
However, most scores do not show signiﬁcant difference from CTRLsum (Automatic Keyword) with large p-values, despite their very different similarities against the reference summaries in terms of ROUGE/
BERTScore (e.g. CTRLsum with oracle keywords).
This implies that the summary quality from different systems powered by strong pretrained models like BART has become difﬁcult to be clearly distinguished by non-expert MTurkers.
We also note that non-expert human judgement for summarization may be unreliable and exhibit poor correlation with expert judgement (Gillick & Liu, 2010; Fabbri et al., 2020).
C ONCLUSION
In this paper we propose a generic framework to perform multi-aspect controllable summarization.
The model is conditioned on keywords to predict summaries during training.
At inference time the control tokens, in the form of keywords or prompts, enable users to interact with models in a very ﬂexible way.
Experiments on ﬁve different control aspects demonstrate the efﬁcacy of our method.
A E XPERIMENTAL S ETUP D ETAILS A. G ENERAL S ETUP
In this section we include additional experimental details left out in the main content due to space limitations.
We ﬁne-tune the pretrained BART LARGE model in all our experiments.
Speciﬁcally we use the bart.large checkpoint from fairseq (Ott et al., 2019).
For all BART-based summarization models, we ﬁne-tune with learning rate 3e-5 and a polynomial learning rate decay schedule, the optimizer is Adam (Kingma & Ba, 2015) and batch size is 64.
Our optimization scheme and hyperparameters follow the BART ﬁne-tuning instructions in fairseq examples.
We train the summarization models with 20k steps on CNNDM, 50k steps on arXiv, and 300k steps on BIGPATENT.
We train the
BERT tagger with learning rate 5e-5, Adam optimizer, and batch size of 128 on all datasets.
Similar to summarization models, the tagger is trained with 20k, 50k, and 300k steps on CNNDM, arXiv, and BIGPATENT respectively.
Also, we adopt a sliding window approach so that the BERT-based tagger is able to handle sequences that are longer than 512 tokens.
For both ROUGE and BERTScore evaluation, we report the F1 measure.
We report the rescaled BERTScore, and the hash code is roberta-large_L17_no-idf_version=0.3.6(hug_trans=3.0.2)-rescaled .
As mentioned in §2.2, we need three hyperparameters for automatic keywords extraction during inference – the number of pre-selected sentences n s , the selection probability threshold (cid:15) , and the maximum number of keywords m max .
We select these hyperparameters for each dataset based on the uncontrolled summarization ROUGE-2 score on validation dataset.
The summarization performance is robust to these hyperparameters in a reasonable range, as shown in Appendix D. Speciﬁcally, we use { n s = 10 , (cid:15)
= 0 .
25 , m max = 30 } for CNNDM, { n s = 10 , (cid:15) = 0 .
15 , m max = 40 } for arXiv, and { n s = 5 , (cid:15) = 0 .
15 , m max = 30 } .
Invention Purpose Summarization.
In the experiment of summarizing invention purpose on patent articles (§4.4).
We examined whether the model would possibly copy source sentences through matching the prompts, we search strings in the form of “the purpose of [some words or phrases] is” among 763 test examples, and only 3 test articles are identiﬁed.
This means the models are not generating by exactly matching prompts most of the time.
A. H UMAN E VALUATION S ETUP
Here we include details about human evaluation experiments in §4.7.
Controlled Summarization.
For controlled summarization, we sample 100 examples for each task, and summaries of each example from all systems are presented together to the human annotator to be scored.
For CNNDM we provide article and summaries, while for BIGPATENT we provide reference and summaries using the reference summary as a surrogate for the source article.
This is because the source patent documents are very long and hard to be read by non-expert humans.
We did not evaluate contribution summarization since it is unrealistic to ask humans to judge contributions of many scientiﬁc papers from various domains.
We tried to hire workers from Amazon Mechanical
Turk ﬁrst, but we failed to obtain reliable results from them – they often ignored the given user intent and tended to score the text as uncontrolled summaries (reﬂected by very poor scores on unimportant-entity summaries because these summaries do not contain the main information of the article), even though we instructed them that the control signal is critical.
Therefore, we ask two independent human annotators through personal correspondence from the authors of this paper.
One of the annotator is a PhD researcher on physics, and the other is a law graduate on intellectual property in the United States.
They are able to follow the given control intent and considered more reliable than the MTurkers.
We take the average of two annotators as the score for each example, and average over all examples to obtain the ﬁnal score.
Uncontrolled Summarization.
For uncontrolled summarization, we sample 100 examples for each dataset, and hire 3 independent workers from Amazon Mechanical Turk to conduct evaluation.
For
CNNDM we provide article and summaries, while for arXiv and BIGPATENT we provide reference and summaries using the reference summary as a surrogate for the source article.
This is because the source patent documents or scientiﬁc papers are very long and hard to be read by non-expert humans.
Summaries of each example from all systems are presented together to the human annotator to be scored.
The median score of 3 workers is taken for each example, and average over all examples is reported.
B A BLATION A NALYSIS OF E NTITY C ONTROL
In Table 3 we observe that CTRLsum achieves a very high success rate ( ∼ 95% ) of entity control, compared to previous work (Fan et al., 2018) which can only succeed 61.2% and 33.8% of the time on lead-3 and full-article entities respectively.
We perform ablation analysis to understand the important gradients that contribute to the success of CTRLsum.
We train CTRLsum with another two architectures in addition to BART: (1) convolutional seq2seq (Gehring et al., 2017) with the same hyperparameters as in (Fan et al., 2018), and (2) transformer seq2seq with the same hyperparameters as the base model in (Vaswani et al., 2017).
Note that the transformer model is trained from scratch without pretraining.
Results are shown in Table 10.
CTRLsum parameterized with a weaker convolutional seq2seq architecture fails to depend on the keywords well with an over 40-point success rate drop, yet the success rate of transformer seq2seq without pretraining only drops around 5 points.
This implies that the transformer seq2seq architecture is critical for CTRLsum to depend on the keywords well, while pretraining can further improves it.
[Table]
C A BLATION A NALYSIS ON K EYWORDS AND P ROMPTS
In the controlling aspects we studied CTRLsum uses control tokens either as keywords alone (entity and length), or as keywords and prompts together (contribution, purpose, QA).
Here we present further results when control tokens are used as prompts, keywords, or both for entity control, contribution control, and NewsQA tasks.
Speciﬁcally for entity control, we use the control tokens “ a summary focused on [entity] is: ” for “prompt” and “prompt + keyword” variants.
In this case success rate is computed excluding the prompt text.
The control tokens for other settings are the same as previous experiments.
Results are shown in Table 11, where keywords and prompts are of different importance for different tasks and are complementary in general.
For example, using prompts to control entities turns out to be difﬁcult with a very low success rate – we ﬁnd that the system fails to understand the prompt and produce summaries appropriately in most cases.
However, prompts contribute the most to contribution summarization with comparable performance with using prompts and keywords together, while removing prompts and using keywords alone suffers from drastic performance drop to trigger the contribution.
For NewsQA task, prompts and keywords demonstrate mixing effectiveness – using either of them alone experiences over 20 F1 points loss compared to using them together.
[Table]
D R OBUSTNESS A NALYSIS OF K EYWORDS E XTRACTION H YPERPARAMETERS
Table 12 shows the ROUGE-2 scores of uncontrolled summarization on the validation set with different keywords extraction hyperparameters.
We use more ﬁne-grained stride size to iterate the m max hyperparameter for CNNDM since its source articles are usually shorter than arXiv and
BIGPATENT.
As observed, the automatic summarization performance is relatively robust to these hyperparameters in a reasonable range.
[Table]
s = 5 , m max =– 17.
(cid:15)
= 0 . 15 , n s = 10 , m max =22.
– – (cid:15) = 0 . 15 , n s = 10 , m max =22.
(cid:15)
= 0 . 15 , n s = 10 , m max =22.
– – (cid:15) = 0 . 15 , n s = 10 , m max =– 18.
(cid:15)
= 0 . 20 , n s = 5 , m max =22.
– – (cid:15) = 0 . 20 , n s = 5 , m max =22.
(cid:15)
= 0 . 20 , n s = 5 , m max =22.
– – (cid:15) = 0 . 20 , n s = 5 , m max =– 17.
(cid:15)
= 0 . 20 , n s = 10 , m max =22.
– – (cid:15) = 0 . 20 , n s = 10 , m max =22.
(cid:15)
= 0 . 20 , n s = 10 , m max =22.
– – (cid:15) = 0 . 20 , n s = 10 , m max =– 17.
(cid:15)
= 0 .
25 , n s = 5 , m max =22.
– – (cid:15) = 0 . 25 , n s = 5 , m max =22.
(cid:15)
= 0 .
25 , n s = 5 , m max =22.
– – (cid:15) = 0 . 25 , n s = 5 , m max =– 17.
(cid:15)
= 0 .
25 , n s = 10 , m max =22.
– – (cid:15) = 0 . 25 , n s = 10 , m max =22.
(cid:15)
= 0 .
25 , n s = 10 , m max =22.
– – (cid:15) = 0 . 25 , n s = 10 , m max =– 17.
(cid:15)
= 0 . 30 , n s = 5 , m max =22.
– – (cid:15) = 0 . 30 , n s = 5 , m max =22.
(cid:15)
= 0 . 30 , n s = 5 , m max =22.
– – (cid:15) = 0 . 30 , n s = 5 , m max =– 17.
(cid:15)
=
n s = 10 , m max =22.
– – (cid:15) = 0 . 30 , n s = 10 , m max =22.
(cid:15)
=
n s = 10 , m max =22.
– – (cid:15) = 0 . 30 , n s = 10 , m max =– 17.
E R ANDOM O UTPUT E XAMPLES
In this section
[Table]
we randomly sample test examples and show the source aticle, reference summary, and the model output from CTRLsum for each control aspect
[Table]
E. E NTITY C ONTROL
For entity control, we randomly sample 3 articles from CNNDM and for each article we randomly select 5 entites as keywords to show the model output.
[Table]
no-ﬂy list.
Now, they’ll be made aware of their status if they apply for redress, with an option to request further information.
The U.S. government is making enhancements to the Department of Homeland Security Traveler Redress Inquiry Program (DHS TRIP) to provide additional transparency and process for U.S. citizens and lawful permanent residents who have been denied boarding on a commercial aircraft because they are on the No Fly List," the Department of Homeland Security said in a statement.
In cases in which travelers included on the list request to receive or submit more information about their status, the government will provide a second, more detailed response, identifying "speciﬁc criterion under which the individual has been placed on the No Fly List," according to the court documents.
An unclassiﬁed summary of that information will be provided "to the extent feasible, consistent with the national security and law enforcement interests at stake," court papers said.
Those who appear on the no-ﬂy list will then have further opportunity to dispute their status in writing, with supporting materials or exhibits, and will receive a ﬁnal written decision from the Transportation Security Administration.
The 2014 ruling that prompted the policy changes had called for passengers on the list to be given the opportunity to dispute their status before a judge.
ReferenceSummary
Americans on the no-ﬂy list will now get info about why they’ve been banned from ﬂights.
ACLU says the policy still denies “meaningful notice, evidence, and a hearing”
CTRLsum
the Fifth Amendment’s :
The new policy is in response to a June ruling that said the old process was in violation of the Fifth Amendment’s guarantee of due process.
Civil Liberties Union : The American Civil Liberties Union says it’s not satisﬁed with the govern- ment’s new policy.
the Transportation Security Administration : The Transportation Security Administration will pro- vide travelers with more information about their status.
A June ruling had called for passengers on the list to be given the opportunity to dispute their status before a judge.
Virginia :
The ACLU is challenging the new policy in federal courts in Oregon and Virginia.
FBI : People on the no-ﬂy list, managed by the FBI’s Terrorist Screening Center, are prohibited from boarding commercial ﬂights.
The ACLU isn’t satisﬁed with the government’s new policy.
[Table]
One
[Table]
Monday night
[Table]
State
[Table]
up
[Table]
it left
[Table]
in
[Table]
season,
[Table]
19th
[Table]
ﬁnal minute
[Table] [Table]
lot of free
[Table]
which made it
[Table]
lot closer than it needed to be
[Table]
the
[Table]
you’ve just got to get it done somehow
[Table]
We’re
[Table] [Table]
That’s where we want to be
[Table]
the
[Table] [Table]
[Table] [Table]
[Table]
and four three
[Table]
All ﬁve
[Table]
State
[Table]
in
[Table] [Table]
been in
[Table]
form as he looks to lead
the
[Table]
to the
[Table]
glory
[Table]
after
[Table] [Table]
most
[Table] [Table]
and came complete with an edible six-pack.
Award-winning: Both Jennifer Lawrence and her cake alter-ego have won awards.
Homage: The cake, which triumphed at a show last November, was inspired by the Hunger Games .
Actress Jennifer Lawrence has also been immortalised in cake, with baker Lara Clarke creating a sweet treat designed to resemble the 24-year-old’s Hunger Games alter-ego, Katniss Everdeen.
The confection, which was baked ahead of the release of Mockingjay Part One in November, met with the approval of Lawrence herself, who, when asked about it, said Ms Clarke was ’incredibly talented’.
Other A-listers to get the culinary treatment include Kevin Bacon, whose likeness was made from bacon
[Table]
and Taylor Swift who featured on a cake.
The Duke and Duchess of Cambridge were turned into a tasty-looking pizza, while President Barack Obama had his likeness sculpted from butter.
Immortalised
[Table]
Duke and Duchess of Cambridge have been turned into pizza in the past.
ribute: Kevin Bacon has a bacon sculpture to his name
[Table]
Dornan
[Table]
been
[Table]
a
[Table]
ReferenceSummary
[Table]
of
[Table]
Cumberbatch has been
[Table]
E. P AIRED E NTITY C ONTROL
The entity control experiments in this paper only consider one entity as the control signal, here we show examples inputting paired entities.
Speciﬁcally, we are interested in the output when one of the paired entities is important and the other is unimportant.
Therefore, we sample 3 articles from
CNNDM and for each article we randomly select one important entity and one unimportant entity as paired keywords to show the model output.
We repeat this sampling ﬁve times for each article to obtain ﬁve different summaries.
[Table]
how
[Table] [Table]
One
[Table]
that detonating a
[Table]
is his No. 1 aspiration because he couldn’t be captured, all evidence would be destroyed and he would be guaranteed to hit his target," the criminal complaint said.
He made a video with a Fort Riley airﬁeld in the background and said ISIS was coming to kill American soldiers, both abroad and in the United States, the complaint said.
Booker acquired components for a bomb and rented a storage locker to store the components, the complaint said.
The plan was for conﬁdential informants to build a bomb and for Booker to drive to Fort Riley and detonate it, the complaint said.
But the bomb was built with "inert" parts and would never explode, the complaint said.
On Friday, the informants and Booker drove to what Booker thought was a little-used utility gate near Fort
Riley, the complaint said.
While Booker was making ﬁnal connections on the "bomb," the FBI arrested him, the complaint said.
He was charged with one count of attempting to use a weapon of mass destruction, one count of attempting to damage property by means of an explosive and one count of attempting to provide material support to the Islamic State of Iraq, a designated foreign terrorist organization.
If convicted, he could face life in prison.
Alexander E. Blair, 28, of Topeka was taken into custody Friday and charged with failing to report a felony.
The FBI said agents interviewed Blair after Booker’s arrest.
Blair said he shared some of Booker’s views, knew of his plans to detonate a vehicle bomb at Fort Riley and loaned him money to rent storage space, according to the FBI’s criminal complaint.
He said he thought Booker would carry out his plan but did not contact authorities, the complaint said.
If convicted, Blair faces a maximum of three years in prison.
ReferenceSummary
Alexander Blair, 28, of Topeka accused of knowing about bomb plot but not contacting authorities.
Fort Riley’s security was never breached and the device was "inert" and not a threat, authorities say.
John T. Booker Jr., 20, of Topeka had acquired bomb parts and made a propaganda video, the Justice Department says.
CTRLsum
Fort Riley ISIS : John T. Booker Jr. is charged with trying to detonate a car bomb at Fort Riley.
A second man, who allegedly knew about the bomb plot but didn’t call authorities, is charged.
Booker said he believed ISIS wanted him to commit a
[Table] [Table]
4-0 win over Queens Park Rangers without looking at the ball.
The former Athletic Bilbao man ﬂicked the ball past Preston’s players without having his eyes on the ball .
Herrera scored in Manchester United’s 2-1 defeat by
Swansea City at the Liberty Stadium in February.
He also netted in Manchester United’s shock 5-3 defeat at the hands of Premier League strugglers Leicester.
ReferenceSummary
Ander Herrera has scored seven goals
[Table]
Man
[Table]
since joining in June.
Herrera’s eyes have been shut when striking the ball for six of his goals.
His superb strike against Yeovil Town has been only goal with eyes open.
Herrera netted a brace in Manchester United’s 3-1 win over Aston Villa .
CTRLsum
Ander Herrera Van Gaal :
Ander Herrera has scored six of his seven goals without looking at the ball .
Louis van Gaal has praised the Spanish midﬁelder’s shooting ability .
Yeovil Town Swansea City : Ander Herrera scored his side’s opening goal against Aston Villa with his eyes shut.
The 25-year-old has also scored without looking at either the ball or the opposition’s net in games against QPR, Leicester City, Swansea and Yeovil Town .
Manchester United Queens Park Rangers : Ander Herrera appeared to have his eyes closed as he scored against Aston Villa on Saturday.
The 25-year-old has scored six of his seven goals without looking at the ball.
Herrera also netted with his eyes open in Manchester United’s 4-0 win over Queens Park Rangers .
Aston Villa MUTV :
Ander Herrera scored a brace against Aston Villa on Saturday.
But the Spanish midﬁelder appeared to have his eyes closed .
Louis van Gaal told MUTV that Herrera should be ’more composed’ .
Herrera Athletic Bilbao : Ander Herrera has scored six of his seven goals with his eyes closed.
The former Athletic Bilbao man joined Manchester United for £29million in the summer .
[Table]
last October 22 after racing to the scene. .
PC Dave Catt said they drew their ‘incapacitating’ sprays fearing the knifeman would be still holding his weapon.
They were greeted by O’Riordan wearing a blood-spattered light blue shirt and holding a cordless phone on which he had phoned for an ambulance.
Mr Catt said O’Riordan admitted: ‘I found out that she was having an affair and I lost it.’.
Mrs O’Riordan was moaning and lying on a bed, holding a towel to her stomach with a deep chest wound and serious wounds to her hand and back.
Paramedics arrived moments later and took her to hospital.
Jurors looked at two screens as images of her husband’s arrest and subsequent detention at a police station were shown.
Growing suspicion: Giving evidence yesterday Alfred Harris – a friend of the couple for more than six years – told how O’Riordan had conﬁded in him that he believed his wife was having an affair.
Yesterday, jurors at Brighton Crown Court (above) were shown dramatic footage of the moment police arrived at the couple’s home to be greeted by a ‘calm’ Mr O’Riordan opening the door.
PC Stuart Kenway told how, as O’Riordan had opened the door, he ‘appeared calm and composed and the situation was surreal’ as he then said: ‘She is in the bedroom – do you want the knife?’
Ofﬁcers were directed to a 7in blade with a black handle which was in the kitchen.
Dr Stephen Drage, an intensive care consultant with Brighton and Sussex
University Hospitals, told the jury how seriously Mrs O’Riordan was hurt.
‘It is quite clear she was bleeding to death,’ he said.
‘She underwent life-saving surgery which took six hours.’.
O’Riordan denies attempted murder.
The trial continues. . .
Sorry we are not currently accepting comments on this article.
ReferenceSummary
[Table]
about
[Table]
E. L ENGTH C ONTROL
For length control
[Table]
we randomly sample 3 test articles from CNNDM and for each article we generate ﬁve different-length summaries
[Table] [Table]
Article
Washington (CNN)An Iranian military observation aircraft ﬂew within 50 yards of an armed U.S. Navy helicopter over the Persian Gulf this month, sparking concern that top Iranian commanders might not be in full control of local forces, CNN has learned.
The incident, which has not been publicly disclosed, troubled U.S. military ofﬁcials because the unsafe maneuver could have triggered a serious incident.
It also surprised U.S. commanders because in recent months Iranian forces have conducted exercises and operations in the region in a professional manner, one U.S. military ofﬁcial told CNN.
"We think this might have been locally ordered," the ofﬁcial said.
The incident took place as the U.S. and other world powers meet with Iran in Switzerland to negotiate a deal limiting Tehran’s nuclear program.
At the same time, Iran has been active in supporting proxies in several hotspots in the Persian Gulf and neighboring regions.
The Navy MH-60R armed helicopter was ﬂying from the deck of the USS Carl Vinson on a routine patrol in international airspace, the ofﬁcial said.
An unarmed Iranian observation Y-12 aircraft approached.
The Iranian aircraft made two passes at the helicopter, coming within 50 yards, before the helicopter moved off, according to the ofﬁcial.
The ofﬁcial said the helicopter deliberately broke off and ﬂew away in a ’predictable’ manner so the Iranians could not misinterpret any U.S. intentions.
The Navy helicopter was in radio contact with the ship during the encounter, but there was no contact between the two aircraft and no shots were ﬁred.
The Navy crew took photos of the incident but the military is not releasing them.
The U.S. administration is considering a potential demarche protest against Iran, the ofﬁcial said.
CNN has reached out to Iranian ofﬁcials but has not received a response.
This type of Iranian observation aircraft generally operates over the Gulf several times a month.
But after the recent incident, U.S. naval intelligence did not see it again for two weeks, leading to the conclusion that the incident may have been ordered by a local commander who was then reprimanded by higher-ups.
The Pentagon has noted for the last several years that most encounters with the Iranian military at sea or in air are conducted professionally, but that some missions run by Iranian Revolutionary Guard Corps forces have been too aggressive against U.S. forces in the area.
The U
[Table]
S
[Table]
military’s concern has been that one of these incidents could escalate into a military encounter.
This incident
[Table]
might have been buffoonery"
[Table]
ofﬁcial
[Table]
, but there is always a risk from such actions.
The incident comes as the Navy patrols the Gulf of Aden to watch for Iranian ships the U.S. believes are trying to bring weapons to resupply the Houthi rebels in Yemen.
The Navy would share such intelligence with Saudi Arabia, a second U
[Table]
S. ofﬁcial told CNN.
[Table]
might
[Table]
full control
[Table]
forces
[Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
performances and music, and the festival itself raises money and awareness for HIV research.
The festival started nearly 40 years ago
[Table]
and the Kanayama Shrine was once a popular place for prostitutes to pray for protection from sexually transmitted infections.
Today, people visit the shrine to pray for everything from fertility and business prosperity to wedded bliss and good health.
In addition to the parade there are performances and music, and the festival itself raises money and awareness for HIV research.
The Kanayama Shrine was once a popular place for prostitutes to pray for protection from sexually transmitted infections.
Today, people visit the shrine to pray for everything from fertility and business prosperity to wedded bliss and good health.
Groups of locals carried three heavy phalluses modelled after a mikoshi portable shrine, which is commonly used in Shinto festivals.
[Table]
Shinto Kanamara Matsuri started as a small tradition but has grown into a popular tourist attraction.
Known as the Festival of the Steel Phallus, participants pray to a god of fertility and protection from infections.
Rainy weather didn’t ruin the mood at this year’s festival, which attracted a large crowd of holidaymakers .
[Table]
the
[Table]
E. C ONTRIBUTION S UMMARIZATION ON S CIENTIFIC P APERS
Here we show three random examples from the arXiv test set.
Note that this is the test set from (Cohan et al., 2018) instead of the contribution test data collected by us, because we want to show the difference between reference summaries (i.e. the paper abstract) in existing standard paper summarization dataset and our output contribution summaries.
We truncate the source articles since they are too long to display.
this be an over therefore may be step how neural systems work @ stimulated a deal of theoretical such as the studies on the effects of the topological properties and the properties of recently that the of synchronized is favored occurs are mechanism of be useful for understanding the way the investigated the dynamical mechanism underlying the inﬂuence of synaptic efﬁcacy on ﬁring synchronization in hh neuron networks .
to do this , we ﬁrst studied the dynamics of the response of hh neuron to excitatory synaptic current .
when the efﬁcacy of synapse is low , namely , strength is weak and duration is short , the limit cycle is stable to the perturbation of the synaptic current .
when synaptic efﬁcacy is high , synaptic current can induce the transition of the neurons from limit cycle to ﬁxed point or transient state .
the transition is determined by dynamics of neuron s ionic channel .
the decrease of synaptic current depresses the feedback of sodium ionic current which is responsible for the initiation of the spike .
for simplicity we will refer to the transitions as spike death .
in neuronal networks , synaptic input of a neuron is the accumulation of the currents received from all presynaptic neurons .
when the coherence of ﬁring time of neurons is enhanced by the excitatory interaction , the synaptic input of neurons transforms from the ﬂuctuating waveform into the pulse shape like the signal produced by one synapse .
if synaptic efﬁcacy is high , the input signal can induce spike death of the neuron .
then spike death disorders the adjustment of the rhythm of neurons and prevents neurons from ﬁring spikes synchronously .
in contrast , for synapses of lower efﬁcacy , the duration of synaptic current is too short to induce spike death of neurons .
additionally , the ﬁring synchronization is different from synchronous activity of oscillators for the existence of the transitions of neuron s state .
the paper is organized as follows .
the hh neuron model and the synaptic coupling were introduced in sec .
the response of a hh neuron to synaptic current was investigated in sec .
the inﬂuence of the dynamics of neurons on ﬁring synchrony was shown in sec .
ReferenceSummary we investigated the inﬂuence of efﬁcacy of synaptic interaction on ﬁring synchronization in excitatory neuronal networks .
we found spike death phenomena , namely , the state of neurons transits from limit cycle to ﬁxed point or transient state .
the phenomena occur under the perturbation of excitatory synaptic interaction that has a high efﬁcacy .
we showed that the decrease of synaptic current results in spike death through depressing the feedback of sodium ionic current .
in the networks with spike death property the degree of synchronization is lower and unsensitive to the heterogeneity of neurons .
the mechanism of the inﬂuence is that the transition of neuron state disrupts the adjustment of the rhythm of neuron oscillation and prevents further increase of ﬁring synchronization .
CTRLsum [ the main contributions of this paper are : ( 1 ) ]: we investigated the dynamical mechanism underlying the inﬂuence of synaptic efﬁcacy on ﬁring synchrony in hodgkin - huxley neuron networks ; ( 2 ) we found that the dynamics of synaptic current plays an important role in determining the stability of ﬁring synchronization .
a three dimensional continuum object its motion is usually described by the amplitudes of different oscillation eigenmodes .
in general , these modes can cause the cantilever to bend in all directions in space .
however , the cantilever is positioned such that the softest ﬂexural modes bend the beam in a plane orthogonal to the surface plane .
we restrict ourselves to the case where only these ﬂexural modes are excited by the drive force .
due to this experimental conﬁguration the cantilever is much more susceptible to the component of the tip - surface force which is orthogonal to the surface plane .
this component of the force is typically the most dominant component and the inﬂuence of lateral force components is considered negligible .
in this case the cantilever acts as a mechanical projector which reacts only to one component of a three dimensional force vector ﬁeld .
the deﬂection @xmath0 of a cantilever of length @xmath1 orthogonal to surface is described by a one dimensional euler - bernoulli equation@xcite @xmath2 where @xmath3 is the young s modulus , @xmath4 is the second moment of area , @xmath5 is the mass per unit length of the cantilever , @xmath6 is the position coordinate along the cantilever beam and @xmath7 is the time variable .
the force term @xmath8 includes the surface forces acting as a point - like load at position @xmath9 , the external drive force and the hydrodynamic damping due to the surrounding medium@xcite .
ReferenceSummary
in atomic force microscopy ( afm ) tip - surface interactions are usually considered as functions of the tip position only , so - called force curves .
however , tip - surface interactions often depend on the tip velocity and the past tip trajectory .
here , we introduce a compact and general description of these interactions appropriate to dynamic afm where the measurement of force is restricted to a narrow frequency band .
we represent the tipsurface interaction in terms of a force disk in the phase space of position and velocity .
determination of the amplitude dependence of tip - surface forces at a ﬁxed static probe height allows for a comprehensive treatment of conservative and dissipative interactions .
we illuminate the fundamental limitations of force reconstruction with narrow band dynamic afm and we show how the amplitude dependence of the fourier component of the force at the tip oscillation frequency , gives qualitative insight into the detailed nature of the tip - surface interaction .
with minimal assumptions this amplitude dependence force spectroscopy allows for a quantitative reconstruction of the effective conservative tip - surface force as well as a position - dependent damping factor .
we demonstrate this reconstruction on simulated intermodulation afm data
[ the main contributions of this paper are : ( 1 ) ]: a comprehensive framework for the representation and analysis of complex surface forces as they are measured by dynamic atomic force microscopy ( afm ) ; ( 2 ) a study of the fundamental limit of force reconstruction with narrow band dynamic afm at ﬁxed probe height and show how minimal assumptions allow for a quantitative reconstruction of the tip - surface interaction .
Article in this paper we discuss the mathematical aspects of the problems originating in the solution of nonlinear systems of hyperbolic partial differential equations .
these equations describe a large variety of physical phenomena , such as , gasdynamics , magnetohydrodynamics ( mhd ) , shallow water equations , elasticity equations , etc . being nonlinear , these systems usually require numerical methods for their solution .
solutions motivates the necessity of the development of reliable numerical methods based on the fundamental mathematical properties of hyperbolic systems .
although such methods are rather well developed for the euler gasdynamic equations in the conservation law form , their extension to more complicated hyperbolic systems is not straightforward .
it requires a mathematical justiﬁcation of the solution uniqueness , a formulation of the selection principles for relevant solutions , and , ﬁnally , an investigation of their physical validity .
most of high - resolution methods for gasdynamic equations use the exact or some of the approximate self - similar riemann problem solutions to determine ﬂuxes through the computational cell surfaces .
similar methods are expected to be developed for various types of hyperbolic systems .
in this case we must construct the elementary self - similar solution using only admissible discontinuities ( entropy consistent , evolutionary , etc . ) .
basically the choice of the solution must be made on the basis of the structure of the solution of the extended problem @xcite .
all mentioned above makes very important the study of discontinuous solutions behavior under vanishing viscosity and dispersion to create a proper background for the development of high - resolution numerical methods for hyperbolic systems more complicated than the euler equations of gasdynamics .
we discuss several analytical and numerical solutions in the mentioned ﬁelds which illustrate the complexity of the selection problem and outline the methods of its solution .
tvd upwind and symmetric differencing schemes have recently become very efﬁcient tool for solving complex multi - shocked gasdynamic ﬂows .
this is due to their robustness for strong shock wave calculations .
the extension of these schemes to the equations of the ideal magnetohydrodynamics is not simple
ﬁrst , the exact solution xcite of the mhd riemann problem is too multivariant to be used in regular calculations .
second , several different approximate solvers @xcite , @xcite , @xcite , @xcite , @xcite , @xcite , and @xcite applied to mhd equations are now at the stage of investigation and comparison .
this investigation requires i ) determination of a proper slope limiting method in the parameter interpolation procedure necessary to obtain nonoscillatory schemes of the order of accuracy higher than one ; ii ) development of an efﬁcient entropy correction method necessary to exclude rarefaction shocks ; and , ﬁnally , iii ) solution of the problem of excluding the origin of nonevolutionary solutions in ideal mhd calculations .
the system of governing equations for a mhd ﬂow of an ideal , inﬁnitely conducting , perfect plasma in the cartesian coordinate system @xmath0 , @xmath1 , @xmath2 with the use of the conventional notations reads ( one ﬂuid approximation ) : @xmath where @xmath4 is the vector of conservative variables and @xmath5 , @xmath6 , and @xmath7 are the ﬂux vectors .
we introduced here the source term @xmath8 in the form @xmath9 this form of the system can be used to satisfy the divergence - free condition by convecting away the magnetic charge from the computational region @xcite .
otherwise , any other well - known method can be used to eliminate the magnetic charge .
to determine a numerical ﬂux @xmath10 normal to the computational cell boundary ( @xmath11 is a unit outward vector normal to the cell surface ) one can use the formulas based on the solution of the linearized problem @xmath12. ] ] here @xmath13 and @xmath14 are the matrices formed by the right and by the left eigenvectors , respectively , of the frozen jacobian matrix @xmath15 the matrix
@xmath16 is a diagonal matrix consisting of the frozen jacobian matrix eigenvalue moduli .
the superscripts @xmath17 and @xmath18 denote the values at the right- and at the left - hand side of the cell boundary .
ReferenceSummary
a number of physical phenomena are described by nonlinear hyperbolic equations .
presence of discontinuous solutions motivates the necessity of development of reliable numerical methods based on the fundamental mathematical properties of hyperbolic systems .
construction of such methods for systems more complicated than the euler gas dynamic equations requires the investigation of existence and uniqueness of the self - similar solutions to be used in the development of discontinuity - capturing high - resolution numerical methods .
this frequently necessitates the study of the behavior of discontinuities under vanishing viscosity and dispersion .
we discuss these problems in the application to the magnetohydrodynamic equations , nonlinear waves in elastic media , and electromagnetic wave propagation in magnetics .
CTRLsum [ the main contributions of this paper are : ( 1 ) ]: the mathematical aspects of the problems originating in the solution of nonlinear systems of hyperbolic partial differential equations ; ( 2 ) the study of discontinuous solutions behavior under vanishing viscosity and dispersion to create a proper background for the development of high - resolution numerical methods for hyperbola systems more complicated than the euler equations of gasdynamics ; and ( 3 ) solution of the problem of excluding the origin of nonevolutionary solutions in ideal magnetohydrodynamics calculations .
E. I NVENTION P URPOSE S UMMARIZATION ON P ATENT F ILINGS
Here we show three random examples from the BIGPATENT test set.
Note that this is the test set from origial BIGPATENT, because we want to show the difference between reference summaries in existing standard dataset and our output purpose summaries.
We truncate the source articles since they are too long to display.
designed mesh and the infection resistance of a monoﬁlament - designed mesh may be produced .
particularly , it has been discovered that a support trellis formed of multiﬁlament yarn wherein the interstitial voids located between adjacent ﬁlaments are enclosed within an infection - impervious matrix exhibits the desired resistance to harboring of infectious matter without signiﬁcant loss of ﬂexibility .
particularly , the matrix , which completely encloses the interstitial voids between the ﬁlaments of the yarn , provides an effective barrier to the passage of infectious matter between the interior and exterior of the yarn .
accordingly , any voids remaining in the yarn after encapsulation of such yarn are enclosed ( and thereby sealed ) within the resultant matrix .
a ﬁrst embodiment of the present invention is shown in ﬁg2 .
particularly , this ﬁrst embodiment includes a support trellis 20 formed of multiﬁlament yarns 22 and 24 which overlap at cross - over junctions 25 .
subsequent to forming of the trellis , such trellis is encapsulated within a matrix 26 , which is preferably a ﬂexible material that continuously surrounds the exterior of the yarns thereby enclosing interstitial voids 27 located between ﬁlaments 28 ( see ﬁg2 a ).
in one embodiment , the matrix is formed from a polymeric resin .
as shown in ﬁg2 a , the resin can be applied to the yarn in such a manner as to not allow the resin to substantially penetrate into the yarn .
particularly , the penetration of the resin can be controlled through the application procedure , e .
g ., quantity of resin applied and / or encapsulating time .
in such an embodiment , the interstitial spaces are enclosed ( rather than ﬁlled ) within the continuous matrix .
however , it is contemplated that the resin can be allowed to penetrate into the yarn , thereby substantially ﬁlling the void space located therein .
in another embodiment of the present invention , individual yarns 29 , as shown in ﬁg3 are encapsulated within matrix 30 prior to forming of the support trellis .
ﬁg3 a shows a compressed yarn 29 which provides a trellis having a reduced thickness .
as a result of the encapsulation , interstitial voids 32 remaining in the yarn are enclosed ( and thereby sealed ) within the matrix .
ReferenceSummary a soft and pliable surgical support mesh exhibiting increased resistance to inhabitation of infectious matter .
the mesh includes a support trellis formed of multiﬁlament yarns wherein the interstitial voids located between the ﬁlaments of said yarns are enclosed within an infection - impervious matrix .
the meshes may be designed to be extremely thin yet retain the requisite strength for repairing soft tissue , which allows for a low proﬁle when folded for delivery .
CTRLsum
[ the purpose of the present invention is ]: to provide a surgical mesh that is resistant to the growth of bacteria and other infectious matter .
this is accomplished by encapsulating the interstitial spaces located between the ﬁlaments of the yarn within a matrix .
ReferenceSummary
the present invention is directed to a multicolor web fed rotary printing press having printing units that can be moved along a linear section of a web by a positioning mechanism to allow easy access to each of the printing units .
the easy access to the printing units signiﬁcantly reduces maintenance costs and downtime of the press .
in addition , the invention provides a printing press that has a reduced overall size and allows the printing units to be arranged in a nested formation during printing operations .
this nested formation reduces fanout and paper waste .
the linear web section can be oriented vertically or horizontally .
CTRLsum [ the purpose of the present invention is ]: to provide a web fed rotary printing press that allows an operator to access a desired part of a printing unit more easily than in the past .
[Table]
spaced from one another around an axis of rotation 60 of the barrel 58 .
each of the cylinder bores 59 is oriented within the barrel 58 parallel with the axis of rotation 60 .
a plurality of piston assemblies 62 are operatively associated with the barrel 58 and each one of the plurality of piston assemblies 62 includes a piston 64 slideably disposed in the respective ones of the plurality of cylinder bores 59 .
each one of the plurality of piston assemblies 62 also has a shoe 66 pivotably attached to one end of each piston 64 in a conventional manner .
the barrel 58 has an end surface 68 that is in mating , sealing contact with the port plate 54 to provide communication between the cylinder bores 58 and the respective inlet and outlet port passages 50 , 52 of the head portion 46 .
a closed chamber 70 is deﬁned in each cylinder bore 59 of the barrel 58 between the end of the piston 64 and the end surface 68 thereof .
referring to ﬁg3 the porting between the barrel 58 and inlet and outlet port passages 50 , 52 of the head portion 46 is more clearly illustrated .
for explanation purposes only , the “ 270 ” degree position illustrated in ﬁg3 relates to a position on the right side of the drawing of ﬁg1 and the “ 0 ” degree position illustrated in ﬁg3 relates to a position on the right side of the drawing of ﬁg2 .
an arcuate slot 72 is deﬁned in the port plate 54 and provides communication between the plurality of closed chambers 70 and the inlet port passage 50 .
a plurality of slots 74 are deﬁned in the port plate 54 circumferentially spaced from the arcuate slot 72 and provides communication between the plurality of closed chambers 70 and the outlet port passage 52 .
ReferenceSummary
a variable displacement axial piston pump is typically used to receive ﬂuid from a tank and supply pressurized ﬂuid through a control valve to move an actuator .
the present variable displacement axial piston pump has a swashplate arrangement that is capable of being angled in two different directions to control the pressure transitions between the low pressure inlet port passage and the higher pressure outlet port passage as cylinder bores in a barrel of a rotating group rotate through trapped volume regions situated between inlet and outlet port passages of the axial piston pump .
movement of the swashplate arrangement in two different directions provides smooth pressure transitions and increases the operating efﬁciency of the variable displacement axial piston pump .
CTRLsum
E. Q UESTION -G UIDED S UMMARIZATION
We randomly sample 3 articles from NewsQA and show ﬁve questions and answers from CTRLsum for each article.
We also show the gold answers to these questions.
[Table]
Revolution
[Table]
a gesture
[Table]
hard
[Table]
he
[Table]
to a more conciliatory relationship with
[Table]
United
[Table]
basic and
[Table]
changes in
[Table]
and
[Table]
to
[Table]
state-run Islamic
[Table]
Agency
[Table]
Relations between
[Table]
and
[Table]
have historically been
[Table]
and have
[Table]
strained
[Table]
recent years over
[Table]
nuclear program
[Table]
that the program exists for peaceful purposes
[Table]
but
[Table]
United States
[Table]
other
[Table]
nations
[Table]
concerned by Iran’s refusal to halt uranium enrichment activities.
[Table]
Shirzad Bozorgmehr
[Table]
to
[Table] [Table]
The reports
[Table]
journalism, where viewers are
[Table]
along
[Table]
process.
[Table]
believe this unique reporting approach
[Table]
worthy of sharing with our CNN.com readers.
Brooklyn, New York (VBS.TV) – Whenever someone at our ofﬁce bitches about being overworked, our stock response is "Beats digging ditches."
While the express intention of the statement is usually not-so-supportive, we think it’s a healthy reminder that at the end of the day, we are all basically professional e-mailers and should be thankful for such.
The wildland ﬁreﬁghters who work for Grayback Forestry in Medford, Oregon, have no such motivational adages because their job is actually digging ditches.
Around active forest ﬁres.
On the sides of mountains.
You can’t even bitch at these guys for having cushy government pensions to fall back on when they get older, because they’re all private-sector contractors.
Which means if they aren’t out ﬁghting forest ﬁres or doing preventative forestry on unburned woods (basically extreme landscaping), they are losing money.
They are the hardest working men in the tree business.
Southern Oregon in the summer is a tinderbox.
Last year the state recorded some 560 wild ﬁres, the majority of which occurred in the seemingly endless sea of trees running across its bottom from the Cascades to the Paciﬁc Coast.
Humidity is next to nonexistent, which is extremely pleasant, but means that even an errant spark from a chainsaw or the proverbial cigarette butt out the car window can set the entire region ablaze.
Flying into the Rogue Valley, there is evidence of past wild ﬁres is everywhere: From the miles-long scar of the"Biscuit" ﬁre stretching past the horizon, to the smaller pockets of charred trees crowding the edge of towns to the blue Wilderness-Fireﬁghter-ribbon bumper stickers ﬂying past on the I-5 to the elaborate wildﬁre and ﬁreﬁghter shrines at a local bar.
See more of Oregon Fire Lines at VBS.TV There are very few places ﬁremen aren’t revered as local heroes, but the wilderness ﬁreﬁghters of Oregon go past people simply risking their lives to help others into a crazy superhero realm where their work regularly prevents entire settlements from being destroyed.
They’re literally the guardians of their communities.
Incidentally, their work is also very beneﬁcial for the forest.
We spent a few days following a crew of Grayback forest-ﬁreﬁghters walk up the sides of what most people would consider a cliff, to chop down underbrush in preparation for a controlled burn.
This is what they like to call "project work" – the light stuff they do between ﬁres.
The work is the hardest and least rewarding work we have ever tried to do.
Unless you consider 12-hour-plus shifts of backbreaking labor, virtually zero outside
[Table]
, and occasional accusations of being shills for the timber industry rewards
[Table]
Which we do not.
This piece was originally produced in August 2010.
[Table]
Article
WASHINGTON (CNN) – The nation’s largest publicly owned utility company may be vulnerable to cyber attacks, according to a new report.
In 2007 President Bush visited the Browns Ferry Nuclear Plant, operated by the Tennessee Valley Authority.
The Tennessee Valley Authority, which supplies power to almost 9 million Americans, "has not fully implemented appropriate security practices to protect the control systems used to operate its critical infrastructures," leaving them "vulnerable to disruption," the Government Accountability Ofﬁce found.
Simply put, that means a skilled hacker could disrupt the system and cause a blackout.
Rep.
James Langevin, a Rhode Island Democrat, fears the problem is much larger than just the TVA.
"If they are not secure, I don’t have a great deal of conﬁdence that the rest of our critical infrastructure on the electric grid is secure," he said.
The TVA operates 52 nuclear, hydroelectric and fossil-fuel facilities in the southeastern United States.
Among the government watchdog agency ﬁndings: • The TVA’s ﬁrewalls have been bypassed or are inadequately conﬁgured • Passwords are not effective • Servers and work stations lack key patches and effective virus protection • Intrusion-detection systems are not adequate • Some locations lack enough physical security around control systems.
The GAO recommends 73 steps to correct the problems in its report to Congress.
In September, CNN ﬁrst aired dramatic footage of a government experiment demonstrating that a cyber attack could destroy electrical equipment.
The experiment, dubbed "Aurora," caused a generator to fall apart and grind to a halt after a computer attack on its control system.
The test was conducted by scientists at the Idaho National
Laboratory.
In October, the North American Electric Reliability Corp. told Congress that 75 percent of utilities had taken steps to mitigate the Aurora vulnerability, but Langevin said it now appears that Congress was misled.
A congressional audit of the electric reliability corporation’s claim cast doubt on the assertion that most utilities were taking steps to ﬁx the problem.
"It appears that they just made those numbers up," Langevin said.
"It is not acceptable.
It is outrageous."
He said the result is there is now no clear picture of how vulnerable utilities are to cyber attacks.
The electric reliability corporation – a nongovernmental group that oversees the power system and comprises members of the industry and some consumers – told CNN it regrets the confusion.
Experts told CNN that Cooper Industries is the only manufacturer of hardware that can close the Aurora vulnerability.
The company estimated it would need to sell about 10,000 devices to ﬁx the problem nationwide.
It has sold just over 100, it told CNN.
Langevin said the federal government may need new powers to require utilities to take corrective actions to close cyber security gaps, and he will press to give those powers to the Federal Energy Regulatory Commission.
The congressman is chairing an Emerging Threats, Cybersecurity, and Science and
Technology subcommittee hearing Wednesday afternoon.
Representatives of the TVA, the GAO, the federal commission and the electric reliability corporation are to appear before the subcommittee.
[Table]