Probing Script Knowledge from Pre-Trained Models (2022)

Abstract: Adversarial attack of structured prediction models faces various challenges such as the difficulty of perturbing discrete words, the sentence quality issue, and the sensitivity of outputs to small perturbations. In this work, we introduce SHARP, a new attack method that formulates the black-box adversarial attack as a search-based optimization problem with a specially designed objective function considering sentence fluency, meaning preservation and attacking effectiveness. Additionally, three different searching strategies are analyzed and compared, , Beam Search, Metropolis-Hastings Sampling, and Hybrid Search. We demonstrate the effectiveness of our attacking strategies on two challenging structured prediction tasks: part-of-speech (POS) tagging and dependency parsing. Through automatic and human evaluations, we show that our method performs a more potent attack compared with pioneer arts. Moreover, the generated adversarial examples can be used to successfully boost the robustness and performance of the victim model via adversarial training.

## introduction a script is a structure that describes a stereotyped sequence of events that happen in a particular scenario @xcite . it allows human to keep track of the states and procedures that are necessary to complete various tasks from daily lives to scientific processes. taking the task of eating in a restaurant as an example. a classic example script for this task may consist of a chain of subevents, such as @xmath0. the script knowledge has shown benefit to many downstream applications, such as story generation @xcite @xcite @xcite , machine reading comprehension @xcite @xcite , commonsense reasoning @xcite @xcite and so on. recent large-scale pre-trained language models (plms) @xcite @xcite have shown competitive performance on many natural language processing tasks. abundant studies have demonstrated that these models either directly capture certain types of syntactic @xcite @xcite , factual @xcite @xcite and commonsense knowledge @xcite @xcite during the pre-training or acquire inductive capability to more efficiently induce such knowledge from natural language text @xcite . however, as another important type of cognitive and schematic knowledge describing human routine activities, scripts are not yet well probed in the language models by prior studies. to investigate how well the pre-trained language models have captured the script knowledge, in this work, we design three probing tasks and language model prompting methods to probe the script knowledge from plms, and further leverage the language model prompting methods to induce the scripts given the main events. specifically, we aim to answer the following two research questions: whether and what script knowledge is captured by the pre-trained language models. to answer this question, we design three sub-tasks to probe the script knowledge, including inclusive sub-event selection (i.e., whether a sub-event is included or excluded in a main event or task), starting sub-event selection (i.e., which sub-event is the start of the script for a particular main event), and sub-event temporal ordering (i.e., predicting a temporal before or after relation between two sub-events). on these sub-tasks, we explore both template-based and soft prompting methods to query the knowledge from pre-trained language models. by investigating their performance gaps to the fine-tuning results, we find that both the inclusive and starting sub-event selection sub-tasks have relatively poorer performance than that of temporal ordering, which is likely due to the lack of relevant objectives to encourage the models to capture such knowledge during pre-training, and further suggests future research directions to enhance the plms to better capture the script knowledge. how to better generate the scripts from these pre-trained models. with the language model prompting methods, we can select the inclusive subevents of a particular script, the starting sub-event and subsequent events by predicting the temporal order among all the inclusive sub-events, which can ultimately generate a sequence of events as the script of a main event. thus, we further design a benchmark dataset to fine-tune the models for the three sub-tasks and evaluate their performance on generating the whole scripts for various main events from diverse domains and topics. the contributions of this work can be summarized as follows: ## experiment setup we take bert-base-uncased @xcite as the target plm to investigate how well it encodes the script language via the three probing tasks. we combine three script datasets, including descript @xcite , omics @xcite and stories (trinh and le, 2018), where each main event is annotated with 7 to 122 scripts written by different crowd-sourcing workers. we sample 60 main events as the evaluation set, 39 main events as the development set and use the remaining 98 main events for training. for the main events in training and development sets, we keep all the scripts, while for each main event in the evaluation set, we only keep the longest script as the target. to create the training samples for the inclusive sub-event selection task, for each script, we use all the ground truth subevents as positive samples and randomly choose 100 times of negative samples from other main events' scripts. for evaluation, as the inclusive sub-event selection requires a pool of all the possible candidate events, we combine the sub-events of all scripts in the evaluation dataset. to create the training samples for the start subevent selection task, we use the first sub-event of each script as the positive sample and all the remaining sub-events from the same script as the negative samples. during the inference, we select the starting sub-event from the inclusive sub-events predicted by the inclusive sub-event selection approach. we use accuracy as the evaluation metric. finally, for the temporal ordering task, we create each training sample based on each sub-event together with one of its following sub-events. we randomly shuffle the order of each pair of subevents and create its corresponding label: "before" or "after". to evaluate the quality of the temporal ordering among all the sub-events, we first generate a script based on the predicted temporal order and then use rouge-l to evaluate the longest common subsequence between the generated script and the gold script. we compare the following approaches for each probing task as well as the script induction: bert pre-trained: directly use the pre-trained bert model to make the predictions on the evaluation set. bert fine-tuning: fine-tune bert with taskspecific training data and evaluate those fine-tuned models on the evaluation set. bert ptuning: following the ptuning framework @xcite , fine-tune the parameters of both bert model and prompt tokens. bert ptuning freeze: only fine-tune the prompt tokens while freezing the parameters of bert model. ## conclusion in this work, we investigate the capability of largescale pre-trained language models (plms) on capturing three aspects of script knowledge: inclusive sub-event knowledge, starting sub-event knowledge and temporal knowledge among the sub-events from the same script. these three types of knowledge can be further leveraged to automatically induce a script for each main event given all the possible sub-events. we use bert as a target plm. by analyzing its performance on script induction as well as each individual probing task, we achieve the conclusions that the stereotypical temporal knowledge among the sub-events is well captured in bert, however the inclusive and starting sub-event knowledge are not well encoded.