Weakly Supervised Extraction of Tasks from Text (2021)

Abstract: In this paper, we propose a novel problem of automatic extraction of tasks from text. A task is a well-defined knowledge-based volitional action. We describe various characteristics of tasks as well as compare and contrast them with events. We propose two techniques for task extraction – i) using linguistic patterns and ii) using a BERT-based weakly supervised neural model. We evaluate our techniques with other competent baselines on 4 datasets from different domains. Overall, the BERT-based weakly supervised neural model generalizes better across multiple domains as compared to the purely linguistic patterns based approach.

## introduction we define a task as a well-defined knowledge-based action with a specific goal and which is carried out within a small time period, often by a single person, a group of persons, a device, or a system. the tasks usually demand some skill and expertise by the human actor(s) performing the task and they are carried out volitionally by the actor(s). the problem of task extraction is to automatically identify mentions of such tasks in text. syntactically, a task can be mentioned as a verb phrase (e.g., implemented a model for weather prediction) or as a noun phrase (model implementation for weather prediction) in a sentence. event extraction @xcite is a popular task in nlp literature. an event is generally defined as a specific occurrence of something happening in a certain time and place which involves one or more participants and can often be described as a change of state. although events are similar to tasks in some aspects, there are certain crucial distinctions (described in detail in section 2) and hence it is important to define and address task extraction as a separate problem. there are several interesting analyses that can be carried out over the extracted tasks from large corpora. similar tasks can be mentioned in different ways and it is important to cluster the tasks together which have similar meanings. by definition, a task needs certain expertise to be carried out and it would be an interesting problem to determine difficulty level for each task. usually, each task is carried out by an actor and this actor often plays a certain generic role such as engineer, banker, farmer etc. also, most tasks need certain skills to be carried out such as various technical or domain concepts, programming languages, certain tools or technologies, etc. such co-occurrence or interdependence between tasks and roles as well as between tasks and various skills, can be studied. task extraction has several useful real-life applications. for example, tasks extracted from resumes capture the fine-grained experience of the candidate and would be quite useful for automatically shortlisting candidates for a certain job requirement. another interesting application of the extracted tasks and their corresponding roles is to automatically augment common sense knowledge. for example, conceptnet @xcite contains knowledge of the form engineer; is capable of; building a bridge or policeman; is capable of; arresting criminals . but the number of such triplets are lim-dataset type examples of tasks resumes verbal we compared the techniques with which [low power cascade amplifiers can be developed]. nominal verified the credibility of ga algorithm in [designing of orthogonal waveform for mimo radar]. techcrunch verbal but kaliszan and robertson realized that commercial security was so backward that just [implementing the established principles of machine vision] and the cloud could create a huge company. nominal remote work is what led to [the development of gitlab's publicly viewable handbook]. patents verbal particularly, [a new e-mail message is created for each software package to be deployed] . nominal the computer implemented method includes performing, using a processor, [a static timing analysis of the integrated circuit]. reuters verbal at noon the bank had [estimated the shortfall at 500 mln stg]. news nominal stock market analysts said today's generally weak stock market plus unwinding of positions after [heavy buying of bat shares in the run-up to the results] caused the fall in the share price. in this paper, we focus on the problem of automatic extraction of tasks from text and propose two techniques for that. the first technique makes use of linguistic patterns and resources such as word-net @xcite . the second technique uses a weakly supervised bert-based neural model and employs the snorkel framework @xcite for automatically creating a labeled training dataset. the rest of the paper is organized as follows -section 2 discusses relevant past work. section 3 describes the two proposed techniques for the automatic extraction of tasks from text. section 4 describes detailed experimental analysis including dataset descriptions, baselines, and evaluation. finally, we conclude in section 5 along with some potential future work. ## related work to the best of our knowledge, ours is the first attempt to introduce the problem of task extraction and propose extraction techniques for it. event extraction @xcite is a related but different problem as compared to task extraction. definition of an event varies depending on its application and context. sims et al. @xcite have defined an event to be what is depicted as actually occurring in text (also referred as realis events). such events are expected to have four important aspects (non-negation, tense, genericity, and modality) which we compare and contrast with tasks in the top four rows of ## task extraction in this section, we discuss the two techniques for extraction of tasks from text. ## experimental analysis in this section, we describe our experiments in detail including datasets, baselines, evaluation metrics and results. ## conclusion and future work in this paper, we have introduced a new nlp task of automatic extraction of tasks from text, highlighted the motivation behind it, and its potential applications. we described various aspects of tasks and highlighted how they compare with another popular nlp task of event extraction. we proposed two techniques for task extraction -i) linguistic patterns and ii) bert-based weakly supervised neural model. we demonstrated effectiveness of our techniques on 4 datasets from different domains and compared them with other competent baselines. given that ours is the first attempt for extracting tasks from text and the approach is only weakly supervised and does not demand any heavy manual annotation efforts, the overall performance is encouraging. however, there is still scope for the improvement which we plan to pursue as a future work. also, we wish to refine the labeling functions to better capture linguistic characteristics of tasks such as volitionality, need for expertise, and execution within a small time period.