Learning to Decompose and Organize Complex Tasks (2021)

Abstract: People rely on digital task management tools, such as email or to-do apps, to manage their tasks. Some of these tasks are large and complex, leading to action paralysis and feelings of being overwhelmed on the part of the user. The micro-productivity literature has shown that such tasks could benefit from being decomposed and organized, in order to reduce user cognitive load. Thus in this paper, we propose a novel end-to-end pipeline that consumes a complex task and induces a dependency graph from unstructured text to represent sub-tasks and their relationships. Our solution first finds nodes for sub-tasks from multiple ‘how-to’ articles on the web by injecting a neural text generator with three key desiderata – relevance, abstraction, and consensus. Then we resolve and infer edges between these subtask nodes by learning task dependency relations. We collect a new dataset of complex tasks with their sub-task graph to develop and evaluate our solutions. Both components of our graph induction solution are evaluated in experiments, demonstrating that our models outperform a state-of-the-art text generator significantly. Our generalizable and scalable end-to-end solution has important implications for boosting user productivity and assisting with digital task management.

## introduction people today increasingly rely on digital modalities and applications to organize, track and complete tasks from their work and life. they depend on email to structure their communications as a way of tracking pending tasks @xcite , issue commands to their digital assistants 1 for timely task reminders @xcite , and use task management applications (bellotti et al., 2004) 2 figure 1 : the subtask graph of "plan a birthday party", where nodes are subtasks, and the directed edges represent the dependencies between subtasks. in this work, we focus on tasks that are complex @xcite , and -which research in micro-productivity has shown @xcite ) -may benefit from thoughtful organization. for example, consider figure 1 , where we show how the complex task "plan a birthday party" foot_0 can be broken down into more manageable pieces and structured by mutual temporal dependencies, in order to create an actionable plan that is simpler and more effective. in this paper we propose to help automate generating such actionable plans in order to reduce cognitive load on users. while prior research @xcite has shown the benefits of tracking and acting on micro-tasks, little effort has been expended on finding automated solutions for actually breaking down complex tasks into tractable sub-tasks. thus we design a novel end-to-end solution that is capable of decomposing complex tasks and structuring sub-task dependencies. we model our end-to-end solution as a graph induction problem, in which we first find nodes to represent sub-tasks, then infer the temporal dependency edges between them, yielding a flow diagram like the one in figure 1 . all of this is done from unstructured text ubiquitously found on the web, making our approach general and scalable. in the first component (that of finding nodes), we learn to synthesize information from multiple 'how-to' articles across the web and generate text fragments for sub-tasks. in particular, we extend a state-of-the-art neural text generator by injecting it with three desiderata for these fragments: relevance (to the complex task), abstraction (by summarizing content in articles), and consensus (for appearing across multiple sources). in the second component (that of finding edges), we infer temporal dependencies between sub-tasks. existing corpora of 'how-to' articles (most notably wikihow @xcite ) do not contain this latent dependency structure. moreover, articles in these corpora are structured and formatted consistently and uniformly, making them ill-suited to our approach, which seeks to synthesize the content of multiple heterogeneous web pages. we therefore devise a simple annotation framework through which we gather a new dataset of complex tasks, and their associated subtasks and mutual dependencies, from multiple 'how-to' web articles using non-expert crowd workers. finally, we use this data to fine-tune our augmented neural text generator, as well as predict dependency edges between the sub-tasks it generates. in experiments, we demonstrate that our optimal solution -which encodes relevance, abstraction and consensus -yields significant improvements over a state-of-the-art text generator on both subtask generation and dependency prediction. the focus of this paper is on complex tasks; however, our research has impact beyond intelligent task management. for example, learning to decompose complex natural language expressions could have impact on complex question answering @xcite , where question decomposition, multi-hop reasoning, information synthesis, and implicit knowledge all play an important role. more generally, the ability to model mappings between short text fragments and elements in multiple documents could benefit research in areas such as topic-focused multidocument summarization @xcite and event timeline extraction of evolving news stories @xcite . in summary, our key contributions are (i) building an end-to-end pipeline for complex task decomposition as a graph induction problem from unstructured text; (ii) constructing a new dataset for complex tasks that contain sub-tasks as well as the temporal dependencies between them; and (iii) extending a neural text generator by injecting signals for relevance, abstraction and consensus, thereby making it more capable at tackling task decomposition. ## problem definition we begin by defining some key concepts. we refer to a task as a text fragment that represents a goal people want to track, remind themselves of, or learn how to do; for example, "buy a christmas present", "eat healthier" or "change a tire". in order to disambiguate the intent of tasks (consider the fragment "harry potter", which could equally refer to "read [the book]" or "watch [the movie]"), we scope our investigation to tasks that contain at least one verb. a task is considered as a complex task when two or more individual steps -themselves also worth tracking, remembering or learning how to do -need to be performed in its completion. therefore, "plan a birthday party", which involves creating a guest list and buying food and beverages (see figure 1 ) is a complex task. while "throw out the trash" is not such a complex task, even though it may involve opening the front door and walking to the trash bins. we refer to the individual steps of a complex task as sub-tasks. sub-tasks may sometimes depend on other subtasks being completed before they can be tackled. consider the example from figure 1 again, which illustrates how one must "set up a time" and "make a guest list" before "send(ing) out invitations". we refer to these relations as temporal dependencies, and are pair-wise notated as sub-task b depending on (←) a. given these key concepts, we define a complex task graph as follows. let the sub-tasks of a complex task t be denoted by st (t) = {s i } n @xmath0. then define g s (t) = (v, e) as the complex task graph of t. here g s (t) is a directed graph, where @xmath1, and e represents the set of temporal dependency edges such that (s i , s j ) ∈ e, s i ← s j . ## data collection to build and evaluate our solutions, we need data. the most relevant existing dataset is wik-ihow @xcite , which is derived from the popular how-to website. however wiki-how, while very useful for parts of our modeling paradigm, is ill-suited to others. namely, its articles are manually curated, with consistent structure and format, making them a mis-match to the heterogeneous, noisy and free-form articles we expect to encounter on the web. moreover, they contain no dependency information between sub-tasks beyond a simple numbered ordering. thus, to support our problem, we need a dataset which (1) contains complex task and their subtasks; (2) encodes dependencies between sub-tasks; and (3) the sub-tasks come from a variety of webpages. note that our goal is to create a dataset that enables model generalization, rather than constructing a comprehensive knowledge base. therefore, rather than exhaustively annotating sub-tasks and their dependencies, we seek only to gather labels for the most important ones. in what follows we will describe the step-by-step construction of our dataset, and how these steps encode our three fundamental desiderata for task-sub-task relationships (see section 2). the dataset we collect, which we call mscom-plextasks foot_3 is being released freely to the research community. collecting complex tasks and how-to articles we begin with logs from the popular -now defunct -task management application wunderlist foot_4 . these logs are privately and respectfully handled by passing them through an enterprise grade, legaland trust-approved pipeline, which anonymizes, aggregates and scrubs all personally identifiable information. this yields a collection of task strings, some of which have associated sub-task metadata (not sub-tasks themselves). we retain those tasks which have at least one sub-task, and contain at least one verb foot_5 (to avoid issues with disambiguating task intent) as a candidate seed pool of complex tasks. it may be noted that while the logs are not publicly available, they play a minimal role in our end-to-end solution. their only purpose is to seed the initial set of complex how-to queries. in order to find relevant articles for each complex task we trawl through a month's worth of logs from a commercial search engine using the 'how-to' query expansion described in section 2.1. further, in order to protect user privacy we discard queries that were issued by fewer than 5 distinct users. to each remaining complex task query, we associate the top-10 clicked urls across all users for the entire month. text in these webpages satisfy our notion of relevance. finding candidate sentences for sub-tasks next, we create a pool of candidate sentences that we hypothesize might contain sub-tasks. specifically, we use an in-house webpage parser to extract section headings and list items from the set of urls previously collected. these types of text fragments often represent very short summaries that are then elaborated on in 'how-to' articles; we thus attempt to restrict our candidate pool to subtasks that capture the notion of abstraction. finally, we also care about consensus across articles, since this allows us to retain only those sub-task which are cited in different sources and are therefore more reliably important. because the same sub-task can be expressed differently in text we perform clustering on the bert @xcite embeddings of candidate sentences and discard those clusters that only contain a single source url. the remaining set of sentences form our pool of candidate sub-tasks. labeling sub-tasks and dependencies given the set of candidate complex task queries and their associated sub-task sentences, we ask crowdworkers to label them. specifically, we guide annotators through a series of questions: (1) is the candidate query about a task? a complex task? (2) if it is a complex task, which candidate sentences represent sub-tasks? (3) does the ordering of sub-tasks matter? if so, assign pairwise temporal dependency labels to sub-tasks. we ask three workers to label each hit, aggregating annotations by majority vote. ## models recall from section 2 that we model complex task decomposition as a graph induction problem over unstructured text. in what follows we will first describe our approach for sub-task node construction, followed by our method for sub-task temporal dependency inference. ## experimental evaluations in this section we aim to answer the following research questions: rq1 can we accurately and automatically generate sub-tasks given an input complex task? rq2 can we correctly identify the temporal dependencies between sub-tasks? we begin by describing datasets, baselines and metrics used in our evaluation. foot_6 ## conclusion and future work in this paper we have tackled the novel problem of decomposing and organizing a complex task from unstructured text. we devised an end-to-end solution that formulated this problem as graph induction in two stages. the first consisted of finding nodes to represent sub-tasks by parsing multiple 'how-to' articles on the web and extracting key text fragments from them. notably, we framed three desiderata for finding these fragments -relevance, abstraction and consensus -and built a custom neural architecture to encode these properties by extending a state-of-the-art text generation system. in the second stage we designed a crowd-sourcing study to collect a new dataset of complex tasks, -consisting of their sub-tasks and the temporal dependency relations between them -then used this dataset to generate sub-task nodes as well as infer the edges between them. in evaluations of both stages we demonstrated the efficacy of our approach by significantly outperforming the stateof-the-art text generator that we extended. this work opens several avenues for future research. in this paper, we have assumed a complex task as given input; we plan to extend our pipeline with the ability to distinguish complex from sim-ple tasks. this extension in turn will allow us to expand the scope of our current system by allowing for recursive task decomposition and organization. meanwhile, although our novel complex task dataset proved a useful resource for modeling sub-task dependency inference, it remains quite small; we hope to increase its size considerably in future work, in order to make it more useful to the broader research community. we also hope to conduct human evaluations of generated sub-tasks in order to gauge their coherence and utility to complex tasks. finally, we hope to test our system in practical, downstream usage by studying the productivity impact of automated task decomposition and organization on real users in their daily lives.