# Task Automation Intelligent Agents: A Review
## 3 Results
### RQ.1
#### 3.1.1 Desktop-Based Task Automation Systems and Intelligent Agents  
The early task automation systems and intelligent agents which were developed were mostly desktop-based. The first task automation system using the programming-by-demonstration approach was Pursuit by [PERSON] [27], which enables users to create abstract programs directly containing variables, loops, and conditionals within the interface. It made programming easy for the users but hardly automated their tasks and needed to be supported by a user study. Being the first of its kind, as a programming-by-demonstration system, Pursuit failed to adhere to standardized usability heuristics, and the user interface was developed based on the developers' experience. One of the main reasons is that [PERSON]'s heuristics were introduced the same year this system was developed, making it hard for researchers to consider usability [28]. SMARTedit, another programming-by-demonstration system, allowed users to automate repetitive text-editing tasks by learning techniques drawn from a machine-learning concept called version space algebra, through which it could learn useful text-editing procedures after only a few demonstrations. The limitations of this system were that it only worked for text editing, and no usability evaluation was performed [29]. Another task automation system that used a similar version of space algebra was CHINLE, which automatically constructed programming-by-demonstration systems for applications based on the interface specifications by [PERSON] and [PERSON] [30]. CHINLE was also unsupported by usability evaluation or user study, built atop SUPPLE, had fixed-length loops, and could not use logical connectives inside conditionals. DocWizard, another programming-by-demonstration system, presented a novel algorithm in the Eclipse platform for automatically capturing follow-me documentation wizards by demonstrations through observing experts performing procedures. The limitation of this system was that it lacked the features for the author to specify where user inputs were required and violated several usability principles, e.g., visibility of system status, user control and freedom, and error prevention [31]. FlashFill was inductive programming to create opportunities for task automation for non-programmers through synthesizing functional or logic programs for general-purpose tasks. However, the major challenges of this research were compositionality (i.e., requiring more usability studies) and making inductive programming more cognitive (i.e., ease of use or a simple and minimalistic design). Other challenges were domain change, validation, and noise tolerance [32]. Photo manipulation tutorials, where the author demonstrates the manipulation using an instrumented version of GIMP that records all the interface and application state changes. The system automatically generates tutorials from the recordings that illustrate the manipulation using images, text, and annotations [33]. These tutorials are not supported by usability evaluation and therefore have no feedback and error correction method; the semantic tool is based on computer vision and needs to learn macros from multiple demonstrations rather than one and to generalize it. Another GUI-based task automation system is Sikuli, which uses a similar visual approach to search and automate GUIs using screenshots. No usability evaluation was performed at any stage of development; therefore, it had the following issues and limitations: it did not work as expected due to theme variation and background changes, and it also had visibility constraints because it operated on visible screen elements and did not accept invisible GUI elements, e.g., hidden underneath other windows, tabs, or scrolling out of view [34]. Another GUI-based intelligent agent is Bespoke, a system synthesizing custom GUIs by observing user demonstrations of command-line applications.