Analysis of Control and Monitoring of RPA bots in Academic Research and RPA Platforms
Creators: name: Kurowski, Kelly
name: Martínez-Rojas, Antonio
name: Reijers, Hajo A.
Publisher: name: Zenodo
Year: 2024
Subjects: Robotic Process Automation
Control and Monitoring
RPA
Software Robots
Review

description: Abstract: This resource presents the analysis derived from a systematic literature review on control and monitoring of software robots in Robotic Process Automation (RPA). We focus on features such as monitoring, scheduling, exception handling, logging, and information display, aiming for a cross-platform and cross-study comparison.

Analysis

In this resource, we detail the outcomes from our systematic literature review [9] focusing on the control and monitoring of RPA bots. Our analysis encompasses both academic studies and technical documentation from leading RPA platforms. We specifically target the features critical for effective RPA management:



Monitoring: The capability for real-time oversight of both system performance and overall health.

Scheduling: The mechanisms available for planning and executing robot tasks.

Exception Handling: Strategies for the management and resolution of unforeseen issues or errors.

Logging: The process and standards for recording activities, messages, or incidents throughout the lifecycle of RPA bots.

Information Display: The methods and tools used to present essential data to users or developers, aiding in decision-making and oversight.


These dimensions serve as the basis for a nuanced comparison across different platforms and research findings.

Academic Research Analysis

We have examined five prominent academic contributions, categorized based on their nature into literature reviews [4], case studies [6,8], and technical proposals [5,7]:



Hartikainen et al. [4] proposed a multivocal literature review to explore what options there are to improve the maintenance of software robots in large-scale robot environments. They developed eleven functional requirements related to monitoring and maintenance and validated these in a real-life context.

Hwang et al. [5] designed a middleware system called MIORPA to control open-source RPA robots. The system offers a job-scheduling algorithm for the distribution of tasks to multiple RPA robots operating across various middleware environments, while also offering a watchdog-based monitoring feature for real-time tracking and management of the status of RPA robots.

Kedziora et al. [6] provide a case study of how robotic process automation governance is managed within Nordea, a major banking group in the Nordic region. Specifically, the study focuses on identifying the maintenance practices of the Operations Team, which include software monitoring, handling scheduling tasks, and addressing simpler bug-fixing duties.

Martínez-Rojas et al. [7] proposed AIRPA, which is a software architecture that facilitates the separation of robot development from AI development. A central component of AIRPA architecture is the RPA Control Room, through which users can oversee and orchestrate the entire lifecycle of robots, with special attention to those robots based on AI models; it also offers a visual representation of metrics and data associated with process execution.

Šimek et al. [8] conducted a case study into the HR onboarding process within two companies where a Robotic Service Orchestration (RSO) platform was used alongside RPA technology. Their aim was to evaluate the benefits of the RSO platform, which facilitates the delivery, management, and execution of business processes regardless of whether they are carried out by a digital or human workforce.


The detailed analysis of the academic approaches can be seen in Table 1 - Academic Approaches (1st sheet of attached Excel). We will briefly discuss the analysis by reflecting on the various features, as expressed in the columns.



Platform: In the Platform column, there are references to RPA platforms such as UiPath Orchestrator and Blue Prism, which indicate the connection ofthese academic works with industry-strength technology. However, it should be noted that none of the proposed technological proposals in any of thestudies have been integrated into such a platform. Either a new proposal was built on top of open source technology, like in the case of Šimek etal. [8], or an entirely new technological environment had to be developed to evaluate the new proposal, such as is the case for the work by Martínez-Rojaset al. [7].

Scheduling: What is highly interesting is that all of the academic approaches describe the planning of robotic jobs as an act that is mostly or even completely manual. Even though in the case study by Hwang et al. [5] and the proposal by Šimek et al. [8] there are references to business and priority rules that relate to scheduling, these still are part of an overall manual approach to plan when the robots work.

Monitoring: Most approaches portray monitoring as a manual task that is based on automatically generated information. Such information is either provided on a fine-granular level (Status), which means that the status of single robots is available, or on an aggregated level (Performance), which means that the information is provided over the whole robotic workforce. For example, in the study by Hartikainen et al. [4], the CoE maintenance team monitors the performance of the robots using tools like UiPath Orchestrator and manually ensures optimal virtual workforce usage in an MS Excel spreadsheet. While information on aggregated performance aspects (e.g., workflows, faulted tasks/robot status, data structure, and data quality) is being provided for monitoring purposes, it is worth noting that the actual monitoring is accomplished manually. There is also one approach that is scored as completely manual, namely the study by Kedziora et al. [6]: Operations Team members seem to manually observe the robots, draw their conclusions, and take action, but the study does not mention the monitored parameters.

Exception Handling: Also with respect to how exceptions are being handled, we can observe the prevalence of manual methods in the academic approaches. In Martínez-Rojas et al. [7], for example, KOs are manually handled in the Control room; In Kedziora et al. [6], it is the Development Team that is responsible for manually reducing the number of exceptions. By contrast, Šimek et al. [8] describe a sophisticated approach where exception handling models are generated during the testing phase, which were then integrated into the real-time version by considerably reducing exceptions.

Logging: As to the logging features of the academic approaches, only three of the approaches provide sufficient information to make an assessment. These three emphasize different types of information that are logged: Hwang et al. [5] describe how the status of separate jobs are recorded, while the study of Hartikainen et al. [4] shows how, for example, the operating environment in which the robots work is recorded as well. Of particular interest is the study by Martínez-Rojas et al. [7], which proposes to log information on how robots have transitioned from state to state.

Information display: Similar to the logging aspect, only three of the approaches provide sufficient information for an assessment. In Hartikainen et al. [4], the Center of Excellence (CoE) creates a dashboard to highlight the quality and benefits of RPA by leveraging software robotic log files. The monitoring tool functions as a dashboard showcasing the benefits, costs, and quality of robot jobs calculated from the robot log files. Martínez-Rojas et al. [7] present a visualization of metrics and data associated with process execution, including plots and statistical data. Šimek et al. [8] use a console for monitoring and measurement provided by Enate, where performance indicators, similar to UiPath, are displayed.


RPA Platforms

Three leading RPA platforms are analyzed. We will briefly characterize these here and subsequently present a feature analysis of each of the platforms.



UiPath [3]: UiPath Orchestrator enables dynamic workload distribution to unattended robots through queues. The platform provides overviews of job and process statuses in real-time, with a focus on highlighting processes executed with errors. It also logs critical information such as time, log level, process details, hostname, host identity, and messages generated by robots. Additionally, it offers dashboards that visualize performance indicators, forecasts, and error alerts.

SS&C Blue Prism [2]: The Blue Prism Control Room provides users with the ability to view and manage Blue Prism activities for all RPA environments in use. In the control room, it is possible to monitor the status of work queue items, as well as the so-called Digital Workers. Within each environment, it is possible to view, filter, and organize session activities.

Automation Anywhere [1]: The Automation Anywhere Control Room provides APIs that enable one to customize the behavior of the software robots.It can be used to manage bot deployments, create and manage user accounts, roles, and queues. The Bot Insight dashboard provides customizable widgets for specific scenarios, including chart filters, chart style, metric-based filters, and date grouping options. The Audit Log records a table of a variety of actions performed by users, including status, time, event type, item name, event started by, source device, and source.


The detailed analysis of the RPA platforms is provided in Table 2 - RPA Platforms (2nd sheet of attached Excel). From the table, it can be derived that BluePrism and Automation Anywhere are very similar to each other, in particular about scheduling, logging, and information display. To highlight the differences, we will review each of the features in more detail.



Platform: Two out of three RPA platforms use a pay-to-use license. This makes it more difficult for any researcher to integrate their technology with these.

Scheduling: Two out of three RPA vendors use automatic updates as a scheduling technique. The BluePrism Control Room manages all scheduling and automatically picks up subsequent updates to bots without altering automation schedules. The Automation Anywhere Control Room automatically and intelligently picks up the subsequent updates to bots, without any need to alter automation schedules. In contrast, Uipath Orchestrator distributes workload to unattended robots and optimizes robot usage through dynamic workload distribution using Queues.

Monitoring: The UiPath Orchestrator provides real-time metrics for monitoring the health and state of the system, including workflows, faulted tasks/robot status, data structure, and data quality. Blue Prism monitors schedules, tasks, work queue items, and Digital Workers. In Automation Anywhere, the Bot Insight analytics platform provides real-time insights and operational intelligence for business processes through dashboards.

Exception Handling: UiPath highlights processes executed with errors, whereas Blue Prism does this with exceptional tasks. In Automation Anywhere, one can analyze log files that hold information about the root cause of the error.

Logging: All RPA vendors log the start and end times of the robot jobs, but there are differences in what other information is logged. UiPath logs the message of the robots, which is considered to be conceptual information (unlike robot execution ID for instance), which can contribute to a better overall understanding of the process.

Information display: Two out of 3 RPA platforms do not prioritize explicit performance indicators – UiPath being the exception.


The results from these analyses will be used in the associated publication [9] to formulate a set of recommendations for industry and academia.

References



Automation Anywhere Control Room Documentation. https://docs.automationanywhere.com/bundle/enterprise-v11.3/page/enterprise/topics/control-room/getting-started/using-control-room.html

BluePrism Control Room: Hub 4.6 and Control Room 4.6 user guide. https://bpdocs.blueprism.com/hub-interact/4-6/en-us/home-control-room.htm

UiPath Orchestrator Documentation. https://docs.uipath.com/orchestrator/standalone/2023.4/user-guide/introduction

Hartikainen, E., Hotti, V., Tukiainen, M.: Improving software robot maintenance in large-scale environments—is center of excellence a solution? IEEE Access 10, 96760–96773 (2022). https://doi.org/10.1109/ACCESS.2022.3205420

Hwang, M.H., Na, U.K., Lee, S., Cho, B., Kim, Y., Lee, D., Shin, J.: Miorpa: Middleware system for open-source robotic process automation. Journal of Computing Science and Engineering 14(1), 19–25 (2020)

Kedziora, D., Penttinen, E.: Governance models for robotic process automation: The case of nordea bank. Journal of Information Technology Teaching Cases 11(1), 20–29 (2021). https://doi.org/10.1177/2043886920937022

Martínez-Rojas, A., Sánchez-Oliva, J., López-Carnicer, J.M., Jiménez-Ramírez, A.: Airpa: An architecture to support the execution and maintenance of ai-powered rpa robots. In: Business Process Management: Blockchain and Robotic Process Automation Forum: BPM 2021 Blockchain and RPA Forum, Rome, Italy, September6–10, 2021, Proceedings. pp. 38–48. Springer International Publishing (2021)

Šimek, D., Šperka, R.: How robot/human orchestration can help in an hr department: A case study from a pilot implementation. Organizacija 52(3), 204–217 (2019). https://doi.org/10.2478/orga-2019-001

Kurowski, K., Martínez-Rojas, A., Reijers, H.A. (2024, May). Control and Monitoring of Software Robots: What Can Academia and Industry Learn From Each Other?. In International Conference on Research Challenges in Information Science. Cham: Springer International Publishing.