Interactive Tools (IT) in Galaxy
Creators: name: Kayikcioglu, Tunc
name: Srikakulam, Sanjay Kumar
name: Maier, Wolfgang
name: Grüning, Björn
Publisher: name: Zenodo
Year: 2023
Subjects: Interactive Tools, Galaxy, workflows, notebooks, Jupyter, R Studio

description: Galaxy is a GUI-based scientific data management and analysis platform that aims to expand the target audience of scientific computing to scientists without advanced computer literacy or access to computing infrastructure. During the operation of a typical asynchronous workflow, the interaction of the user triggers a job submission to a hosted computing system, the results of which are then reported back to the user upon completion of the task. However, such an asynchronous interaction scheme is not compatible with scientific software that require user interaction during the execution itself (synchronous), e.g. many data visualization software. Interactive Tools on Galaxy (IT) leverage tool and job components of Galaxy by taking advantage of the latest containerization solutions to provide access to interactive executable software. With ITs, Galaxy is enabling the more and more common use-case of asynchronous workflows while keeping the possibility to bridge at any time to synchronous workflows and reusing all the other Galaxy data management features. Dynamic visual analytics applications, such as those using R Shiny, Dash or Ruby on Rails can be incorporated into Galaxy through an extension of the standard Galaxy tool framework. A developer needs to provide a container image (e.g., Docker) of the underlying application and a Galaxy tool specification (a.k.a. ‘wrapper’). The existing job management system of Galaxy handles scheduling the execution of the container using its existing job management system. Interactive access through a built-in proxy is provided by presenting the user an auto-generated URL via the usual Galaxy GUI, which is forwarded to an internal port bound by the application of interest. The application will start in a user-specified configuration with access to input datasets. Users can then use the application interactively and commit the results of their work in the form of regular Galaxy output datasets. ITs can be seamlessly integrated into Galaxy: If the IT has been executed as part of a workflow, steps downstream of the IT will be scheduled once the expected outputs from the IT (whether generated through automated execution or through a human in the loop) become available. Irrespective of whether it is part of a Galaxy workflow, the saved efforts of the user will be recorded within their analysis history, and the exported ‘notebooks’ (the Jupyter ‘scripts’) can be shared with others as needed just as any other dataset. Such ITs provide a reproducible and resumable computing system which can leverage not only the computation infrastructure, but also the storage system. As our scientific knowledge deepens, cross-disciplinary collaboration is becoming increasingly important for discovery, because it is rarely possible using only one set of skills. Galaxy currently supports sharing of research assets, such as workflows and histories, with other users and even with non-Galaxy users. However, these sharing capabilities are more limited to sharing a snapshot of the work rather than enabling virtual real-time collaboration. In contrast, the IT framework enables real-time visual analytics collaboration among authorized users. As such, we expect ITs to have a similar enabling impact on collaborative teams as the multi-access office suite tools that make collaborative editing of research outputs possible. It is important to note that these simultaneous collaborative capabilities are being provided solely by our framework and do not require any modifications to the individual software for support at all. As a case study, consider a user developing a simple machine learning model for the first time, a task involving a series of commands executed on some test data sets. Galaxy enables those exploratory research scenarios for example with the integration of Jupyter Notebooks, a popular open source web application that can be used to create and share documents containing source code and its products. A local deployment or usage of such an online resource would potentially present technical and security concerns and prohibit common usage scenarios such as institutional policies or hardware with restrictive configurations. Transfer of training data would also require the user to transfer large amounts of data over the network. The user would also likely need to purchase and/or configure a suitable graphics processor to be able to fully test her code and the model. ITs alleviate all of these issues as the users would immediately gain access to a pre-configured system with access to their other data in the Galaxy storage system. Up to this date, our public instance, https://live.usegalaxy.eu, already had 20,000 IT jobs executed. We think that ITs can provide an opportunity to integrate much more interactive scientific use-cases and visualizations and will increasingly gain popularity in the community. In this talk we will walk through the integration and practical usage of the ITs with the aim of gradually increasing interest in deployment of new ITs and supporting more interesting and diverse use-cases.