dataesr/scientific-paragraphs-categorization huggingface dataset:
The interest in the ComBat approach was also evaluated in the recent study of Da-Ano et al. 35  , where four versions of the nonparametric ComBat were compared in their ability to harmonize radiomic features in a multicenter context, including two clinical datasets. The first dataset was composed of 119 patients suffering from locally advanced cervical cancer and contained MR and PET images from three different centers. The second involved 98 patients with locally advanced laryngeal cancer from 5 centers who underwent contrast-enhanced computed tomography. Among the four versions, one version identified a reference center, in addition to the conventional version, on which radiomic features were transformed. The other two versions used conventional versions, but with the addition of Boostrap and Monte-Carlo strategies for improved robustness in the estimation.
They showed that all four versions of ComBat showed a contribution in removing machine differences, and improving the prediction performance of the given outcome. In addition, the version using a reference site gave the best results. For example, Modified ComBat resulted in a 6% improvement in balanced accuracy compared to untransformed data for the random forest algorithm in the prediction of local failure in locally advanced cervical cancer. When using ComBat in the 5 centers dataset, they were confronted with the fact that the machine parameters were very heterogeneous. Following this observation, they would have had to manually assign a batch to each image, leading to more than 15 labels, which they did not consider realistic due to the limited number of patients. We have shown from Figs.
3 and S1 that there is limited sense to affect to a same batch images coming from a single center but for which devices or acquisition parameters differ, even though centers tend to harmonize meaningful parameters in terms of image interpretation as shown earlier. This study, therefore, highlighted the urgent need to define an alternative for batch assignment, as already mentioned. For this purpose, Da-Ano et al. 35  proposed an unsupervised hierarchical clustering technique applied directly to radiomic features. Using this technique, they were able to correctly cluster the patients in the dataset from the three centers with homogeneous acquisition parameters per center into three different clusters. Only one patient was misclassified. Then, they applied clustering to the dataset with heterogeneous parameters to establish the ComBat "batch" labels.
e. , the mean and variance). Our code available at the following address https:// github. com/ Alxal ine/ ComSc an answers this problem by following the philosophy of scikit-learn with a fit and transform function. The hardest part in using Combat is that there are not always ground truths about the batch labels, in particular in the case of very heterogeneous data as it is in a multicentric context. The advantage of using clustering to determine the batch is that it becomes possible to know whether the imaging data not seen during the training stage lies outside the distribution of the training data. This does not solve the generalizability problem in a general way but gives an idea of the space in which the imager must be located for a developed radiomic signature to be applicable. This study poses some limitations.
As well, ComBat was applied in a very heterogeneous context, with stratification by institution, rather than by MR provider/software/sequence parameters, which is a highly unfavorable case, even if already used as such in recent studies  40. We did not consider the discretization step as a variable parameter and have fixed it to a fixed bin width  29, 67. For the classification step, we did not try to establish the best model but put the emphasis on understanding the influence of each strategy on the radiomic features harmonization; that is why we have created separate models with either first-order or second-order features. Furthermore, we did not evaluate the shape features, which can also be affected by the acquisition parameters. Finally, the potential of AutoComBat should be further investigated for other datasets and other clinical tasks.
Acknowledgements We would like to acknowledge The  Cancer Genome Atlas  and  The Cancer Imaging Archive. This work was supported by  ITMO PhysiCancer , the  Fondation pour la Recherche Médicale  (FRM; No. DIC20161236437 ),  Amazon Web Services (AWS) , and by the  French Government  under the  Programme d'investissements d'avenir. Figure 1 has been designed using resources from https:// www. flati con. com.
In this paper we introduce ATLAS, a model-driven, toolsupported framework for the systematic engineering of MRS that facilitates the exploration and tradeoff analysis of candidate MRS designs and CI algorithms. Driven by insights derived from recent robotics surveys  [14, 16] , ATLAS is underpinned by the following principles. First, the ATLAS domain-specific language (DSL) enables the specification of (i) the MRS mission, including both functional and non-functional requirements; and (ii) the characteristics of individual robots comprising the MRS, including architec-ture, internal behaviours, and capabilities (e. g. , use of energy efficient or reliable sensors). Second, the ATLAS code generation engine consumes the MRS mission and system specifications, and produces the necessary infrastructure (i.
• A prototype open-source ATLAS tool and case study repository, both available on our project webpage at https://www. github. com/jrharbin-york/ATLAS. The design space exploration provided by ATLAS can be used to iteratively refine the system design as part of its systematic process. For example, ATLAS can show through tradeoff analysis that certain CI algorithms are not viable for the target robotic missions, or that some software components could lead to system configurations that do not satisfy the mission requirements. Therefore, ATLAS can support a cost-benefit analysis between CI choice and mission requirements. Although we do not attempt to trace the underlying cause of violations, visual inspection via the the user interface provided by the simulator provides a mechanism for error tracing and performance analysis.
Overview The high-level ATLAS architecture is depicted in Fig. 2. At its core, lies the ATLAS DSL (Sect. 3. 2) that enables the specification of the robotic's team structure and mission objectives, complemented by a model-driven code generation engine for the specialisation of a middleware (Sect. 3. 4), a simulator-specific logical interface, and CI templates that facilitate tradeoff analysis of CI algorithms (Sect. 3. 5). The middleware facilitates the communication between MRS team members and between the collective intelligence algorithms. These algorithms implement the high-level command and control which is responsible for the completion and adequate performance of the mission. For instance, for our healthcare use case from Sect. 2, the CI is responsible for issuing the initial tasks to be performed by each robot or for reassigning tasks when a team member has a low battery level.
The low-level logic of robotic team members is implemented using MRS implementation-specific functionality. Low-level behaviours, including station keeping, inter-robot avoidance and path navigation, are mediated by state held in the target robotic platform (e. g. , in publish-subscribe databases). Therefore, although Fig. 2  illustrates the highlevel communication, the yellow MRS nodes on the right contain additional state and communication flows which are specific to the target robotic platform. This key ATLAS feature of using a middleware to coordinate the high-level CI command and control exploits the modular architecture of modern robotic systems and leverages low-level support from the publish-subscribe communication protocol underpinning widely used robotic platforms including ROS  [11]  and MOOS-IvP  [12].
We note that, in principle, domain experts and system testers could employ other formalisms than the ATLAS DSL to represent these dimensions of their robotic mission and architecture. However, the following steps, including code generation and middleware interfacing, are fairly integrated with this specification. As expected, a corresponding set of model-tomodel transformation activities must take place to produce the required artefacts. Since this activity is not within the scope of this work, we leave it for future research. The invocation of the ATLAS code generation engine in Step 2 uses the full information in the supplied models to automatically generate (i) a middleware configuration that enables the communication of the various system components; (ii) CI templates that will be used by the middleware to orchestrate the robotic team; (iii) interface code that enables the middleware to communicate directly with the target robotic simulator and extract the required information from the simulators; and (iv) the necessary configuration files for the target robotic simulator.
In our prototype implementation, these transformations are performed using the Epsilon Generation Language (EGL)  [22]. Fig. 3  Stepwise MDE ATLAS methodology Next, in Step 3, engineers implement the logic and coordination of the robotic team. To achieve this, they populate the CI templates generated in Step 2 with suitable code that realises their selected CI algorithms. This separation of concerns between the implementation of the high-level behaviour (demonstrated by the CI algorithm) and the lowlevel functionality of a robot has two key benefits. First, it reduces the effort to analyse multiple CI algorithms without changing the underlying low-level system behaviour. Second, it enables reuse of already available low-level code developed for individual robotic team members that has been developed independently.
Our current ATLAS implementation supports the generation of CI templates in Java. Further details about the CI implementation, alternative implementation strategies and choices for development language are presented in Sect. 3. 5. Once the necessary artefacts have been instantiated, the MRS simulation is automatically executed in Step 4. ATLAS records simulation results related to the MRS mission requirements in the form of logs comprising both messages exchanged between system components and events relevant to system goals and requirements that occurred during the simulation. The analysis of these results permits the computation of metrics that support the selection of the most effective MRS designs and the identification of the most suitable CI algorithm for the target mission.
DSL core concepts The primary concepts of the ATLAS DSL consist of the Mission (Fig. 4 ) and the VariationProgram (Fig. 5 ). The Mission part includes the Goals that should be accomplished. Each goal is connected to the Region where it takes place and includes the specific GoalAction that should be executed for the fulfilment of the goal. Example actions include Patrolling an area, Avoiding an obstacle, etc. Although the DSL covers the vast majority of actions available in the current ROS and MOOS robotic environments, this is an extensibility point of the DSL. Accordingly, interested users can extend the GoalAction type to introduce new actions and the appropriate model-to-text (M2T) transformations to generate the corresponding robotic simulator specific implementations, which may be required for new use cases.
As these are scenario-specific, the implementation of the action per goal is delegated to the user in Step 3 of the ATLAS methodology (cf. Fig. 3 ). For example, the GoalAction named TrackDistances will track the relative distances of robots to each other and check for possible intersection with environmental objects. This information is recorded by the middleware allowing the generation of related metrics for further analysis in Step 5. Robotic simulators provide bespoke implementations of various behaviours such as navigating to specific coordinates (e. g. , using the MoveBaseGoal and WayPoint behaviours in ROS and MOOS-IvP, respectively), and avoiding other robots and obstacles. These behaviours can be employed by low-level simulator code to facilitate the specification and execution of a robotic mission.
Searching for optimal robot configurations Another key feature of the ATLAS DSL is the support for analysing candidate system configurations through design space exploration. To achieve this, ATLAS leverages concepts from the domain of software product line engineering  [23] [24] [25] , enabling the specification of alternative candidate designs as a VariationGroup with a defined cardinality (see Fig. 5 ). The members of a variation group are components, at the same level of the system specification hierarchy, that can participate in the mission. Accordingly, variation groups can be formed by SystemComponent subclasses such as robots and their subcomponents. For example, if there are different types of Sensor or Battery that a robot can use for a mission, those subcomponents can form another variation group.
To this end, ATLAS leverages recent MDE advances in model similarity estimation through robust hashing  [18]  and uses simulated annealing  [27]  to guide the selection of the most promising and informative model configurations. Robust hashing of MDE models  [18]  builds on techniques developed for the comparison of large datasets and enables the estimation of model similarity considering both the contents of model elements (e. g. , attributes, operations) and their relative position with regards to other model elements. Given the set of models to compare, we use the robust hashing technique to produce a similarity matrix between these models. The similarity score between a pair of models ranges from 0 (completely dissimilar models) to 100 (identical models). For further details, we refer interested readers to  [18].
g. , the components can be computing platform and programming language independent). Irrespective of the target robotic simulator, the ATLAS middleware comprises (i) a simulator-specific interface that enables ATLAS to connect to the simulator, and subscribe and publish messages/topics; (ii) a highly efficient message broker (e. g. , ActiveMQ  [34] ) to facilitate fast inter-robot communication and interaction with the CI algorithm; and (iii) a CI mapping software module that converts the high-level CI commands into message/topic changes for the target simulator. ATLAS uses the mission model (e. g. , Fig. 6 ) and automatically generates these components through a sequence of M2T transformations. The current ATLAS implementation fully supports the Gazebo/ROS and MOOS-IvP simulators. Clearly, for each target simulator the M2T transformation and required components are developed only once and can be reused thereafter.
We discuss below the concrete instantiations for these two simulators. The full details of the M2T transformations are available on our project webpage. ATLAS middleware for MOOS-IvP: The M2T transformation uses the mission model and configures appropriately the ATLAS middleware to enable monitoring the MRS status for the specified goals. Also, MOOS-specific configuration files are produced to represent the robot configurations, properties and behaviours necessary to run the simulation. Within MOOS-IvP, each robot is represented as a community comprising a set of C++ software modules that provides the functionality of robot components. Each community is served by an individual publish-subscribe database (MOOSDB) which contains key-value pairs. When robots communicate, these key-value pairs form a communication channel through which one robot will publish a message and subscribed robots will receive the update and act accordingly.
Example messages that can be sent/received include sensor detection events, speed values and location information. The communication between the ATLAS middleware and the simulator occurs through MOOSDBInterface, a generic MOOS software component that enables interfacing directly with robot communities. The simulator did not support this functionality. Thus, we developed this reusable software component which can now be instrumented through the middleware to publish/subscribe to messages within MOOSDB databases, e. g. , receiving updated robot coordinates, activating the return home behaviour upon mission completion. Messages received by the MOOS-IvP simulator are kept in the message broker (ActiveMQ) until they are automatically processed and translated into update nodes mapped to the initial mission goals.
6 ). Engineers are expected to specialise those templates with the appropriate logic to develop the target CI algorithm (cf. Step 3 in Fig. 3 ). Although this CI generation facility is provided for convenient development, the decoupling of the CI into a separate process provides considerable flexibility. If system testers wish to exploit CI algorithms already implemented in other languages, it would be possible for users to use a custom CI instead of ATLAS-specific generated stubs. To use this external CI, the users would need to provide parsing and communications mechanisms by providing an API to their custom CI and to interface it to the middleware. Further discussion of this functionality is presented in Sect. 4. 1. The communication of fully fledged CI instances with the ATLAS middleware is underpinned by the inversion of control programming paradigm  [35].
ATLAS prototype tool The prototype ATLAS model-driven tool uses the Epsilon family of languages  [22]  to perform the MDE tasks. In particular, the DSL is implemented with EMF  [37]. We use Epsilon's EMF Model Java API to check the variation groups and produce the candidate MRS designs (Sect. 3. 3) and the Epsilon Generation Language to generate the ATLAS middleware (Sect. 3. 4), the configuration files needed by the target simulator and the CI template (Sect. 3. 5). We also use ActiveMQ  [34]  to link the MRS simulator, CI algorithm and the middleware. The open-source ATLAS source code and the full experimental results summarised next are available at https://www. github. com/jrharbin-york/ATLAS. Finally, a video showing the execution of ATLAS in a marine case study that involves a team of unmanned underwater vehicles (UUVs) (see Section 5.
Code generation We provide below further information about the code generation functionality provided in ATLAS. Simulator Interface Generation: The code for the simulator interface is automatically generated using EGL, which consumes the model and generates a loader interface to con-figure the middleware for the target use case. This loader contains instantiates objects from the model, representing the configuration of robots and sensors etc. to be loaded by the middleware. A Mission object is returned from the loader class, which the middleware uses internally to maintain the state and support further processing. Collective Intelligence Generation: Templates are also generated for the collective intelligence algorithms using EGL, based upon the information provided in the model. ATLAS users should populate these templates with suitable code that delivers the expected functionality (e.
g. , see Listing 1). Currently, we assume that in the ATLAS architecture, the CI runs centrally upon a single computer which receives messages from the robotic team and coordinates the CI (i. e. , the CI is not distributed). Users interested in exploiting CI algorithms already available in other programming languages (e. g. , C++, Python) could achieve this by using their custom CI instead of the generated templates. However, parsing and communication mechanisms (e. g. , a client-server architecture) must be provided to enable the interaction between their custom CI and the middleware.
Case studies We followed the standard practice in empirical software engineering  [38, 39]  and evaluated ATLAS using three distinct case studies: one from the domain of healthcare robotics using ROS/Gazebo  [11]  and two other from the domain of UUVs using the MOOS-IvP robotic simulator  [12]. More specifically, the three case studies selected are: (1) the healthcare robotic mission presented in Sect. 2; (2) an object detection UUV mission developed by the ATLAS team and described in Sect. 5. 2. 1; and (3) a variant of the Bo-Alpha mission  [40]  developed by the MOOS-IvP community and included within the standard package of the simulator, which we describe in Sect. 5. 2. 2.
Experimental setup We performed a wide range of experiments for the healthcarebased use case described in Sect. 2 using the ROS simulator and the two UUV-based use cases (Sects. 5. 2. 1 and 5. 2. 2) using the MOOS-IvP simulator. The experimental setup for the healthcare use case involved running ATLAS and its supporting scripts for both CI variants (standard and energyaware) and all 60 possible configurations described in the variation group example from Sect. 2. Similarly, we executed a corresponding number of experiments for the object detection and Bo-Alpha UUV missions. The experimental setup in these instances involved running ATLAS and MOOS-IvP in both missions for the two CI variants (standard and advanced; standard and energy-based, respectively) and all possible configurations  (48 and 16, respectively).
Moreover, results were derived by automatically analysing the simulator-produced logs and reducing potential user bias. We also support reproducibility of our findings by making our analysis scripts available in the project's repository. External validity threats could reduce the generalisation of ATLAS. We reduce this threat class by using established MDE practices and tools including EMF  [37] , Epsilon  [22]  and ActiveMQ  [34]. The experimental evaluation involved three case studies (one provided by the MOOS-IvP community), while the use of two distinct robotic simulators reduces further this threat. Finally, the evaluation of ATLAS both in Gazebo/ROS and MOOS-IvP demonstrates the generality of our framework and its capability to support multiple robotic simulators. However, further experiments are needed to establish the applicability, feasibility and scalability of ATLAS in domains and applications with characteristics different from those used in our evaluation.
Model-driven engineering for robotics Developing model-driven solutions for the robotics domain is an established area, which has produced several results over the years  [42] [43] [44]. The majority of the proposed domainspecific modelling languages deals only with specific robot functions such as perception or control, while there are some model-driven toolchains like RobotML  [8] , BRICS  [10] , SmartSoft  [7] , and Robochart  [9]  which provide multiple modelling notations to be used together when developing a robotic system. For a detailed description of different approaches to model-driven engineering of robots, the reader is referred to  [16]  and  [45]. Despite the available literature on the application of MDE to robotics, the engineering of MRS is still inadequately investigated.
Model-driven engineering has also been used in the context of design space exploration. Model-driven approaches specify the design problem as a set of models and model transformations and generate input for other tools that perform the space exploration. The Gaspard Framework  [62]  focuses on the design of massively parallel embedded systems. It uses automatic model refinement and model transformations to allow for design space exploration to evaluate performance characteristics through simulations. Neema et al. [63]  propose the DESERT tool suite, which supports constrained-based design space exploration and model synthesis for embedded systems. Moreover, the OCTOPUS toolset  [29]  supports the design of software-intensive embedded systems and provides an intermediate representation for the specification of design problems.