# A Survey on Malleability Solutions for High-Performance Distributed Computing
## 4 MPI Malleability
### Flex-MPI  
Flex-MPI [30] is a library, built on top of MPICH [18], to improve _Single Program Multiple Data_ (SPMD) application performance using a performance-aware dynamic reconfiguration where users set performance objectives and application constraints. When an application is executed using Flex-MPI, information related to performance is automatically gathered and analyzed to decide if a reconfiguration would benefit the performance. Furthermore, Flex-MPI also takes care of load-balancing issues, selecting how data will accordingly be partitioned and redistributed among processes. Flex-MPI is composed of the following modules, which work together under a reconfiguration policy:  
* Monitoring: Takes care of gathering data from the application using the library PAPI [31] to retrieve information from hardware counters, and PMPI to collect information from MPI calls. Metrics are collected for each process and are preserved during a context switch. * Computational prediction model (CPM): Predicts how the application will perform from gathered data when executing on different processor configurations. The result is the expected computation and communication time for the following iterations. It also calculates the expected time for the creation/finalization process and data redistribution. * Dynamic process management: Controls process creation and termination. Processes are divided between _initial_, which are in the first set of processes created, or _dynamical_, which are created in execution time. Flex-MPI will always have at least an _initial_ number of processes working and this group can not be terminated. To create a process, the function MPI_Comm_spawn is called with an MPI_Info object to select where the process will be bound. Processes are created one by one to allow fine-grained control over the processes' number and, therefore, satisfy performance constraints. This makes the termination of processes different to the _initial_ ones easier; therefore only _dynamic_ processes can be removed [30]. A new process is created with a new inter-communicator connected to the main group of processes, which in turn, is merged into an intra-communicator with the function MPI_Intercomm_merge. So, a new main processes group is started where the recently created process is added, knowing this operation as a \"merge\" method [32]. The intra-communicator in Flex-MPI is called XMPI_COMM_WORLD and is updated after each process creation or finalization. Processes are created only if these criteria are met:  
* There are idle processors in the system. * Current performance does not meet the user-given performance objective. * CPM predicts a performance improvement to satisfy the user-given performance objective. * In order to remove a dynamic process, XMPI_COMM_WORLD is released and recreated without the process that is being terminated, which ends the compute and calls MPI_Finalize. The rest of the processes recreate XMPI_COMM_WORLD by invoking the API function MPI_Group_incl, without the removed process. Processes are removed only if all this criterion is met:  
* Current performance does not meet the user-given performance objective. * CPM predicts a performance reduction that satisfies the user-given performance objective. * Load balancing: Ensures that data are partitioned among processes having a workload according to the processor they are bound. This module is based on techniques described at [33; 34], and creates a partition concerning the computing power of each processor. * Data redistribution: Redistributes data according to the model set by the load balancing module. The distribution is performed completely by this module and users do not need to describe the data structures; however, Flex-MPI only handles the following types of data: one-dimensional, two-dimensional, and dense or sparse matrices with block-based one-dimensional domain decomposition. The reconfiguration policy, which controls the previous modules, follows these steps to decide if a resize is necessary:  
1. Activation of the monitoring module. 2.