# A Metaheuristic Framework with Experience Reuse for Dynamic Multi-Objective Big Data Optimization
## 3 The Proposed Framework
### Dynamic Multi-Objective Big Data Optimization-Oriented Framework  
As a metaheuristic software framework for dynamic multi-objective big data optimization, this framework aims to (1) design and apply modern dynamic metaheuristics for solving DMDDOPs, (2) provide a platform for optimization researchers to conduct experimental research while quickly building the whole research workflow, and (3) offer researchers the maximal flexibility to utilize or extend the integrated modules so that they can focus on their concerns without being distracted by other factors or details of implementation. Figure 1 shows the overall framework of JDBDO. JDBDO is designed in a modularized way to maintain flexibility. The framework is split into several modules from the perspective of the modern research workflow. No matter whether algorithms, test problems, or others, the tools provide one or more interfaces and implementations to offer researchers sufficient flexibility to use the integrated algorithms or override the existing algorithms to try their ideas. Thus, they could focus only on the modules of interest and ignore the rest of the framework. In addition, the modules with oblique lines in the background in the figure indicate that they are under development and will be released in the future, and the modules with dotted line borders indicate that they support user customization and extensibility. From a vertical perspective, the framework consists of three layers. The infrastructure layer at the bottom of Figure 1 provides research support. The Hadoop Distributed File System (HDFS) provides a primary data storage environment for the framework, and a fully distributed Hadoop cluster and Spark cluster form the distributed and parallel computing environment. The function of the _Data Server_ is to provide a data engine to organize raw data and manage streaming data for problem updates. In addition, test suites for DMDDOPs, DMOPs, and MOPs are embedded in the _Problem Set_. The _Algorithm Set_ contains different types of metaheuristics composed of dynamic EAs and dynamic SI algorithms. The second layer shows the basic workflow for solving DMDBOPs by JDBDO. The serial numbers in the figure represent the steps for solving a DMDDOP. The _Data Preprocessor_ module extracts update information from the _Data Server_ and updates the problem, and then the dynamic algorithm restarts for the next optimization run. The _Experience Reuse Method_ uses historical information to help the algorithm construct the initial population when restarting. Then, the algorithm solves the new problem. After that, the nearly optimal PS and problem features obtained at the end of every optimization run are used for updatingthe archive in the _Experience Reuse Method_. It should be noted that the _Problem Change Information_ and the _Solving Process Information_ in Figure 1 are two abstract concepts intended to provide the user with an idea for constructing an experience reuse method. In concrete implementation, we adopt the problem feature information and the final population information of this problem as historical experience for updating the archive in the _Experience Reuse Method_. However, users can develop different experience reuse methods based on this pattern without being limited to using the two types of information we adopt. The interface layer tries to provide a user-friendly interface for the integrated methods in the framework. Users can build the whole workflow by configuring the _Optimizer Runner_ module, in which the desired algorithm and test problem can be selected and set. The _Analytic Tools_ module is designed to meet the analytic demands of users in the process of problem optimization. When solving a dynamic problem, users can monitor the nearly optimal PF curve for the current problem with the _Dynamic Visualizer_. Alternatively, users can obtain the nearly optimal PF of a static problem using the _Static Visualizer_ after the algorithm is terminated.