# Two-Level Algorithm Combining Bayesian Optimization and Swarm Intelligence for Variable-Size Optimal Layout Problems

Juliette Gamot, Mathieu Balesdent, Arnault Tremolet, Romain Wuilbercq,

Nouredine Melab, El-Ghazali Talbi

► To cite this version:

Juliette Gamot, Mathieu Balesdent, Arnault Tremolet, Romain Wuilbercq, Nouredine Melab, et al. "Two-Level Algorithm Combining Bayesian Optimization and Swarm Intelligence for Variable-Size Optimal Layout Problems." GECCO '23 Companion: Companion Conference on Genetic and Evolutionary Computation, Jul 2023, Lisbonne, Portugal. pp.139-142, 10.1145/3583133.3596997. hal-04381343

HAL Id: hal-04381343

https://inria.hal.science/hal-04381343

Submitted on 27 Jun 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Two-Level Algorithm Combining Bayesian Optimization and Swarm Intelligence for Variable-Size Optimal Layout Problems

Juliette Gamot, Mathieu Balesdent, Arnault Tremolet, Romain Wuilbercq,

Nouredine Melab, El-Ghazali Talbi

► To cite this version:

Juliette Gamot, Mathieu Balesdent, Arnault Tremolet, Romain Wuilbercq, Nouredine Melab, et al. "Two-Level Algorithm Combining Bayesian Optimization and Swarm Intelligence for Variable-Size Optimal Layout Problems." GECCO '23 Companion: Companion Conference on Genetic and Evolutionary Computation, Jul 2023, Lisbonne, Portugal. pp.139-142, 10.1145/3583133.3596997. hal-04381343

HAL Id: hal-04381343

https://inria.hal.science/hal-04381343v1

Submitted on 27 Jun 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Two-Level Algorithm Combining Bayesian Optimization and Swarm Intelligence for Variable-Size Optimal Layout Problems

Juliette Gamot

ONERA/DTIS

Palaiseau, France

INRIA

Villeneuve d'Ascq, France

juliette.gamot@onera.fr

Mathieu Balesdent

ONERA/DTIS

Palaiseau, France

mathieu.balesdent@onera.fr

Arnault Tremolet

ONERA/DTIS

Palaiseau, France

arnault.tremolet@onera.fr

Romain Wuilbercq

ONERA/DTIS

Palaiseau, France

romain.wuilbercq@onera.fr

Nouredine Melab

INRIA

Villeneuve d'Ascq, France

nouredine.melab@inria.fr

El-Ghazali Talbi

INRIA

Villeneuve d'Ascq, France

el-ghazali.talbi@inria.fr

## ABSTRACT

The design process of complex engineering systems may involve problems in which the number and type of design variables and constraints vary throughout the optimization process based on the values of dimensional variables. This category of problems is called Variable-Size Design Space optimization. A well-known application is the optimal layout problem which requires to place a variable number of components into a container. The dual objective is here to optimize the list of components in addition to their placements within the container. In this paper, a two-level algorithm is described to solve the aforementioned optimal layout problems. This algorithm combines the strength of a Swarm Intelligence algorithm based on a virtual-force system and a discrete Bayesian Optimization algorithm purposely adapted to tackle the dimensional aspect of this problem. The implementation of the two-level algorithm is discussed and the proposed approach is applied to the layout optimization of a satellite module. The performance of the algorithm is then analyzed with respect to several occupation rates of the system. This approach is also compared with a Hidden-Genes Genetic Algorithm.

## KEYWORDS

Multilevel Optimization, Bayesian Optimization, Swarm Intelligence, Variable-Size Design Space Problem, Optimal Layout

### ACM Reference Format:

Juliette Gamot, Mathieu Balesdent, Arnault Tremolet, Romain Wuilbercq, Nouredine Melab, and El-Ghazali Talbi. 2023. Two-Level Algorithm Combining Bayesian Optimization and Swarm Intelligence for Variable-Size Optimal Layout Problems. In *Genetic and Evolutionary Computation Conference Companion (GECCO '23 Companion)*, July 15-19, 2023, Lisbon, Portugal, 4 pages. https://doi.org/10.1145/3583133.3596997

## 1 INTRODUCTION

The optimal layout design of a complex system consists in placing components (instruments and equipment) into an available space. This key aspect has a critical impact on the overall system performance. This type of problems belongs to the class of NP-hard problems due to its mathematical complexity, and is therefore theoretically not solvable by classical optimization methods (e.g. gradient-based methods for instance). During the past 20 years, various methods have been developed in order to solve layout optimization problems. Mostly, metaheuristic algorithms have been used [12], sometimes hybridized with other techniques [21], but also multi-agent methods [20] and Machine Learning techniques [3]. In most of real-world engineering design problems, many architecture choices have to be made and have an impact on the problem definition (number of variables to be optimized, number of constraints). For instance, a possible choice would be either to place in the container two small tanks of fuel or a large one. The number of components is consequently not the same in both cases and hence, the number of design variables will vary from one sub-problem to another. Allowing the number of components to vary during the optimization may provide a better solution at the end. This type of problems is referred to as variable-sized design space (VSDS) problems in the literature [14]. For illustration purposes, the layout optimization of satellite module, refered as Satellite Module Layout Problem (SMLP) in the literature, is derived [21] without loss of generality and is extended in this paper to consider variable-sized problems. Most of the aforementioned algorithms are designed to handle fixed-size design space problems. Some previous research has been conducted on VSDS problems. Pelamatti et al. [14] extended Bayesian Optimization (BO) in order to address mixed-variables problems as well as VSDS problems. However, BO algorithms are often used to handle a few dozens of design variables and are not applicable to optimize few hundreds of design variables. Moreover, metaheuristics have been extended in order to tackle VSDS problems [1, 5, 8, 15]. Even if metaheuristics methods can directly tackle layout problems with hundreds of dimensions and constraints, it has been highlighted in [11, 16] that pure metaheuristic techniques based on genetic algorithms may

<footnote>Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the owner/author(s).</footnote>

<footnote>GECCO '23 Companion, July 15-19, 2023, Lisbon, Portugal</footnote>

<footnote>© 2023 Copyright held by the owner/author(s).</footnote>

<footnote>ACM ISBN 979-8-4007-0120-7/23/07.</footnote>

<footnote>https://doi.org/10.1145/3583133.3596997</footnote>
# Two-Level Algorithm Combining Bayesian Optimization and Swarm Intelligence for Variable-Size Optimal Layout Problems

Juliette Gamot

ONERA/DTIS

Palaiseau, France

INRIA

Villeneuve d'Ascq, France

juliette.gamot@onera.fr

Mathieu Balesdent

ONERA/DTIS

Palaiseau, France

mathieu.balesdent@onera.fr

Arnault Tremolet

ONERA/DTIS

Palaiseau, France

arnault.tremolet@onera.fr

Romain Wuilbercq

ONERA/DTIS

Palaiseau, France

romain.wuilbercq@onera.fr

Nouredine Melab

INRIA

Villeneuve d'Ascq, France

nouredine.melab@inria.fr

El-Ghazali Talbi

INRIA

Villeneuve d'Ascq, France

el-ghazali.talbi@inria.fr

## ABSTRACT

The design process of complex engineering systems may involve problems in which the number and type of design variables and constraints vary throughout the optimization process based on the values of dimensional variables. This category of problems is called Variable-Size Design Space optimization. A well-known application is the optimal layout problem which requires to place a variable number of components into a container. The dual objective is here to optimize the list of components in addition to their placements within the container. In this paper, a two-level algorithm is described to solve the aforementioned optimal layout problems. This algorithm combines the strength of a Swarm Intelligence algorithm based on a virtual-force system and a discrete Bayesian Optimization algorithm purposely adapted to tackle the dimensional aspect of this problem. The implementation of the two-level algorithm is discussed and the proposed approach is applied to the layout optimization of a satellite module. The performance of the algorithm is then analyzed with respect to several occupation rates of the system. This approach is also compared with a Hidden-Genes Genetic Algorithm.

## KEYWORDS

Multilevel Optimization, Bayesian Optimization, Swarm Intelligence, Variable-Size Design Space Problem, Optimal Layout

### ACM Reference Format:

Juliette Gamot, Mathieu Balesdent, Arnault Tremolet, Romain Wuilbercq, Nouredine Melab, and El-Ghazali Talbi. 2023. Two-Level Algorithm Combining Bayesian Optimization and Swarm Intelligence for Variable-Size Optimal Layout Problems. In *Genetic and Evolutionary Computation Conference Companion (GECCO '23 Companion)*, July 15–19, 2023, Lisbon, Portugal, 4 pages. https://doi.org/10.1145/3583133.3596997

## 1 INTRODUCTION

The optimal layout design of a complex system consists in placing components (instruments and equipment) into an available space. This key aspect has a critical impact on the overall system performance. This type of problems belongs to the class of NP-hard problems due to its mathematical complexity, and is therefore theoretically not solvable by classical optimization methods (e.g. gradient-based methods for instance). During the past 20 years, various methods have been developed in order to solve layout optimization problems. Mostly, metaheuristic algorithms have been used [12], sometimes hybridized with other techniques [21], but also multi-agent methods [20] and Machine Learning techniques [3]. In most of real-world engineering design problems, many architecture choices have to be made and have an impact on the problem definition (number of variables to be optimized, number of constraints). For instance, a possible choice would be either to place in the container two small tanks of fuel or a large one. The number of components is consequently not the same in both cases and hence, the number of design variables will vary from one sub-problem to another. Allowing the number of components to vary during the optimization may provide a better solution at the end. This type of problems is referred to as variable-sized design space (VSDS) problems in the literature [14]. For illustration purposes, the layout optimization of satellite module, refered as Satellite Module Layout Problem (SMLP) in the literature, is derived [21] without loss of generality and is extended in this paper to consider variable-sized problems. Most of the aforementioned algorithms are designed to handle fixed-size design space problems. Some previous research has been conducted on VSDS problems. Pelamatti et al. [14] extended Bayesian Optimization (BO) in order to address mixed-variables problems as well as VSDS problems. However, BO algorithms are often used to handle a few dozens of design variables and are not applicable to optimize few hundreds of design variables. Moreover, metaheuristics have been extended in order to tackle VSDS problems [1, 5, 8, 15]. Even if metaheuristics methods can directly tackle layout problems with hundreds of dimensions and constraints, it has been highlighted in [11, 16] that pure metaheuristic techniques based on genetic algorithms may

<footnote>Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the owner/author(s).
GECCO '23 Companion, July 15–19, 2023, Lisbon, Portugal
© 2023 Copyright held by the owner/author(s).
ACM ISBN 979-8-4007-0120-7/23/07.
https://doi.org/10.1145/3583133.3596997</footnote>
present some weaknesses for solving problems with complex com-
binatorial spaces or high number of constraints. Hybridization are
often preferred to solve such problems. Among them, two-level al-
gorithms have been developed to solve complex problems in which
several decision makers are involved [10, 18]. In that respect, in the
present paper, a two-level algorithm is presented in order to ad-
dress VSDS optimal layout problems. This algorithm combines the
strength of a dedicated swarm intelligence algorithm called Compo-
nent Swarm Optimization algorithm based on a virtual-force system
[7] and a Bayesian Optimization algorithm purposely adapted to
tackle the dimensional aspect of the VSDS problem [14].

The rest of the paper is organized as follows: Section 2 describes the formulation of the VSDS optimal layout problem. Section 3 introduces the proposed two-level algorithm, and describes both levels of the approach. The algorithm is applied to the SMLP in Section 4. The results are shown and compared to a Hidden Genes GA.

## 2 FORMULATION OF VARIABLE-SIZE DESIGN SPACE OPTIMAL LAYOUT PROBLEMS

As a representative example of optimal layout problems, without
loss of generality, the satellite module layout problem is considered
in this paper. The aim is to find the optimized layout that minimizes
the inertia of the satellite module composed with N components
on one bearing plate. Some components can be decomposed into
several sub-components and the choice of a subdivision varies the
number of components along with the number of design variables
and constraints. Indeed, a catalogue of components is given to the
designed optimization algorithm which must optimize the list of
the components as well as their positions in the container.

In order to propose a mathematical formulation of the aforemen-
tioned mixed-variable and VSDS problem, the design variables are
introduced and split into 3 categories [14]: continuous variables
**x** that correspond to the centers of inertia of the components as
well as their orientations, discrete variables **z** that are the plate
on which position each component (if there are several bearing
plates in the satellite module), and dimensional variables **w** that
are responsible for the choice of the sub-components to be part
of the layout. According to their values, the number and type of
continuous and discrete variables that are optimized in the problem
may vary. Consequently, the single-objective VSDS problem can be
mathematically formulated as follows in the case of the SMLP:

$$
\min_{x,z,w} I_{tot}(x, z, w)
$$

$$
\text{where} \quad \mathbf{x} \in F_x(\mathbf{w}) \subseteq \mathbb{R}^{n_x(\mathbf{w})}, \mathbf{z} \in F_z(\mathbf{w}) \subseteq \mathbb{N}^{n_z(\mathbf{w})}, \mathbf{w} \in F_w
$$

$$
\text{s.t.} \quad h_{\text{overlap}}(\mathbf{x}, \mathbf{z}, \mathbf{w}) = 0 \tag{1}
$$

$$
g_{CG}(\mathbf{x}, \mathbf{z}, \mathbf{w}) \le 0
$$

$$
g_{\text{functional}}(\mathbf{x}, \mathbf{z}, \mathbf{w}) \le 0
$$

where *I*<sub>tot</sub> is the objective function corresponding to the global inertia of the module. *h*<sub>overlap</sub> is the equality constraint ensuring that there is no overlap between the components and between the components and the container. It is defined by a *n*<sub>overlap</sub>(*w*)-dimensional vector. *g*<sub>CG</sub> is the inequality constraint which aim is to position the global center of gravity (CG) in a tolerance zone centered at the geometrical center of the container. It is defined

by a *n*<sub>CG</sub>(*w*)-dimensional vector. Finally, *g*<sub>functional</sub> ensures that incompatible components are spaced from a certain distance (e.g. radiation threshold). It is defined by a *n*<sub>functional</sub>(*w*)-dimensional vector. One can note that the design space of the continuous variables **x** and discrete variables **z** are functions of the dimensional variable values.

3 TWO-LEVEL ALGORITHM FOR VARIABLE-SIZE DESIGN SPACE OPTIMAL LAYOUT PROBLEMS

**3.1 General Structure of the Algorithm**

The proposed algorithm consists in optimizing the dimensional
variables **w** separately from the continuous and discrete variables
**x**, **z**. More specifically, in the case of the optimal layout problem
detailed in Section 2, the upper stage of the algorithm is responsible
for the choice of the components to be integrated into the system,
while the lower stage is responsible for the optimization of the lay-
out of the chosen set of components. The methodology embedded
with each of the two stages of the algorithm is introduced below:

*   **Lower level:** Component Swarm Optimization based on a virtual-force system (CSO-VF) algorithm [7] designed to solve FSDS optimal layout problems.

*   **Upper level:** Bayesian Optimization used to deal with the dimensional variables [14].

Figure 1 illustrates the structure of the two-level algorithm and both levels are detailed in the following sections.

<figure><img src="image_2.png" /><figcaption>Figure 1: Structure of the two-level algorithm.</figcaption></figure>

3.2 Lower Level: CSO-VF algorithm

The aim of the lower level is to optimize the fixed-size design space (FSDS) optimal layout problem i.e. with a fixed set of components determined by the upper level. The proposed approach is a swarm intelligence algorithm based on a virtual-force system to solve the highly constrained optimization problem. This strategy is herein adopted in [7] to solve optimal layout problems. In the proposed CSO-VF algorithm, each component is defined by its dynamic features: its position and orientation, its translational and rotational speeds, and its translational and rotational accelerations. A resultant force and a torque are applied to it. The goal of forces and torques are to solve the constraints and to minimize the objective
present some weaknesses for solving problems with complex com-
binatorial spaces or high number of constraints. Hybridization are
often preferred to solve such problems. Among them, two-level al-
gorithms have been developed to solve complex problems in which
several decision makers are involved [10, 18]. In that respect, in the
present paper, a two-level algorithm is presented in order to ad-
dress VSDS optimal layout problems. This algorithm combines the
strength of a dedicated swarm intelligence algorithm called Compo-
nent Swarm Optimization algorithm based on a virtual-force system
[7] and a Bayesian Optimization algorithm purposely adapted to
tackle the dimensional aspect of the VSDS problem [14].

The rest of the paper is organized as follows: Section 2 describes the formulation of the VSDS optimal layout problem. Section 3 introduces the proposed two-level algorithm, and describes both levels of the approach. The algorithm is applied to the SMLP in Section 4. The results are shown and compared to a Hidden Genes GA.

## 2 FORMULATION OF VARIABLE-SIZE DESIGN SPACE OPTIMAL LAYOUT PROBLEMS

As a representative example of optimal layout problems, without loss of generality, the satellite module layout problem is considered in this paper. The aim is to find the optimized layout that minimizes the inertia of the satellite module composed with N components on one bearing plate. Some components can be decomposed into several sub-components and the choice of a subdivision varies the number of components along with the number of design variables and constraints. Indeed, a catalogue of components is given to the designed optimization algorithm which must optimize the list of the components as well as their positions in the container.

In order to propose a mathematical formulation of the aforementioned mixed-variable and VSDS problem, the design variables are introduced and split into 3 categories [14]: continuous variables **x** that correspond to the centers of inertia of the components as well as their orientations, discrete variables **z** that are the plate on which position each component (if there are several bearing plates in the satellite module), and dimensional variables **w** that are responsible for the choice of the sub-components to be part of the layout. According to their values, the number and type of continuous and discrete variables that are optimized in the problem may vary. Consequently, the single-objective VSDS problem can be mathematically formulated as follows in the case of the SMLP:

$$
\begin{array}{ll}
\min_{\mathbf{x},\mathbf{z},\mathbf{w}} & I_{\text{tot}}(\mathbf{x}, \mathbf{z}, \mathbf{w}) \\
\text{where} & \mathbf{x} \in F_x(\mathbf{w}) \subseteq \mathbb{R}^{n_x(\mathbf{w})}, \mathbf{z} \in F_z(\mathbf{w}) \subseteq \mathbb{N}^{n_z(\mathbf{w})}, \mathbf{w} \in F_w \\
\text{s.t.} & \mathbf{h}_{\text{overlap}}(\mathbf{x}, \mathbf{z}, \mathbf{w}) = 0 \\
& g_{\text{CG}}(\mathbf{x}, \mathbf{z}, \mathbf{w}) \le 0 \\
& g_{\text{functional}}(\mathbf{x}, \mathbf{z}, \mathbf{w}) \le 0
\end{array}
\qquad (1)
$$

where $I_{tot}$ is the objective function corresponding to the global inertia of the module. $h_{overlap}$ is the equality constraint ensuring that there is no overlap between the components and between the components and the container. It is defined by a $n_{overlap}(\mathbf{w})$-dimensional vector. $g_{CG}$ is the inequality constraint which aim is to position the global center of gravity (CG) in a tolerance zone centered at the geometrical center of the container. It is defined

by a $n_{CG}(\mathbf{w})$-dimensional vector. Finally, $g_{functional}$ ensures that incompatible components are spaced from a certain distance (e.g. radiation threshold). It is defined by a $n_{functional}(\mathbf{w})$-dimensional vector. One can note that the design space of the continuous variables $\mathbf{x}$ and discrete variables $\mathbf{z}$ are functions of the dimensional variable values.

## 3 TWO-LEVEL ALGORITHM FOR VARIABLE-SIZE DESIGN SPACE OPTIMAL LAYOUT PROBLEMS

### 3.1 General Structure of the Algorithm

The proposed algorithm consists in optimizing the dimensional variables **w** separately from the continuous and discrete variables **x**, **z**. More specifically, in the case of the optimal layout problem detailed in Section 2, the upper stage of the algorithm is responsible for the choice of the components to be integrated into the system, while the lower stage is responsible for the optimization of the layout of the chosen set of components. The methodology embedded with each of the two stages of the algorithm is introduced below:

* **Lower level:** Component Swarm Optimization based on a virtual-force system (CSO-VF) algorithm [7] designed to solve FSDS optimal layout problems.

* **Upper level:** Bayesian Optimization used to deal with the dimensional variables [14].

Figure 1 illustrates the structure of the two-level algorithm and both levels are detailed in the following sections.

<figure><img src="image_2.png" /><figcaption>Figure 1: Structure of the two-level algorithm.</figcaption></figure>

### 3.2 Lower Level: CSO-VF algorithm

The aim of the lower level is to optimize the fixed-size design space (FSDS) optimal layout problem i.e. with a fixed set of components determined by the upper level. The proposed approach is a swarm intelligence algorithm based on a virtual-force system to solve the highly constrained optimization problem. This strategy is herein adopted in [7] to solve optimal layout problems. In the proposed CSO-VF algorithm, each component is defined by its dynamic features: its position and orientation, its translational and rotational speeds, and its translational and rotational accelerations. A resultant force and a torque are applied to it. The goal of forces and torques are to solve the constraints and to minimize the objective
function. Indeed, a dedicated attractive, repulsive or gradient-based force (and possibly a torque) is associated to each of the constraints and to the objective function. By the end of an iteration, a Newton's Second Law of Motion is applied in order to update the position and orientation of the whole swarm. The forces and torques are detailed as follows (for a component *i*) and illustrated in Figure 2:

* $F_{container,i}$: An attractive force applied if a component does not belong entirely to the container, illustrated in Figure 2(a).

* $F_{overlap,i}$, $T_{overlap,i}$: A repulsive force and a torque, applied in case of overlapping between components, illustrated in Figure 2(a).

* $F_{func,,i}$, $T_{func,,i}$: A repulsive force and a torque, applied if two functionally incompatible components are too close, illustrated in Figure 2(a).

* $F_{CG,i}$: A gradient-based force, applied if the center of gravity is not in the tolerance zone, illustrated in Figure 2(b).

* $F_{inertia,i}$, $T_{inertia,i}$: A gradient-based force and torque, applied to minimize the global inertia, illustrated in Figure 2(b).

<figure><img src="image_3.png" /><figcaption>Figure 2: Virtual-force system</figcaption></figure>

Complementary operators are detailed in [7]. This algorithm differs from classical particle swarm algorithm in that it is not a population-based algorithm, one instance of this swarm intelligence algorithm corresponds to a one container optimal layout resolution.

## 3.3 Upper Level: Bayesian Optimization for Categorical Variables

This level aims to select the combination of components (variables *w*) to be part of the layout among all the possible configurations given to the algorithm. The dimensional variables are referred to as categorical variables because although they can be expressed as discrete variables, they do not have any direct order relationship. The proposed method for this level is a Bayesian Optimization algorithm based on Gaussian Process (GP) surrogate modeling adapted to tackle solely categorical variables [14, 17].

GP surrogate modeling is a technique for modeling complex and expensive-to-evaluate systems using GP regression. It involves using a GP to approximate the relationship between inputs and outputs of a system (e.g. a black-box function), based on a limited set of observations of the system (i.e. Design of Experiment, DoE). The GP model is then used as a surrogate for the system, allowing for predictions at new input points without the need to run the expensive simulation or evaluation of the black-box function. GP is then enriched during the optimization process in order to converge to the optimum (during $N_{it}$ iterations). The enrichment process involves an auxiliary optimization process of an infill criterion in

order to find the best valuable candidate to be evaluated at the lower value of the criterion. Most of the surrogate modeling techniques as well as Bayesian Optimization algorithm frameworks have been developed to model and optimize continuous problems [9]. However, those techniques have recently been adapted to handle mixed continuous-discrete problems. They are reviewed in [2, 13, 19].

## 4 SMLP RESULTS

The algorithm is implemented and applied to the SMLP composed with 12 generic components subdivided up to 50 components in the catalogue. The geometry of the components are detailed in [6]. The subdivisions correspond to an overall combinatory of 3888 possible lists of components. Two occupation rates (O.R.) are considered: 30% and 50% of the plate. Increasing the occupation rate of the container also increases the difficulty to solve the constraints. The two-level algorithm (referred to as BO+CSO-VF) is compared with a GA enhanced by a Hidden-Variables Mechanism as described in [6] (referred to as HVM for GA). This algorithm is based on a GA enhanced by a mechanism that modifies the implementation of the chromosomes and the operators in order to deal with the VSDS aspect of the problem. Both algorithms are implemented and run 10 times each. The two algorithms are compared with the same total number of function evaluations. Figure 3 shows optimal layouts obtained for the two occupation rates and the two compared algorithms. Figure 4 corresponds to the median of the convergence curves (in addition to the interquartile range) with respect to the objective evaluations. The convergence curves are plotted from the beginning of the Bayesian Optimization process. Table 1 sums up the numerical results obtained for both algorithms and both occupation rates.

First of all, both algorithms allow to tackle the problem at hand, providing optimal feasible layouts. For both occupation rates, the two-level algorithm provides better final objective functions in terms of mean and median over the 10 repetitions (nearly -10%), but also in terms of best optimal layout obtained. Moreover, the two-level algorithm provides a smaller interquartile range and standard deviation in comparison to the HVM for GA (about -85%) that illustrates better robustness to initialization. These trends can be due to the virtual-force system and operators of the lower level of the BO+CSO-VF algorithm which provide dedicated techniques to address the constraints easier than for the hidden-variables method in which the constraints are handled thanks to Constraint-Dominance [4]. The illustrations of the plates show that the two-level algorithm converges toward subdivisions providing more components to be laid out, for a same occupation rate.

## 5 CONCLUSION

In this paper, variable-size design space optimal layout problems are described and mathematically formulated. A two-level algorithm is developed, hybridizing a swarm intelligence based on a virtual-force system and a Bayesian Optimization algorithm. It is successfully applied to a satellite module optimal layout problem and compared to a Hidden-Genes GA. Even if both algorithms provide feasible optimal layout, the two-level algorithm proves to have better performance in terms of final objective function, convergence speed and robustness to the initialization.
function. Indeed, a dedicated attractive, repulsive or gradient-based force (and possibly a torque) is associated to each of the constraints and to the objective function. By the end of an iteration, a Newton's Second Law of Motion is applied in order to update the position and orientation of the whole swarm. The forces and torques are detailed as follows (for a component *i*) and illustrated in Figure 2:

* $F_{container,i}$: An attractive force applied if a component does not belong entirely to the container, illustrated in Figure 2(a).

* $F_{overlap,i}, T_{overlap,i}$: A repulsive force and a torque, applied in case of overlapping between components, illustrated in Figure 2(a).

* $F_{func,,i}, T_{func,,i}$: A repulsive force and a torque, applied if two functionally incompatible components are too close, illustrated in Figure 2(a).

* $F_{CG,i}$: A gradient-based force, applied if the center of gravity is not in the tolerance zone, illustrated in Figure 2(b).

* $F_{inertia,i}, T_{inertia,i}$: A gradient-based force and torque, applied to minimize the global inertia, illustrated in Figure 2(b).

<figure><img src="image_3.png" /><figcaption>Figure 2: Virtual-force system</figcaption></figure>

Complementary operators are detailed in [7]. This algorithm differs from classical particle swarm algorithm in that it is not a population-based algorithm, one instance of this swarm intelligence algorithm corresponds to a one container optimal layout resolution.

## 3.3 Upper Level: Bayesian Optimization for Categorical Variables

This level aims to select the combination of components (variables *w*) to be part of the layout among all the possible configurations given to the algorithm. The dimensional variables are referred to as categorical variables because although they can be expressed as discrete variables, they do not have any direct order relationship. The proposed method for this level is a Bayesian Optimization algorithm based on Gaussian Process (GP) surrogate modeling adapted to tackle solely categorical variables [14, 17].

GP surrogate modeling is a technique for modeling complex and expensive-to-evaluate systems using GP regression. It involves using a GP to approximate the relationship between inputs and outputs of a system (e.g. a black-box function), based on a limited set of observations of the system (i.e. Design of Experiment, DoE). The GP model is then used as a surrogate for the system, allowing for predictions at new input points without the need to run the expensive simulation or evaluation of the black-box function. GP is then enriched during the optimization process in order to converge to the optimum (during $N_{it}$ iterations). The enrichment process involves an auxiliary optimization process of an infill criterion in

order to find the best valuable candidate to be evaluated at the lower value of the criterion. Most of the surrogate modeling techniques as well as Bayesian Optimization algorithm frameworks have been developed to model and optimize continuous problems [9]. However, those techniques have recently been adapted to handle mixed continuous-discrete problems. They are reviewed in [2, 13, 19].

## 4 SMLP RESULTS

The algorithm is implemented and applied to the SMLP composed with 12 generic components subdivided up to 50 components in the catalogue. The geometry of the components are detailed in [6]. The subdivisions correspond to an overall combinatory of 3888 possible lists of components. Two occupation rates (O.R.) are considered: 30% and 50% of the plate. Increasing the occupation rate of the container also increases the difficulty to solve the constraints. The two-level algorithm (referred to as BO+CSO-VF) is compared with a GA enhanced by a Hidden-Variables Mechanism as described in [6] (referred to as HVM for GA). This algorithm is based on a GA enhanced by a mechanism that modifies the implementation of the chromosomes and the operators in order to deal with the VSDS aspect of the problem. Both algorithms are implemented and run 10 times each. The two algorithms are compared with the same total number of function evaluations. Figure 3 shows optimal layouts obtained for the two occupation rates and the two compared algorithms. Figure 4 corresponds to the median of the convergence curves (in addition to the interquartile range) with respect to the objective evaluations. The convergence curves are plotted from the beginning of the Bayesian Optimization process. Table 1 sums up the numerical results obtained for both algorithms and both occupation rates.

First of all, both algorithms allow to tackle the problem at hand, providing optimal feasible layouts. For both occupation rates, the two-level algorithm provides better final objective functions in terms of mean and median over the 10 repetitions (nearly -10%), but also in terms of best optimal layout obtained. Moreover, the two-level algorithm provides a smaller interquartile range and standard deviation in comparison to the HVM for GA (about -85%) that illustrates better robustness to initialization. These trends can be due to the virtual-force system and operators of the lower level of the BO+CSO-VF algorithm which provide dedicated techniques to address the constraints easier than for the hidden-variables method in which the constraints are handled thanks to Constraint-Dominance [4]. The illustrations of the plates show that the two-level algorithm converges toward subdivisions providing more components to be laid out, for a same occupation rate.

## 5 CONCLUSION

In this paper, variable-size design space optimal layout problems are described and mathematically formulated. A two-level algorithm is developed, hybridizing a swarm intelligence based on a virtual-force system and a Bayesian Optimization algorithm. It is successfully applied to a satellite module optimal layout problem and compared to a Hidden-Genes GA. Even if both algorithms provide feasible optimal layout, the two-level algorithm proves to have better performance in terms of final objective function, convergence speed and robustness to the initialization.
<table><caption>Table 1: Numerical results obtained for both BO+CSO-VF and HVM for GA algorithms, 10 repetitions.</caption><thead><tr><th></th><th>OR</th><th>HVM for GA</th><th>BO+CSO-VF</th><th>Relative improvement</th></tr></thead><tbody><tr><td rowspan="2">Mean of the final objective function</td><td>30%</td><td>1.553e7</td><td><strong>1.411e7</strong></td><td>-9.15%</td></tr><tr><td>50%</td><td>2.353e7</td><td><strong>2.134e7</strong></td><td>-9.36%</td></tr><tr><td rowspan="2">Best run (objective function)</td><td>30%</td><td>1.483e7</td><td><strong>1.389e7</strong></td><td>-6.33%</td></tr><tr><td>50%</td><td>2.191e7</td><td><strong>2.112e7</strong></td><td>-3.65%</td></tr><tr><td rowspan="2">Final Standard Deviation</td><td>30%</td><td>7.020e5</td><td><strong>1.122e5</strong></td><td>-84.33%</td></tr><tr><td>50%</td><td>1.422e6</td><td><strong>1.766e5</strong></td><td>-87.58%</td></tr><tr><td rowspan="2">Nb of explored subdivisions (mean)</td><td>30%</td><td>467 (/3888)</td><td>100* (/3888)</td><td>-78.58%</td></tr><tr><td>50%</td><td>468 (/3888)</td><td>100* (/3888)</td><td>-78.63%</td></tr></tbody></table>

<figure><img src="image_4.png" /><figcaption>Figure 3: Optimal layouts for both algorithms and occupation rates.</figcaption></figure>

<figure><img src="image_5.png" /><figcaption>Figure 4: Convergence curves (median and interquartile range) for both algorithms and both occupation rates.</figcaption></figure>

REFERENCES

[1] O. Abdelkhalik and A. Gad. 2012. Dynamic-size multiple populations genetic algorithm for multigravity-assist trajectory optimization. *Journal of Guidance, Control, and Dynamics* 35(2) (2012), 520–529.

[2] T. Bartz-Beielstein and M. Zaefferer. 2017. Model-based methods for continuous and discrete global optimization. *Applied Soft Computing* 55 (2017), 154–167.

[3] P. Burggraaf, J. Wagner, and B. Heinbach. 2021. Bibliometric Study on the Use of Machine Learning as Resolution Technique for Facility Layout Problems. *IEEE Access* 9 (2021), 22569–22586.

[4] C.A.C. Coello and E.M. Montes. 2002. Constraint-handling in genetic algorithms through the use of dominance-based tournament selection. *Advanced Engineering Informatics* 16(3) (2002), 193–203.

[5] A. Gad and O. Abdelkhalik. 2011. Hidden genes genetic algorithm for multi-gravity-assist trajectories optimization. *AIAA Journal of Spacecraft and Rockets* 48(4) (2011), 629–641.

[6] J. Gamot, M. Balesdent, A. Tremolet, R. Wulbercq, N. Melab, and E. G. Talbi. 2023. Hidden-Variables Genetic Algorithm for Variable-Size Design Space Optimal Layout Problems with Application to Aerospace Vehicles. *Engineering Applications of Artificial Intelligence* 121 (2023), 105941.

[7] J. Gamot, R. Wulbercq, M. Balesdent, A. Tremolet, N. Melab, and E.-G. Talbi. 2022. Component Swarm Optimization Using Virtual Forces for Solving Layout Problems. *Swarm Intelligence: 13th International Conference, ANTS 2022, Málaga, Spain, November 2–4, Proceedings* Cham: Springer International Publishing. (2022), 292–299.

[8] B. Hutt and K. Warwick. 2007. Synapsing variable-length crossover: Meaningful crossover for variable-length genomes. *IEEE transactions on evolutionary computation* 11 (1) (2007), 118–131.

[9] D.R. Jones, S. Matthias, and J. Welch William. 1998. Efficient global optimization of expensive black-box functions. *Journal of Global Optimization* 13 (1998), 455–492.

[10] E. Kieffer. 2019. Co-evolutionary Hybrid Bi-level Optimization. *Doctoral dissertation, University of Luxembourg, Esch-sur-Alzette, Luxembourg* (2019).

[11] N. Krasnogor and J. Smith. 2005. A tutorial for competent memetic algorithms: model, taxonomy, and design issues. *IEEE transactions on Evolutionary Computation* 9(5) (2005), 474–488.

[12] J. Liu, H. Zhang, K. He, and S. Jiang. 2018. Multi-objective particle swarm optimization algorithm based on objective space division for the unequal-area facility layout problem. *Expert Systems With Applications* 102 (2018), 179–192.

[13] M. Meckesheimer, R.R. Barton, T. Simpson, F. Limayem, and B. Yannou. 2001. Metamodeling of combined discrete/continuous responses. *AIAA Journal* 39(10) (2001), 1950–1959.

[14] J. Pelamatti, L. Brevault, M. Balesdent, E.G. Talbi, and Y. Guerin. 2021. Bayesian optimization of variable-size design space problems. *Optimization and Engineering* 22 (2021), 387–447.

[15] M.M. Ryerkerk, R.C. Averill, K. Deb, and E.D. Goodman. 2017. Solving metameric variable-length optimization problems using genetic algorithms. *Genetic Programming and Evolvable Machines* 18 (2017), 247–277.

[16] S. Salcedo-Sanz. 2009. A survey of repair methods used as constraint handling techniques in evolutionary algorithms. *Computer science review* 3(3) (2009), 175–192.

[17] P. Saves, N. Bartoli, Y. Diouane, T. Lefebvre, J. Morlier, and C. David et al. 2022. Bayesian optimization for mixed variables using an adaptive dimension reduction process: applications to aircraft design. *AIAA SciTech Forum* 2022 (2022), 0082.

[18] A. Sinah, P. Malo, and K. Deb. 2017. A review on bilevel optimization: From classical to evolutionary approaches and applications. *IEEE transactions on Evolutionary Computation* 22(2) (2017), 276–295.

[19] L.P. Swiler, P.D. Hough, P. Qian, X. Xu, C. Storlie, and H. Lee. 2014. Surrogate models for mixed discrete-continuous variables. *Constraint Programming and Decision Making* (2014), 181–102.

[20] H. Tarkesh, A. Atighehchian, and A.S. Nookabadi. 2009. Facility layout design using virtual multi-agent system. *Journal of Intelligent Manufacturing* 20 (4) (2009), 347.

[21] Y.S. Wang, H.-F. Teng, and Y.-J. Shi. 2009. Cooperative co-evolutionary scatter search for satellite module layout design. *Engineering Computations* 26, 7 (2009), 761–785.

<footnote>*: The explored subdivisions for the BO+CSO-VF algorithm is 100 due to simulation budget (DoE and Iterations of the BO)</footnote>
<table><caption>Table 1: Numerical results obtained for both BO+CSO-VF and HVM for GA algorithms, 10 repetitions.</caption><thead><tr><th></th><th>OR</th><th>HVM for GA</th><th>BO+CSO-VF</th><th>Relative improvement</th></tr></thead><tbody><tr><td rowspan="2">Mean of the final objective function</td><td>30%</td><td>1.553e7</td><td><strong>1.411e7</strong></td><td>-9.15%</td></tr><tr><td>50%</td><td>2.353e7</td><td><strong>2.134e7</strong></td><td>-9.36%</td></tr><tr><td rowspan="2">Best run (objective function)</td><td>30%</td><td>1.483e7</td><td><strong>1.389e7</strong></td><td>-6.33%</td></tr><tr><td>50%</td><td>2.191e7</td><td><strong>2.112e7</strong></td><td>-3.65%</td></tr><tr><td rowspan="2">Final Standard Deviation</td><td>30%</td><td>7.020e5</td><td><strong>1.122e5</strong></td><td>-84.33%</td></tr><tr><td>50%</td><td>1.422e6</td><td><strong>1.766e5</strong></td><td>-87.58%</td></tr><tr><td rowspan="2">Nb of explored subdivisions (mean)</td><td>30%</td><td>467 (/3888)</td><td>100* (/3888)</td><td>-78.58%</td></tr><tr><td>50%</td><td>468 (/3888)</td><td>100* (/3888)</td><td>-78.63%</td></tr></tbody></table>

<figure><img src="image_4.png" /><figcaption>Figure 3: Optimal layouts for both algorithms and occupation rates.</figcaption></figure>

<figure><img src="image_5.png" /><figcaption>Figure 4: Convergence curves (median and interquartile range) for both algorithms and both occupation rates.</figcaption></figure>

REFERENCES

[1] O. Abdelkhalik and A. Gad. 2012. Dynamic-size multiple populations genetic algorithm for multigravity-assist trajectory optimization. *Journal of Guidance, Control, and Dynamics* 35(2) (2012), 520–529.

[2] T. Bartz-Beielstein and M. Zaefferer. 2017. Model-based methods for continuous and discrete global optimization. *Applied Soft Computing* 55 (2017), 154–167.

[3] P. Burggraaf, J. Wagner, and B. Heinbach. 2021. Bibliometric Study on the Use of Machine Learning as Resolution Technique for Facility Layout Problems. *IEEE Access* 9 (2021), 22569–22586.

[4] C.A.C. Coello and E.M. Montes. 2002. Constraint-handling in genetic algorithms through the use of dominance-based tournament selection. *Advanced Engineering Informatics* 16(3) (2002), 193–203.

[5] A. Gad and O. Abdelkhalik. 2011. Hidden genes genetic algorithm for multi-gravity-assist trajectories optimization. *AIAA Journal of Spacecraft and Rockets* 48(4) (2011), 629–641.

[6] J. Gamot, M. Balesdent, A. Tremolet, R. Wulbercq, N. Melab, and E. G. Talbi. 2023. Hidden-Variables Genetic Algorithm for Variable-Size Design Space Optimal Layout Problems with Application to Aerospace Vehicles. *Engineering Applications of Artificial Intelligence* 121 (2023), 105941.

[7] J. Gamot, R. Wulbercq, M. Balesdent, A. Tremolet, N. Melab, and E.-G. Talbi. 2022. Component Swarm Optimization Using Virtual Forces for Solving Layout Problems. *Swarm Intelligence: 13th International Conference, ANTS 2022, Málaga, Spain, November 2–4, Proceedings* Cham: Springer International Publishing. (2022), 292–299.

[8] B. Hutt and K. Warwick. 2007. Synapsing variable-length crossover: Meaningful crossover for variable-length genomes. *IEEE transactions on evolutionary computation* 11 (1) (2007), 118–131.

[9] D.R. Jones, S. Matthias, and J. Welch William. 1998. Efficient global optimization of expensive black-box functions. *Journal of Global Optimization* 13 (1998), 455–492.

[10] E. Kieffer. 2019. Co-evolutionary Hybrid Bi-level Optimization. *Doctoral dissertation, University of Luxembourg, Esch-sur-Alzette, Luxembourg* (2019).

[11] N. Krasnogor and J. Smith. 2005. A tutorial for competent memetic algorithms: model, taxonomy, and design issues. *IEEE transactions on Evolutionary Computation* 9(5) (2005), 474–488.

[12] J. Liu, H. Zhang, K. He, and S. Jiang. 2018. Multi-objective particle swarm optimization algorithm based on objective space division for the unequal-area facility layout problem. *Expert Systems With Applications* 102 (2018), 179–192.

[13] M. Meckesheimer, R.R. Barton, T. Simpson, F. Limayem, and B. Yannou. 2001. Metamodeling of combined discrete/continuous responses. *AIAA Journal* 39(10) (2001), 1950–1959.

[14] J. Pelamatti, L. Brevault, M. Balesdent, E.G. Talbi, and Y. Guerin. 2021. Bayesian optimization of variable-size design space problems. *Optimization and Engineering* 22 (2021), 387–447.

[15] M.M. Ryerkerk, R.C. Averill, K. Deb, and E.D. Goodman. 2017. Solving metameric variable-length optimization problems using genetic algorithms. *Genetic Programming and Evolvable Machines* 18 (2017), 247–277.

[16] S. Salcedo-Sanz. 2009. A survey of repair methods used as constraint handling techniques in evolutionary algorithms. *Computer science review* 3(3) (2009), 175–192.

[17] P. Saves, N. Bartoli, Y. Diouane, T. Lefebvre, J. Morlier, and C. David et al. 2022. Bayesian optimization for mixed variables using an adaptive dimension reduction process: applications to aircraft design. *AIAA SciTech Forum* 2022 (2022), 0082.

[18] A. Sinah, P. Malo, and K. Deb. 2017. A review on bilevel optimization: From classical to evolutionary approaches and applications. *IEEE transactions on Evolutionary Computation* 22(2) (2017), 276–295.

[19] L.P. Swiler, P.D. Hough, P. Qian, X. Xu, C. Storlie, and H. Lee. 2014. Surrogate models for mixed discrete-continuous variables. *Constraint Programming and Decision Making* (2014), 181–102.

[20] H. Tarkesh, A. Atighehchian, and A.S. Nookabadi. 2009. Facility layout design using virtual multi-agent system. *Journal of Intelligent Manufacturing* 20 (4) (2009), 347.

[21] Y.S. Wang, H.-F. Teng, and Y.-J. Shi. 2009. Cooperative co-evolutionary scatter search for satellite module layout design. *Engineering Computations* 26, 7 (2009), 761–785.

<footnote>*: The explored subdivisions for the BO+CSO-VF algorithm is 100 due to simulation budget (DoE and Iterations of the BO)</footnote>