## 2 Background
### Selecting a MOEA
#### 2.2.2 HHLA and HHRL  
Learning Automata-based Multi-Objective Hyper-Heuristic with a Ranking scheme Initialization [3] implements a learning automata whose action is to select a LLH at each decision point, while the optimization problem is solved. There are two versions available of this algorithm: HHLA and HHRL. The only difference resides in the fact that HHRL employs an initialization process used in order to reduce the number of LLHs in the pool.  
Algorithm 2 illustrates how both hyper-heuristics work. First, all of the initialization process is performed (Line 6). The algorithm continues while a stopping criterion is not reached; this hyper-heuristic applies the current LLH \(h_{i}\) to the current population (\(Pop\)) during \(g\) generations producing a new offspring (\(Pop^{\prime}\)). In the following, \(Pop\) and \(Pop^{\prime}\) are combined to generate the new current population \(Pop\). In Line 11, this HH verifies whether it is time to switch or not to another LL: this is performed by verifying if is there an improvement in the Hypervolume value (compared to previous iterations). If it is the case, the current LLH keeps running, and, if not, the reinforcement learning scheme updates the transition matrix \(P\). Finally, another LLH is selected by the \(\varepsilon\)-RoulleteGreedy method from \(A\) considering the transition matrix \(P\).  
The \(\varepsilon\)-RoulleteGreedy method focuses on exploring different transition pairs by performing a given number of trials in order to get a better view of LLH pairwise performance at the early stage. Then, it becomes more and more greedy exploiting the accumulated knowledge.  
As mentioned before, the only difference between HHLA and HHRL lies in the initialization method. Algorithm 3 describes this process. First, the method creates a random population of solutions (Line 5) and the transition matrix \(P\) (Line 6), which describes the selection probabilities of transitions between LLHs. If HHLA is being run (Line 7), all LLHs are allowed to execute. Otherwise, only allowed LLH is selected.  
```
1 Input:
2 Problem;
3\(H\): set of LLHs \(\{h_{1},...,h_{i},...,h_{n}\}\);
4\(g\) fixed number of generations;
5 begin
6\([A,P,Pop,h_{i}]\gets Initialization(H)\);
7 whileA stopping criterion is not reacheddo
8\(Pop^{\prime}\leftarrow\) ApplyMetaHeuristic(\(h_{i}\),\(Pop\),\(g\));
9\(Pop\leftarrow\)\(Pop^{\prime}\)//acceptance criterion;
10//decide whether to switch to another metaheuristic;
11 ifswitch()then
12 LearningAutomataUpdateScheme(\(P\));
13\(h_{i}\leftarrow\) SelectMetaheuristic(\(P\), A);
14
15
16 end if
17
18 end for
19 returnPop
20
21 end while
```  
**Algorithm 2**HHLA and HHRL pseudocode, adapted from [3].  
For this purpose, the set of LLHs (\(H\)) is reduced in order to eliminate poor-performing LLHs. This works as follows: First, all LLHs are executed in sequence for a number of stages. Every time an LLH executes, HHRL computes the resulting population Hypervolume, computed using the same reference points. The scheme counts how many times an LLH becomes the best one in all stages. These counts are then used to determine which LLH should compose the allowed set \(A\). LLHs with a performance worse than the average are not allowed to compose \(A\). The algorithm continues by selecting a current LLH \(h_{i}\) according to the \(\varepsilon\)-RoulleteGreedy and returning all the generated information.