## 4 Proposed QL-Driven ABC-Based Solving Approaches
### Food Source Representation and Evaluation  
A suitable solution representation is based on the work of [73]. A candidate solution \(Sol\), i.e., food source, jointly specifies the production job sequence and the maintenance operations' positions on each machine and is codded by a two-field structure. The first field is a vector \(\mathrm{S}=\left\{j,j\in\left[1,n\right]\right\}\) representing the production job order. The second field represents the scheduling of maintenance operations on each machine by a binary matrix \(P\) of \(m\times n\) size. An element \(P\left[i,j\right]\) of \(P\) indicates if a maintenance intervention is scheduled on machine \(M_{i}\) after job \(j\), and then \(P\left[i,j\right]=1\), or not, and then \(P\left[i,j\right]=0\). Let  
\[\mathrm{S}=\left(1\;\;\;9\;\;3\;\;8\;\;5\;\;6\;\;7\;\;4\;\;\;2\;\;0\right)\text{, and}\]  
\[P=\left(\begin{matrix}0&1&0&0&1&0&0&0&1\\ 0&0&1&0&0&1&0&0&0\\ 0&1&0&0&0&1&0&0&0\end{matrix}\right)\]  
represent a solution of an integrated flowshop scheduling problem with \(m=3\) machines and \(n=10\) jobs. Then, the execution order of the ten jobs and maintenance operations on the three machines is the following:  
Machine 1: 1, 9, M\({}_{11}\), 3, 8, 5, M\({}_{12}\), 6, 7, 4, 2, M\({}_{13}\), 0.  
Machine 2: 1, 9, 3, M\({}_{21}\), 8, 5, 6, M\({}_{22}\), 7, 4, 2, 0.  
Machine 3: 1, 9, M\({}_{31}\), 3, 8, 5, 6, M\({}_{32}\), 7, 4, 2, 0.  
Figure 1: Flow chart of the IQABC-LE algorithm.  
```
0: Population size (PopSize), Onlooker bees percentage (onlook%), Number of generations (max_iterations), Abandonment number of trials (limit) Begin\(Ebees[1]\gets INEH()\)\(\triangleright\) Population initialization step for\(i\gets 1\) to alpha%PopSize do\(Ebees[i]\gets ModifiedNeth()\) endforfor\(j\gets 1\) to (PopSize-alpha%PopSize-1) do\(Ebees[j]\gets RandomGeneration()\) endfor\(iter\gets 1\) while\(iter\leq max\_iterations\)do\(i\leftarrow\)\(1\)\(\triangleright\) employed bees phase  NeighborEbee\(\leftarrow\) QL(Ebees[i]) \(\triangleright\) Qlearning selection (Algorithm 3) if NeighborEbee\(\geq\) Ebees[i] then  Ebees[i]\(\leftarrow\) NeighborEbee endfor for i\(\leftarrow\)\(1\) to onlook%PopSize do\(\triangleright\) onlooker bees phase  Olook[i]\(\leftarrow\) probaSelection(Ebees)  Olook \(\leftarrow\) ILS(Olook[i]) if Olook \(\geq\) Olook[i] then  Ebees[i]\(\leftarrow\)-Olook endif endfor for i\(\leftarrow\)\(1\) to PopSize do\(\triangleright\) scout bees phase  if CheckLimit(Ebees[i],limit)=True then  Ebees[i]\(\leftarrow\)-RandomGeneration() endif endfor endwhile Return Best Solution() End
```  
**Algorithm 1** IQABC-LE()  
The objective is to find a solution \(Sol\) that minimizes the amount of nectar (the measure of the quality of a solution), i.e., the best sequencing of jobs and maintenance operations to be processed for each machine in order to minimize the total completion time of the schedule, \(C_{max}\), taking into account maintenance interventions planning under learning and deteriorating effects. Considering the learning and the deteriorating effects, and the fact that the total completion time \(C_{max}\) depends mainly on the sum of the job processing times and maintenance durations (see Formula (3)), the \(C_{max}\) while considering the learning effect and both effects simultaneously are given by Formulas (4) and (5), respectively.  
\[C_{max}=IT+\Sigma_{j=1}^{j=n}p_{mj}+\Sigma_{c=1}^{c=k_{m}}pm_{mc} \tag{3}\]  
\[C_{max}=IT+\Sigma_{j=1}^{j=n}p_{mj}+\Sigma_{c=1}^{c=k_{m}}pm_{mc}(c)=IT+\Sigma_ {j=1}^{j=n}p_{mj}+\Sigma_{c=1}^{c=k_{m}}(pm_{mc}\times c^{\alpha}) \tag{4}\]  
\[C_{max}=IT+\Sigma_{j=1}^{j=n}p_{mj}(t)+\Sigma_{c=1}^{c=k_{m}}pm_{mc}(c)=IT+ \Sigma_{j=1}^{j=n}(p_{mj}+\beta\times t)+\Sigma_{c=1}^{c=k_{m}}(pm_{mc}\times c ^{\alpha}) \tag{5}\]  
where: \(IT\) refers to the total idle time of the last machine \(m\) waiting for jobs arrival and \(k_{m}\) is the number of inserted maintenance activities on the last machine \(m\).