# Transfer Learning for Operator Selection: A Reinforcement Learning Approach
## 4 Experimental Results
### The Problem and Datasets  
This study has been conducted to demonstrate the gain/benefit of transfer learning using an adaptive operator selection scheme built with an implementation of Q learning algorithm. Both the selection scheme and the RL (i.e., Q learning) are embedded within a standard ABC algorithm equipped with three recent state-of-the-art binary operators; _binABC_[25], _ibinABC_[26], and _disABC_[27] to solve the set union knapsack problem (SUKP), where Q learning is implemented and integrated into the ABC algorithm to allow the agent (i.e., the ABC algorithm) to learn how to adaptively select one of these three operators.  
The family of _knapsack_ problems includes renown combinatorial optimisation problem sets used to test the efficiency and performance of problem-solving algorithms. They are known as an NP-Hard problems with respect to complexity and are very instrumental in modelling and solving real-world industrial problems. SUKP is a special form of _knapsack_ problem, which holds NP-Hard complexity level [28]. This problem is chosen as the test-bed in this study to demonstrate the success of proposed approach. It requires a set of items to be optimally composed in subsets so as to gain the maximum benefit. Given a set of \(n\) elements, \(U=\{u_{i}|i=1,\ldots,n\}\) with a non-negative weight set, \(W=\{w_{i}|i=1,\ldots,n\}\) and a set of \(m\) items, \(\mathcal{S}=\{U_{i}|j=1,\ldots,m\}\) with a profit set, \(\mathcal{P}=\{p_{j}>0|j=1,\ldots,m\}\), a subset of \(A\subseteq\mathcal{S}\) is sought to be found such that it maximises the profit subject to that the sum of the weights of selected items is not to exceed the capacity constraint, \(C\). The formal structure of the problems is as follows:  
\[\max\quad P(A)=\sum_{j\in A}p_{j} \tag{6}\]  
\[\text{s.t.}\quad W(A)=\sum_{i\in\bigcup_{j\in A}\,U_{j}}w_{i}\leq C,\quad A \subseteq\mathcal{S}. \tag{7}\]  
The problem is represented in real numbers and needs to be represented in binary form to enable binary operators in search algorithms such as binary ABC [3]. Following the details of the problem and the approach introduced by [29], a binary vector, \(B=\{b_{j}|j=1,..,m\}\in\{0,1\}\), is defined to be used as the set of decision variables, where \(b_{j}=1\) if an item is selected, \(b_{j}=0\), otherwise. The model of the problem can be reformulated as follows:  
\[\max\quad f(B)=\sum_{j=1}^{m}b_{j}p_{j} \tag{8}\]  
\[\text{s.t.}\quad W(A_{B})=\sum_{i\in\bigcup_{j\in A_{B}}u_{j}}w_{i}\leq C. \tag{9}\]  
The main goal is to find the best binary vector, \(B\), which provides the subset of items with the maximum profit.  
The problem instances of SUKP chosen in this study are collected from recently published literature. [PERSON] et al. [30] have introduced 30 benchmarking problem instances of SUKP as tabulated in Table 1 with all configuration details, where three different configurations presented varying with comparative status of \(m\) and \(n\); (i) \(m>n\), (ii) \(m<n\), and (iii) \(m=n\)), while \(w\in\{0.10,0.15\}\) and \(y\in\{0.75,0.85\}\) represent the density of elements and the rate between the capacities and the sum of weights of elements, respectively. As seen, each set of problem instances includes 10 instances varying with \(m,n,w\), and \(y\) values. More details can be found in [30; 31].