## 4 Simulations
Moreover, palletizing and depalletizing tasks of single parcels are performed by inserting or removing the parcel boxes from the top layer. These assumptions derived from industrial practice are exploited to strengthen the robustness of our box detection algorithm. Hence, in this work, we consider a single type of boxes of known size, but the proposed approach can be easily extended to handle different box formats. Moreover, we can estimate the equation of the top plane of the highest layer of parcel boxes by applying a RANSAC-based estimation method when the pallet layer is full, i.e., when no box has been removed yet. Hence, we assume that the top plane is always known with respect to the robot. Parcel box detection is constrained to the points \(p_{ij}\) which are within a small distance from the plane. Parcel boxes are then detected according to the following steps [29]:

#### 5.1.1 Edge Detection and Candidate Boxes Computation

Edges are detected in both the intensity and the depth image acquired by the camera. In particular, discontinuities in the intensity image are detected by applying the standard Canny edge detector, with upper and lower thresholds \(U_{\text{canny}}=120\) and \(L_{\text{canny}}=60\). Conversely, depth discontinuities are defined as the pixels whose corresponding points belong to the top plane of the pallet (with a tolerance \(t_{\text{inlier}}=5\) cm) for which there exists at least a pixel in the neighborhood which does not satisfy the same condition. Straight lines are fitted to the union of the two discontinuity images using the Hough Line Transform. Lines are organized in a connectivity graph by connecting them at intersections. A set of candidate boxes \(\mathbb{B}\) is generated by locating cycles of length 4 in the graph, i.e., quadrilaterals in the image. A candidate box is only accepted if the edges approximately intersect at a right angle, with tolerance \(\theta_{\text{box}}=10^{\circ}\). Moreover, the length of the edges must correspond to the expected size of the box top face (with tolerance \(\sigma_{\text{box}}=2\) cm). In the case of a mixed pallet, multiple possible box sizes could be acceptable in this step. Figure 6: Detection of parcel boxes and their separation gaps. #### 5.1.2 Genetic Optimization Algorithm

As the set of candidate boxes \(\mathbb{B}\) detected in the previous step may contain many spurious or overlapping boxes, a genetic optimization algorithm is used to locate the best subset of boxes \(\mathbb{S}\). The optimization aims at maximizing the area \(F(\mathbb{S})\) of the image (in pixels) which is covered by exactly one of the boxes. In particular, the objective function to be optimized is:

\[F(\mathbb{S})=\sum_{S\in\mathbb{S}}A(S)-\gamma_{O}\sum_{S\in\mathbb{S}}O(S)- \gamma_{I}\sum_{S\in\mathbb{S}}I(S)-\gamma_{C}\sum_{S\in\mathbb{S}}\sum_{S^{ \prime}\in\mathbb{S}^{\prime}\
eq S}C\big{(}S^{\prime},S\big{)} \tag{3}\]

The first term is the summation of the areas \(A(S)\) (in pixels) of the candidate box \(S\in\mathbb{S}\), and the remaining three terms are penalty terms where \(O(S)\) is the number of pixels of \(S\), which do not belong to the top plane of the pallet, \(I(S)\) is the difference between the quadrilateral area and the expected area of the top face of a box, and \(C(S,S^{\prime})\) is the overlapping area between box candidates \(S\) and \(S^{\prime}\). Coefficients \(\gamma_{O}=2\), \(\gamma_{I}=8\) and \(\gamma_{C}=2\) are parameters weighting each penalty term. The genetic optimization algorithm operates on a population of \(G_{\text{pop}}=100\) individuals \(\mathbb{S}_{i}\), each representing a subset of the candidate box \(\mathbb{B}\). The population is initialized by selecting random subsets of candidate boxes. We define two mutation operators \(\text{Mutation1}(\mathbb{S})\) and \(\text{Mutation2}(\mathbb{S})\), and a crossover operator \(\text{Crossover}(\mathbb{S}_{1},\mathbb{S}_{2})\).