## 3 Methods and Materials: The Proposed Pipeline
### GAIA: The Customer Sentiment Analysis System  
As reported in Figure 3, the proposed GAIA system also embeds a skeleton generator block. Subsequently, through a deep hybrid network (recurrent-convolutive), the time series of the junction points of the skeleton associated with each traced customers will be learned in order to characterize their \"sentiment\". Preliminarily, the innovative human skeleton generator is described. Several researchers have proposed a solution to build a skeleton map of the human subject for pose estimation, robotic applications, computer vision, activities recognition, etc. [27; 28; 29; 30; 31; 32; 33; 34]. The authors for this application propose a novel method to extract the skeleton of the human subject embedded in the input calibrated frames \(I(x,y,t)\). The proposed skeleton generator is based on the usage of Mask-R-CNN [35] architecture properly enhanced by means of the Criss-Cross attention block [36]. Other authors have used Mask-R-CNN for skeleton and human pose estimation [37] but the architecture we designed is completely different with respect to ones reported in the scientific literature as an embedded ad hoc self-attention mechanism to improve the performance of the overall deep network. In the following Figure 5, a detailed diagram of the skeleton generator based on the implemented Criss-Cross Mask-R-CNN is reported. As reported in Figure 5, the Mask-R-CNN backbone of the proposed solution has been modified according to our development. Specifically, the embedded convolutional architecture (\"CNN\" clock of the diagram reported in Figure 5) is based on the ResNet-101 [26] backbone in which we have interpolated a Criss-Cross attention layer [36] as per Figure 5. This block is needed to extract the visual features of the input video frames to be passed to the downstream layer of the Mask-R-CNN, which will be able to generate the skeleton of the tracked human as well as the related bounding box segmentation mask. More details about the modified ResNet-101 are included in the Criss-Cross attention layer (which we have named as \"ResNet-CC\"). The pre-calibrated input frame \(I(x,y,t)\) is preprocessed by the input layer of the ResNet-CC, and the visual features are extracted from the residual layers of the deep network. If we define with \(H\) the visual features extracted from the  
Figure 5: The designed Criss-Cross Mask-R-CNN used for human skeleton generator. residual layers of the so-designed deep architecture, the features extraction enhanced by the Criss-Cross attention processing, performed by ResNet-CC, can be formalized as follows. Given a space of visual features \(H\ \in\ R^{(C\times W\times H)}\), \(C\) is the original number of channels, while \(W\times H\) represents the spatial dimension of the features of the feature map generated through the convolutional layers of the ResNet-101 backbone. The Criss-Cross processing layer preliminary applies two \(1\times 1\) convolutional blocks to \(H\) to generate two feature maps \(F_{1}\) and \(F_{2}\), which belong to \(R^{(C\times W\times H)}\), where \(C^{\prime}\) represents the reduced number of channels due to the dimensional reduction performed by convolutional blocks. After that, we defined an \"Affinity\" function suitable to generate the feature attention map \(A_{M}\in R^{[(H+W-1)\times(W\times H)]}\). After that, for each position \(u\) in the spatial dimension of \(F_{1}\), a vector \(F_{(1,u)}\in R\times C^{\prime}\) is extracted, and the set \(\Omega_{u}\in R^{[(H+W-1)\times C^{\prime}]}\) is defined by extracting characteristic vectors from \(F_{2}\) for each position \(u\in F_{2}\), such that \(\Omega_{(j,u)}\in R\times C^{\prime}\) is the \(i\)-th element of \(\Omega_{u}\).