# Scene Description for Visually Impaired People with Multi-Label Convolutional SVM Networks
## 2 Proposed Methodology
### A. Convolution Layer  
In this section, we present the SVM convolution technique for the first convolution layer. The generalization to subsequent layers is straightforward. In a binary classification setting, the training set \(\{X_{i},y_{i}\}_{i=1}^{M}\) is supposed to be composed of \(M\) positive and negative RGB images and the corresponding class labels are set to \(y_{i}\in\{+1,-1\}\). The positive images contain the object of interest, whereas the negatives ones represent background. From each image \(X_{i}\), we extract a set of patches of size \(h\times h\times 3\) and represent them as feature vectors \(x_{i}\) of dimension \(d\), with \(d=h\times h\times 3\). After processing the \(M\) training images, we obtain a large training set \(Tr^{(1)}=\{x_{i},y_{i}\}_{i=1}^{m^{(1)}}\) of size \(m^{(1)}\) as shown in Figure 2.  
Next, we learn a set of SVM filters on different sub-training sets \(Tr^{(1)}_{sub}=\{x_{i},y_{i}\}_{i=1}^{I}\) of size \(I\) randomly sampled from the training set \(Tr^{(1)}\). The weight vector \(w\in\mathcal{R}^{d}\) and bias \(b\in\mathcal{R}\) of each SVM filter are determined by optimizing the following problem [39; 40]:  
\[\underset{w,b}{\text{min}}w^{T}w+C\sum_{i=1}^{I}\xi(w,b;x_{i},y_{i}) \tag{1}\]  
where \(C\) is a penalty parameter, which can be estimated through cross-validation. As loss function, we use \(\xi(w,b;x_{i},y_{i})=\text{max}\big{(}1-y_{i}\big{(}w^{T}x_{i}+b\big{)},0\big{)}\) referred as the hinge loss. After training, we represent the weights of the SVM filters as \(\left\{w_{k}^{(1)}\right\}_{k=1}^{n^{(1)}}\), where \(w_{k}^{(1)}\in\mathcal{R}^{k\times h\times 3}\) refers to \(k\)th-SVM filter weight matrix, while \(n^{(1)}\) is the number of filters. Then, the complete weights of the first convolution layer are grouped into a filter bank of four dimensions \(W^{(1)}\in\mathcal{R}^{k\times h\times 3\times n^{(1)}}\).  
In order to generate the feature maps, we simply convolve each training image \(\{X_{i}\}_{i=1}^{M}\), with the obtained filters as is usually done in standard CNN to generate a set of 3D hyper-feature maps \(\left\{H_{i}^{(1)}\right\}_{i=1}^{M}\). Here \(H_{i}^{(1)}\in\mathcal{R}^{n^{(1)}\times c^{(1)}\times n^{(1)}}\) is the new feature representation of image \(X_{i}\) composed of \(n^{(1)}\) feature maps (Figure 3). To obtain the \(k\)th feature map \(h_{ki}^{(1)}\), we convolve the \(k\)th filter with a set of sliding windows of size \(h\times h\times 3\) (with a predefined stride) over the training image \(X_{i}\) as shown in Figure 4:  
\[h_{ki}^{(1)}=f\big{(}X_{i}*w_{k}^{(1)}\big{)},\ k=1,\ldots,\ n^{(1)} \tag{2}\]  
Figure 1: Proposed multi-label convolutional support vector machine (M-CSVM) architecture for detecting the presence of multiple objects in the image.  
where \(*\) is the convolution operator and \(f\) is the activation function.  
In the following algorithm, we present the implementation of this convolutional layer. The generalization to subsequent convolution layer is simply made by considering the obtained feature maps as new input to the next convolution layer.  
Figure 3: Supervised feature map generation.  
Figure 2: Training set generation for the first convolution layer.