# SSNet: Learning Mid-Level Image Representation Using Salient Superpixel Network
## 3 The Proposed Method for Image Representation
### Image Representation with the Weighted Second-Order Pooling  
#### 3.2.1 The Proposed Feature Weighting Method  
In this section, we will present the proposed feature weighting method, which can be considered as an improvement of the generalized max pooling approach [11]. Next, we first introduce the generalized max pooling approach. Generalized Max-Pooling Method  
In order to alleviate the side effect of the burstiness, [PERSON] et al. [11] propose the generalized max-pooling (GMP) method, which equalizes the contribution of the frequently occurring and rarely occurring descriptors to the image representation. Let \(X=\{x_{1},\cdots,x_{N}\}\) be a set of \(N\) feature descriptors extracted from an image and \(\varphi_{n}=\varphi(x_{n})\) denote the \(D\)-dimensional encoding result of the \(n\)th descriptor. Assuming that \(\varphi^{gmp}\) denotes the GMP representation, the GMP method enforces the dot-product between each feature encoding result \(\varphi_{n}\) and GMP representation \(\varphi^{gmp}\) to be a constant \(c\):  
\[\varphi_{n}^{T}\varphi^{gmp}=c,\ n=1,\cdots,N. \tag{13}\]  
Here, the constant \(c\) can be set as an arbitrary value, as the final representation \(\varphi^{gmp}\) typically need to be \(L2\) normalized. Therefore, the authors assign \(c=1\). Equation (13) ensures that each feature descriptor makes the same contribution to the final pooled representation. Further, let \(\varPhi\) denote a \(D\times N\) matrix that contains \(N\)\(D\)-dimensional feature encoding results, i.e., \(\varPhi=[\varphi_{1},\cdots,\varphi_{N}]\). Then, Equation (13) can be represented in matrix form:  
\[\varPhi^{T}\varphi^{gmp}=_{N} \tag{14}\]  
where \({}_{N}\) denotes a \(N\)-dimensional vector of all ones. This is a linear system of \(N\) equations with \(D\) unknowns. In general, this system might not have a solution. Thus, Equation (14) can be transformed into a least-squares regression problem with the additional constraint that \(\varphi^{gmp}\) has a minimal norm in the case of an infinite number of solutions:  
\[\varphi^{gmp}= \begin{array}{c}argmin\big{\|}\varPhi^{T}\varphi_{-N}\big{\|} ^{2}\\ \varphi\end{array} . \tag{15}\]  
This problem has a simple closed-form solution:  
\[\varphi^{gmp}=\left(\varPhi^{T}\right)_{N}^{+}=\left(\varPhi\varPhi^{T} \right)^{+}\varPhi_{N}. \tag{16}\]  
Note that \(\varPhi_{N}=\sum\limits_{n=1}^{N}\varphi(x_{n})=\varphi^{sum}\) is the sum-pooled representation. Hence, the proposed GMP can be considered as the result of projecting \(\varphi^{sum}\) onto the \(\left(\varPhi\varPhi^{T}\right)^{+}\). Since the \(\left(\varPhi\varPhi^{T}\right)^{+}\) is not a continuous operation, the authors further introduce \(\varphi^{gmp}_{\lambda}\), the regularized GMP, and try to obtain a stable solution:  
\[\varphi^{gmp}_{\lambda}= \begin{array}{c}argmin\big{\|}\big{\|}\varPhi^{T} \varphi_{-N}\big{\|}^{2}+\lambda\big{\|}\varphi^{2}\big{\|}\big{\|}\\ \varphi\end{array} . \tag{17}\]  
This is a ridge regression problem whose solution is  
\[\varphi^{gmp}_{\lambda}=\left(\varPhi\varPhi^{T}+\lambda\right)^{-1}\varPhi_ {N} \tag{18}\]where \(\lambda\) is a regularization parameter and \(E\) is an identity matrix. If Equation (18) is rewritten as the weighted pooling form  
\[\varphi_{\lambda}^{gmp}=\Phi\alpha_{\lambda} \tag{19}\]  
where \(\alpha_{\lambda}\) is the vector of weights, and let \(\varphi=\Phi\alpha\) in Equation (17), then the weight \(\alpha_{\lambda}\) can be obtained by:  
\[\alpha_{\lambda}= \begin{array}{c}argmin\{\big{\|}\big{\|}\,\Phi^{T}\Phi \alpha_{-N}\big{\|}^{2}+\lambda\big{\|}\Phi\alpha^{2}\big{\|}\big{\|}\\ a\end{array} \big{\|}\big{\|}\big{\}}. \tag{20}\]  
Let \(K=\Phi^{T}\Phi\), which represents a \(N\times N\) matrix of descriptor-to-descriptor similarities; then, Equation (20) can be represented as:  
\[\alpha_{\lambda}= \begin{array}{c}\arg min\{\big{\|}\big{\|}K\alpha_{-N} \big{\|}^{2}+\lambda a^{T}Ka\big{\}}\\ a\end{array} .