## 5 Feature Selection and Regression Techniques
\begin{table}
\begin{tabular}{c c c c c c c c c} \hline \hline  & \multicolumn{2}{c}{**Pyramid(8)**} & \multicolumn{2}{c}{**Responsiveness (25)**} & \multicolumn{2}{c}{**Readability(23)**} & \multicolumn{2}{c}{**Responsiveness(6)**} \\ \hline  & **canon** & **canon** & **canon** & **canon** & **robust** & **robust** & **nonneg** & **nonneg** \\ \hline
**Feature** & **A** & **B** & **A** & **B** & **A** & **B** & **A** & **B** \\ \hline R2 & 4.8e + 1 & 2.4e + 1, 5.3e + 1 & 4.8e + 1 & 4.7e + 1, 8.5e + 1 & 4.5e + 1 & 6.5e + 0, -7.1e + 0 & & \\ SU4 & 3.7e + 1 & 3.9e + 1, 5.4e + 1 & 3.7e + 1 & & 4.0e + 1 & -6.9e + 0, 2.0e + 1 & 2.3e + 1 & \\ Coverage & 3.6e + 1 & -3.9e - 1, -1.6e + 0 & 3.6e - 1 & -2.4e - 1, -1.3e + 0 & 2.7e - 1 & 4.3e - 1, -2.0e - 1 & 1.2e - 1 & 7.4e - 2, 2.3e - 2 \\ Bigrams & -4.0e - 1 & & & -4.0e - 1 & & -3.6e - 1 & & 5.9e - 3, 3.5e - 3 \\ Bigrams P2P & 5.9e - 1 & & & 5.9e - 1 & & 3.0e + 0 & & \\ Coverage P2P & -7.2e - 1 & & & -7.2e - 1 & & -3.2e + 0 & & \\ log2(1+Term Overlap) & 8.9e - 2 & 1.9e - 1 & 8.9e - 2 & 9.3e - 2 & & 2.9e - 2 & 0.0e + 0 \\ Norm Term Overlap & -6.6e - 1 & -1.0e + 0 & -6.6e - 1 & & 7.6e - 2 & & \\ Redundant 1 & & -2.2e - 4 & & & & -3.0e - 5 & & \\ Redundant 2 & & & & -2.0e - 4 & 6.7e - 4 & & & \\ Term Entropy & & 1.0e - 1 & & & -5.3e - 2 & -5.3e - 2 & 2.2e - 1 & & \\ -log2(sent length) & 1.2e + 0 & & & 1.2e + 0 & & -2.2e - 3 & 4.4e - 1 & & \\ Neg Sent Entropy & -9.6e - 1 & & -9.6e - 1 & & 1.6e - 1 & -3.8e - 1 & & \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Features used when predicting scores of both human and machine summaries. The value \(r\) in the weight functions is \(r=\frac{\text{resid}}{\text{tune-}s\sqrt{1-h}}\), where \(resid\) is the vector of residuals from the previous iteration, \(tune\) is a tuning constant that is divided into the residual vector before computing weights, \(h\) is the vector of leverage values from a least-squares fit, and \(s\) is an estimate of the standard deviation of the error term given by \(s=\frac{MAD}{0.6745}\). Here \(MAD\) is the median absolute deviation of the residuals from their median. The constant \(0.6745\) makes the estimate unbiased for the normal distribution. If there are \(p\) columns in \(A\), the smallest \(p\) absolute deviations are excluded when computing the median. **Non-Negative Least Squares** (nonneg): This is essentially ordinary least squares, since we are trying to minimize \(||Ax-b_{i}||\), but the variation here is that \(x\) is restricted to having non-negative entries. In order to avoid having features ignored, we first multiply all feature columns by \(\pm 1\) to ensure each is positively correlated with the response vector. We used Matlab's lsqnonneg, which also uses an iterative procedure. The procedure starts with a set of possible basis vectors and computes the associated dual vector \(\lambda\). It then selects the basis vector corresponding to the maximum value in \(\lambda\) and swaps out that vector in exchange for another possible candidate. This continues until \(\lambda\leq 0\), at which point a solution is reached. **Canonical Correlation** (canon): This method seeks a linear combination of the columns of \(A\) that has maximum correlation with a linear combination of the columns of \(B\). As in [20], we can form the covariance matrix \(\Sigma\) of the matrix \((A,B)\) and partition it as follows:  
\[\Sigma=\left( \begin{array}{cc}\Sigma_{AA}&\Sigma_{AB}\\ \Sigma_{BA}&\Sigma_{BB}\end{array} \right)\]  
where \(\Sigma_{AB}=\Sigma_{BA}^{\prime}\). Then, \(\rho^{2}\), the squared correlation between \(A\) and \(B\), is defined to be the maximum squared correlation between arbitrary linear combinations of the columns of \(A\) and \(B\), say \(v^{\prime}A\) and \(w^{\prime}B\). This is given by:  
\[\rho^{2}=\frac{(\text{cov}[v^{\prime}A,w^{\prime}B])^{2}}{\text{var}[v^{\prime }A]\,\text{var}[w^{\prime}B]}=\frac{(v^{\prime}\Sigma_{AB}w)^{2}}{(v^{\prime} \Sigma_{AA}v)(w^{\prime}\Sigma_{BB}w)}\]