## 5 Feature Selection and Regression Techniques  
For the TAC AESOP task, we submitted four sets of predictions to each of the four different subtasks, for a total of sixteen. Each submission was based on a supervised learning algorithm performed on its own subset of features. We limited our focus here to two variations of linear regression (non-negative least squares and robust regression), and also canonical correlation, an eigenvalue method, but would like to explore other methods in the future. In particular, we chose to start with the current methods due to their ease of of application and interpretability. In order to predict the quality of an individual summary, we took advantage of previous years' data in an interesting way. We had 13 predictors, arising from the seven linguistic features and six ROUGE-like content features. For each submission, our goal was to create a model using some subset of these features. We tested every possible combination of these 13 features. For each of the \(2^{13}-1\) combinations, we fit three models to the data. Since our ultimate goal was to predict the quality of the 2011 summaries, we trained each of these three models on the 2009 TAC data and predicted the quality of summaries in the 2010 data. For each of the three different regression methods (described below), we used the combination of features that was best able to predict the 2010 summary quality as our feature set for predicting 2011. This gave us three sets of features, each one tailored to a particular regression method. We then used each of the three combinations of features, together with its method, to train a model on the TAC 2010 summaries. This gave us the coefficients to use for predicting the quality of the TAC 2011 summaries. The values of the coefficients are given in Tables 2 and 3 for each of our sixteen submissions. For each subset of the covariates, we used three different methods of regression (canonical correlation, robust least squares, and non-negative least squares) to fit a predictive model. What follows are short descriptions of these methods. The input data in each case is an \(n\times 13\) matrix \(A\), consisting of columns of predictors, and an \(n\times 3\) matrix \(B\), consisting of columns \(b_{1},b_{2}\) and \(b_{3}\) of the three human scores given to the summaries (overall responsiveness, pyramid score, and linguistic quality score). If we used standard linear regression (ordinary least squares), we would be looking to find \(x\) such that \(||Ax-b_{i}||\) is minimized (where \(b_{i}\) is the particular column of human metrics we are trying to predict). Each of the three methods uses a variation on this theme. **Robust Least Squares** (robust): This method is finding \(x\) such that \(||w(Ax-b_{i})||\) is minimized, where \(w\) is a function that increases the weight of certain observations and decreases the weight of others. Here the function \(w\) is applied component-wise to the vector \(Ax-b_{i}\). We used Matlab's robustfit, which follows an iterative re-weighting procedure. Its default weighting function, \"bisquare\", and default tuning constant, \(4.685\), were also used. \"Bisquare\" is defined as \(w=(|r|<1)(1-r^{2})^{2}\), where each operation is done component-wise.