## 2 Related Work
A graph-based method is presented in [12], where the authors compute term weights to give more credit to essential words in reference sentences (_i.e._, sentences from a human-generated summary) when used to automatically measure the quality of a human generated summary. This competitive approach uses centrality score as well as sentence clustering to weight co-occurring words in a sentence. The modifications of their approach were developed after the TAC data were released and are a slight variation of what the authors prepared for TAC 2011. The results of this approach are given in Figures 2-4 and are labelled 10, 13, 20 and 24. Systems 1-3 are the baseline evaluation systems (see Results section). Our systems are numbers 6, 8, 23 and 25. Systems 12 (AutoSummENG) and 18 (MeMoG) are due to [PERSON] _et al_. [8]. System 4 is VERT-F, due to [PERSON] _et al_. The vertical bar shown for each system is a 95% confidence interval for its Pearson correlation with the human metric listed at the top of the figure. Figure 2: Pearson correlation with pyramid top 14 AESOP metrics all peers and no models (2011). Systems 1-3 are the baseline evaluation systems (see Results section). Our systems are numbers 6, 8, 23 and 25. Systems 12 (AutoSummENG) and 18 (MeMoG) are due to [PERSON] _et al_. [8]. System 4 is VERT-F, due to [PERSON] _et al_. The vertical bar shown for each system is a 95% confidence interval for its Pearson correlation with the human metric listed at the top of the figure. Figure 3: Pearson correlation with readability top 14 AESOP metrics all peers and no models (2011). Systems 1 and 2 are the baseline evaluation systems (ROUGE-2 and ROUGE-SU4, respectively). Our systems are numbers 6, 8, 23 and 25. The vertical bar shown for each system is a 95% confidence interval for its Pearson correlation with the human metric listed at the top of the figure. Figure 4: Pearson correlation with responsiveness all peers and no models (2008). The following two papers consider the challenge of evaluating machine-produced summaries without model human-written summaries. [PERSON] and [PERSON] [13] propose using Latent Semantic Analysis (LSA) for summarization evaluation. The idea is that LSA can identify the most important topics in a reference text (full document or abstract) and that a summary can be judged based on how similar its topics are to those in the reference text. They produce similarity scores by computing angles between singular vectors and combinations of singular vectors and achieve good results with model summaries and separately with full text documents. In [14], [PERSON] _et al._ expand on the work of [PERSON] and [PERSON] [15] by employing Jensen-Shannon divergence to contrast the probability distribution of a summary with that of its reference text. They show that substituting the full document for the model summary works almost as well in certain cases (generic and topic-based multi-document summarization) but performs much worse in others (summarization of biographical information and summarization of opinions in blogs). We believe our work is the first to simultaneously combine the different components we used for the scoring of text summaries. We used a novelty metric, content features and various linguistic features to create a family of algorithms able to accurately predict the responsiveness and readability of a text summary.