## 4 Readability of Web Documents
### Bias Correction in Readability on the Web  
The good correlation values obtained when cleaning the documents on the basis of the ACCB and DSC content extraction filters indicate a linear relation between the readability value on the actual content and the index as it is computed on the automatically created extract. However, looking at the individual values in Table 2, ACCB, for instance, tends to overestimate the level of difficulty for readability.  
\begin{table}
\begin{tabular}{l r r r r r} \hline \hline \multirow{2}{*}{**Source**} & \multirow{2}{*}{**Number of documents**} & \multicolumn{4}{c}{**SMOG**} \\ \cline{3-6}  & & **Gold standard** & **Full** & **ACCB** & **DSC** \\ \hline BBC News & \(337\) & \(4.8323\) & \(4.0569\) & \(4.9360\) & \(4.8052\) \\ The Economist & \(53\) & \(5.0578\) & \(4.2486\) & \(5.1433\) & \(5.0835\) \\ Herald Tribune & \(300\) & \(5.0477\) & \(4.0891\) & \(5.0650\) & \(5.0412\) \\ MSNBC News & \(197\) & \(4.8949\) & \(4.4675\) & \(4.9050\) & \(4.8491\) \\ Yahoo News & \(227\) & \(4.9416\) & \(4.2063\) & \(4.7563\) & \(4.7670\) \\ Total & \(1114\) & \(4.9344\) & \(4.1793\) & \(4.9385\) & \(4.8820\) \\ \hline \hline \end{tabular}
\end{table}
Table 2: Average SMOG index value for web documents based on the actual article text, the full text and after cleaning the document using the content extraction methods ACCB and DSC.  
\begin{table}
\begin{tabular}{l r r} \hline \hline
**Choice of text sample** & **Correlation** & **MSE** \\ \hline Full document & \(0.3220\) & \(0.6624\) \\ ACCB & \(0.8406\) & \(0.0277\) \\ DSC & \(0.8612\) & \(0.0277\) \\ \hline \hline \end{tabular}
\end{table}
Table 3: Correlation and MSE in measuring readability for different choices of text samples.  
Our hypothesis is that these deviations are systematic and are caused by the bias of CE algorithms to achieve either a better precision or better recall when determining the main content. If this hypothesis is correct, we can correct the deviations by learning a functional relation between the true readability values obtained on the gold standard and the estimates obtained via the CE methods.  
To learn such a functional relation, we sampled a random subset of 100 documents from our test corpus to train a linear regression model on the data. Using the results of this model we can correct the bias in the computation of the readability. To furthermore check if such a bias correction in readability would be sufficient to replace CE methods entirely, we also trained a model on the SMOG index values of the full documents. This lead to the following adjusted bias-correction (\(bc\)) formulae for SMOG on different text samples:  
\[SMOG_{full}^{bc} =0.9865\cdot SMOG_{full}+0.8241 \tag{5}\] \[SMOG_{ACCB}^{bc} =0.9284\cdot SMOG_{ACCB}+0.3573\] (6) \[SMOG_{BSC}^{bc} =0.9525\cdot SMOG_{BSC}+0.2718 \tag{7}\]  
We evaluate the bias correction by computing again MSE for the adjusted values on the remaining 1.014 documents. The results in Table 4 show that the bias correction reduces MSE in all series. The biggest relative improvement for SMOG is obtained on the full documents. However, the best absolute value is achieved for the bias corrected SMOG index on the DSC filter. For the ACCB content extraction filter the improvements are still valid but do not reach the quality of DSC.