# Solving Multi-Document Summarization as an Orienteering Problem
## 6 Experiments
### Evaluation Results
Figure 7: ROUGE-L scores (F-measure) on DUC 2004 (Task 2). \begin{table}
\begin{tabular}{l c c c c c c c c c c c} \hline \hline  & & & & & & & & & & & & \\ \multirow{2}{*}{**System ID**} & **R-1** & **R-2** & **R-3** & **R-4** & **R-L** & **R-W** & \multicolumn{3}{c}{**Relative Improvement of**} \\ \cline{4-11}  & **MDS-OP** & **0.386142** & 0.08799 & 0.031144 & 0.013086 & **0.33438** & 0.15 & - & - & - & - & - & - \\  & 0.3212 & 0.06402 & 0.02011 & 0.00694 & 0.28472 & 0.12639 & +0.22 & +37.44 & +54.87 & +88.56 & +17.45 & +17.39 \\  & 0.3798 & **0.92152** & **0.03859** & **0.01689** & 0.32803 & 0.14707 & +1.78 & +4.51 & \({}^{-13}\).22 & \({}^{-22}\).5 & \({}^{-1}\).94 & +0.88 \\  & 0.37584 & 0.0839 & 0.02675 & 0.01068 & 0.3339 & **0.14853** & +2.74 & +4.87 & +16.43 & +42.53 & +0.14 & \({}^{-0.11}\) * \\  & 0.3319 & 0.07652 & 0.02762 & 0.01278 & 0.28452 & 0.12568 & +16.34 & +14.99 & +12.76 & +2.39 & +17.52 & +18.05 \\  & 0.37422 & 0.08033 & 0.02555 & 0.01013 & 0.32356 & 0.14308 & +3.19 & +9.54 & +21.89 & +29.18 & +3.34 & +3.69 \\  & 0.31288 & 0.0513 & 0.01364 & 0.00469 & 0.26949 & 0.11886 & +23.61 & +71.52 & +128.33 & +179.02 & +24.08 & +24.82 \\  & 0.36282 & 0.07763 & 0.02637 & 0.01232 & 0.32299 & 0.14339 & +6.43 & +13.35 & +18.10 & +6.22 & +3.53 & +3.47 \\  & 0.38877 & 0.07028 & 0.00207 & 0.00865 & 0.30801 & 0.13787 & +9.12 & +25.2 & +52.14 & +5.87 & +5.65 & +7.61 \\  & 0.3643 & 0.07987 & 0.02743 & 0.01253 & 0.31921 & 0.14289 & +6 & +10.17 & +13.54 & +4.44 & +4.75 & +3.77 \\  & 0.34142 & 0.34182 & 0.02599 & 0.01094 & 0.26622 & 0.13149 & +13.1 & +12.63 & +19.83 & +19.62 & +12.88 & +12.46 \\  & 0.37155 & 0.08528 & 0.02713 & 0.01073 & 0.32281 & 0.1441 & +3.93 & +3.18 & +14.8 & +42.19 & +3.58 & +29.6 \\  & 0.24312 & 0.01806 & 0.00265 & 0.00074 & 0.21411 & 0.09549 & +64.93 & +38.721 & +107.52 & \({}^{-1}\).1668.3 & \({}^{-1}\).17 & +55.37 * \\  & 0.3918 & 0.05853 & 0.03138 & 0.00377 & 0.30147 & 0.33399 & +13.85 & +50.338 & +133.77 & +247.11 & +10.92 & +11.23 \\  & 0.36444 & 0.08608 & 0.30422 & 0.01653 & 0.29638 & 0.13124 & +11.46 & +2.22 & -9.52 & -19.96 & +12.07 & +13.05 \\  & 0.31263 & 0.05376 & 0.014 & 0.00547 & 0.27008 & 0.17145 & +23.51 & +63.67 & +122.46 & +139.23 & +23.81 & +26.32 \\  & 0.36763 & 0.08255 & 0.02843 & 0.01212 & 0.31964 & 0.14306 & +5.04 & +6.59 & +9.55 & +7.97 & +4.61 & +3.71 \\  & 0.33962 & 0.07135 & 0.02342 & 0.01019 & 0.26276 & 0.12144 & +13.7 & +23.32 & +32.98 & +28.42 & +25.11 & +22.17 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: F-measure scores of ROUGE-1, ROUGE-2, and ROUGE-3, ROUGE-4, ROUGE-L, and ROUGE-W-1.2 metrics of MDS-OP, the baseline, and the participating systems at DUC 2004 (Task 2). The highest values are written in bold. The highest and the lowest improvements (%) of MDS-OP are indicated by \(\star\) and \(\star\), respectively. _Algorithms_**2018**, _11_, 96 \begin{table}
\begin{tabular}{l c c c c c c} \hline \hline \multirow{2}{*}{**System ID**} & \multirow{2}{*}{**AutoSummENG**} & \multirow{2}{*}{**MeMoG**} & \multirow{2}{*}{**NPowER**} & \multicolumn{3}{c}{**Relative Improvement of MDS-OP (\%)**} \\ \cline{4-7}  & & & & **AutoSummENG** & **MeMoG** & **NPowER** \\ \hline MDS-OP & **0.2157** & **0.2521** & **1.9942** & - & - & - \\ \hline MMS1 & 0.1751 & 0.1988 & 1.8441 & +23.19 & +26.81 & +8.14 \\ MMS2 & 0.1909 & 0.222 & 1.9054 & +12.99 & +13.56 * & +4.66 * \\ MMS3 & 0.164 & 0.1848 & 1.8039 & +31.52 & +36.42 & +10.55 \\ MMS5 & 0.1778 & 0.1944 & 1.8436 & +21.32 & +29.68 & +8.17 \\ MMS8 & 0.1925 & 0.2185 & 1.9046 & +12.05 * & +15.38 & +4.7 \\ MMS9 & 0.1657 & 0.1797 & 1.8013 & +30.18 & +40.29 & +10.71 \\ MMS11 & 0.1688 & 0.1836 & 1.8125 & +27.78 & +37.31 & +10.02 \\ MMS12 & 0.1475 & 0.1651 & 1.7453 & +46.24 * & +52.7 * & +14.26 * \\ MMS13 & 0.1607 & 0.1801 & 1.7911 & +34.23 & +39.98 & +11.34 \\ MMS15 & 0.1744 & 0.2004 & 1.8446 & +23.68 & +25.8 & +8.11 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Scores of MDS-OP and the participating systems on MultiLing 2015 (MMS task) obtained with evaluation methods AutoSummENG, MeMoG, and NPowER.