Average	Kp20K	Krapivin	Inspec	NUS	SemEval	DUC
Model	5	10	O	M	5	10	O	M	5	10	O	M	5	10	O	M	5	10	O	M	5	10	O	M	5	10	O	M
One2One variants
RNN-O2O-KP20k	29.7	28.6	32.8	14.9	33.1	27.9	35.3	11.1	32.0	27.0	35.5	11.2	28.5	32.5	33.7	23.1	40.2	35.9	43.4	17.7	32.9	34.6	35.2	16.4	11.4	13.7	13.4	9.7
RNN-O2O-KP20k-nocopy	6.2	6.5	6.1	6.7	8.3	8.5	8.4	8.6	5.5	5.8	5.0	5.8	4.9	5.3	5.3	5.4	10.3	10.9	10.4	11.1	8.4	8.4	7.5	9.0	0.0	0.0	0.0	0.0
RNN-O2O-KP20k+MagKP-ALT	31.1	30.5	34.3	13.1	32.4	27.4	34.7	8.8	32.3	28.1	35.6	9.3	32.2	37.6	38.4	20.6	40.2	38.2	43.5	14.8	35.4	35.4	37.4	16.5	14.3	16.5	16.5	8.8
BigrRN N-O2O-KP20k	31.9	31.0	35.4	13.1	35.5	29.5	38.1	9.0	34.2	29.1	38.8	9.2	31.0	36.2	37.3	20.2	42.6	39.2	45.8	15.1	34.4	36.1	37.1	17.1	13.5	16.2	15.1	8.2
BigrRN N-O2O-magKP20k-ALT	31.5	30.7	34.5	11.8	33.1	27.9	35.3	7.8	32.3	28.1	36.4	8.1	32.2	37.1	37.8	18.6	41.2	38.1	44.9	13.3	35.6	35.7	36.0	15.2	14.4	17.2	16.2	7.8
TF-O2O-KP20k	28.2	27.3	31.2	15.1	34.5	28.9	37.4	11.7	29.5	26.4	33.0	11.2	28.0	30.8	32.6	23.6	37.6	36.1	41.1	17.5	32.9	33.0	35.1	19.3	6.9	8.4	7.8	7.1
TF-O2O-KP20k-nocopy	22.0	20.5	23.9	17.3	28.5	24.0	31.2	18.9	24.7	20.8	28.9	15.9	16.4	18.1	18.4	19.0	33.8	30.2	35.5	23.9	25.3	25.9	25.8	22.0	3.4	4.0	3.7	4.2
TF-O2O-KP20k+MagKP-ALT	29.9	29.0	32.5	13.9	33.8	28.2	36.5	10.0	32.7	27.6	34.9	9.5	28.9	33.5	34.0	22.3	39.3	37.8	41.9	15.7	33.7	34.8	35.7	18.2	10.8	12.1	11.9	7.7
One2Seq variants (trained in Pres-Abs)
RNN-O2S-KP20k	29.6	29.9	32.3	22.2	31.2	26.1	31.2	16.3	30.9	26.9	33.5	18.1	32.8	38.7	38.8	32.5	37.3	36.6	39.2	25.2	33.5	35.0	36.2	26.7	11.9	16.0	15.0	14.5
RNN-O2S-KP20k-nocopy	7.1	7.0	7.5	7.0	10.4	10.2	11.1	10.2	8.1	7.9	9.8	7.9	4.4	4.5	4.5	4.5	11.0	10.6	11.0	10.6	8.8	8.6	8.8	8.6	0.1	0.1	0.1	0.1
RNN-O2S+Kp20k+MagKP-ALT	28.1	29.1	31.0	22.0	28.2	23.8	28.2	15.3	28.0	25.8	30.6	17.0	32.9	40.3	39.9	33.4	35.1	33.2	36.4	23.6	30.6	33.1	34.0	26.4	13.5	18.4	17.0	16.2
BigrRN N-O2S-KP20k	28.8	29.0	31.6	22.7	30.2	25.7	30.4	16.7	29.8	26.4	32.4	18.2	31.6	37.5	38.1	32.5	37.4	35.7	39.7	27.0	32.5	33.7	35.3	27.2	11.6	15.0	14.1	14.6
BigrRN N-O2S-KP20k-Alt	29.1	29.5	31.5	21.2	28.2	23.7	28.2	15.0	28.9	25.6	30.9	16.9	34.9	41.1	40.1	32.0	35.9	34.3	37.6	24.0	32.0	33.8	34.8	24.0	14.5	18.5	17.4	15.5
TF-O2S-KP20k	30.4	29.4	32.9	25.5	34.6	29.0	36.2	21.5	32.4	28.1	36.4	21.8	31.5	36.6	36.9	34.6	40.1	37.3	42.3	32.0	33.9	34.2	34.8	30.9	10.1	11.4	11.0	11.9
TF-O2S-KP20k-nocopy	25.6	24.5	27.1	23.7	32.3	29.0	33.9	27.9	28.5	25.1	31.5	24.1	23.2	24.6	25.3	24.7	36.9	34.5	37.5	33.5	27.4	28.4	29.5	26.8	5.2	5.1	5.2	5.2
TF-O2S-KP20k+MagKP-ALT	32.7	32.2	35.9	24.9	36.8	30.2	37.7	18.9	35.2	29.9	37.6	20.1	32.2	38.8	39.4	35.6	41.8	39.2	44.1	29.3	35.6	36.5	38.7	29.9	14.8	18.4	17.7	15.8
TRANS+One2Seq (trained in Pres-Abs)
MagKP-LN-ONLY	26.1	26.7	27.6	25.0	28.1	25.1	28.0	20.5	27.8	26.4	28.7	23.8	29.6	34.3	34.3	35.1	33.5	34.0	34.9	31.6	28.9	30.3	30.2	29.1	8.7	10.0	9.6	10.2
MagKP-Nsmall-ONLY	22.3	23.2	23.3	22.5	20.8	19.8	20.9	18.6	25.2	24.3	26.0	23.2	30.8	34.0	33.9	33.1	26.2	27.0	27.0	26.1	24.1	26.2	24.8	26.3	6.9	7.6	7.5	7.7
MagKP-Nlarge-ONLY	21.9	23.1	23.6	22.3	20.4	19.6	21.1	18.2	24.8	23.5	25.6	22.3	32.6	36.2	36.1	35.2	26.0	26.6	28.1	26.2	21.4	25.0	23.3	24.4	6.2	7.7	7.3	7.8
MagKP-ONLY	25.0	26.8	27.3	24.5	25.3	22.3	25.5	18.4	26.2	25.1	28.0	21.7	31.3	38.7	37.2	37.0	29.9	31.0	31.3	28.3	26.5	30.3	29.5	28.4	11.0	13.3	12.5	13.4
MagKP-LN-MX	31.5	30.8	34.4	24.6	35.5	29.3	36.9	19.7	34.2	28.6	37.9	20.2	31.5	38.0	37.8	34.6	41.7	38.7	44.6	29.6	32.7	35.0	34.5	28.9	13.3	15.4	14.8	14.6
MagKP-Nsmall-MX	31.0	30.7	33.5	26.2	35.3	29.2	36.5	21.1	34.1	28.7	37.0	22.7	31.6	38.2	37.2	37.0	40.6	38.5	42.6	32.6	33.4	36.2	35.5	30.6	11.1	13.4	12.4	13.3
MagKP-Nlarge-MX	31.8	31.0	34.7	26.2	36.3	30.0	37.1	20.7	34.9	29.7	36.9	23.0	31.8	37.8	38.3	37.7	41.9	39.5	44.8	31.4	34.4	35.2	37.6	30.5	11.3	13.8	13.3	13.6
MagKP-MX	32.2	32.0	34.9	26.4	36.3	29.8	37.4	19.7	35.0	30.0	37.3	22.5	32.0	39.8	38.7	38.1	41.4	39.8	44.6	31.0	34.8	36.7	36.9	31.2	13.5	16.0	14.8	16.0
MagKP-LN-FT	32.0	31.3	34.6	25.7	36.2	29.8	37.7	20.8	34.9	29.6	36.2	21.2	32.1	38.0	38.2	36.4	41.4	39.4	44.9	30.6	35.4	36.2	36.8	30.9	11.9	14.9	14.1	14.4
MagKP-Nsmall-FT	32.6	31.8	35.7	26.2	36.4	30.1	37.7	20.4	35.7	30.5	39.4	22.1	32.6	38.4	38.8	37.1	43.0	39.5	45.6	30.9	34.9	35.6	37.4	30.9	13.3	16.5	15.3	16.0
MagKP-Nlarge-FT	33.5	32.8	36.6	25.8	37.0	30.4	37.9	19.6	36.6	30.6	38.9	21.7	33.3	39.5	39.8	36.5	44.0	40.2	47.9	29.9	34.3	36.4	35.9	30.9	16.0	19.6	19.2	16.1
MagKP-FT	33.6	32.3	36.3	26.5	37.1	30.5	38.3	20.4	36.1	30.6	38.4	22.5	32.4	38.1	38.5	36.0	43.9	40.1	46.0	32.4	36.6	37.0	39.2	31.8	15.1	17.5	17.5	15.7
Abstractive Neural Generation
CopyRN N (Meng et al.)
CopyRN N* (Yuan et al.)
Cot R N (Chen et al.)
ParaNet + CoAtt (Zhao and Zhang)
CatSeq TG-2RF† (Chan et al.)
KG-KE-KR-M† (Chen et al.)
CatSeq Q (Yuan et al.)