Average	Kp20K	Inspec	Krapivin	NUS	SemEval	DUC
Model	10	50	M	10	50	M	10	50	M	10	50	M	10	50	M	10	50	M	10	50	M
One2One variants
RNN-O2O-KP20k	4.3	8.7	13.6	6.7	13.1	19.6	7.8	13.7	20.7	4.2	8.2	13.6	4.7	11.2	17.5	2.4	6.1	10.0	0.0	0.0	0.3
RNN-O2O-KP20k-nocopy	1.2	3.6	6.2	2.4	5.8	9.8	1.2	4.0	8.4	1.5	2.4	3.4	1.5	5.5	9.7	0.4	3.7	6.1	0.0	0.0	0.0
RNN-O2O-KP20k+MagKP-ALT	5.0	9.6	14.7	6.5	13.1	19.3	8.2	16.1	23.7	5.1	9.1	15.8	7.0	13.8	18.7	3.2	5.2	10.0	0.0	0.1	0.8
BIGRNN-O2O-KP20k	7.4	13.9	19.2	11.4	20.2	27.8	12.7	22.5	31.7	5.6	11.3	16.8	10.1	18.6	24.8	4.3	10.1	13.7	0.1	0.4	0.4
BIGRNN-O2O-magkp20k-ALT	6.4	11.9	17.6	9.0	16.5	23.3	12.3	20.7	28.9	6.2	10.8	16.9	7.6	15.1	23.4	3.6	8.2	12.8	0.0	0.0	0.1
TF-O2O-KP20k	7.4	14.0	20.4	12.6	22.1	30.8	12.8	23.8	33.0	4.1	9.2	15.8	10.1	18.9	27.0	4.8	9.9	15.7	0.0	0.2	0.2
TF-O2O-KP20k-nocopy	8.8	15.4	22.3	14.0	24.4	33.5	14.9	25.7	38.9	4.7	9.9	13.4	13.2	22.0	30.9	6.1	10.7	17.2	0.0	0.0	0.1
TF-O2O-KP20k+MagKP-ALT	8.0	14.5	21.1	12.4	21.6	30.2	13.0	23.7	34.5	5.2	11.6	18.5	11.4	20.0	27.1	6.1	10.1	16.1	0.1	0.2	0.2
One2Seq variants (trained in PRES-ABS)
RNN-O2S-KP20k	2.2	2.5	2.5	2.7	3.2	3.3	2.8	3.3	3.3	3.3	3.7	3.7	2.6	2.9	3.0	1.6	1.7	1.7	0.0	0.0	0.0
RNN-O2S-KP20k-nocopy	3.1	4.1	4.2	4.9	6.7	6.8	4.6	6.3	6.4	1.8	2.1	2.1	4.5	6.2	6.2	2.6	3.6	3.6	0.0	0.0	0.0
RNN-O2S+Kp20k+MagKP-ALT	2.0	2.6	2.7	2.6	3.1	3.3	2.4	3.3	3.5	3.4	4.8	4.8	1.4	2.5	2.6	2.0	2.0	2.1	0.0	0.0	0.0
BIGRNN-O2S-KP20k	1.6	1.7	1.7	1.9	2.0	2.0	2.6	2.6	2.6	1.9	2.1	2.1	2.2	2.3	2.3	1.0	1.0	1.0	0.0	0.0	0.0
BIGRNN-O2S-KP20k+MagKP-ALT	2.6	3.0	3.0	3.2	3.7	3.8	4.6	5.2	5.2	3.3	3.5	3.5	2.9	3.7	3.7	1.8	1.9	1.9	0.0	0.0	0.0
TF-O2S-KP20k	7.0	9.8	10.2	11.1	15.0	15.5	12.1	16.6	17.5	4.2	6.7	7.1	9.0	12.5	12.7	5.9	8.3	8.4	0.0	0.0	0.0
TF-O2S-KP20k-nocopy	7.9	11.0	11.5	13.2	18.1	18.5	14.3	19.8	20.9	4.5	6.4	6.5	10.5	15.1	15.8	5.2	6.9	7.2	0.0	0.0	0.0
TF-O2S-KP20k+MagKP-ALT	9.4	12.1	12.8	13.7	16.3	16.9	15.7	19.1	20.5	9.3	14.1	14.5	12.1	15.2	16.3	5.4	7.4	8.1	0.1	0.3	0.3
TRANS+OneSeq +MAGKP (trained in PRES-ABS)
MagKP-LN-ONLY	4.0	5.8	7.2	6.6	8.9	10.1	6.9	10.9	13.1	2.2	3.8	5.4	6.1	8.0	10.1	2.4	3.0	4.2	0.0	0.0	0.0
MagKP-Nsmall-ONLY	2.7	3.9	4.6	3.7	4.7	5.3	6.3	8.8	9.7	4.0	5.7	6.3	1.0	2.5	3.3	1.5	1.6	2.5	0.0	0.0	0.3
MagKP-Nlarge-ONLY	3.3	4.3	4.7	3.4	4.6	5.0	8.3	10.2	11.0	4.2	5.8	6.5	2.2	2.9	3.2	1.6	2.1	2.7	0.0	0.0	0.0
MagKP-ONLY	4.2	5.4	6.7	4.7	6.0	7.6	8.8	11.6	13.5	6.0	7.8	9.7	2.9	3.9	5.3	2.6	3.3	3.8	0.1	0.1	0.1
MagKP-LN-MX	8.6	11.4	12.1	12.9	16.7	17.2	15.0	19.2	19.7	5.5	8.6	9.7	12.1	15.6	17.2	6.3	8.1	8.4	0.0	0.0	0.1
MagKP-Nsmall-MX	8.6	11.3	12.4	13.0	16.7	17.9	15.1	18.5	20.6	6.7	11.0	12.8	12.2	15.2	16.2	4.3	6.1	6.9	0.1	0.1	0.1
MagKP-Nlarge-MX	8.9	10.8	11.7	13.2	15.6	16.5	15.5	19.0	21.3	7.6	10.7	11.9	11.6	13.8	14.5	5.2	5.7	6.1	0.0	0.0	0.0
MagKP-MX	8.8	10.4	11.6	12.3	14.3	15.2	15.7	17.3	19.7	8.9	12.1	13.7	10.6	12.5	13.7	5.2	6.2	6.7	0.2	0.3	0.3
MagKP-LN-FT	8.0	10.7	11.1	12.9	16.3	16.8	13.6	16.7	17.7	5.8	9.4	9.8	10.2	14.4	14.6	5.4	7.3	7.8	0.0	0.0	0.0
MagKP-Nsmall-FT	9.7	12.4	13.9	14.2	18.0	19.3	16.8	19.9	22.8	9.0	13.1	14.4	11.9	15.5	17.4	6.0	7.9	9.2	0.2	0.2	0.2
MagKP-Nlarge-FT	9.9	13.2	14.2	15.0	19.0	20.3	17.2	21.2	23.6	9.4	13.3	14.1	11.2	16.7	17.5	6.6	8.8	9.5	0.3	0.3	0.4
MagKP-FT	9.6	12.9	13.7	14.8	18.9	19.8	17.2	21.4	23.1	7.9	12.3	13.6	11.5	16.2	16.9	6.3	8.5	8.8	0.2	0.2	0.2