Table 10: Test accuracy results of grid searches for various settings. The results are computed over the last 40 training rounds (with 1000 training rounds in total). The highest test accuracy among different learning rates is marked in cyan for both algorithms.
Setting	PFL	SFL
10-2.0	10-1.5	10-1.0	10-0.5	10-2.5	10-2.0	10-1.5	10-1.0
MNIST, Logistic, C=1, K=5	90.95	89.81	87.78	84.33	90.68	87.05	81.38	78.27
MNIST, Logistic, C=1, K=20	88.98	87.00	86.06	82.49	85.87	76.88	72.82	72.14
MNIST, Logistic, C=1, K=50	86.99	85.50	85.52	81.77	78.45	69.67	67.00	67.09
MNIST, Logistic, C=2, K=5	91.72	91.48	90.70	89.84	91.97	90.79	86.68	82.43
MNIST, Logistic, C=2, K=20	91.31	90.54	90.10	89.66	90.29	84.86	78.94	75.18
MNIST, Logistic, C=2, K=50	90.69	89.89	89.78	89.20	86.45	79.24	73.89	70.85
MNIST, MLP, C=1, K=5	91.47	89.51	93.06	91.68	95.13	95.74	95.22	67.44
MNIST, MLP, C=1, K=20	78.70	82.53	90.89	87.46	95.79	94.79	91.24	51.06
MNIST, MLP, C=1, K=50	71.61	78.20	88.73	85.52	94.83	92.46	84.62	26.71
MNIST, MLP, C=2, K=5	93.32	94.35	95.66	94.58	95.40	96.14	96.14	93.00
MNIST, MLP, C=2, K=20	93.18	94.59	95.62	90.42	96.33	96.25	95.24	55.47
MNIST, MLP, C=2, K=50	93.40	94.40	95.28	87.78	96.34	95.36	91.99	43.36
MNIST, MLP, C=1, K=5	68.72	95.46	98.07	98.05	98.28	98.76	98.91	98.29
MNIST, MLP, C=1, K=20	58.20	94.30	98.02	97.41	98.43	98.94	98.68	9.84
MNIST, MLP, C=1, K=50	53.61	93.00	97.23	97.49	98.60	98.77	97.98	9.95
MNIST, MLP, C=2, K=5	96.40	98.43	98.90	98.67	98.62	98.95	98.91	98.55
MNIST, MLP, C=2, K=20	97.88	98.69	98.86	98.50	98.94	99.17	98.96	97.74
MNIST, MLP, C=2, K=50	98.07	98.65	98.93	98.12	98.95	98.99	98.85	97.88
FMNIST, LeNet-5, C=1, K=5	50.88	71.24	81.83	83.26	82.76	85.34	85.31	82.02
FMNIST, LeNet-5, C=1, K=20	39.64	69.26	80.73	81.74	78.44	80.84	80.24	61.50
FMNIST, LeNet-5, C=1, K=50	37.56	68.40	79.73	80.53	73.84	76.51	75.95	42.07
FMNIST, LeNet-5, C=2, K=5	70.59	81.00	85.44	85.16	80.94	84.51	85.59	83.83
FMNIST, LeNet-5, C=2, K=20	77.85	84.00	85.70	85.00	80.09	83.20	83.88	77.65
FMNIST, LeNet-5, C=2, K=50	81.24	84.17	86.33	84.36	77.73	82.53	81.39	71.13
FMNIST, CNN, C=1, K=5	75.57	83.44	86.98	86.40	86.83	88.66	87.82	85.01
FMNIST, CNN, C=1, K=20	73.84	82.17	85.86	85.11	83.62	87.37	86.14	73.85
FMNIST, CNN, C=1, K=50	72.94	81.26	84.67	82.41	81.75	85.39	82.42	10.00
FMNIST, CNN, C=2, K=5	83.20	86.58	88.26	87.24	86.41	88.71	89.19	87.06
FMNIST, CNN, C=2, K=20	85.12	87.46	88.80	87.08	85.39	89.24	88.48	83.65
FMNIST, CNN, C=2, K=50	85.93	87.82	88.61	86.64	85.53	88.63	87.12	79.50
CIFAR-10, VGG-9, C=1, K=5	12.51	25.67	34.89	28.77	43.79	53.73	58.63	10.00
CIFAR-10, VGG-9, C=1, K=20	11.51	22.81	28.79	21.10	34.55	27.11	10.00	10.00
CIFAR-10, VGG-9, C=1, K=50	10.14	20.58	21.95	10.00	23.41	10.00	10.00	10.00
CIFAR-10, VGG-9, C=2, K=5	38.50	50.99	57.09	53.46	58.35	68.75	71.24	10.00
CIFAR-10, VGG-9, C=2, K=20	54.70	63.97	68.07	61.92	60.82	67.93	66.50	10.00
CIFAR-10, VGG-9, C=2, K=50	62.72	67.57	69.11	64.21	62.27	67.52	62.59	10.00
CIFAR-10, ResNet-18, C=1, K=5	11.46	19.72	26.50	25.45	33.49	37.77	42.04	36.59
CIFAR-10, ResNet-18, C=1, K=20	10.56	18.48	23.41	20.86	18.99	26.41	32.70	10.00
CIFAR-10, ResNet-18, C=1, K=50	10.55	17.39	22.42	17.86	15.46	17.57	19.39	10.00
CIFAR-10, ResNet-18, C=2, K=5	23.45	32.71	43.24	47.45	41.24	51.51	65.86	68.17
CIFAR-10, ResNet-18, C=2, K=20	28.02	42.36	58.07	64.03	42.83	58.43	70.55	58.29
CIFAR-10, ResNet-18, C=2, K=50	36.22	46.69	64.75	69.71	47.57	65.33	68.40	46.80
CINIC-10, VGG-9, C=1, K=5	10.36	22.56	29.58	21.02	36.87	42.26	43.27	10.00
CINIC-10, VGG-9, C=1, K=20	10.13	19.09	22.83	15.00	28.88	31.54	10.00	10.00
CINIC-10, VGG-9, C=1, K=50	10.37	16.85	17.92	10.00	22.12	10.00	10.00	10.00
CINIC-10, VGG-9, C=2, K=5	33.52	42.24	46.12	43.43	43.64	49.28	50.64	10.00
CINIC-10, VGG-9, C=2, K=20	44.04	48.26	47.80	42.40	48.54	52.10	47.55	10.00
CINIC-10, VGG-9, C=2, K=50	42.87	46.93	48.29	40.75	48.93	51.11	41.19	13.04
CINIC-10, ResNet-18, C=1, K=5	10.64	17.16	23.15	21.37	29.52	32.70	33.55	29.92
CINIC-10, ResNet-18, C=1, K=20	10.29	14.65	18.01	16.68	19.91	20.42	29.57	10.00
CINIC-10, ResNet-18, C=1, K=50	10.18	13.65	16.27	14.35	14.56	17.72	26.30	10.00
CINIC-10, ResNet-18, C=2, K=5	22.59	28.03	35.78	39.35	34.06	38.73	43.57	44.67
CINIC-10, ResNet-18, C=2, K=20	27.18	34.74	43.40	43.46	30.27	37.96	46.13	36.81
CINIC-10, ResNet-18, C=2, K=50	30.36	37.05	40.74	38.90	33.77	43.20	42.32	11.35