SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Noisy Labels. We also experiment in a model with label noise. Here, the label of any instance can be independently flipped with some probability  $p$ , as a result the true bag label sum  $k^*$  has distribution over  $\{0, \dots, q\}$ . In this case the SDP algorithms are not applicable and we omit them. Tables 9 and 10 give the test accuracy scores for Algorithm 2 and rand LTF (R) with label flip probability  $p = \{0.1, 0.25, 0.5\}$  for centered and general Gaussians. Like the balanced case, here also we evaluate both the solutions of Algorithm 2 on the test data and report the better number.
We observe that our algorithms perform significantly better in terms of accuracy than the comparative methods in all the bag distribution settings. Further, our algorithms have much lower error bounds on their accuracy scores. For the standard gaussian case, Algorithm 2 outperforms Algorithm 3 for larger  $m$ , possibly since with larger number of bags Algorithm 2 (which has higher sample complexity) can perform to its full potential. Conversely, we can observe that with larger bag sizes and dimensions, Algorithm 3 outperforms Algorithm 2 for smaller  $m$ .
For the noisy cases, from Tables 9 and 10 we observe that while the test accuracy degrades with large noise, it is fairly robust to small amounts of noise. This robustness is intuitive and we provide an explanation for the same in Appendix H (Lemma H.2).
Table 2: Comparison of Algorithms A3, A2, rand. LTF (R) and SDP (S) on  $N\left( {\mathbf{0},\mathbf{I}}\right)$  feature-vectors.
d	q	k	A3	A2	R	S
10	3	1	95.52	98.17	76.0	68.04
10	10	8	98.58	97.68	75.46	-
10	50	35	99.06	97.95	74.18	-
50	3	1	89.26	95.15	59.86	57.55
50	10	8	96.73	94.36	61.58	-
50	50	35	97.82	94.94	61.04	-
Table 3: Comparison of Algorithm A2, rand. LTF (R) and SDP algorithms (S) without offset
d	q	k	A2	R	S
10	2	1	98.12	78.26	88.40
10	3	1	98.27	77.16	67.31
10	10	5	97.9	78.66	-
10	10	8	97.87	77.64	-
10	50	25	97.87	76.67	-
10	50	35	97.9	77.17	-
50	2	1	95.64	61.25	57.83
50	3	1	95.21	61.15	58.69
50	10	5	95.59	55.06	-
50	10	8	94.34	63.17	-
50	50	25	95.16	55.76	-
50	50	35	94.74	61.02	-
(a)  $N(\mathbf{0},\Sigma)$  feature-vectors.
Table 4: Comparison of Algorithm A4, rand. LTF (R) and SDP algorithms (S) with offset
d	q	k	A4	R	S
10	2	1	92.49	77.23	82.76
10	3	1	93.67	74.96	69.62
10	10	5	94.43	76.42	-
10	10	8	92.5	65.87	-
10	50	25	92.83	70.34	-
10	50	35	92.92	65.75	-
50	2	1	94.84	59.34	58.25
50	3	1	94.7	58.36	59.37
50	10	5	95.15	57.67	-
50	10	8	92.49	56.46	-
50	50	25	94.51	54.53	-
50	50	35	94.45	57.76	-
(a)  $N(\mathbf{0},\Sigma)$  feature-vectors.
d	q	k	A4	R	S
10	2	1	93.08	78.24	88.01
10	3	1	94.62	71.49	67.57
10	10	5	93.1	70.59	-
10	10	8	94.47	67.85	-
10	50	25	93.93	65.03	-
10	50	35	93.31	66.04	-
50	2	1	94.17	59.66	61.01
50	3	1	94.41	59.92	64.24
50	10	5	94.06	56.12	-
50	10	8	92.3	59.67	-
50	50	25	93.41	55.72	-
50	50	35	94.08	56.97	-
(b)  $N(\pmb {\mu},\pmb {\Sigma})$  feature-vectors.
Table 5: Our algorithms A1 and A2 vs. rand. LTF (R) on  $N(\mathbf{0},\mathbf{I})$  feature-vectors.
d	q	k	m	A3	A2	R
10	3	1	100	82.37±0.13	90.2±0.07	73.46±0.68
10	3	1	500	91.2±0.03	96.34±0.01	73.26±0.34
10	3	1	2000	95.52±0.01	98.17±0.0	76.0±0.15
10	10	8	100	94.28±0.02	87.48±0.59	74.73±0.2
10	10	8	500	96.96±0.01	95.29±0.01	74.76±0.13
10	10	8	2000	98.58±0.0	97.68±0.0	75.46±0.12
10	50	35	100	95.79±0.01	89.99±0.07	73.34±0.5
10	50	35	500	98.26±0.0	95.63±0.03	72.83±0.22
10	50	35	2000	99.06±0.0	97.95±0.01	74.18±0.22
50	3	1	100	67.75±0.11	61.21±0.63	55.34±0.11
50	3	1	500	80.78±0.05	89.45±0.02	56.72±0.17
50	3	1	2000	89.26±0.02	95.15±0.01	59.86±0.09
50	10	8	100	85.7±0.02	60.4±0.53	56.26±0.12
50	10	8	500	94.09±0.01	86.84±0.04	60.37±0.11
50	10	8	2000	96.73±0.0	94.36±0.01	61.58±0.07
50	50	35	100	90.21±0.02	62.8±0.52	55.84±0.1
50	50	35	500	95.68±0.01	89.09±0.03	59.69±0.15
50	50	35	2000	97.82±0.0	94.94±0.01	61.04±0.14
Table 6: SDP Algorithm (S) on standard gaussian feature vectors
d	m	S
10	50	67.86±6.49
10	100	66.25±5.87
10	200	68.04±6.82
50	50	58.09±3.24
50	100	56.46±1.83
50	200	57.55±3.27
Table 7: Our algorithms A2 vs. rand. LTF (R) without offset
d	q	k	m	A2	R
10	2	1	100	91.06±0.07	77.32±0.27
10	2	1	500	96.77±0.01	76.22±0.36
10	2	1	2000	98.12±0.01	78.26±0.12
10	3	1	100	91.2±0.1	74.25±0.34
10	3	1	500	96.2±0.01	76.12±0.29
10	3	1	2000	98.27±0.0	77.16±0.24
10	10	5	100	90.29±0.11	66.84±0.9
10	10	5	500	96.08±0.01	74.16±0.62
10	10	5	2000	97.9±0.01	78.66±0.17
10	10	8	100	89.18±0.2	74.49±0.33
10	10	8	500	95.57±0.02	77.34±0.19
10	10	8	2000	97.87±0.01	77.64±0.19
10	50	25	100	90.48±0.05	62.52±0.64
10	50	25	500	95.89±0.02	64.87±0.79
10	50	25	2000	97.87±0.0	76.67±0.19
10	50	35	100	89.69±0.06	70.79±0.23
10	50	35	500	95.87±0.01	76.09±0.07
10	50	35	2000	97.9±0.01	77.17±0.15
50	2	1	100	69.54±0.52	55.16±0.12
50	2	1	500	90.94±0.02	57.9±0.15
50	2	1	2000	95.64±0.0	61.25±0.15
50	3	1	100	66.99±0.38	54.6±0.12
50	3	1	500	90.63±0.02	59.2±0.17
50	3	1	2000	95.21±0.0	61.15±0.12
50	10	5	100	61.92±0.36	53.49±0.07
50	10	5	500	90.29±0.03	55.28±0.15
50	10	5	2000	95.59±0.0	55.06±0.17
50	10	8	100	60.66±0.35	56.92±0.08
50	10	8	500	87.01±0.03	61.72±0.09
50	10	8	2000	94.34±0.01	63.17±0.09
50	50	25	100	59.33±0.58	54.31±0.05
50	50	25	500	89.24±0.03	54.53±0.15
50	50	25	2000	95.16±0.01	55.76±0.18
50	50	35	100	60.68±0.55	56.61±0.15
50	50	35	500	88.63±0.03	57.73±0.08
50	50	35	2000	94.74±0.01	61.02±0.09
(a) Centered Gaussian
d	q	k	m	A2	R
10	2	1	100	91.25±0.19	74.75±0.46
10	2	1	500	96.26±0.02	76.86±0.23
10	2	1	2000	98.18±0.01	78.32±0.23
10	3	1	100	90.44±0.12	70.64±1.09
10	3	1	500	96.04±0.04	73.32±1.1
10	3	1	2000	97.92±0.01	75.14±0.47
10	10	5	100	89.36±0.23	66.08±0.98
10	10	5	500	95.48±0.02	69.46±0.99
10	10	5	2000	97.86±0.01	70.41±0.86
10	10	8	100	85.97±0.3	63.39±0.84
10	10	8	500	94.02±0.1	69.71±0.58
10	10	8	2000	97.4±0.01	69.86±0.8
10	50	25	100	87.92±0.28	60.65±0.6
10	50	25	500	95.5±0.03	61.73±0.8
10	50	25	2000	97.57±0.02	70.48±0.69
10	50	35	100	89.12±0.18	59.08±0.52
10	50	35	500	95.2±0.03	60.67±0.62
10	50	35	2000	97.6±0.01	62.86±0.75
50	2	1	100	69.08±0.46	55.44±0.21
50	2	1	500	89.46±0.12	57.35±0.32
50	2	1	2000	94.99±0.03	58.68±0.15
50	3	1	100	66.42±1.59	57.24±0.46
50	3	1	500	89.96±0.23	57.22±0.28
50	3	1	2000	95.6±0.05	59.8±0.84
50	10	5	100	65.76±1.07	54.71±0.13
50	10	5	500	90.2±0.14	56.57±0.25
50	10	5	2000	95.27±0.04	57.43±0.13
50	10	8	100	62.48±0.7	62.27±0.82
50	10	8	500	88.04±0.29	61.4±0.49
50	10	8	2000	94.44±0.06	61.82±0.54
50	50	25	100	65.07±0.81	54.78±0.14
50	50	25	500	87.72±0.28	55.22±0.22
50	50	25	2000	94.97±0.02	53.98±0.08
50	50	35	100	66.12±0.67	56.97±0.28
50	50	35	500	87.86±0.17	59.57±0.3
50	50	35	2000	94.33±0.03	56.97±0.32
(b) General Gaussian
Table 8: SDP Algorithm S without offset
d	m	q	S
10	50	2	71.82±8.01
10	100	2	82.53±5.25
10	200	2	88.39±3.83
10	50	3	68.94±5.40
10	100	3	68.28±4.53
10	200	3	67.31±6.50
50	50	2	57.32±2.68
50	100	2	58.72±2.92
50	200	2	57.83±2.67
50	50	3	59.18±3.32
50	100	3	57.86±2.91
50	200	3	58.69±3.42
(a) Centered Gaussian
d	m	q	S
10	50	2	77.44±6.71
10	100	2	84.31±5.43
10	200	2	90.10±4.24
10	50	3	72.08±6.72
10	100	3	71.58±8.39
10	200	3	70.80±7.67
50	50	2	59.58±3.97
50	100	2	59.95±3.24
50	200	2	61.12±4.38
50	50	3	63.90±7.61
50	100	3	62.69±6.89
50	200	3	62.39±7.83
(b) General Gaussian
Table 9: Our algorithms A2 vs. rand. LTF (R) with label flip noise (mentioned in bracket) for Centered Gaussians
d	q	k	m	A2(0.1)	R(0.1)	A2(0.25)	R(0.25)	A2(0.5)	R(0.5)
10	2	1	100	81.96±0.4	72.17±0.86	66.9±0.77	63.44±0.85	57.69±0.35	57.4±0.23
10	2	1	500	93.08±0.03	77.86±0.22	76.36±1.19	65.99±0.53	57.2±0.37	59.24±0.56
10	2	1	2000	97.09±0.01	78.21±0.09	91.22±0.09	77.09±0.35	57.64±0.41	60.73±0.47
10	3	1	100	79.94±0.54	67.69±0.85	63.58±0.88	62.53±0.78	56.24±0.3	59.83±0.36
10	3	1	500	92.43±0.05	74.82±0.16	74.0±0.82	67.78±1.09	58.37±0.43	58.86±0.21
10	3	1	2000	96.53±0.01	76.22±0.16	88.51±0.13	73.74±0.6	58.68±0.39	59.6±0.55
10	10	5	100	75.87±0.98	59.41±0.57	62.86±0.65	60.47±0.63	57.69±0.38	56.38±0.32
10	10	5	500	91.97±0.07	69.36±1.07	74.61±0.9	63.14±0.49	57.7±0.28	61.17±0.56
10	10	5	2000	95.78±0.01	74.52±0.67	87.93±0.24	67.22±0.75	58.04±0.39	60.28±0.73
10	10	8	100	72.66±0.66	71.26±0.3	61.11±0.58	67.88±0.71	56.77±0.35	60.96±0.76
10	10	8	500	89.6±0.1	75.16±0.25	65.65±0.74	73.61±0.45	57.7±0.32	59.26±0.5
10	10	8	2000	94.98±0.02	76.48±0.14	78.16±0.78	76.39±0.22	57.78±0.51	57.48±0.3
10	50	25	100	76.86±0.98	60.43±0.54	63.2±0.92	61.48±0.6	56.06±0.27	59.76±0.42
10	50	25	500	91.4±0.09	65.14±0.95	72.86±1.11	61.6±0.35	57.98±0.28	57.18±0.28
10	50	25	2000	96.36±0.01	70.18±0.75	87.59±0.13	65.52±0.83	57.05±0.16	58.78±0.36
10	50	35	100	74.32±0.7	69.02±0.44	60.0±0.6	65.95±0.7	58.81±0.27	58.44±0.4
10	50	35	500	91.23±0.05	75.32±0.25	69.86±0.96	69.33±0.37	58.76±0.54	57.96±0.22
10	50	35	2000	95.39±0.01	75.46±0.13	84.6±0.23	75.25±0.26	56.95±0.28	59.66±0.53
50	2	1	100	58.14±0.36	55.05±0.15	54.43±0.12	54.15±0.11	54.57±0.15	54.78±0.09
50	2	1	500	82.97±0.07	57.46±0.27	59.14±0.36	54.14±0.06	53.73±0.08	53.72±0.09
50	2	1	2000	91.99±0.02	59.86±0.26	73.26±0.41	56.35±0.17	54.2±0.08	54.7±0.13
50	3	1	100	59.07±0.32	54.72±0.13	54.66±0.12	53.8±0.08	53.45±0.08	55.07±0.1
50	3	1	500	79.51±0.22	56.96±0.17	57.44±0.15	54.99±0.1	53.55±0.08	53.41±0.06
50	3	1	2000	90.66±0.01	60.26±0.17	66.04±0.58	56.88±0.13	54.28±0.1	54.69±0.14
50	10	5	100	56.3±0.29	54.32±0.07	54.77±0.12	53.9±0.12	53.92±0.11	54.68±0.13
50	10	5	500	75.21±0.62	54.48±0.06	57.21±0.26	54.14±0.12	53.68±0.08	52.5±0.04
50	10	5	2000	90.68±0.02	55.72±0.24	65.65±0.56	56.06±0.13	53.21±0.05	55.06±0.11
50	10	8	100	55.76±0.18	57.33±0.15	54.34±0.08	56.62±0.17	53.72±0.05	54.93±0.1
50	10	8	500	65.55±0.57	60.97±0.14	55.0±0.21	58.98±0.15	54.7±0.11	53.82±0.06
50	10	8	2000	87.17±0.04	62.39±0.11	60.28±0.26	61.82±0.14	54.76±0.17	53.41±0.05
50	50	25	100	54.6±0.06	54.16±0.11	55.45±0.17	55.12±0.12	53.1±0.04	53.91±0.07
50	50	25	500	77.1±0.22	53.81±0.07	56.34±0.26	54.44±0.09	53.4±0.05	53.8±0.09
50	50	25	2000	90.28±0.03	55.11±0.1	65.29±0.42	55.26±0.08	53.43±0.08	54.43±0.16
50	50	35	100	55.33±0.1	55.45±0.13	54.51±0.14	54.08±0.07	52.94±0.04	54.24±0.05
50	50	35	500	72.95±0.43	59.5±0.16	56.18±0.16	57.85±0.1	53.71±0.11	54.28±0.12
50	50	35	2000	89.07±0.02	60.11±0.16	62.83±0.59	57.27±0.11	54.51±0.13	53.99±0.1
Table 10: Our algorithms A2 vs. rand. LTF (R) with label flip noise (mentioned in bracket) for General Gaussians
d	q	k	m	A2(0.1)	R(0.1)	A2(0.25)	R(0.25)	A2(0.5)	R(0.5)
10	2	1	100	86.68±0.46	68.35±0.96	65.88±0.83	60.62±0.66	59.48±0.48	58.82±0.37
10	2	1	500	93.34±0.09	77.69±0.48	82.68±0.79	67.5±0.93	60.55±0.57	61.5±0.71
10	2	1	2000	96.94±0.02	75.5±0.16	90.02±0.24	76.6±0.24	60.26±0.53	60.05±0.61
10	3	1	100	80.05±0.93	64.76±1.07	64.66±0.8	62.96±0.98	62.46±0.6	60.63±0.43
10	3	1	500	91.83±0.12	70.26±1.42	74.9±1.18	65.09±0.96	59.86±0.49	58.69±0.38
10	3	1	2000	96.12±0.03	73.05±0.97	88.23±0.11	63.38±0.71	60.22±0.51	60.06±0.39
10	10	5	100	79.69±1.2	65.48±0.77	63.53±0.78	63.28±0.85	64.81±0.75	59.51±0.64
10	10	5	500	91.9±0.11	71.2±1.0	71.65±1.33	61.41±0.62	62.1±0.85	61.56±0.45
10	10	5	2000	96.21±0.03	71.95±0.71	86.2±0.28	63.04±0.93	60.43±0.71	59.52±0.4
10	10	8	100	72.46±1.1	59.4±0.47	64.74±0.92	61.22±0.57	59.93±0.75	63.45±0.87
10	10	8	500	87.61±0.47	61.66±0.79	64.32±0.68	62.22±0.74	65.04±0.65	64.1±0.61
10	10	8	2000	94.53±0.07	63.79±0.83	77.46±1.37	61.28±0.58	61.05±0.75	64.82±0.87
10	50	25	100	80.2±0.85	60.49±0.5	62.72±0.93	59.3±0.62	58.45±0.49	59.32±0.48
10	50	25	500	91.96±0.22	61.28±0.57	72.0±1.05	58.47±0.64	59.89±0.9	58.85±0.31
10	50	25	2000	96.38±0.04	63.74±0.67	84.59±0.78	62.78±0.72	59.84±0.48	60.17±0.57
10	50	35	100	69.83±1.44	60.74±0.45	62.45±0.71	59.02±0.24	62.27±0.65	60.51±0.72
10	50	35	500	90.07±0.14	62.02±0.96	66.85±1.68	61.81±1.09	61.67±0.93	62.31±0.6
10	50	35	2000	95.24±0.03	62.2±1.02	81.07±0.65	61.24±0.5	61.13±0.7	60.29±0.49
50	2	1	100	61.4±0.99	57.38±0.25	57.99±0.35	55.72±0.12	56.0±0.22	54.19±0.09
50	2	1	500	82.7±0.37	56.61±0.15	60.44±0.73	56.8±0.14	56.3±0.32	54.22±0.08
50	2	1	2000	91.5±0.07	57.45±0.22	74.38±0.66	56.52±0.16	58.35±0.17	54.0±0.13
50	3	1	100	58.07±0.42	58.16±0.26	59.48±0.35	59.29±0.35	57.95±0.72	56.86±0.23
50	3	1	500	77.58±0.45	57.2±0.24	61.05±0.55	56.24±0.35	56.85±0.44	56.77±0.23
50	3	1	2000	89.25±0.14	55.3±0.18	71.13±0.66	56.78±0.3	56.98±0.3	55.62±0.22
50	10	5	100	59.26±0.5	54.08±0.1	57.16±0.31	54.97±0.08	56.82±0.36	55.3±0.16
50	10	5	500	75.83±0.65	55.05±0.13	59.42±0.5	54.61±0.09	59.04±0.72	54.56±0.07
50	10	5	2000	88.63±0.13	54.74±0.18	68.27±0.56	55.38±0.13	60.22±0.6	54.48±0.1
50	10	8	100	58.23±0.7	60.75±0.48	57.71±0.28	60.52±0.53	55.46±0.2	59.23±0.53
50	10	8	500	68.01±1.04	60.06±0.32	58.77±0.32	61.39±0.54	58.76±0.88	60.28±0.85
50	10	8	2000	85.53±0.17	61.66±0.42	62.99±1.22	61.56±0.67	57.54±0.44	59.17±0.54
50	50	25	100	59.24±0.5	54.38±0.08	59.94±0.56	53.92±0.08	57.92±0.48	54.84±0.14
50	50	25	500	80.22±0.81	54.26±0.1	57.32±0.4	53.97±0.12	55.14±0.22	55.18±0.2
50	50	25	2000	90.32±0.12	54.03±0.1	64.51±0.91	54.95±0.15	57.8±0.33	55.2±0.13
50	50	35	100	57.1±0.34	56.03±0.31	58.14±0.6	57.16±0.36	55.18±0.24	56.69±0.27
50	50	35	500	72.81±0.83	58.09±0.44	59.59±0.43	56.23±0.21	57.36±0.62	58.39±0.32
50	50	35	2000	88.48±0.18	55.86±0.39	61.99±0.6	57.32±0.22	57.42±0.4	58.0±0.31
Table 11: Our algorithms A4 vs. rand. LTF (R) with offset
d	q	k	m	A4	R	d	q	k	m	A4	R
10	2	1	100	89.41±0.09	73.83±0.52	10	2	1	100	88.36±0.24	74.23±0.39
10	2	1	500	92.2±0.18	76.7±0.21	10	2	1	500	92.22±0.38	77.04±0.3
10	2	1	2000	92.49±0.26	77.23±0.17	10	2	1	2000	93.08±0.32	78.24±0.24
10	3	1	100	89.15±0.09	71.78±0.77	10	3	1	100	88.47±0.39	68.16±0.96
10	3	1	500	92.5±0.09	72.66±0.42	10	3	1	500	93.09±0.21	72.18±0.4
10	3	1	2000	93.67±0.13	74.96±0.1	10	3	1	2000	94.62±0.16	71.49±0.3
10	10	5	100	89.43±0.08	64.12±0.75	10	10	5	100	82.44±1.08	64.65±0.86
10	10	5	500	93.31±0.1	73.14±0.83	10	10	5	500	90.75±0.41	68.96±0.72
10	10	5	2000	94.43±0.14	76.42±0.29	10	10	5	2000	93.1±0.38	70.59±0.78
10	10	8	100	85.16±0.31	64.0±0.88	10	10	8	100	84.11±0.97	64.71±0.96
10	10	8	500	90.73±0.11	68.29±0.58	10	10	8	500	92.71±0.17	65.91±1.16
10	10	8	2000	92.5±0.17	65.87±0.77	10	10	8	2000	94.47±0.26	67.85±1.05
10	50	25	100	88.68±0.08	62.41±0.47	10	50	25	100	86.78±0.72	60.89±0.57
10	50	25	500	92.41±0.11	62.66±0.65	10	50	25	500	91.64±0.25	63.56±0.75
10	50	25	2000	92.83±0.15	70.34±0.75	10	50	25	2000	93.93±0.13	65.03±1.14
10	50	35	100	87.0±0.23	62.8±0.64	10	50	35	100	85.69±0.87	59.46±0.66
10	50	35	500	92.26±0.14	65.4±0.75	10	50	35	500	91.96±0.3	66.64±1.08
10	50	35	2000	92.92±0.19	65.75±0.78	10	50	35	2000	93.31±0.33	66.04±0.8
50	2	1	100	67.1±0.9	54.03±0.12	50	2	1	100	70.05±0.96	55.58±0.23
50	2	1	500	90.51±0.01	57.79±0.17	50	2	1	500	90.16±0.04	58.88±0.31
50	2	1	2000	94.84±0.02	59.34±0.19	50	2	1	2000	94.17±0.08	59.66±0.24
50	3	1	100	59.98±0.46	55.47±0.07	50	3	1	100	63.86±1.03	58.19±0.37
50	3	1	500	89.64±0.04	55.86±0.18	50	3	1	500	86.73±0.56	58.3±0.36
50	3	1	2000	94.7±0.01	58.36±0.26	50	3	1	2000	94.41±0.1	59.92±0.42
50	10	5	100	63.14±0.62	53.58±0.08	50	10	5	100	64.76±1.18	54.76±0.13
50	10	5	500	89.88±0.02	54.24±0.1	50	10	5	500	86.8±0.51	55.0±0.14
50	10	5	2000	95.15±0.01	57.67±0.22	50	10	5	2000	94.06±0.04	56.12±0.23
50	10	8	100	55.52±0.19	58.26±0.2	50	10	8	100	62.47±0.99	60.43±0.48
50	10	8	500	84.76±0.06	57.08±0.16	50	10	8	500	82.48±0.7	60.34±0.8
50	10	8	2000	92.49±0.03	56.46±0.23	50	10	8	2000	92.3±0.09	59.67±0.57
50	50	25	100	61.96±0.36	55.44±0.21	50	50	25	100	59.94±1.25	54.34±0.17
50	50	25	500	88.96±0.02	55.74±0.16	50	50	25	500	88.3±0.23	54.11±0.14
50	50	25	2000	94.51±0.01	54.53±0.1	50	50	25	2000	93.41±0.13	55.72±0.23
50	50	35	100	61.06±0.28	56.4±0.19	50	50	35	100	65.29±1.37	58.16±0.63
50	50	35	500	88.32±0.04	56.7±0.14	50	50	35	500	84.11±1.16	56.98±0.34
50	50	35	2000	94.45±0.01	57.76±0.2	50	50	35	2000	94.08±0.07	56.97±0.4
(a) Centered Gaussian
(b) General Gaussian
Table 12: SDP Algorithm S with offset
d	m	q	S
10	50	2	73.16±7.26
10	100	2	79.09±9.21
10	200	2	82.76±7.02
10	50	3	67.80±5.78
10	100	3	68.52±6.28
10	200	3	69.62±6.90
50	50	2	59.30±3.17
50	100	2	59.04±3.27
50	200	2	58.25±2.68
50	50	3	58.17±3.32
50	100	3	57.94±2.72
50	200	3	59.37±3.14
(a) Centered Gaussian
d	m	q	S
10	50	2	75.96±5.05
10	100	2	80.76±6.47
10	200	2	88.01±5.08
10	50	3	69.84±6.78
10	100	3	66.95±5.62
10	200	3	67.57±6.12
50	50	2	61.41±3.51
50	100	2	59.87±4.60
50	200	2	61.01±3.17
50	50	3	63.88±6.37
50	100	3	62.62±5.61
50	200	3	64.24±6.64
(b) General Gaussian
G Class ratio estimation for LTFs
The work of [13] studies the problem of matching the classifier label proportion using a single sampled bag which they call class-ratio (CR) learning as distinct from LLP. Indeed, in LLP the goal is to learn an accurate instance-level classifier from multiple sampled bags, whereas CR-learning does not guarantee instance-level performance. Further, similar to Prop. 18 of [13], CR learning LTFs over Gaussians is easy: for a bag  $B = \{\mathbf{x}^{(i)}\}_{i=1}^n$  of iid Gaussian points, a random unit vector  $\mathbf{r}$  has distinct inner products  $\{s_i := \mathbf{r}^\top \mathbf{x}^{(i)}\}_{i=1}^n$  with probability 1. The LTFs  $\{\mathrm{pos}\left(\mathbf{r}^\top \mathbf{x} - s\right) \mid s \in \{-\infty, s_1, \ldots, s_n\}\}$  achieve all possible target label proportions  $\{j/n\}_{j=0}^n$ , and one can then apply the generalization error bound in Thm. 4 of [13].
H Analysis of a Mixture of Label Sums
Definition H.1 (Mixed Bag Oracle). Given a set of bag oracles  $\operatorname{Ex}(f, \mathcal{D}, q, k)$  for  $k \in \{0, \dots, q\}$  and  $\mathbf{p} = (p_0, \dots, p_q) \in \Delta^q$  where  $\Delta^q$  is a  $q$ -simplex, a mixed bag oracle  $\operatorname{Ex}(f, \mathcal{D}, q, \mathbf{p})$  samples a bag size  $k$  from Multinoulli( $\mathbf{p}$ ) distribution<sup>2</sup> and then samples a bag from  $\operatorname{Ex}(f, \mathcal{D}, q, k)$ .
Let  $\pmb{\Sigma}_{D}$  be the covariance matrix of difference of a pair of vectors sampled u.a.r without replacement from  $\operatorname{Ex}(f, \mathcal{D}, q, \mathbf{p})$  and  $\pmb{\Sigma}_{B}$  be the covariance matrix of vectors sampled u.a.r from  $\operatorname{Ex}(f, \mathcal{D}, q, \mathbf{p})$ . If  $\pmb{\Sigma}_{Dk}$  is the covariance matrix of difference of a pair of vectors sampled u.a.r without replacement from  $\operatorname{Ex}(f, \mathcal{D}, q, k)$  and  $\pmb{\Sigma}_{Bk}$  be the covariance matrix of vectors sampled u.a.r from  $\operatorname{Ex}(f, \mathcal{D}, q, k)$  then we have the following
$$
\boldsymbol {\Sigma} _ {B} = \sum_ {k = 0} ^ {q} p _ {k} ^ {2} \boldsymbol {\Sigma} _ {B k} \quad \boldsymbol {\Sigma} _ {D} = \sum_ {k = 0} ^ {q} p _ {k} ^ {2} \boldsymbol {\Sigma} _ {D k} \tag {54}
$$
Using the above, we prove the following geometric error bound which is analogous to Lemma 4.1.
Lemma H.2. For any  $\varepsilon, \delta \in (0,1)$ , if  $m \geq O\left((d / \varepsilon^4) \log (d / \delta)(\lambda_{\max} / \lambda_{\min})^4 q^4 \left(1 / \sum_{k=1}^{q-1} p_k^2\right)^2\right)$ , then  $\hat{\mathbf{r}}$  computed in Step 3 of Alg. 2 satisfies  $\min \{\|\hat{\mathbf{r}} - \mathbf{r}_*\|_2, \|\hat{\mathbf{r}} + \mathbf{r}_*\|_2\} \leq \varepsilon$ , w.p.  $1 - \delta / 2$ .
H.1 Proof of Lemma H.2
We define and bound the following useful quantities based on  $q$ ,  $\lambda_{\mathrm{max}}$ ,  $\lambda_{\mathrm{min}}$  and  $\mathbf{p}$ .
Definition H.3. Define, (i)  $\kappa_{1}(k) := \left(\frac{2k}{q} - 1\right)^{2}\frac{2}{\pi}$  so that  $0 \leq \kappa_{1}(k) \leq 2/\pi$ , (ii)  $\kappa_{2}(k) := \frac{1}{q-1}\frac{k}{q}\left(1 - \frac{k}{q}\right)\frac{16}{\pi}$  so that  $\frac{16}{\pi q^{2}} \leq \kappa_{2}(k) \leq \frac{4}{\pi(q-1)}$  whenever  $1 \leq k \leq q-1$ , (iii)  $\kappa_{3}(\mathbf{p}) := \frac{\sum_{k=0}^{q} p_{k}^{2} \kappa_{2}(k)}{\sum_{k=0}^{q} p_{k}^{2} - \sum_{k=0}^{q} p_{k}^{2} \kappa_{1}(k)}$  so that  $\frac{16\sum_{k=1}^{q-1} p_{k}^{2}}{\pi q^{2}\sum_{k=0}^{q} p_{k}^{2}} \leq \kappa_{3}(\mathbf{p})$ , and (iv)  $\theta(\mathbf{p}) := \frac{2\lambda_{\max}}{\lambda_{\min}}\left(\frac{1}{2\sum_{k=0}^{q} p_{k}^{2} - \max(0,2\sum_{k=0}^{q} \kappa_{1}(k) - \sum_{k=0}^{q} p_{k}^{2}\kappa_{2}(k))} + \frac{1}{\sum_{k=0}^{q} p_{k}^{2} - \sum_{k=0}^{q} p_{k}^{2}\kappa_{1}(k)}\right)$  so that  $\theta(\mathbf{p}) \leq \frac{3\lambda_{\max}}{(1-2/\pi)\lambda_{\min}\left(\sum_{k=0}^{q} p_{k}^{2}\right)}$ .
Lemma H.4. The ratio  $\rho (\mathbf{r})\coloneqq \mathbf{r}^{\top}\pmb{\Sigma}_{D}\mathbf{r} / \mathbf{r}^{\top}\pmb{\Sigma}_{B}\mathbf{r}$  is maximized when  $\mathbf{r} = \pm \mathbf{r}_{*}$ . Moreover,
$$
\rho (\mathbf {r}) = 2 + \frac {\gamma (\mathbf {r}) ^ {2} \sum_ {k = 0} ^ {q} p _ {k} ^ {2} \kappa_ {2} (k)}{\sum_ {k = 0} ^ {q} p _ {k} ^ {2} - \gamma (\mathbf {r}) ^ {2} \sum_ {k = 0} ^ {q} p _ {k} ^ {2} \kappa_ {1} (k)} \quad \text {w h e r e} \quad \gamma (\mathbf {r}) := \frac {\mathbf {r} ^ {\top} \boldsymbol {\Sigma} \mathbf {r} _ {*}}{\sqrt {\mathbf {r} ^ {\top} \boldsymbol {\Sigma} \mathbf {r}} \sqrt {\mathbf {r} _ {*} ^ {\top} \boldsymbol {\Sigma} \mathbf {r} _ {*}}} a n d
$$
$$
\mathbf {r} ^ {\top} \boldsymbol {\Sigma} _ {B} \mathbf {r} = \mathbf {r} ^ {\top} \boldsymbol {\Sigma} \mathbf {r} \left(\sum_ {k = 0} ^ {q} p _ {k} ^ {2} - \gamma (\mathbf {r}) ^ {2} \sum_ {k = 0} ^ {q} p _ {k} ^ {2} \kappa_ {1} (k)\right),
$$
$$
\mathbf {r} ^ {\top} \boldsymbol {\Sigma} _ {D} \mathbf {r} = \mathbf {r} ^ {\top} \boldsymbol {\Sigma} \mathbf {r} \left(2 \sum_ {k = 0} ^ {q} p _ {k} ^ {2} - 2 \gamma (\mathbf {r}) ^ {2} \sum_ {k = 0} ^ {q} p _ {k} ^ {2} \kappa_ {1} (k) + \gamma (\mathbf {r}) ^ {2} \sum_ {k = 0} ^ {q} p _ {k} ^ {2} \kappa_ {2} (k)\right)
$$
$^{2}$ Section 2.3.2 (p. 35) of 'Machine Learning: A Probabilistic Perspective' (by K. Murphy)
Proof. The proof follows directly from Lemma 4.4 which gives us the expression for  $\Sigma_{Bk}$  and  $\Sigma_{Dk}$  and (54). Once the expression for  $\rho (\mathbf{r})$  is obtained, it is easy to see that since  $|\gamma (\mathbf{r})|\leq 1$ ,  $\rho (\mathbf{r})$  maximizes when  $\gamma (\mathbf{r}) = \pm 1$  and thus when  $\mathbf{r} = \pm \mathbf{r}_{*}$ .
Proof. (of Lemma H.2) By Lemma 3.1, taking  $m \geq O\left((d / \varepsilon_1^2)\log (d / \delta)\right)$  ensures that  $\| \mathbf{E}_B\| _2\leq$ $\varepsilon_{1}\lambda_{\mathrm{max}}$  and  $\| \mathbf{E}_D\| _2\leq \varepsilon_1\lambda_{\mathrm{max}}$  w.p. at least  $1 - \delta$  where  $\mathbf{E}_B = \hat{\pmb{\Sigma}}_B - \pmb {\Sigma}_B$  and  $\mathbf{E}_D = \hat{\pmb{\Sigma}}_D - \pmb {\Sigma}_D$  We start by defining  $\hat{\rho} (\mathbf{r})\coloneqq \frac{\mathbf{r}^\top\hat{\pmb{\Sigma}}_D\mathbf{r}}{\mathbf{r}^\top\hat{\pmb{\Sigma}}_B\mathbf{r}}$  which is the equivalent of  $\rho$  using the estimated matrices. Observe that it can be written as  $\hat{\rho} (\mathbf{r}) = \frac{\mathbf{r}^\top\Sigma_B\mathbf{r} + \mathbf{r}^\top\mathbf{E}_B\mathbf{r}}{\mathbf{r}^\top\Sigma_D\mathbf{r} + \mathbf{r}^\top\mathbf{E}_D\mathbf{r}}$  Using these we can obtain the following bound on  $\hat{\rho}$  : for any  $\mathbf{r}\in \mathbb{R}^d$ $|\hat{\rho} (\mathbf{r}) - \rho (\mathbf{r})|\leq \theta (\mathbf{p})\varepsilon_1|\rho (\mathbf{r})|$  w.p. at least  $1 - \delta (*)$  as long as  $\varepsilon_1\leq \frac{(\sum_{k = 0}^q p_k^2 - \sum_{k = 0}^q p_k^2\kappa_1(k))}{2}\frac{\lambda_{\min}}{\lambda_{\max}}$  , which we shall ensure. This is obtained as follows.
Define  $a(\mathbf{r}) \coloneqq \mathbf{r}^{\mathsf{T}}\pmb{\Sigma}_{D}\mathbf{r}, b(\mathbf{r}) \coloneqq \mathbf{r}^{\mathsf{T}}\pmb{\Sigma}_{B}\mathbf{r}, e(\mathbf{r}) \coloneqq \mathbf{r}^{\mathsf{T}}\pmb{E}_{D}\mathbf{r}, f(\mathbf{r}) \coloneqq \mathbf{r}^{\mathsf{T}}\pmb{E}_{B}\mathbf{r}$ . Thus, we get that  $|e(\mathbf{r})| \leq \varepsilon_{1}\lambda_{\max}, |f(\mathbf{r})| \leq \varepsilon_{1}\lambda_{\max}, |a(\mathbf{r})| \geq \lambda_{\min} \left( \sum_{k=0}^{q} p_{k}^{2} - \sum_{k=0}^{q} p_{k}^{2}\kappa_{1}(k) \right)$  and  $|b(\mathbf{r})| \geq \lambda_{\min} \left( 2\sum_{k=0}^{q} p_{k}^{2} - \max \left( 0, 2\sum_{k=0}^{q} p_{k}^{2}\kappa_{1}(k) - \sum_{k=0}^{q} p_{k}^{2}\kappa_{2}(k) \right) \right)$ . Notice that  $\hat{\rho}(\mathbf{r}) / \rho(\mathbf{r}) = (1 + e(\mathbf{r}) / a(\mathbf{r})) / (1 + f(\mathbf{r}) / b(\mathbf{r}))$ . Taking  $\varepsilon_{1} \leq \frac{(\sum_{k=0}^{q} p_{k}^{2} - \sum_{k=0}^{q} p_{k}^{2}\kappa_{1}(k))}{2}\frac{\lambda_{\min}}{\lambda_{\max}}$  allows us to claim that  $|\hat{\rho}(\mathbf{r}) - \rho(\mathbf{r})| \leq \varepsilon_{1}\theta(\mathbf{p})\rho(\mathbf{r})$ .