SlowGuess/NeurIPS_2023_OCR huggingface dataset:
where  $t_{\mathrm{min}}$  and  $t_{\mathrm{max}}$  are the minimum and maximum creation timestamp within the considered full input data and  $m$  is the desired total number of colors. In this way, we have  $m$  equidistant time intervals such that messages (relatively) close to one another by creation time belong to the same color and the most recent messages are in the  $m$ -th color. In this way, we can also run Fair Diversity Maximization (FDM) for a desired distribution of number of messages within a color, i.e.,  $k_{i}$ , for  $\forall i = 1,2,\dots,m$ , either by fairness (all  $k_{i}$ 's equal) or by recency ( $k_{i}$  increases with  $i$ ). In order to transform the message into a multi-dimensional space, we use BERT-model semantic embeddings [12] in our experiments (presented in the paper). Sentences and paragraphs embedding is an active area of NLP research [34, 36]. Although, the quality of sentences and paragraphs semantic embedding is out-of-scope for this work and we assume the messages are already given in the metric space, we have demonstrated (in Section D.2.4) the trend of the results is preserved by using other state-of-the-art (such as MPNETv2 [35], T5 [29], RoBERTa [23] or MiniLM [38]) and traditional (like GloVe [32]) pre-trained models for semantic embeddings. In the paper, the input "pool" of messages for the DM and FDM algorithms in the experiments is set to one month as we want to present to a user messages that are selected from a timely and fresh period and the DM and FDM approximation algorithms make a decision for the most relevant, but also diverse and fairly distributed (per color) messages. We present results from January 2018 Reddit dataset that consists of 21,474 samples, divided into four colors<sup>6</sup> and our experiments show there is a consistency in the results using input data from other periods.
We also utilize the MovieLens movie reviews dataset [18] from 2001, where the movie titles are semantically embedded into a metric space. In addition to the color determined by the creation time of the review (determined in the same way as explained for the Reddit dataset), for MovieLens dataset, we also assign a color based on the movie genre. There are 18 movie genres: {'Western', 'Documentary', 'Mystery', 'Film-Noir', 'Thriller', 'Crime', 'Adventure', 'Musical', 'Sci-Fi', 'War', 'Action', 'Fantasy', 'Comedy', "Children's", 'Animation', 'Horror', 'Romance', 'Drama'}
Description of the experiments. The main experiments we run in this work are the followings.
- First we show that if we run DM on the data, the results are not balanced as we want, see Section 5 (main part, Figure 1) and Section D.2.1 (Figures 2 and 3), and thus we need to resort to FDM. Further, we show that using FDM the diversity does not decrease by much. This is shown in Section 5 (main part, Table 2) and Section D.2.2 (Tables 4 and 5).
- Second, we show the effectiveness of the core-sets. We run our core-set construction algorithms (developed in Section 2 and Section 4) on each color independently, to get a smaller size dataset. We then run the FDM optimization once on the union of the core-sets and once on the whole data. We measure the diversity loss and runtime improvement achieved by the use of core-sets. The results are given in Section 5 (main part, Table 3) and Section D.2.3 (Table 6).
- The difference of the results (in the SUM-PAIRWISE notion of diversity) if we use alternative message embeddings are given in Section D.2.4 (Table 7).
- Last, the practical comparison of our algorithm with the state-of-the-art algorithm of Ceccarello et al. [8] for core-set construction respect with to SUM-PAIRWISE diversity is given in Section D.2.5 (Table 8).
4The messages in this dataset are given with a creation time (time stamp) and message IDs, but with no content / message text and the message content is extracted by using the Reddit API: https://github.com/reddit-archive/redisit/wiki/0Auth2. We request the messages for several months (using a similar script as the one used by the dataset creators) as we want to provide fresh messages from the recent month (our search pool) in our output.
see also: https://www.tensorflow.org/datasets/catalog/reddit and https://huggingface.co/datasets.reddit_tifu
Based on the quarter (roughly a week) to which a message belongs within this month.
D.2 Additional experiment results
D.2.1 Fairness of the DM algorithms outcome and justification of FDM need
While the DM algorithms produce diverse results, they often do not have the balance in the fairness we prefer to have. When we make the color division based on time periods ( $m = 4$ , quarters within a month), DM algorithms produce imbalanced outcome ( $k = 20$ ) for colors presence, see Figure 1 (main part) and Figure 2. The conclusions are two-fold: (i) the most present messages are not the most recent ones and (ii) the results are also not equally balanced per quarter. In the case of MovieLens where the color is the movie genre ( $m = 18$  genres), the most present movies genres in the most optimal  $k = 50$  items are Drama and Comedy across all diversity distances and some genres are not present as shown in Figure 3. This highlights the need for the colored version of the algorithms (FDM) to achieve the desired fairness.
(a) MovieLens (SUM-PAIRWISE)
(b) MovieLens (SUM-NN)
(c) MovieLens (MIN-PAIRWISE)
Figure 2: DM algorithm outcomes for all colors  $(m = 4)$  with equidistant time period as fairness colors in the MovieLens dataset  $(k = 20$  items).
(a) MovieLens (SUM-PAIRWISE)
Figure 3: DM algorithm outcomes for top 5 colors with genres as fairness colors ( $m = 18$ ) in the MovieLens dataset ( $k = 50$  items).
D.2.2 DM vs. FDM results
Reddit dataset. When we run FDM, we gain in fairness (or balancedness), but we lose in diversity over the DM expressed in different metrics. How much the diversity becomes worse when we apply the FDM, for the Reddit dataset, is shown in Table 2. Experiments show that we lose around  $1\%$  for SUM-PAIRWISE DIST; from few percent up to no more than  $20\%$  for SUM-NN DIST and around  $50\%$  for MIN-PAIRWISE DIST (due to its fragility) for various distributions per color, while achieving the desired per colors distribution, both in balancedness (uniform  $k_{i} = const$ ) and recency ( $k_{i}$  increasing linearly;  $k_{m}$  most recent items).
MovieLens dataset. The results for the MovieLens dataset follow the trend of the results of Reddit dataset, when we compare the diversity loss (and fairness gains). This is given in Table 4, comparable to Table 2 in the main part of the paper. In the cases with time as a color, experiments show that on diversity, we lose less than  $1\%$  for SUM-PAIRWISE DIST; few percents for SUM-NN DIST and around  $50 - 70\%$  for MIN-PAIRWISE DIST, while achieving the desired per color distribution. For this dataset, we also run experiments where the number of colors is larger (genres;  $m = 18$ ) and we also
see a similar trend of diversity loss of around  $2 - 3\%$  for for SUM-PAIRWISE DIST; few percents for SUM-NN DIST and around  $50 - 55\%$  for MIN-PAIRWISE DIST as shown in Table 5.
Table 4: The loss of diversity (% Div. loss) between DM vs. FDM on the full data, expressed as a relative change, of the concerned SUM-PAIRWISE, SUM-NN and MIN-PAIRWISE distances for uniform (upper part) or increasing (lower part) color values  $k_{i}$  for MovieLens dataset with time period within January 2001 as colors ( $m = 4$ ).
DM vs. FDM	SUM-PAIRWISE	SUM-NN	MIN-PAIRWISE
colorski	∑ki	% Div. loss	% Div. loss	% Div. loss
[2,4,6,8]	20	0.14%	1.46%	58.93%
[3,6,9,12]	30	0.02%	3.33%	68.94%
[4,8,12,16]	40	0.03%	1.62%	68.32%
[5,10,15,20]	50	0.02%	1.70%	67.64%
[6,12,18,24]	60	0.02%	1.15%	67.36%
Table 5: The loss of diversity (% Div. loss) between DM vs. FDM on the full data, expressed as a relative change, of the concerned SUM-PAIRWISE, SUM-NN and MIN-PAIRWISE distances for uniform color values  $k_{i}$  for MovieLens dataset with genres as colors ( $m = 18$ ).
DM vs. FDM	SUM-PAIRWISE	SUM-NN	MIN-PAIRWISE
colorski	∑ki	% Div. loss	% Div. loss	% Div. loss
[2,2,...,2]	36	2.84%	9.51%	52.63%
[3,3,...,3]	54	2.88%	5.78%	54.04%
[4,4,...,4]	72	2.87%	11.38%	53.09%
[5,5,...,5]	90	2.93%	12.52%	52.25%
18
D.2.3 Effectiveness of the core-sets results
Reddit dataset. We also run experiments to show the power of data summarization in terms of core-sets. The results in terms of diversity loss are on-par or with marginal difference if we apply the FDM to the union of core-sets, compared to FDM applied to the full data. See Table 3 for the results on both SUM-PAIRWISE and SUM-NN distances. (In the case of SUM-NN DIST, there are cases where one is marginally better than the other.) Moreover, running FDM to the union of core-sets is orders of magnitude faster compared to the same algorithm applied to the full data.
MovieLens dataset. As shown in Table 6, the comparison of the diversity outcome of the FDM applied on the full data versus the FDM applied on the data summary presented by the core-sets show similar trend as in the Reddit dataset (Table 3): few percent in diversity is lost for SUM-PAIRWISE notion, while for the SUM-NN the results are mixed either in favor of the full data outcome or the core-set in terms of diversity. In the case of MIN-PAIRWISE the loss of diversity is either  $0\%$  in a case the two points connected with the minimum distance are in the union of the core-sets, or it can be a higher value in the opposite case due to the fragility of the MIN-PAIRWISE DIST.
For example, FDM on the core-sets is occasionally better in diversity compared to the full data. One should note that the higher approximation factor contributes to this.
Table 6: The loss of diversity (\% Div. loss) expressed as a relative change of diversity distances, and the running time gains ( $\times$  times faster) of the FDM when applied to the union of core-sets compared to FDM applied to the full data for SUM-PAIRWISE, SUM-NN and MIN-PAIRWISE distances for uniform or increasing color values  $k_{i}$  for MovieLens dataset for January 2001 and  $m = 4$ .
FDM full data vs. core-sets	SUM-PAIRWISE	SUM-NN	MIN-PAIRWISE
colorski	∑ki% Div. loss Time gain (×) % Div. loss Time gain (×) % Div. loss Time gain (×)
[2,2,2,2]	8	1.16%	1515.18	2.81%	1225.64	0.00%	1734.86
[3,3,3,3]	12	1.38%	967.96	-0.16%	604.15	26.51%	1130.37
[4,4,4,4]	16	0.79%	617.67	-1.62%	325.73	46.17	874.17
[5,5,5,5]	20	0.61%	455.46	0.00%	207.71	26.75	686.42
[6,6,6,6]	24	0.62%	277.63	-0.23%	128.25	26.75	582.37
[2,4,6,8]	20	0.83%	412.02	-2.94%	205.90	9.25%	1515.18
[3,6,9,12]	30	0.77%	143.14	0.04%	83.48	0.68%	967.96
[4,8,12,16]	40	0.90%	121.84	5.30%	41.73	11.79%	617.67
[5,10,15,20]	50	0.70%	30.68	0.05%	25.92	0.00%	455.46
[6,12,18,24]	60	0.75%	18.40	-8.85%	13.07	43.38%	277.63
D.2.4 Experiment results with alternative semantic embeddings
In this part, we show that using different embeddings for the messages produce similar trend in the obtained results. Table 7 present the comparison results in terms of diversity loss in the notion of SUM-PAIRWISE DIST if we apply a FDM algorithm in comparison with the standard DM algorithm for various semantic message embeddings. The results show that we have a negligible loss of diversity, while we have the desired fairness (in terms of the requested  $k_{i}$ 's); irrelevant of the employed pre-trained model to embed the messages in a multi-dimensional space by using 6 state-of-the-art pre-trained models for embedding a message or a paragraph: BERT [12]; T5 [29]; MPNETv2 [35]; MiniLM [38]; RoBERTa [23] and GloVe [32]. We use one of the highest performance and popular pre-trained models (without further training), however the quality of semantic sentence & paragraph embeddings using Large Language Models (LLM) is out-of-scope for this work.
D.2.5 Comparison of core-set construction algorithms
In this section for the SUM-NN notion of diversity, we first compare the size of the obtained core-sets of the algorithm proposed in this paper and the one from [8], in practice, for Reddit data. The results are given in Table 8. The results show that core-set size of the algorithm from [8] can indeed be very high providing very little summarization, while the core-set of our proposal is indeed much smaller (columns 3 and 4). As a result, our algorithm is few orders of magnitude faster than the core-set construction algorithm from [8] (column 5). This is true while the diversity loss of our algorithm is negligible (less than  $1\%$  to few percent).
Table 7: The loss of diversity (\% Div. loss) between DM vs. FDM on the full data, expressed as a relative change, of the concerned SUM-PAIRWISE DIST for uniform or increasing color values  $k_{i}$  for six popular semantic embeddings. The results in the main part of the paper and in the other figures in this appendix are based on BERT [12] message embeddings.
DM vs. FDM	BERT [12]	T5 [29]	MPNETv2 [35]
colorski	∑ki	% Div. loss	% Div. loss	% Div. loss
[2,2,2,2]	8	1.22%	1.15%	0.32%
[3,3,3,3]	12	0.98%	0.60%	0.32%
[4,4,4,4]	16	0.50%	0.30%	0.20%
[5,5,5,5]	20	0.47%	0.29%	0.11%
[6,6,6,6]	24	0.19%	0.20%	0.05%
[2,4,6,8]	20	0.42%	0.26%	0.08%
[3,6,9,12]	30	0.29%	0.23%	0.14%
[4,8,12,16]	40	0.25%	0.08%	0.06%
[5,10,15,20]	50	0.16%	0.10%	0.02%
[6,12,18,24]	60	0.12%	0.10%	0.03%
MiniLM [38]	RoBERTa [23]	GloVe [32]
% Div. loss	% Div. loss	% Div. loss
[2,2,2,2]	8	1.14%	1.27%	1.11%
[3,3,3,3]	12	0.71%	0.17%	0.47%
[4,4,4,4]	16	0.38%	0.06%	0.13%
[5,5,5,5]	20	0.32%	0.14%	0.03%
[6,6,6,6]	24	0.21%	0.05%	0.10%
[2,4,6,8]	20	0.38%	0.31%	0.43%
[3,6,9,12]	30	0.35%	0.18%	0.38%
[4,8,12,16]	40	0.27%	0.19%	0.31%
[5,10,15,20]	50	0.32%	0.14%	0.32%
[6,12,18,24]	60	0.33%	0.15%	0.13%
Table 8: The size of the union of core-sets produced by the algorithm of [8] and the size of the union of core-sets produced by our algorithm. The loss of diversity (\% Div. loss) expressed as a relative change of diversity distances, and the running time gains ( $\times$  times faster) of the FDM when applied to the union of core-sets produced by our algorithm vs. the one from [8] for SUM-PAIRWISE distance for uniform or increasing color values  $k_{i}$  for the Reddit dataset.
Corsets size comparison	∪i core-sets size	core-sets compute	FDM (∪i core-sets)
colorski	∑ki	[8] alg.	our alg.	Time gain (×)	% Div. loss	Time gain (×)
[2,2,2,2]	8	12791	40	17.19	3.89%	1055.62
[3,3,3,3]	12	13486	56	9.20	0.62%	529.96
[4,4,4,4]	16	13857	72	5.28	0.26%	458.05
[5,5,5,5]	20	13993	154	3.65	1.69%	246.58
[6,6,6,6]	24	14305	164	2.78	0.27%	129.73