Original: T WO - DIMENSIONAL VISUALIZATION OF LARGE DOCUMENT LIBRARIES USING T -SNE
Revised: T WO - DIMENSIONAL VISUALIZATION OF LARGE DOCUMENT LIBRARIES USING T -SNE

Original: A BSTRACT
Revised: A BSTRACT

Original: We benchmarked different approaches for creating 2D visualizations of large document libraries, using the MEDLINE (PubMed) database of the entire biomedical literature as a use case (19 million scientific papers).
Revised: We benchmarked different approaches for creating 2D visualizations of large document libraries, using the MEDLINE (PubMed) database of the entire biomedical literature as a use case (19 million scientific papers).

Original: Our optimal pipeline is based on log-scaled TF-IDF representation of the abstract text, SVD preprocessing and t-SNE with uniform affinities, early exaggeration annealing, and extended optimization.
Revised: Our optimal pipeline is based on log-scaled TF-IDF representation of the abstract text, SVD preprocessing and t-SNE with uniform affinities, early exaggeration annealing, and extended optimization.

Original: The resulting embedding distorts local neighborhoods but shows meaningful organization and rich structure on the level of narrow academic fields
Revised: The resulting embedding distorts local neighborhoods but shows meaningful organization and rich structure on the level of narrow academic fields

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Many academic fields that traditionally did not have to deal with high-dimensional data analysis now face the challenge of having rich datasets with millions of samples.
Revised: Many academic fields that traditionally did not have to deal with high-dimensional data analysis now face the challenge of having rich datasets with millions of samples.

Original: One example is digital humanities, with digital libraries now providing millions of documents.
Revised: One example is digital humanities, with digital libraries now providing millions of documents.

Original: Here we focus on two-dimensional visualization of such libraries based on the document text (Schmidt, 2018).
Revised: Here we focus on two-dimensional visualization of such libraries based on the document text (Schmidt, 2018).

Original: An alternative approach is to base visualization on the citation graph (Noichl, 2021)
Revised: An alternative approach is to base visualization on the citation graph (Noichl, 2021)

Original: (see also opensyllabus.org , paperscape.org , or connectedpapers.com ), which is beyond the scope of our analysis.
Revised: (see also opensyllabus.org , paperscape.org , or connectedpapers.com ), which is beyond the scope of our analysis.

Original: One of the most popular dimensionality reduction algorithms for 2D visualization is t-SNE (van der
Revised: One of the most popular dimensionality reduction algorithms for 2D visualization is t-SNE (van der

Original: Maaten & Hinton, 2008), based on the earlier SNE (Hinton & Roweis, 2002).
Revised: Maaten & Hinton, 2008), based on the earlier SNE (Hinton & Roweis, 2002).

Original: Modern implementations (Linderman et al., 2019; Artemenkov & Panov, 2020) allow to run t-SNE on millions of samples.
Revised: Modern implementations (Linderman et al., 2019; Artemenkov & Panov, 2020) allow to run t-SNE on millions of samples.

Original: Closely related algorithms include LargeVis (Tang et al., 2016) and UMAP (McInnes et al., 2018).
Revised: Closely related algorithms include LargeVis (Tang et al., 2016) and UMAP (McInnes et al., 2018).

Original: These methods have already been used to visualize document corpora such as the
Revised: These methods have already been used to visualize document corpora such as the

Original: HathiTrust library ( n = 14 M) (Schmidt, 2018) or its subsets (Kobak et al., 2019), but there has been no systematic benchmark of different algorithms and processing choices.
Revised: HathiTrust library ( n = 14 M) (Schmidt, 2018) or its subsets (Kobak et al., 2019), but there has been no systematic benchmark of different algorithms and processing choices.

Original: Here we develop a t-SNE pipeline to produce a visualization (Figure 1) of the MEDLINE database of scientific articles on life science and biomedicine (used by the PubMed search engine, pubmed.ncbi.nlm.nih.gov ).
Revised: Here we develop a t-SNE pipeline to produce a visualization (Figure 1) of the MEDLINE database of scientific articles on life science and biomedicine (used by the PubMed search engine, pubmed.ncbi.nlm.nih.gov ).

Original: We benchmark different text preprocessing methods, embedding algorithms (t-SNE and UMAP) and t-SNE parameters, and show how to overcome challenges arising from the extremely large sample size ( n = 19 M papers).
Revised: We benchmark different text preprocessing methods, embedding algorithms (t-SNE and UMAP) and t-SNE parameters, and show how to overcome challenges arising from the extremely large sample size ( n = 19 M papers).

Original:  M ETHODS 2. T -SNE ALGORITHM , AFFINITIES , AND EXAGGERATION
Revised:  M ETHODS 2. T -SNE ALGORITHM , AFFINITIES , AND EXAGGERATION

Original: The t-SNE objective is to find a low-dimensional representation
Revised: The t-SNE objective is to find a low-dimensional representation

Original: y i ∈ R 2 of high-dimensional points x i ∈ R d that would preserve pairwise similarities (affinities) p ij between points.
Revised: y i ∈ R 2 of high-dimensional points x i ∈ R d that would preserve pairwise similarities (affinities) p ij between points.

Original: The affinities are normalized to sum to 1 and are defined as follows:
Revised: The affinities are normalized to sum to 1 and are defined as follows:

Original: [Equation]
Revised: [Equation]

Original: The variance σ 2 i of the Gaussian similarity kernel is chosen to yield a pre-specified value of the perplexity P = 2 H of p j | i values,
Revised: The variance σ 2 i of the Gaussian similarity kernel is chosen to yield a pre-specified value of the perplexity P = 2 H of p j | i values,

Original: where H = − (cid:80) j ̸ = i p
Revised: where H = − (cid:80) j ̸ = i p

Original: j | i log 2 p j | i is the entropy.
Revised: j | i log 2 p j | i is the entropy.

Original: The affinities decay exponentially with distance, taking infinitesimal values for the majority of pairs of points.
Revised: The affinities decay exponentially with distance, taking infinitesimal values for the majority of pairs of points.

Original: Modern t-SNE implementations (van Der Maaten, 2014; Linderman et al., 2019) explicitly truncate the affinities and use the k -nearest-neighbor (kNN) graph of the data with k = 3 P as the input.
Revised: Modern t-SNE implementations (van Der Maaten, 2014; Linderman et al., 2019) explicitly truncate the affinities and use the k -nearest-neighbor (kNN) graph of the data with k = 3 P as the input.

Original: The affinity matrix can be further simplified by replacing Gaussian similarity kernel with the kNN graph adjacency matrix, by defining v j | i
Revised: The affinity matrix can be further simplified by replacing Gaussian similarity kernel with the kNN graph adjacency matrix, by defining v j | i

Original: = 1 if point j is within k nearest neighbors of point i andotherwise.
Revised: = 1 if point j is within k nearest neighbors of point i andotherwise.

Original: These uniform affinities with k = P / 3 typically yield a layout very similar to Gaussian affinities with perplexity P (B ¨ ohm et al., 2020), but require kNN graph with 9x smaller value of k .
Revised: These uniform affinities with k = P / 3 typically yield a layout very similar to Gaussian affinities with perplexity P (B ¨ ohm et al., 2020), but require kNN graph with 9x smaller value of k .

Original: In the low-dimensional space, affinities q ij are defined using the Cauchy similarity kernel:
Revised: In the low-dimensional space, affinities q ij are defined using the Cauchy similarity kernel:

Original: [Equation]
Revised: [Equation]

Original: and the loss function is the Kullback-Leibler divergence (cid:80) ij p ij log( p ij /q ij ) .
Revised: and the loss function is the Kullback-Leibler divergence (cid:80) ij p ij log( p ij /q ij ) .

Original: This loss is optimized using gradient descent, which makes close neighbors feel attractive forces, while other points feel repulsive forces.
Revised: This loss is optimized using gradient descent, which makes close neighbors feel attractive forces, while other points feel repulsive forces.

Original: One standard optimization trick is to increase the strength of all attractive forces by a factor ρ = 12 in the first 250 iterations.
Revised: One standard optimization trick is to increase the strength of all attractive forces by a factor ρ = 12 in the first 250 iterations.

Original: This ‘early exaggeration’ (van der Maaten & Hinton, 2008) helps avoiding cluster fragmentation.
Revised: This ‘early exaggeration’ (van der Maaten & Hinton, 2008) helps avoiding cluster fragmentation.

Original: Recent work (Kobak & Berens, 2019; B ¨ ohm et al., 2020) showed that in some cases, particularly for large datasets, it may be beneficial to keep some exaggeration on until the end of optimization.
Revised: Recent work (Kobak & Berens, 2019; B ¨ ohm et al., 2020) showed that in some cases, particularly for large datasets, it may be beneficial to keep some exaggeration on until the end of optimization.

Original: While high exaggeration helps preserving continuous manifold structures and emphasizes large clusters (B ¨ ohm et al., 2020), it comes with a price of distorting local neighborhoods.
Revised: While high exaggeration helps preserving continuous manifold structures and emphasizes large clusters (B ¨ ohm et al., 2020), it comes with a price of distorting local neighborhoods.

Original: 2. MEDLINE DATASET AND EXPERIMENTAL SETUP
Revised: 2. MEDLINE DATASET AND EXPERIMENTAL SETUP

Original: We downloaded the full MEDLINE database ( ∼ 250 GB) as XML files using the bulk download service ( www.nlm.nih.gov/databases/download/pubmed_medline.html ) on 26.01.2021.
Revised: We downloaded the full MEDLINE database ( ∼ 250 GB) as XML files using the bulk download service ( www.nlm.nih.gov/databases/download/pubmed_medline.html ) on 26.01.2021.

Original: We used the Python xml package to extract PubMed ID, title, abstract, language, journal name, and publication date of all ∼ 30 M papers.
Revised: We used the Python xml package to extract PubMed ID, title, abstract, language, journal name, and publication date of all ∼ 30 M papers.

Original: We filtered out all 4.7 M non-English papers and all 7.6 M English papers with abstracts shorter than 250 or longer than 4000 symbols, leaving 19,016,308 papers for further analysis.
Revised: We filtered out all 4.7 M non-English papers and all 7.6 M English papers with abstracts shorter than 250 or longer than 4000 symbols, leaving 19,016,308 papers for further analysis.

Original: We labeled the dataset by selecting 21 frequent terms among journal names: bioinformatics, biology, cancer, cardiology, cell, chemistry, ecology, engineering, environmental, food, genetics, immunology, material, microbiology, neuroscience, nutrition, pediatric, psychiatry, psychology, surgery, and veterinary.
Revised: We labeled the dataset by selecting 21 frequent terms among journal names: bioinformatics, biology, cancer, cardiology, cell, chemistry, ecology, engineering, environmental, food, genetics, immunology, material, microbiology, neuroscience, nutrition, pediatric, psychiatry, psychology, surgery, and veterinary.

Original: Papers were assigned a label if their journal title contained that term (journal titles containing more than one term were assigned randomly to one of them), and left unlabeled otherwise.
Revised: Papers were assigned a label if their journal title contained that term (journal titles containing more than one term were assigned randomly to one of them), and left unlabeled otherwise.

Original: This resulted in 5,147,755 labeled papers (27%).
Revised: This resulted in 5,147,755 labeled papers (27%).

Original: All experiments were performed in Python.
Revised: All experiments were performed in Python.

Original: We used Scikit-learn 0.24.1 (Pedregosa et al., 2011) for TF-IDF vectorizer, count vectorizer, and for truncated SVD.
Revised: We used Scikit-learn 0.24.1 (Pedregosa et al., 2011) for TF-IDF vectorizer, count vectorizer, and for truncated SVD.

Original: We used openTSNE 0.6.0 (Poliˇcar et al., 2019), a Python reimplementation of FIt-SNE (Linderman et al., 2019), and UMAP 0.5.
Revised: We used openTSNE 0.6.0 (Poliˇcar et al., 2019), a Python reimplementation of FIt-SNE (Linderman et al., 2019), and UMAP 0.5.

Original: (McInnes et al., 2018).
Revised: (McInnes et al., 2018).

Original: We used default parameters, unless specified.
Revised: We used default parameters, unless specified.

Original: All experiments were run on a server with 384 GB of RAM and 64 double-threaded 2.90 GHz CPU cores.
Revised: All experiments were run on a server with 384 GB of RAM and 64 double-threaded 2.90 GHz CPU cores.

Original: Parsing the XML files took ∼ 10 h. Computing TF-IDF representation took 1 h. Truncated
Revised: Parsing the XML files took ∼ 10 h. Computing TF-IDF representation took 1 h. Truncated

Original: SVD took 4 h, t-SNE affinity calculation took 2 h, and subsequent t-SNE optimization took 2 h for the default 750 iterations, 4 h for 2250 iterations, and over 3 days for 10,250 iterations.
Revised: SVD took 4 h, t-SNE affinity calculation took 2 h, and subsequent t-SNE optimization took 2 h for the default 750 iterations, 4 h for 2250 iterations, and over 3 days for 10,250 iterations.

Revised: The code is available at https://github.com/berenslab/pubmed-tsne-iclr .

Original:  R ESULTS
Revised:  R ESULTS

Original: TF-IDF
Revised: TF-IDF

Original: The abstracts were transformed into the bag-of-words representation C ij , with elements denoting the number of times word j appears in abstract i .
Revised: The abstracts were transformed into the bag-of-words representation C ij , with elements denoting the number of times word j appears in abstract i .

Original: This was a sparse matrix of 19,016,308 × 4,114,381 size (0.003% non-zero elements).
Revised: This was a sparse matrix of 19,016,308 × 4,114,381 size (0.003% non-zero elements).

Original: We tried various normalization approaches and used kNN classification accuracy to assess the quality of each representation (Table 1).
Revised: We tried various normalization approaches and used kNN classification accuracy to assess the quality of each representation (Table 1).

Original: We obtained the highest kNN accuracy (0.71) with the TF-IDF (term frequency – inverse document frequency) representation (Jones, 1972) with log-scaling, as defined in the scikit-learn implementation:
Revised: We obtained the highest kNN accuracy (0.71) with the TF-IDF (term frequency – inverse document frequency) representation (Jones, 1972) with log-scaling, as defined in the scikit-learn implementation:

Original: [Equation]
Revised: [Equation]

Original: where n is the total number of documents.
Revised: where n is the total number of documents.

Original: In scikit-learn, the TF term is substituted by word counts
Revised: In scikit-learn, the TF term is substituted by word counts

Original: C ij for computational simplicity and instead, X ij is row-normalized such that each row has norm 1.
Revised: C ij for computational simplicity and instead, X ij is row-normalized such that each row has norm 1.

Revised: as an approximate average abstract length (the average was 106 words).

Original: [Table]
Revised: [Table]

Revised: C ij / (cid:80) k

Revised: C

Original: [Equation]
Revised: [Table] [Table] [Table] [Table]

Revised: IDF without

Original: [Table]
Revised: [Table]

Revised: -scaling

Original: SVD It is computationally infeasible to use the entire X ij matrix for t-SNE.
Revised: C ij instead of 1 + log 2 C ij in Eq. [Table] TF [Table]

Revised: IDF with log-scaling

Original:  We therefore used singular value decomposition (SVD) to reduce dimensionality down to 300 — the largest number we could use given our memory resources.
Revised: X ij , Eq. [Table] SVD It is computationally infeasible to use the entire X ij matrix for t-SNE. We therefore used singular value decomposition (SVD) to reduce dimensionality down to 300 — the largest number we could use given our memory resources.

Original: We employed SVD instead of PCA to avoid centering the sparse matrix.
Revised: We employed SVD instead of PCA to avoid centering the sparse matrix.

Original: We did not observe any difference in performance between the randomized and arpack SVD solvers and used the latter solver for all further processing (randomized solver was 25% faster but used up 22% more RAM).
Revised: We did not observe any difference in performance between the randomized and arpack SVD solvers and used the latter solver for all further processing (randomized solver was 25% faster but used up 22% more RAM).

Original: After SVD, the kNN accuracy decreased only slightly, to 0.66 (Figures 2b, A4).
Revised: After SVD, the kNN accuracy decreased only slightly, to 0.66 (Figures 2b, A4).

Original: At the same time, kNN recall (fraction of preserved k = 10 nearest neighbors, measured on a random subset of 500 papers)
Revised: At the same time, kNN recall (fraction of preserved k = 10 nearest neighbors, measured on a random subset of 500 papers)

Original: (Lee & Verleysen, 2009) was only 0.05, meaning that
Revised: (Lee & Verleysen, 2009) was only 0.05, meaning that

Original: SVD was unable to accurately preserve nearest neighbors in our dataset (Figure A4).
Revised: SVD was unable to accurately preserve nearest neighbors in our dataset (Figure A4).

Original: See Table A for 10 nearest neighbors in the TF-IDF and SVD spaces for one exemplary paper.
Revised: See Table A for 10 nearest neighbors in the TF-IDF and SVD spaces for one exemplary paper.

Original: t-SNE affinities We found it impossible to use default t-SNE affinities with our sample size due to memory constraints, so we used uniform affinities (as implemented in openTSNE) with k = 10 .
Revised: t-SNE affinities We found it impossible to use default t-SNE affinities with our sample size due to memory constraints, so we used uniform affinities (as implemented in openTSNE) with k = 10 .

Original: Using a n = 2 M subset of the data, we confirmed that the resulting layout was nearly identical to using default affinities with perplexity 30 (Figure A5).
Revised: Using a n = 2 M subset of the data, we confirmed that the resulting layout was nearly identical to using default affinities with perplexity 30 (Figure A5).

Original: Default affinities required 2.7x more RAM and took 1.05x more time.
Revised: Default affinities required 2.7x more RAM and took 1.05x more time.

Original: t-SNE exaggeration We experimented with different values of exaggeration, ρ ∈ { 1 , 2 , 4 } , and found that ρ > 1 led to lower kNN recall (Figure 2a), in agreement with B¨ohm et al.
Revised: t-SNE exaggeration We experimented with different values of exaggeration, ρ ∈ { 1 , 2 , 4 } , and found that ρ > 1 led to lower kNN recall (Figure 2a), in agreement with B¨ohm et al.

Original: (2020), and also lower kNN accuracy (Figure 2b).
Revised: (2020), and also lower kNN accuracy (Figure 2b).

Original: Visually, ρ = 2 led to somewhat increased cluster separation,  whereas ρ = 4 lumped all points together (Figure A6), and neither provided clear benefits compared to ρ = 1 .
Revised: Visually, ρ = 2 led to somewhat increased cluster separation, [Figure] whereas ρ = 4 lumped all points together (Figure A6), and neither provided clear benefits compared to ρ = 1 .

Original: We conclude that exaggeration was not helpful for visualizing the dataset studied here.
Revised: We conclude that exaggeration was not helpful for visualizing the dataset studied here.

Original: [Figure] Without exaggeration, kNN accuracy after t-SNE was 0.57, while kNN recall was 0.014 compared to the SVD representation (Figure 2).
Revised: Without exaggeration, kNN accuracy after t-SNE was 0.57, while kNN recall was 0.014 compared to the SVD representation (Figure 2).

Original: We observed that kNN recall after t-SNE strongly decreased with increasing the sample size (Figure A7): from 0.2 at n = 2000 to 0.01 at n = 19 M.
Revised: We observed that kNN recall after t-SNE strongly decreased with increasing the sample size (Figure A7): from 0.2 at n = 2000 to 0.01 at n = 19 M.

Original: UMAP UMAP (McInnes et al., 2018), a recent and very popular neighbor embedding method, yields very similar results to t-SNE with increased exaggeration, such as ρ = 4 (B ¨ ohm et al., 2020).
Revised: UMAP UMAP (McInnes et al., 2018), a recent and very popular neighbor embedding method, yields very similar results to t-SNE with increased exaggeration, such as ρ = 4 (B ¨ ohm et al., 2020).

Original: Using a n = 2 M subset of the data, we confirmed that the UMAP embedding was qualitatively similar to the t-SNE embedding with ρ = 4 (Figure A8), and its kNN recall and kNN accuracy were both in between t-SNE values with ρ = 2 and ρ = 4 , and worse than t-SNE without exaggeration (Figure A9).
Revised: Using a n = 2 M subset of the data, we confirmed that the UMAP embedding was qualitatively similar to the t-SNE embedding with ρ = 4 (Figure A8), and its kNN recall and kNN accuracy were both in between t-SNE values with ρ = 2 and ρ = 4 , and worse than t-SNE without exaggeration (Figure A9).

Original: We used a subset of the data for this experiment because we were unable to run
Revised: We used a subset of the data for this experiment because we were unable to run

Original: UMAP on the entire dataset due to memory constraints (UMAP uses Pynndescent for kNN graph construction and it needs more RAM than Annoy, employed by openTSNE).
Revised: UMAP on the entire dataset due to memory constraints (UMAP uses Pynndescent for kNN graph construction and it needs more RAM than Annoy, employed by openTSNE).

Original: Number of gradient descent iterations By default, gradient descent in openTSNE runs foriterations (with learning rate n/ 12 , Belkina et al., 2019).
Revised: Number of gradient descent iterations By default, gradient descent in openTSNE runs foriterations (with learning rate n/ 12 , Belkina et al., 2019).

Original: We observed that while kNN accuracy plateaued after around 300 iterations (Figure 2b), kNN recall did not plateau within 750 iterations (Figure 2a), so we used 2250 iterations for most experiments.
Revised: We observed that while kNN accuracy plateaued after around 300 iterations (Figure 2b), kNN recall did not plateau within 750 iterations (Figure 2a), so we used 2250 iterations for most experiments.

Original: Running optimization even longer increased kNN recall further (to 0.03 with respect to SVD, 0.01 with respect to TF-IDF, both after 10,250 iterations, Figure 2c), but took a long time: 4 hours for 2250 but over 3 days for 10, iterations.
Revised: Running optimization even longer increased kNN recall further (to 0.03 with respect to SVD, 0.01 with respect to TF-IDF, both after 10,250 iterations, Figure 2c), but took a long time: 4 hours for 2250 but over 3 days for 10, iterations.

Original: Note that the run time does not scale linearly because FIt-SNE approximation slows down when the embedding increases in size (Linderman et al., 2019).
Revised: Note that the run time does not scale linearly because FIt-SNE approximation slows down when the embedding increases in size (Linderman et al., 2019).

Original: We felt that 2250 iterations provided a reasonable trade-off between accuracy and runtime.
Revised: We felt that 2250 iterations provided a reasonable trade-off between accuracy and runtime.

Original: Early exaggeration annealing By default, the early exaggeration factor is switched off afteriterations.
Revised: Early exaggeration annealing By default, the early exaggeration factor is switched off afteriterations.

Original: We hypothesized that this abrupt transition may potentially ‘trap’ some points in suboptimal locations.
Revised: We hypothesized that this abrupt transition may potentially ‘trap’ some points in suboptimal locations.

Original: If so, gradually annealing early exaggeration factor down to 1 during iterations 125–250 could potentially improve the final embedding.
Revised: If so, gradually annealing early exaggeration factor down to 1 during iterations 125–250 could potentially improve the final embedding.

Original: Using annealing, we observed almost no improvement in kNN recall (Figure 2a,c), however this metric may not be very sensitive to a minority of papers or clusters located suboptimally.
Revised: Using annealing, we observed almost no improvement in kNN recall (Figure 2a,c), however this metric may not be very sensitive to a minority of papers or clusters located suboptimally.

Original: We visually investigated the effect of annealing and found that while overall the embeddings with and without annealing looked very similar, there were some subtle changes on a smaller scale.
Revised: We visually investigated the effect of annealing and found that while overall the embeddings with and without annealing looked very similar, there were some subtle changes on a smaller scale.

Original: One such example is shown in Figure 3.
Revised: One such example is shown in Figure 3.

Original: Without annealing (Figure 3a), there was an isolated island with predominantly blue papers (labeled ‘materials’) that ceased to exist when the exaggeration was annealed.
Revised: Without annealing (Figure 3a), there was an isolated island with predominantly blue papers (labeled ‘materials’) that ceased to exist when the exaggeration was annealed.

Original: Instead, its points got fused with two bigger regions of blue points (Figure 3b,c), suggesting that annealing helped them to drift to more meaningful locations.
Revised: Instead, its points got fused with two bigger regions of blue points (Figure 3b,c), suggesting that annealing helped them to drift to more meaningful locations.

Original: Two further examples of this effect can be seen in Figures A10 and A11.
Revised: Two further examples of this effect can be seen in Figures A10 and A11.

Original: For our final pipeline (Figure 1), we combined long optimization (10,250 iterations) with the early exaggeration annealing.
Revised: For our final pipeline (Figure 1), we combined long optimization (10,250 iterations) with the early exaggeration annealing.

Original:  D ISCUSSION
Revised:  D ISCUSSION

Original: Here we developed a pipeline for making 2D visualizations of large document libraries, using the
Revised: Here we developed a pipeline for making 2D visualizations of large document libraries, using the

Original: MEDLINE database of the entire biomedical literature as a use case.
Revised: MEDLINE database of the entire biomedical literature as a use case.

Original: The pipeline is based on
Revised: The pipeline is based on

Original: TF-IDF representation, SVD preprocessing, and t-SNE with uniform affinities, early exaggeration annealing, and extended optimization.
Revised: TF-IDF representation, SVD preprocessing, and t-SNE with uniform affinities, early exaggeration annealing, and extended optimization.

Original: Our final embedding (Figures 1, A1–A3) shows meaningful structure with good topic separation.
Revised: Our final embedding (Figures 1, A1–A3) shows meaningful structure with good topic separation.

Original: The closest neighbors were strongly distorted compared to the TF-IDF representation (Table A2), suggesting that such an embedding may not be suitable for literature queries.
Revised: The closest neighbors were strongly distorted compared to the TF-IDF representation (Table A2), suggesting that such an embedding may not be suitable for literature queries.

Original: However, bigger-scale structure was meaningful, emphasizing small clusters containing 100s–1000s of papers on clearly defined topics.
Revised: However, bigger-scale structure was meaningful, emphasizing small clusters containing 100s–1000s of papers on clearly defined topics.

Original: In ongoing work, we found that random projection (Schmidt, 2018) outperformed SVD in terms of kNN recall, but resulted in poor t-SNE embeddings; future work is needed to explain this observation.
Revised: In ongoing work, we found that random projection (Schmidt, 2018) outperformed SVD in terms of kNN recall, but resulted in poor t-SNE embeddings; future work is needed to explain this observation.

Original: It will also be interesting to replace TF-IDF with a pretrained language model such as PubMed
Revised: It will also be interesting to replace TF-IDF with a pretrained language model such as PubMed

Original: BERT (Gu et al., 2021).
Revised: BERT (Gu et al., 2021).

Original: A CKNOWLEDGMENTS
Revised: A CKNOWLEDGMENTS

Original: We thank Sebastian Damrich and Jan Niklas B ¨ ohm for comments and suggestions.
Revised: We thank Sebastian Damrich and Jan Niklas B ¨ ohm for comments and suggestions.

Original: This research was funded by the Deutsche Forschungsgemeinschaft (KO6282/2-1; through a Heisenberg Professorship, BE5601/4-1; and through the Excellence Cluster 2064 “Machine Learning: New Perspectives for Science”, 390727645), and by the German Ministry of Education and Research (01IS18039A).
Revised: This research was funded by the Deutsche Forschungsgemeinschaft (KO6282/2-1; through a Heisenberg Professorship, BE5601/4-1; and through the Excellence Cluster 2064 “Machine Learning: New Perspectives for Science”, 390727645), and by the German Ministry of Education and Research (01IS18039A).

Original: The authors thank the International Max Planck Research School for Intelligent Systems (IMPRSIS) for supporting Rita Gonz´alez M´arquez.
Revised: The authors thank the International Max Planck Research School for Intelligent Systems (IMPRSIS) for supporting Rita Gonz´alez M´arquez.

Original: A A PPENDIX A. S UPPLEMENTARY T ABLES
Revised: A A PPENDIX A. S UPPLEMENTARY T ABLES

Original: [Table]
Revised: [Table]

Original: Neighbor10:
Revised: Neighbor10:

Original: RNA-SeqintheCollaborativeCross.
Revised: RNA-SeqintheCollaborativeCross.

Original: A. S UPPLEMENTARY F IGURES
Revised: A. S UPPLEMENTARY F IGURES

Original: [Figure]
Revised: [Figure]

Original: NN
Revised: NN

Original: [Figure]
Revised: [Figure]

Original: and memory
Revised: and memory

Original: [Figure]
Revised: [Figure]

Original: e c
Revised: e c

Original: [Figure]
Revised: [Figure]

Original: ( k =)
Revised: ( k =)

Original: [Figure]
Revised: [Figure]