GlycanML: A Multi-Task and Multi-Structure Benchmark for Glycan Machine Learning
Authors: Minghao Xu
Yunteng Geng
Yihang Zhang
Ling Yang
Jian Tang
Wentao Zhang
Venue: ICLR 2025 Poster (2025)
Keywords: Glycan Machine Learning
Representation Learning
Multi-Task Learning
Primary area: datasets and benchmarks
TL;DR: We propose the GlycanML benchmark with diverse types of glycan understanding tasks, multiple representation methods of glycan structures, and a multi-task learning testbed.

Abstract: Glycans are basic biomolecules and perform essential functions within living organisms. The rapid increase of functional glycan data provides a good opportunity for machine learning solutions to glycan understanding. However, there still lacks a standard machine learning benchmark for glycan property and function prediction. In this work, we fill this blank by building a comprehensive benchmark for Glycan Machine Learning (GlycanML). The GlycanML benchmark consists of diverse types of tasks including glycan taxonomy prediction, glycan immunogenicity prediction, glycosylation type prediction, and protein-glycan interaction prediction. Glycans can be represented by both sequences and graphs in GlycanML, which enables us to extensively evaluate sequence-based models and graph neural networks (GNNs) on benchmark tasks. Furthermore, by concurrently performing eight glycan taxonomy prediction tasks, we introduce the GlycanML-MTL testbed for multi-task learning (MTL) algorithms. Also, we evaluate how taxonomy prediction can boost other three function prediction tasks by MTL. Experimental results show the superiority of modeling glycans with multi-relational GNNs, and suitable MTL methods can further boost model performance. We provide all datasets and source codes at https://github.com/GlycanML/GlycanML and maintain a leaderboard at https://GlycanML.github.io/project

Published as a conference paper at ICLR 2025
GLYCANML: A MULTI-TASK AND MULTI-STRUCTURE BENCHMARK FOR GLYCAN MACHINE LEARNING
Minghao Xu 12.3
Yunteng Geng'
Yihang Zhang
Ling Yang'
Jian Tang2.45
2,3,4,5
Wentao Zhang
equal contribution
ccorresponding author
Pekknng University
Mila Québec AI Institute
3 BiooGeomety
*HEC Montréal
CCFAR Al Research Chair
contacts: minghao. stu. pkw.cdu.ccn wentao watpoo//hhan@puuu.uu....
ABSTRACT
Glycans are basic biomolecules and perform essential functions within living or- ganisms. The rapid increase of functional glycan data provides good opportunity for machine learning solutions to glycan understanding. However, there still lacks a standard machine learning benchmark for glycan property and function prediction. In this work, fill this blank by building comprehensive benchmark for Glycan Machine Learning (GLYCANML). The GLYCANML benchmark consists di- verse types of tasks including glycan taxonomy prediction, glycan immunogenicity prediction, glycosylation type prediction, and protein-glyen -lyyann interaction prediction. Glycans can be represented by both sequences and graphs in GLYCANML, which enables us to extensively evaluate soquence-based models and graph neural net- works (GNNs) on benchmark tasks. Furthermore, by concurrently performing eight glycan taxonomy prediction tasks, we introduce the GLYCANML-NV for multi-task learning (MTL) algorithms. Also, we evaluate how taxonomy predic- tion can boost other three function prediction tasks by MTL. Experimental results show the superiority of modeling glycans with multi-relational GNNs, and suitable MTL methods can further boost model performance. We provide all datasets and source codes at [at tttpps:// [attps://github. com http://aww......caaaaaaaaaaaaa/a
testbed
GlycamMM and main- tain leaderboard at https:: httpp://///aacaa.....ooooooooooa
INTRODUCTION
Glycans are fundamental biomolecules whose significance are comparable to the biomolecules of central dogma, i.e., DNAs, RNAs and proteins. They regulate inflammatory responses (Hochrein ctt al. 2022), enable the recognition and communication between cells (Zhang) (006)), preserve stable blood sugar levels (Bermingham al., 2018), etc. They perform their functions mainly by interacting with other biomolecules, e.g., binding with antigens to form glycan epitopes. Thanks to the advance of tigg-throughhut sequencing techniques of glycans (Yan et al. 2019 Lee et al. [2009). large number of glycan data are accessible, e.g., the more than 240 thousand glycans stored in the GlyTour Gln database (Tiemeyer et al., (017)). This progress enables glycan function analysis by machine learning methods which are essentially data-driven.
There are some existing works that employ machine learning models to predict the species origins of glycans (Bojar et al. [22T]] Burkholz et al. (202I). glycosylation phenomenon (Pakhrin et al. 202T
Li Li al. 2022) and ability of glycans induce immune response (Wang et | [022 Lundstrøm et al. 2022). These works mainly aim to solve one or several related glycan understanding problems. However, there still lacks a comprehensive benchmark studying the general effectiveness of various machine learning models on predicting diverse glycan properties and functions, which hinders the progress of machine learning for glycan understanding. As a matter of fact, comprehensive benchmark studies greatly facilitate the machine learning research of other biomolecules like small molecules (Wu et al. 20188) Townshend et al.) 2020). proteins (Rao et al.) 2019 Xu et al. 2022) and
nucleic acids (Wang al. 2023 Nguuen al. 2024)

---

Published as a conference paper at ICLR 2025
Therefore, in this work, we take the initiative of building Glyean Machine Learning (GLYCANML) benchmark featured with diverse types of tasks and multiple glycan representation structures. The GLYCANML benchmark consists of 11 benchmark tasks for understanding important glycan prop- erties and functions, including glycan taxonomy prediction, glyean immunogenicity prediction, glycosylation type prediction, and protein-glycan interaction prediction. For each task, carefully split the benchmark dataset to evaluate the generalization ability of machine learning models in real-world scenarios. For example, in glycan taxonomy prediction, we leave out the glycans with unseen structural motifs during training for validation and test, which simulates the classification of newly discovered glycans nature with novel molecular structures.
The GLYCANML benchmark accommodates two glycan representation structures, i.e., glycan tok- enized sequences and glycan planar graphs. For each structure, we adopt suitable machine learn- ing models for representation learning, where sequence encoders such as CNN (He et al.) (016).
LSTM (Hochreiter Schmidhuber 1997) and Transformer (Vaswani et al.) 2017) are employed
for glycan sequence encoding, and both homogeneous GNNs (Kipf & Welling 2017 7, Veličkovid
ct al. 2017| Xu et al. 2018) and heterogeneous GNNs (Gilmer et al. 2017] Schlichtkrul et al. 2018
Vashishth et al. (2019) are used to encode glycan graphs. In addition, five typical small molecule encoders (Ying et al. 2021] Rampášek et al., [002] Ross et al.) 2022 Wang et al. 2022] Lu et al.
2023) and performant pre-trained small molecule encoders (Ying al. [022] Wang al. [2022]
are also studied for all-atom glycan modeling. We evaluate each model on all benchmark tasks to study its general effectiveness.
The GLYCANML benchmark also provides a testbed, namely GLYCANML-MTL. for multi-task learning (MTL) algorithms, where an MTL method is asked simultaneously solve eight glyean laxonomy prediction problems which are highly correlated. The performance on this testbed measures how well an MTL method can transfer the knowledge learned from different glycan taxonomies, e.g., transferring between species-level classification and genus-level classification. Taking step further, we study the effect of MTL. on boosting three function prediction tasks (i.e., immunogenicity, glycosylation and interaction prediction) by concurrently performing taxonomy prediction.
Benchmark results show that the RGCN model (Schlichtkrull et al., 2018). typical heterogeneous GNN, performs best most benchmark tasks, and simple two-layer CNN can surprisingly achieve competitive performance by using only condensed sequential information of glycan structures. The MTL methods with elaborate gradient redirection and task reweightng strategies can further enhance the performance of glycan taxonomy prediction, and learning the immunogenicity and glycosylation type of glycans jointly with their taxonomies also achieves benefits, showing the potential of MTL on boosting glycan understanding. We hope the GLYCANML benchmark will spark the interest of studying glycoseience with machine learning.
2 RELATED WORK
Glycan machine learning. With the expanding size of experimental glycomics datasets, the inte- gration of machine learning techniques into glycoinformatics shows considerable promise (Bojar) & Lisacek 2022 Li al. 2022). Early approaches use traditional machine learning algorithms
(e.g.. SVMs) to learn patterns from mass spectrometry data (Kumozaki et al. 2015 Liang et al. 2014). predict glycosylation sites (Caragea al. 2007 Li et al. [015 Pitti et al. 2019), and classify glycans (Yamanishi et al. 2007). Recently, thanks to the advancements in deep learning and new glycomics datasets, there has been rise in studies applying deep learning to glycan and glycosyla- tion modeling. These works seek to identify N N-glycosylated sequon (Pakhrin et al.) [222]) model
glycan 3D structures (Bánkestad et al., 2023 Chen et al. 2024) and predict various glycan properties and functions (Bojar et 2020 Burkholz et al. 2021 Dai et al. 2021 Lundstrøm et al. 2022
Carpenter et al. 2022 Alkuhlani et 2023).
However, there still lacks comprehensive benchmark that incorporates diverse types of glycan understanding tasks and different glycan modeling methods like sequence-based and graph-based methods. Also, it is unknown how multi-task learning (MTL) influences the learning of glycan property prediction. In this work, we fill these blanks by introducing the GLYCANML benchmark with multiple task types, multiple representation schemes of glycan structures, and an MTL testbed.
Biological machine learning benchmarks. To evaluate the performance of different machine learning methods in modeling biomolecules, it is necessary to establish large-scale standardized
2

---

Published as a conference paper at ICLR 2025
Monosaccharides:
Glycosidic bonds: a1-3, u1-6, B1-2, 2, B1-4...
Domain: Fukarya
Receptor on Immone Cell
Kingdom: Plantae
Glyeun
Phylum: Angiosperms
e1-6
B1-4
B1-4
Class: Dicotyledons
m1-3
Order: Sapindales
Glycan
Family: Sapindacene
Antigon
Genus:
Immunogenic Non inmunogenic
Species: pscudoplatamus
(a) Glycan Taxonomy Prediction
(b) Glyean Immunogenicity Prediction
Asparagine
Glycan
Prutein
Asn
Binding affinity
Profein
S N-gh gllcossylation
(Z-score): 0.375
D-glyeosylation
glycan
Glyean
(c) Glycosylation Type Prediction
(d) Protein-Glyean Interaction Prediction
Figure 1: Illustration of benchmark tasks. (a) Predicting the biological taxonomy of glycans at eight
levels. (b) Judging whether glycan is immunogenic or not in organisms. (c) Analyzing how glycan glycosylates its target protein. (d) Given a protein and glycan, predicting their binding affinity.
benchmarks. MoleculeNet (Wu et al. 2018) is a widely-used benchmark of small molecule modeling, evaluating the efficacy of both traditional machine learning and deep learning on predicting molecular properties. In the field of protein modeling, the renowned CASP (Kryshtafovych et al. 2023)
competition is dedicated to establishing standards for protein structure prediction. Also, benchmark datasets are constructed for machine learning guided protein engineering (Rao et al., 2019 Dallago
al. 2021), protein design (Gao et al. (2022) and protein function annotation (Xu et al. [002 Zhang
et al. 2022). Benchmark datasets are also established for other biomolecules like DNAs (Ji et al.
2021
Nguyen al., (0044 and RNAs (Wang al. 2023). In this work, we take the initiative of building glycan machine learning benchmark for comprehensive glycan understanding.
3
BENCHMARK TASKS
The GLYCANML benchmark consists of benchmark tasks, including glycan taxonomy prediction, glycan immunogenicity prediction, glycosylation type prediction, and pootiinglyyan interaction prediction, as illustrated Figure [
We summarize the information of all tasks in Table
3.1 GLYCAN TAXONOMY PREDICTION
Scientific significance. The taxonomy of glycans lays the foundation for glycomics research (Aoki- Kinoshita 2008). Biologists commonly classify glycans based on their origin under the hierarchical system of domain, kingdom, phylum, class, order, family, genus and species. Such a systematic classification helps us compare the similarities and differences between glycans, which further facilitates the study of glycan structures and functions. Also, the glycan taxonomy helps us understand the process of biological evolution. By comparing the structures of glycans in different organisms, we can infer their phylogenetic relationships and possible changes that may occur during evolution. Therefore, it is very helpful to have accurate glycan taxonomy predictor based machine learning.
Task definition. We study glycan taxonomy prediction on domain, kingdom, phylum, class, order, family, genus and species levels, leading to eight individual tasks. These tasks formulated as classification problems with 4, 11, 39, 101, 210, 415, 922 and 1,737 biological categories, respectively. Taking the class imbalance into consideration, report Macro-F1 score for each task.

---

Published as a conference paper at ICLR 2025
Table 1: Benchmark task descriptions. We list each task along with its type, the average number of monosaccharides in each glycan for this task (in (sd format), dataset statistics, and evaluation metric. Abbr, Mono.: Monosaccharides.
Task
Task type
#Mono. per glycan
#Sample
#Tratn/Validation//es
Metric
Taxonomy prediction of Domain Taxonomy prediction of Kingdom Taxonomy prediction of Phyyuum
Classification
6.39
13,209
11,01011
Macro-F
Classification
6.39gs5
13,209
11,010/1,8007
Macro-FI
Classification
6,39
13,209
11.010/1280/9119
Mucro-FI
Taxonomy prediction Class Taxonomy prediction Taxonomy prediction Family Taxonomy prediction of Genus Taxonomy prediction of Speciex Immunogenicity prediction
Classification
6,39(3.5
13,209
11,010/1,28099
Macro-FI
Classification
6,39
13,209
11.010/1.280/919
Macro-FI
Classification
6.39(3
13,209
11,01011. 12200919
Macro-FI
Classification
6.39gusn
13,209
11,010/1,200/19
Macro-FI
Classification
6. 6.39
13,209
11,010/1,80099
Macro- Fl
Rinary classification
7 7.30
1,320
1,026/149/45
AUPRC
Glycouylation type prediction Protein-Glycan interaction prediction
Classification
9.04,3
1,683
1,356/163/164
Mazro- FF
Regression
6.56,
564,647
442,796/58,887/66
Spearman's 's
Benchmark dataset. We collect the glycans the SugarBase database (Bojar et al., 2020b) that fully annotated with domain, kingdom, phylum, class, order, family, genus and species labels, with 13,209 glycans in total. We adopt a motif- bastif-based method for dataset splitting, which well fits the real-world scenario where the machine learning models trained the glycans with existing motifs are applied to predict the functions the glycans with newly discovered motifs (Porter et al.) 2010) Klamer et al., 2017). Specifically, we represent each glycan with the frequencies of popular motifs (i.e., those frequently occurring substructures in glycans), where the motif list proposed by Thomès al. (202T) employed. Based on such representations, we cluster all glycans in the dataset by K-means (K 10), where 8 clusters are assigned to training, and the remaining two clusters are respectively utilized for validation and test.
3.2 GLYCAN IMMUNOGENICITY PREDICTION
Scientific significance. Predicting the immunogenicity glycans is of great significance for vaccine design and disease treatment. (1) Glycans are key components in many vaccines, especially in bacterial vaccines (Kaplonek et al. 2018). By predicting the immunogenicity of glycans, researchers can design more eifective vaccine formulations.
(2) In addition, certain glycans can inhibit tumor growth by activating the immune system (Amon et al. 2014). and therefore accurately predicting glycan immunogenicity can help optimize tumor treatment strategies.
Task definition. We formulate this task as a binary classification problem, i.e., predicting whether a glycan is immunogenic or We evaluate with the AUPRC metric measure the trade-off between precision and recall of a model on immunogenic glycans.
Benchmark dataset. We select all glycans in the SugarBase (Bojar al. 2020b whose
immunogenicity is annotated based on evidences in literature, summing up to 1,320 glycans. As in glycan taxonomy prediction, we use the motif-based dataset splitting scheme to derive training, validation and test splits with an 8:1:1 ratio. In this way, we evaluate models' generalization ability across structurally distinct glycans.
3.3
GLYCOSYLATION TYPE PREDICTION
Scientific significance. Glycans are a class of macromolecules with diverse biological activities, including immune system regulation, antitumor effects, antíviral effects, etc. By predicting the type of glycosylation, researchers can better understand the relationship between glycan structure and its functions. Understanding the structure- fututtoof relationship is crucial for designing and synthesizing glycan derivatives with specific biological activities (Bieberich) 2014).
Task definition. Given glycan, aim at predicting whether it forms N-glycosylation, O- glycosylation or maintains a free state, formulated as a three-way classification problem. The Macro-F1 score is used for evaluation.
Benchmark dataset. We traverse the GlyConnect database (Alocci et al. 2018) and select out all glycans with glycosylation annotations, with 1,683 glycans in total. Upon these data, again
4

---

Published as a conference paper at ICLR 2025
Mannal--yyyXX/tii22)MMMmbb hytMMMaaaattiCiiAcccctttteoooa (uuaaaaccbbb///uaaaaaajiiAAAA graph
Glyean LUPAC-condensed sequence
a1-3
B1-4
81-4
Tokenization
Man
at-3
Xyl
b1-2
Man
bl-4
GleNAc
b144
Fac
al-3
GleNAe
(a) Glycan tokenized sequence
(h) Glycan planar graph
Figure 2: Illustration of glycan representations. (a) The glycan tokenized sequence is derived
by tokenizing the IUPAC-condensed sequence. (b) The glycan planar graph constructed by transforming the IUPAC-condensed sequence to graph.
employ the motif-based dataset splitting scheme (introduced in Section 3.1 validation and test splits with an 8:1:1 ratio. This task again assesses the generalization ability across the glycans with distinct structures.
construct training,
3.4 PROTEIN-GLYCAN INTERACTION PREDICTION
Scientific significance. The interactions between proteins and glycans play crucial role in cellular signaling, affecting cell growth, differentiation, and apoptosis (Villalobo et al. (006). For example, glycans are one of the main components of the extracellular matrix (ECM). which interact with proteins such as collagen, laminin, and fibronectin to form the structural framework of ECM, providing physical support and passing biochemical signals to cells (Cohen 2015). Understanding these interactions helps reveal how cells respond to external signals.
Task definition. Given a protein and glycan, this task aims to regress their binding affinity, where the Z-score transformed relative fluorescence unit represents binding affinity. For this task, we adopt the Spearman's correlation coefficient as the evaluation metric to measure how well a model ranks a set of protein-glycan pairs with different binding affinitics.
Benchmark dataset. This benchmark dataset is built upon 564,647 protein-glycan interactions deposited in the LectinOracle database (Lundstrøm al. 2022). It is desired to have model that can well generalize to new proteins against training ones, considering the continuous discovery of new proteins by sequencing techniques. Therefore, split the dataset based on protein sequence similarity. Specifically, we first cluster all protein sequences using MMseqs2 (Steinegger & Söding 2017) (minimum sequence identity within each cluster: 0.5), and then we derive training, validation and test proteins by splitting all clusters with an 8:1:1 ratio. Finally, samples of protein-glycan pairs are split according to the protein splits, deriving the benchmark dataset for protein-gih -lycan interaction prediction. This splitting scheme is consistent with practical vaccine design applications where newly discovered lectins (proteins) could be used the target for glycan binding.
4 METHODS
4.1
REPRESENTATIONS
In the GLYCANML benchmark, we adopt two glycan-specific representation structures, i.e., glycan tokenized sequence and glycan planar graph, as illustrated in Figure
Glycan tokenized sequence. A glycan is commonly represented by an IUPAC-condensed sequence. For example, in the sequence "Glc(al-4)CC -4)Glc", two glucoses are connected an tio bond, and this structure is the basic component of starch, typical glycan. To process such sequences with machine learning models, a straightforward way tokenize the IUPAC-condensed sequence. Specifically, we regard each monosaccharide (e.g., Glc), each glycosidic bond (e.g. al-4), and each bracket that indicates glycan branching (i.e., "I" and "I"") as a single token, which derives the glycan tokenized sequence, denoted x, {s; \ Various sequence encoders like Transformers (Vaswani et al. 2017) can then be applied to such tokenized sequences for glycan representation learning.
Glycan planar graph. Essentially, IUPAC-condensed sequence describes the branching structure of glycan, in which the part between brackets "I" and "I" denotes a side branch the main branch, illustrated in Figure This structure is well represented by planar graph 2gg (V, E), ) in which
5

---

Published as a conference paper at ICLR 2025
nodes V denotes monosaccharides, and edges & denotes glycosidic bonds. Nodes and edges in this graph are represented by one-hot feature vectors to indicate the type of monosaccharide and glycosidic bond. In this way, graph neural networks (GNNs) are readily used for glycan modeling.
4.2 BASELINES
We include four types of models in our benchmark, i.e., sequence encoders for modeling glycan tokenized sequences, homogeneous and heterogeneous GNNs for modeling glycan planar graphs and all-atom molecular encoders for modeling the all-atom molecular graphs of glycans, with 12 baseline models in total. Their detailed architectures are provided in Appendix A
Sequence encoders. We study the performance of four typical sequence encoders. Inspired by the success of shallow CNNs in modeling biological sequences like protein sequences (Shanchsazzadch investigate (1) 2-layer shallow CNN along with (2) a deep residual network (ResNet) (He et al.] 2016) with 8 hidden layers. These two CNN models mainly focus on capturing local information in glycan sequences. To investigate the importance of long context modeling for glycan understanding, we also include (3) a 3-layer bidirectional LSTM (Hochreiter & Schmidhuber 1997) and (4) 4-layer Transformer encoder /Vaswani al. 2017).
al. 2020 Xu et al. 2022).
Homogeneous GNNs. Upon glycan planar graphs, standard GNNs designed for homogeneous graph modeling can be readily used to learn glycan representations. In our benchmark, three typical homogeneous GNNs, i... GCN (Kipl & Welling, 2017), GAT (Velickovćć et al., (017) and GIN (Xu
al. 2018), serve as baselines, and they are all configured with message passing layers.
Heterogeneous GNNs. As matter of fact, modeling glycans as homogeneous graphs is suboptimal, in which the rich information within glycosidic bonds fully ignored. To capture the complete information in glycan graphs, it is more proper to view them as heterogeneous graphs and employ heterogeneous GNNs for representation learning. Therefore, we adapt three popular heterogeneous GNNs. i.e. MPNN (Gilmer et al. 2017). RGCN (Schlichtkrull al. (018) and CompGCN (Vashishth ctt al. 2019). to model glycan graphs, where each model equipped with 3 message passing layers.
All-atom molecular encoders. Essentially, glycans are kind of macromolecules with hundreds of atoms, and thus we can directly use all-atom molecular encoders for small molecules to model glycans. In this benchmark, include five typical small molecule encoders, Graphormer (Ying et al. 202T), GraphGPS (Rampášek et al. (022). MolFormer (Ross et al. 2022), MolCLR (Wang
al. 2022 and Uni-Mol+ (Lu al. 2023
Pre-trained all-atom molecular encoders. Regarding the success of pre-training in small molecule modeling, also evaluate two performant pre-trained small molecule encoders, the pre-trained Graphormer (Ying et al. 2021) and the pre-trained MoICLR (Wang et al. 2022
4.3 MODEL PIPELINES
Depending on inputs, the benchmark tasks of GLYCANML can be solved with two model pipelines.
Single-glycan prediction. This pipeline handles the tasks that predict the properties of individual glycans, including glycan taxonomy prediction, glycan immunogenicity prediction, and glycosylation type prediction. For each task, the glycan representation vector is first extracted by glycan encoder and then passed to an MLP head for task-specific prediction.
Protein-glycan interaction prediction. Because of the additional input of protein, protein- protein-glycan interaction prediction requires a different pipeline. Given a protein and a glycan, we first extract the protein representation with protein encoder (e.g., the ESM-11b protein language model (Rives et al.) 2021 used in this work) and extract the glycan representation with a glycan encoder, and these two representations are then concatenated and sent to MLP head for interaction prediction.
4.4 MULTI-TASK LEARNING
In GLYCANML, the glycan taxonomy prediction tasks classify glycans under the hierarchical system from domain to species. These tasks are highly correlated and well-suited for multi-task learning (MTL) where related tasks learned together for better generalization performance (Zhang & Yang

---

Published as a conference paper at ICLR 2025
2021). Therefore, we integrate eight glycan taxonomy prediction tasks in GLYCANML as testbed for MTL algorithms, named as the GLYCANML-MTL. benchmark.
On this benchmark, we study 8 representative MTL methods. All these methods use the network architecture with hard parameter sharing (Zhang & Yang 202T), where all tasks share a common backbone encoder, and each task owns its individual prediction head. We introduce these methods below, with an abbreviation after each one.
Naive MTL (N-MTL): The most straightforward way to perform MTL is sum up the losses of all tasks with equal weights and optimize the model with this loss summation. Denoting the losses of GLYCANML.-MTL tasks as L, (i (i=1l ,,,",,8),, the naive MTL loss defined as: CN-MTL E. C.
Gradient Normalization (GN) (Chen et al., 2018): However, regarding all tasks equally is suboptimal, considering the varying difficulties different tasks. Therefore, this method employs a weighted loss summation CeN = Ei=1 w,C,, where the weights satisfy: Σ W; = 8. The main idea of gradient normalization is that different tasks should be trained at similar rates (i.e., similar speed of convergence). To achieve this goal, authors first deem the L.2 norm of per-task gradient as the training rate of the task: ri Now;Calle (0 denotes model parameters), and all tasks are then pushed to have similar training rates by optimizing the loss [(w), .. ws) = E ||r; Fll2 (F = (Ein r8) For each training step, this loss is first optimized w.t.t loss weights {w; ) 1. and, using the updated loss weights, the whole model is optimized by Cax.
Temperature Scaling (TS) (Kendall et al., 2018): For classification tasks, the sharpness of categorical distribution represents prediction uncertainty. further implying task difficulty. Inspired by this fact, the TS method weighs different tasks by scaling their classification logits. this way. each task loss is defined as CTS is the classification logit of sample æ at class y, and a, denotes the task-specific temperature parameter for scaling. The temperature parameters are learned along with the whole model.
log mox (fo ([(()/))/)) (i (i= 1, 8), where fo(v/ar)
Uncertainty Weighting (UW) (Kendall et al., 2018): Kendall et al. (2018) shows that the lemperature-scaled losses above can be approximated by weighted summation of unscaled losses: Luw = Σ. L./oī + log og where the weighting parameters {or are learnable. This method also weighs different tasks based on the uncertainty of task predictions.
Dynamic Weight Averaging (DWA) (Liu et al. 2019 The loss scales along training can well indicate task convergence. Therefore, this method employs the ratio of consecutive losses to weigh different tasks: wx(t) = 8- (t) 1)), where (t) denotes the loss of task at training step L. In this way, more weights are assigned to the tasks with slower convergence.
Dynamic Task Prioritization (DTP) (Guo et al., 2018): This method maintains a key per- formance indicator (KPI) K; (4)() for each task along training (moving average of classification accuracy on our benchmark) and weighs different tasks in a focal loss (Lin et al. 2017) manner: w,(t) (1 (*) log (t). where 7 is the focusing hyperparameter for task i. Such task reweighting scheme pays more attention to difficult tasks with low KPI.
Nash Bargaining Solution (Nash) (Navon et al. 2022):
To address gradient conflicts among
tasks, this method combines task gradients using the Nash Bargaining Solution, ensuring Pareto-optimal and proportionally fair updates. We denote the gradient matrix as G [VeL), VoLs]. The task weights 1 are obtained by solving (G G w where is a regularization coefficient. This method adaptively balances tasks based on their gradient interactions.
Conflict-Averse Gradient descent (CAGrad) (Liu et al., 2021 CAGrad resolves gradient conflicts in MTL by balancing tasks based on the worst-case improvement. In this method, the average gradient is defined as go
( - VoL.)/8, and the task weights {w;} 18 obtained by minimizing F(w) T go ggl, where it defines g ( w. VoC.)8 and l with E [o, 1). The parameters are updated as O, 0,-1 aggo + Tg..| gw). This method dynamically balances tasks and ensures convergence to the optimal average loss.
7

---

Published as a conference paper at ICLR 2025
Table 2: Benchmark results single-task learning. We report mean (std) for each experiment. Three color scales of blue denote the first, second and third best performance. Abbr, Immuno: Immunogenicity; Glycos: Glycosylation. * denotes a pre-trained model.
Madel
Donsin (M
Kingdom
Plylum
Clas
Taxonomy
Order
Family
Geous (Macno-F))
Speces (Macre-F /i
Inmana (AEPPC))
Galyo (Macre-F))
Interaction (Spaissee's
Wiighted Moen Rtank
(Macre-d
(M
(MacreFF1)
(Macrs-) F
Suquencu Enosder:
Shallow CNN
0.62
0.599asse
0.388uuo
0:542
0,202amse
0.
0.115aom
0. 0.77kaash 0.75 (19
0,858 am
8:25
6.56
BesNet
0,505ars
0,33
0 a.00
0.165aa
0.1Zaam
0.73aoo oon
0,919
0.373
591
LSTM
0,36
0.772
0.14bgma
0.778
0.8622
8:288aum
6.33
Transforaier
0.72%
0:244
Horoogmeenns GNN
OCN GAT
0665
0:527axw
0.325
0.257amm s012
0.112amx
0,395yosc:
0.00
0:934
11.09
0.30
0.190go0
0.115uaxs
0.103
0.197am
PNA
(.20%
0.114guue
0.391aam
01771
0/5lasm
11.09
Hetengeaees
MPNN
auta
0.272mams
0.35pam
0.66kgoee
0.13
0.104
0.644um 780
0.10c aw
BGUN
k251 en
CompteN
0.19
0.13$
n B
Al-Attn Molccalas Escedes
Graphernce
0.6
0.47aa
0.261oa0
0 0.20
0.112x00
0.0800ax
0154 aoi
0,665 pase
0,88
16.34
GraphtaPS
B447/amm
0 0.261
0.13Appe
0.300mmm
0105
0.67 cm
0.99%
12.94
MalFormer
0 0.402amm
0.372omm
0 0240a0
0,177amm
0.145omm
0.108am
0.873xam 0.67
12.75
M
8
0.79%pav
0.17aa
0.20%
0778
0.924
11:55
Uni-Mal+
0.227
0.174axme
0.00
0,017
0.056
0.789 pam
0.885x
0.44
Pre-imined Malcculer
Graphermer
0 0.50laams
0.342am0
0.118aw
0.100pmm::
007
0.743
0,910
0.279
5.25
0.900aas
0.77
0.11aar
0.00
5 EXPERIMENTS
5.1 EXPERIMENTAL SETUPS
Model setups. For glycan taxonomy, immunogenicity and glycosylation type prediction, upon the glycan representation extracted by the glycan encoder, use 2-layer MLP with ReLU activation to perform prediction. For protein-glycan interaction prediction, we use the ESM- lb protein language model (Rives et al. (021) to extract protein representation, and, upon the concatenation of protein and glycan representations, the binding affinity is predicted by a 2-layer MLP with ReLU activation.
Training setups. We conduct every experiment on three seeds (0, 1 and 2) and report the mean and standard deviation of results. We train with an Adam optimizer (learning rate: 5 x 10-4, weight decay: 1 x 10-3) for 50 epochs on taxonomy, immunogenicity and glycosylation type prediction and for 10 epochs on interaction prediction. The batch size is set as 32 for interaction prediction and 256 for other tasks. For model training, we use cross entropy loss train taxonomy and glycosylation type prediction tasks, use binary cross entropy loss to train immunogenicity prediction, and adopt mean squared error to train interaction prediction. For model selection, 10 times of validation are uniformly performed along the training process, and the checkpoint with the best validation performance is selected for test. For multi-task learning (MTL.,, the focusing parameter 7 of the dynamic task prioritization (DTP) method is set as 2.0, and the model selection of all MTL. methods is based on the mean accuracy over all tasks on the validation set. We conduct all experiments on a local server with 100 CPU cores and 4 NVIDIA GeForce RTX 4090 GPUs (24GB). Our implementation is based on the Py PyTocch (Paszke et al. 2019) deep learning library (BSD-style license) and TorchDrug (Zhu eet al. 2022) drug discovery platform (Apache-2.0 license).
5.2
BENCHMARK RESULTS ON SINGLE-TASE LEARNING
In Table 2. we report the single-task performance of 18 representative glycan encoders. We measure the comprehensive performance of a model with its weighted mean rank over all tasks, where each taxonomy prediction task weighs 1/8 and each of the other three tasks weighs 1, so as to balance between different types of tasks. Based on these results, we highlight the following findings:
Multi-relational GNNs show superiority in glycan modeling. Two typical multi-relational GNNs, i.e., RGCN and CompGCN, respectively rank first and second place in terms of weighted mean rank. Especially, RGCN achieves the best performance 6 out of 11 benchmark tasks.
8

---

Published as a conference paper at ICLR 2025
Table 3: Benchmark results multi-task learning. We report mean (sid) for each experiment. Two
color scales of blue denote the first and second best performance.
Method
Domain
Kingdom
Plyfum
Class
Order
Family
Genus
Species
Mean Macro-F1
Backhone encuder: Shallow CNN
Tingle-Tssk
(.659
0.559pum2a
0.388
0.342
0,238
0.200
0.149
0.115
0.327
N-MIL
0.619
0.549muom
0.333
0.344ggmss
0.213
0.167au2041
0.150puann
0.12
0.312
GN
0.6466mmmm
0.497mam5
0. 0.344pasc
0.312
0.233
0,206
0.1 15175p
0.137
0.319pamr
TS
0.6422usm
0 0.609
0 p0301))
0.355
0.24%
0.187o
0.141
0.113
0.3 0.336
UW
0.632
0.620
0.409 euazs
0.320
0.230
0.179))
0.152muam
0.118pommo
pu332puaes
DWA
0.619us:2)
0.536amms
0.330gg244)
38
0.099
0.305
DTP
0.633
0.556quzz
0. (ow4))
14
p.309
Nash
0.674m
0 (0542
0 1,m0s
0.269
0. 0,240,
0.16
0 m.34
CAGrad
0.622%..00
0. 0.526ppaa
0. 388
0:34 12.00)
0.240
0. 0.199.
O. 6-4g0om
D. 54
0.326po15
Backhone encoder: RGCN
SingleTTast Task
0.647mas
0.462
m.373
0.25
0,203
0.164ma00
0. 0.16ppom
0.360
N-MTL
0.644
0.6-66uas0
0.402
0,376
0.231
0.1 0.190))507)
0.154quar
0.1 0.119pmmm
0.345puao
GN
0.626
0.612mesm
0.379
0.36%)mmm
0.233ya.0oc
0.169 (.169
0.1 48qa0)
0.1190m
p.331
TS
(.640)
0 0.652
(omns)
0.388
0.2677
0.20M
0.368
0.6322us:4
0.568
0.395
0.245
0.199
0.1 (.138D
qu34
DWA
0.616 0.587
0:400
8.34
0.230a.0m
0.177.a.22
0.151
10poss
0.331 quakts
DTP
0.647
0.637
0. 3.999ommm
0.370
0.20
0, 0,228
0. 190000
0. 188
0. 0.362
Nash
a.678
0. 0.585
0.379
0.368
a.
0, (aa5))
0.2 18218
0. 172
0.373
CAGrud
0.633
0.672maa0
0.319
0.224
0.184
0.146
0. 1.10p0
0.338
Therefore, is beneficial model a glycan as a multi-relational graph, where different types of glycosidic bonds are deemed as different relations between monosaccharides.
A simple shallow CNN is surprisingly effective. is observed that the 2-layer shallow CNN ranks among the top three 5 out of 11 tasks, and it ranks fifth place in terms of weighted mean rank. Therefore, such a shallow CNN model is sufficient to produce informative glycan representations and achieve competitive performance, aligning with previous findings that shallow CNNs can well model biological sequences (Shanehsazzadeh et al. 2020 Xu et al. (022).
It important to utilize glycosidic bond information. We can observe clear performance gains of heterogencous GNNs over homogeneous GNNs glycan modeling, where in terms of weighted mean rank, three heterogeneous GNNs rank 1st, 2nd and 16th places, while four homogeneous GNNs rank 6th, 9th, 12th and 13th places. Compared homogeneous GNNs that regard all glycosidic bonds as the same, heterogeneous GNNs fully utilize glycosidic bond information by individually treating each type of bonds, leading obvious benefits.
Small molecule encoders can hardly handle glycan modeling. Though performing well on small molccule modeling, all-atom molecular encoders with or without pre-training are not competitive the GLYCANML benchmark. Therefore, the small molecule encoders originally designed to model tens of atoms cannot well model a macromolecule like a glycan with hundreds of atoms, calling for specifically designed models for all-atom glycan modeling.
5.3 BENCHMARK RESULTS ON MULTI-TASK LEARNING
In Table we report the benchmark results of different MTL methods against finglc-task learning. We select shallow CNN and RGCN as the backbone encoder, and all MTL methods are evaluated on each of them. According to benchmark results, we have the findings below:
The Nash bargaining solution (Nash) method performs best. The Nash method achieves the best performance under both shallow CNN and RGCN backbones in terms of mean Macro-F1 score, and it outperforms single-task learning with a clear margin (i.e. 4.28% relative improve- ment in mean Macro-F1 score) when using shallow CNN as backbone encoder. These results demonstrate the superiority of Nash on balancing the learning signals from different glycan taxonomy prediction tasks.
The temperature scaling (TS) approach is the runner-up. On both shallow CNN and RGCN, the TS approach achieves the second best mean Macro-Fl score, and it clearly surpasses single- task learning with a 2.75% % relative improvement in mean Macro-F1 score under the shallow CNN

---

Published as a conference paper at ICLR 2025
backbone. Therefore, the TS approach is also good selection to understand the hierarchical taxonomies of glycans with a single model.
MTL methods are not always beneficial. On shallow CNN, only the Nash, TS and UW methods outperform single-task learning in terms of mean Macro-F1 score; on RGCN, only the Nash, TS and DTP methods outperform single-task learning terms of mean Macro-F1 score. Actually, most MTL methods lead to performance decrease compared to single-task learning. These results suggest the high difficulty of balancing between different glycan taxonomy prediction tasks. More efforts are thus required to boost the MTL performance on the GLYCANML-MMTL testbed, which we leave as one of our major future works.
5.4 EFFECT OF MULKI-TASK LEARNING ON GLYCAN FUNCTION PREDICTION
In this part, we study the effect of MTL on three glycan function prediction tasks, ie., immuno- genicity, glycosylation and interaction prediction. Considering that evolutionarily similar glycans tend to have similar functions, we study how gly- function prediction can be benefited by jointly learning the evolutionary taxonomy of glycans. Specifically, regard each of three function pre- diction tasks as the center task and eight taxonomy prediction tasks as auxiliary tasks, and the center task is trained along with eight auxiliary tasks to perform MTL (the losses of center and auxiliary tasks are summed up for model training).
Table 4: Benchmark results on multi-task learn- ing for glycan function prediction. We report mean (std) for each experiment.
Immonogenicity
Glycosylation (Macro-FI)
Interaction (Spearman's ' p)
Method
(AUPRC)
Backbone encoder: Shallow
Single-Task
0.776ma271
p.898
0.261
Multi-Taas
0.792ma22
0.912
0.257
Backbone encoder: RGCN
Single-Taskk Multi- Task
0.780uu000
D. 0.948
0.262
0.801
0.966
0.259
In Table 4 we compare this multi-task method with the single-task baseline. Under both Shallow CNN and RGCN encoders, MTL performs better on immunogenicity and glycosylation type prediction. while single-task learning performs better on interaction prediction. These results show the potential of MTL on boosting glycan function prediction, and also inspire future research efforts further improve its effectiveness.
6 CONCLUSIONS AND FUTURE WORK
In this work, we build comprehensive benchmark GLYCANML for glycan machine learning. It consists diverse types of glycan understanding tasks, including glycan taxonomy prediction, glycan immunogenicity prediction, glycosylation type prediction, and protein-glycan interaction prediction. In GLYCANML, we support two representation methods of glycans, iee., glycan tokenized sequences and glycan planar graphs, enabling glycan modeling with sequence encoders and graph neural networks (GNNs). Additionally, on eight highly correlated glycan taxonomy prediction tasks, set up testbed GLYCANMIL-MTL. to compare different multi-task learning (MTL.) algorithms. Also, we study how taxonomy prediction can boost other three function prediction tasks by MTL. According to the benchmark results, multi-relational GNNs show great promise for glycan modeling, and well-designed MTL methods can further boost model performance.
In the future, we will apply glycan machine learning models important real-world glycan-related tasks. For example, for vaccine design, we can employ well well-performing immunogenicity predictors and protein-glyce interaction predictors to virtually screen glycan candidates that can most effectively induce immune response.
10

---

Published as a conference paper at ICLR 2025
REFERENCES
Alhasan Alkuhlani, Walaa Gad, Mohamed Roushdy, and Abdel-Badeeh M Salem. Gnngly: Graph neural networks for glyean classification. IEEE Access, 2023.
Davide Alocci, Julien Mariethoz, Alessandra Gastaldello, Elisabeth Gasteiger, Niclas G Karlsson, Daniel Kolarich, Nicolle H Packer, and Frédérique Lisacek. Glyconnect: glycoproteomies goes visual, interactive, and analytical. Journal of proteome research, 18(2):664 677, 2018.
Ron Amon, Eliran Moshe Reuven, Shani Leviatan Ben-Arye, and Vered immune recognition and response. Carbohydrate research, 389:115-122, 2014.
Glycans in
Kiyoko F Aoki-Kinoshita. An introduction to bioinformatics for glycomics research. PLoS computa- tional biology, 4(5)xe 2008.
Maria Bånkestad, Keven M Dorst, Göran Widmalm, and Jerk Rönnols. Carbohydrate nmr chemical shift predictions using e (3) equivariant graph neural networks. arXiv preprint urXiv:231.112657 2023.
Mairead L Bermingham, Marco Colombo, Stuart McGurnaghan, Luke AK Blackbour, Frano Vučković, Maja Pučić Baković, Irena bobojevvêéAkmaütt Gordan Lauc, Felix Agakov, Anna S Agakova, et al. N-glycan profile and kidney disease in type diabetes. Diabetes care, 41(1):79-87, 2018.
Erhard Bieberich. Synthesis, processing, and function of n-glycans in n-glycoproteins. Glycobiology of the nervous system, pp. 47-70, 2014.
Daniel Bojar and Frederique Lisacek. Glycoinformatics in the artificial intelligence era. Chemical Reviews, 122(20):597 2022.
Daniel Bojar, Diogo M Camacho, and James J Collins. Using natural language processing to learn the grammar of glycans. bioRxiv, pp. 2020-01, 2020a.
Daniel Bojar, Rani K Powers, Diogo M Camacho, and James Collins. Swectorigins: Extracting evolutionary information from glycans. bioRxiv, pp. 2020-04, 2020b.
Daniel Bojar, Rani K Powers, Diogo M Camacho, and James J Collins. Deep-learning resources for
studying glycan-mediated host-microbe interactions. Cell Host & Microbe, 29(1):132-144, 2021.
Rebekka Burkholz, John Quackenbush, and Daniel Bojar. Using graph convolutional neural networks to learn a representation for glycans. Cell Reports, 35(11), 2021.
Cornelia Caragea, Jivko Sinapov, Adrian Silvescu, Drena Dobbs, and Vasant Honavar, Glycosylation site prediction using ensembles of support vector machine classifiers. BMC bioinformatics, 8:1-13, 2007.
Eric J Carpenter, Shaurya Seth, Noel Yue, Russell Greiner, and Ratmir Derda. Glynet: multi-task neural network for predicting 2022.
glycan interactions. Chemical Science, 13(22):6669 6686,
Zhao Chen, Vijay Badrinarayanan, Chen-Y Yu Lee, and Andrew Rabinovich. Gradnorm: Gradient normalization for adaptive loss balancing in deep multitask networks. In International conference on machine learning. pp. 803. PMLR, 2018.
Zizhang Chen, Ryan Paul Badman, Bethany Lachele Foley, Robert Woods, and Pengyu Hong. Glyconmr: Dataset and benchmark of carbohydrate-specing nmr chemical shift for machine learning research. Journal of Data-centric Machine Learning Research, 2024.
Miriam Cohen. Notable aspects of glycan-proteis interactions. Biomolecules, 5()):206-20072. 2015.
Bowen Dai, Daniel Mattox, and Chris Bailey- Kellogg. Attention please: modeling global and local context in glycan cenncture-function relationships. bioRxiv, pp. 2021-10, 2021.
11

---

Published as a conference paper at ICLR 2025
Christian Dallago, Jody Mou, Kadina E Johnston, Bruce J Wittmann, Nicholas Bhattacharya, Samuel Goldman, Ali Madani, and Kevin K Yang. Flip: Benchmark tasks fitness landscape inference for proteins. bioRxiv, 2021.
Zhangyang Gao, Cheng Tan, and Stan Li. Alphadesign: A graph protein design method and benchmark on alphafolddb. arXiv preprint vrXiv:2202.01000 2022.
Justin Gilmer, Samuel S Schoenholz, Patrick F Riley, Oriol Vinyals, and George E Dahl. Neural message passing for quantum chemistry. In International conference on machine learning, pp. 7263-1272, PMLR, 2017.
Michelle Guo, Albert Haque, De-An Huang, Serena Yeung, and Li Fei-Fei. Dynamic task priori- tization for multitask learning. In Proceedings of the European conference on computer vision (ECCV), pp. 270-287, 2018.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 770-778, 2016.
Sophia M Hochrein, Hao Wu, Miriam Eckstein, Laura Arrigoni, Josip S Herman, Fabian Schumacher, Christian Gerecke, Mathias Rosenfeldt, Dominic Grün, Burkhard Kleuser, et al. The glucose transporter glut3 controls helper 17 cell responses through eyrcolytic-epigenen reprogramming. Cell metabolism, 34(4):516-532, 2022.
Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9(8):
1735-1780, 1997.
Yanrong Ji, Zhihan Zhou, Han Liu, and Ramana V Davuluri. Dnabert: encoder representations from transformers model for dna-language in genome. Bioinformatics, 37 (15):2112-2120,, 2021.
pre-trained bidirectional
Paulina Kaplonek, Naeem Khan, Katrin Reppe, Benjamin Schumann, Madhu Emmadi, Marilda Lisboa, Fei-Fei Xu, Adam DJ Calow, Sharavathi G Parameswarappa, Martin Witzenrath, al. Improving vaccines against streptococcus pneumoniae using synthetic glycans. Proceedings of the National Academy of Sciences, 115(52)) 13358, 2018.
Alex Kendall, Yarin Gal, and Roberto Cipolla. Multi-task learning using uncertainty to weigh losses for scene geometry and semantics. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 7482-7491, 2018.
Thomas N Kipf and Max Welling. Semi Semi-supervised classification with graph convolutional networks. International Conference on Learning Representations, 2017.
Zachary Klamer, Ben Staal, Anthony R Prudden, Lin Liu, David F Smith, Geert-Jan Boons, and Brian Haab. Mining high-complexity motifs in glycans: new language to uncover the fine specificities of lectins and glycosidases. Analytical chemistry, (9(22) 123422-23122 2017.
Andriy Kryshtafovych, Torsten Schwede, Maya Topf, Krzysztof Fidelis, and John Moult. Critical assessment of methods of protein structure prediction (casp)-roun) xv. Proteins: Structure, Function, and Bioinformatics, 2023.
Shotaro Kumozaki, Kengo Sato, and Yasubumi Sakakibara. A machine learning based approach to de novo sequencing of glycans from tandem mass spectrometry spectrum. IEEE/ACM transactions computational biology and bioinformatics, 1266::267712744, 2015.
Kyung Jin Lee, Jin-Hee Jung, Jung Mi Lee, Yangkang So, Ohsuk Kwon, Nico Callewaert, Hyun Ah Kang, Kisung Ko, and Doo- Doo-Byouu Byoung Oh. High-throughput quantitative analysis of plant n-glycan using a dna sequencer. Biochemical and Biophysical Research Communications, 380(2):223-229, 2009.
Fuyi Li, Chen Li. Mingjun Wang, Geoffrey Webb, Yang Zhang, James C Whisstock, and Jiangn- ing Song. Glycomine: a machine learning-bas based approach for predicting n-, c-and o-linked glycosylation in the human proteome. Bioinformatics, 1411):4111-1111 2015.
12

---

Published as a conference paper at ICLR 2025
Haining Li, Austin WT Chiang, and Nathan E Lewis. Artificial intelligence in the analysis of glycosylation data. Biotechnology Advances, 60:108008, 2022.
Suh-Yuen Liang, Sz-Wei Wu, Tsung-Hsien Pu, Fang- Yu Chang, and Kay-Hooi Khoo. An adaptive workflow coupled with random forest algorithm to identify intact n-glycopeptides detected from mass spectrometry. Bioinformatics, 10(13):19881111 2014.
Tsung-Y Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. Focal loss for dense object detection. In Proceedings of the IEEE international conference on computer vision, pp. 2980-2988, 2017.
Bo Liu, Xingchao Liu, Xiaojie Jin, Peter Stone, and Qiang Liu, Conflict-averse gradient descent for multi-task learning. Advances in Neural Information Processing Systems, 34:
2021.
Shikun Liu, Edward Johns, and Andrew Davison. End-to-end multi- learning with attention. In Proceedings of the IEEF/CVF conference on computer vision and patlern recognition, pp. 1871-1880, 2019.
Shuqi Lu, Zhifeng Gao, Di He, Linfeng Zhang, and Guolin Ke. Highly accurate quantum chemical property prediction with uni-mol+, arXiv preprint arXiv:2303. 16982, 2023.
Jon Lundstrøm, Emma Korhonen, Frédérique Lisacek, and Daniel Bojar. Lectinoracle: generalizable deep learning model for lectin-glycan binding prediction. Advanced Science, 9(1)21003807, 2022.
Aviv Navon, Aviv Shamsian, Idan Achituve, Haggai Maron, Kenji Kawaguchi, Gal Chechik, and Ethan Fetaya. Multi-task learning bargaining game. arXiv preprint 2rXiv:2202.01017 2022.
Eric Nguyen, Michael Poli, Matthew G Durrant, Armin W Thomas, Brian Kang, Jeremy Sullivan, Madelena Y Ng, Ashley Lewis, Aman Patel, Aaron Lou, et al. Sequence modeling and design from molecular genome scale with evo. bioRxiv, pp. 2024-02- 2024.
Subash C Pakhrin, Kiyoko F Aoki-Kinoshita, Doina Caragea, and Dukka Kc. Deepnglypred: deep neural network-bass based approach for human n-nkneed glycosylation site prediction. Molecules, 26(23):7314, 2021.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, pigh-peeformance deep learning library. Advances in neural information processing systems, 32, 2019.
Thejkiran Pitti, Ching-Tai Chen, Hsin-Nan Lin, Wai-Kok Choong, Wen-Lian Hsu, and Ting-Y! Yi Sung. N-glyde: a two-stage n-linked glycosylation site prediction incorporating gapped dipeptides and patter--base based encoding. Scientific reports, 9(1::19975 2019.
Andrew Porter, Tingting Yue, Lee Heeringa, Steven Day, Edward Suh, and Brian B Haab. A motif-based analysis of glycan array data to determine the specificities of glycan-binding proteins. Glycobiology, 20(3):369-380 2010.
Ladislav Rampášek, Michael Galkin, Vijay Prakash Dwivedi, Anh Tuan Luu, Guy Wolf, and Do- minique Beaini. Recipe for general, powerful, scalable graph transformer. Advances in Neural Information Processing Systems, 35: 35:14501-14515, 2022.
Roshan Rao, Nicholas Bhattacharya, Neil Thomas, Yan Duan, Peter Chen, John Canny, Pieter Abbeel, and Yun Song. Exaluating protein transfer learning with tape. Advances in neural information processing systems, 32, 2019.
Alexander Rives, Joshua Meier, Tom Sercu, Siddharth Goyal, Zeming Lin, Jason Liu, Demi Guo, Myle Ott, C Lawrence Zitnick, Jerry Ma, et al. Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences. Proceedings of the National Academy of Sciences. 2e201199 2021.
Jerret Ross, Brian Belgodere, Vijil Chenthamarakshan. Inkit Padhi, Youssef Mroueh, and Payel Das. Large-scale chemical language representations capture molecular structure and properties. Nature Machine Intelligence, 4(12): 2e0:11266226 2022.
13

---

Published as a conference paper at ICLR 2025
Michael Schlichtkrull, Thomas N Kipf, Peter Bloem, Rianne Van Den Berg, Ivan Titox, and Max Welling. Modeling relational data with graph convolutional networks. The semantic web: 15th international conference, ESWC 2018, Heraklion, Crete, Greece, June 3-7, 2018, proceedings 15, pp. 593-677. Springer, 2018.
Amir Shanehsazzadeh, David Belanger, and David Dohan. Is transfer learning necessary for protein landscape prediction? arXiv preprint arXiv:201 2020.
Martin Steinegger and Johannes Söding. Mmseqs2 enables sensitive protein sequence searching for the analysis of massive data sets. Nature biotechnology, 35(1 (5(11):1026-102 2017.
Luc Thomès, Rebekka Burkholz, and Daniel Bojar. Glycowork: A python package for glycan data science and machine learning. Glycobiology, 31 (1110)12200122) 2021.
Michael Tiemeyer, Kazuhiro Aoki, James Paulson, Richard D Cummings, William York, Niclas G Karlsson, Frederique Lisacek, Nicolle H Packer, Matthew P Campbell, Nobuyuki P Aoki, et al. Glytoucan: an accessible glycan structure repository. Glycobiology, 27(10):915 919, 2017.
Raphael JL Townshend, Martin Vögele, Patricia Suriana, Alexander Derry, Alexander Powers, Yianni Laloudakis, Sidhika Balachandar, Bowen Jing, Brandon Anderson, Stephan Eismann, et al. Atom3d: Tasks molecules in three dimensions. arXiv preprint urXiv:2012.04033: 2020.
Shikhar Vashishth, Soumya Sanyal, Vikram Nitin, and Partha Talukdar. Composition-based multi- relational graph convolutional networks. arXiv preprint arXiv:
1911 1. 1.03022, 2019.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention all you need. Advances in neural information processing systems, 30, 2017.
Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. Graph attention networks, arXiv preprint arXiv:1710.10903 2017.
Antonio Villalobo, Aitor Dogales-Gonzálee and Hans-J Gabius. A guide to signaling pathways con- necting protein-glycan interaction with the emerging versatile effector functionality of mammalian lectins. Trends in Glycoscience and Glycotechnologyy 18(99):1-33) 2006.
Oriol Vinyals, Samy Bengio, and Manjunath Kudlur. Order matters: Sequence to sequence for sets. arXiv preprint arXiv: 1ee.:/5111...,11 2015.
Xi Wang, Ruichu Gu, Zhiyuan Chen, Yongge Li, Xiaohong Ji, Guolin Ke, and Han Wen. Uni-rna: universal pre-trained models revolutionize ma research. bioRxiv, pp. 2023-07, 2023.
Yu Wang, Hui Wang, Meijie Hou, Yaojun Wang, Dongbo Bu, Chunming Zhang, Chuncui Huang. and Shiwei Sun. Glycan immunogenicity prediction based on graph neural network. In 2021 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), pp. 348-353, IEEE, 2021.
Yuyang Wang, Jianren Wang, Zhonglin Cao, and Amir Barati Farimani. Molecular contrastive learning of representations via graph neural networks. Nature Machine Intelligence, 403):799-277, 2022.
Zhenqin Wu, Bharath Ramsundar, Evan N Feinberg, Joseph Gomes, Caleb Geniesse, Aneesh S Pappu, Karl Leswing, and Vijay Pande. Moleculenet: benchmark for molecular machine learning. Chemical science, (1)::513-300. 2018.
Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How powerful are graph neural networks? arXiv preprint arXiv: 101000080000026 2018.
Minghao Xu, Zuobai Zhang, Jiarui Lu, Zhaocheng Zhu, Yangtian Zhang, Ma Chang, Runcheng Liu, and Jian Tang. Peer: a comprehensive and multi-task benchmark for protein sequence understanding, Advances in Neural Information Processing Systems, 35:35156-35173, 2022.
Yoshihiro Yamanishi, Francis Bach, and Jean- Philippe Vert. Glycan classification with tree kernels. Bioinformatics, 23(10): 121-11221 2007.
14

---

Published as a conference paper at ICLR 2025
Maomao Yan, Yuyang Zhu, Xueyun Liu, Yi Lasanajak, Jinglin Xiong, Jingqiao Lu, Xi Lin, David Ashline, Vernon Reinhold, David F Smith, et al. Next-generation glycan microarray enabled by dna-coded glycan library and next-generation sequencing technology. Analytical chemistry, 91 (14):9221-9228) 2019.
Chengxuan Ying, Tianle Cai, Shengjie Luo, Shuxin Zheng, Guolin Ke, Di He, Yanming Shen, and Tie- Yan Liu. Do transformers really perform badly for graph representation? Advances in neural information processing systems, 34:28877-28888, 2021.
Xiao-Lian Zhang. Roles of glycans and glycopeptides in immune system and immune-related diseases. Current medicinal chemistry, 13(10):1111-147) 2006.
Yu Zhang and Qiang Yang. A survey on multi-task learning. IEEE Transactions on Knowledge and Data Engineering. 34(12):5586 5609, 2021.
Zuobai Zhang, Minghao Xu, Arian Jamasb, Vijil Chenthamarakshan, Aurelie Lozano, Payel Das, and Jian Tang. Protein representation learning by geometric structure pretraining. arXiv preprint arXiv:2203.066/25, 2022.
Zhaocheng Zhu, Chence Shi, Zuobai Zhang, Shengehao Liu, Minghao Xu, Xinyu Yuan, Yangtian Zhang, Junkun Chen, Huiyu Cai, Jiarui Lu, et al. Torchdrug: A powerful and flexible machine learning platform for drug discovery. arXiv preprint arXi::200.008320, 2022.
15

---

Published as a conference paper at ICLR 2025
A
DETAILS OF MODEL ARCHITECTURES
Table 5: Architectures of baseline models. Abbr, Params.: parameters; dim.: dimension; conv.:
convolutional; attn.: attention; concat.: concatenate. * denotes a pre- tre-trained model.
Model
Inpot Layer
Hidden Layers
Output Layer
AParams.
Sequence encoders
Shallow CNN
128-dim. taken emherkting
x comv layers (hidkien dim:: 128:
pooling
191.7K
kernel size: 5; stride: 1: packing: 2)
512-dim, token embeckding
residual blocks thidden dim.; 512:
ResNet
112-imm dim. positional embedding
kemnel size; 3; stride; I: packing: 1)
artentive weighted sum over all tokens
11,4M
bidirectional LSTM layers
weighted
all
LSTM
640-dim. token emitckling
(hidden dim.: 640)
linear (output dim.: 640) Tanh
26.7M
Transformer
512-fim. positional embedding
512-dim. takes emheddding
Transforme blocks (hidden dim. 512;
heads: activation GELU)
linear (outpet dim. 512) Tanh
upon
21.4M
Homogeneous GNNS
GCN
128 dim. node embedding
3 GCN layers
& pooling concat, max pooling
8.
GAT
node embedding
GAT layers
GIN
128 dim, mode embodding
3 x GIN layers
concal. mean & max pooling
117.4K
PNA
128 dim, node embedding
3 x PNA layers
Set2Set pooling (tsteps: 3)
2.6M
Heterogencous GNNs
MPNN
128-dim, node & odge embeaiding
3 x MPNN layers
Se12Sct pooling (Hatpps: 3)
4.0M
RGCN
128-dim, node embbaiding
3 RGCN layers
concall mean & max pooling
4.2M
CompGCN
128-dim. node erbeedding
CompUCN layers
max pooling
1504K
AB-Atom Molecular Encoders
Graphormer
768 dim. mode embedding
12 Graphormer
readout virtual node
47.7M
GruphGPS
512-dim. edge embedding
GraphGPS layers
concat, max pooling
512 dim. node embodding
MolFormer
512dim. pasitional emhoddding
6 x MolFurmer layers
concall mean & max pooling
18.7M
MolCER
300-dim. node embedding
MelCIR-GiN R-GG layers
& pooling
645.3K
Uni-Mole
512-dim. emhheddng
64-dim, pair umbeolinng
Uni Mal+ layers
concal. mean & max pooling
11.2M
Pre-trained Molecular Encoders
Gruphormer*
768-dim. mode embedding
12 x Grapharmer layers
readout on virtual node
4777M
MoCCLR*
300-dim. node embedding
5 x MelCLR GIN layers
concat. mean & max pooling
645.3K
We provide the detailed architectures of baseline models in Table For the input layer of each model, we employ an embedding layer to get the representation of each token/node; an additional positional embedding layer used by ResNet, Transformer and MolFormer to represent sequential information; MPNN adopts an additional edge embedding layer to represent glycosidic bonds. We follow the implementation of different sequence and graph encoding layers TorchDrug (Zhu et al.) 2022) to construct hidden layers. For the output layer of sequence encoders, we follow the implementation TorchDrug to readout a glycan-level representation from token-level representations. We use the concatenation of mean and max pooling the output layer of GCN, GAT, GIN, RGCN, CompGCN, GraphGPS, MolFormer, MolCLR and Uni-Mol+ for its superior performance in our experiments. Following the original design of PNA and MPNN, Set2Set pooling (Vinyals et al. 2015) serves as their output layers. We follow the default architecture of PNA, Graphormer, GraphGPS, MolFormer, MolCIR and Uni-Mol+ in their original work.
16


Reviews:
[{"review_id": "wTTGj1OT3H", "reviewer": "Reviewer_5NHt", "summary": "This paper aims to and succeed in providing a standard machine learning benchmark for **glycans**. It will clearly benefit the future research in studying glycan property and function prediction via application of machine learning models, and it will make positive impacts on bioinformatics and biological intelligence community. The proposed benchmark **GLYCANML** consists of multiple essential task in glycans research, and the benchmark also offers diverse data modalities, mainly sequence representations and graph representations. Additionally, the *GLYCANML-MTL* testbed is designed to support multi-task learning (MTL), where glycan taxonomy tasks are addressed collectively to assess knowledge transfer across related prediction tasks.", "questions": "See above weaknesses.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 2, "strengths": "1. **Having a clear as well as straightforward motivation and contribution**. The benchmark is highly relevant for advancing glycan-related machine learning research, offering a platform to assess various models on tasks that are both scientifically and practically significant in fields like immunology and molecular biology.\n2. **Well-written paper; addressing each detail of GLYCANML thoroughly**. The paper is comprehensive, including detailed dataset construction, task definitions, and experimental protocols. It also provides a robust comparison of baseline models, which supports its claims of model performance and suitability.\n3. **High benchmark quality: multiple tasks and diverse modalities provided; allowing multitask learning and multimodal training**. The *GLYCANML-MTL* setup for multi-task learning offers a unique opportunity to test MTL methods in a real-world scenario with related biological tasks, demonstrating the potential to improve performance through shared learning.", "weaknesses": "1. **Broader Impact of the Datasets**. From my understanding the dataset is very good resource for the application related to Glycans. However, for machine learning, it is not clear how this dataset is different from previous ones and what new insights it can bring to the domain of graph learning or multi-task learning.\n2. **Lack of Multimodal Encoders**. The benchmark has sequence and graph representation, and it is good to show the performance of multimodal molecular encoders like MolFormer [1].\n3. **More Pretrained Model Performance**. The paper did not explicitly say whether using the checkpoints for Graphormer or not, and if indeed using the pretrained checkpoint, it will also surpise me that a pretrained model even fail to match the standard sequence models and graph models. Also, other pretrained models, such as MolCLR [2], can be presented.\n\n[1] Ross, Jerret, et al. \"Large-scale chemical language representations capture molecular structure and properties.\" _Nature Machine Intelligence_ 4.12 (2022): 1256-1264.\n[2] Wang, Yuyang, et al. \"Molecular contrastive learning of representations via graph neural networks.\" _Nature Machine Intelligence_ 4.3 (2022): 279-287.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "zZ47q94Qig", "reviewer": "Reviewer_iSYu", "summary": "This paper presents a benchmark for evaluating different graph encoders and multi-task learning frameworks on glycan data, covering four tasks with corresponding curated datasets. The benchmark results reveal interesting findings, such as the effectiveness of shallow CNNs as strong baselines and all-atom encoders fail to achieve better performance than GNN-based models.", "questions": "- Why is the sequence identity threshold set to 0.5? A common choice would be 0.3 in protein-related studies [1,2].\n\n[1] Evaluating Representation Learning on the Protein Structure Universe, ICLR2024.\n\n[2] Learning from Protein Structure with Geometric Vector Perceptrons, ICLR2021", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The application of machine learning on glycans is interesting and unexplored. The curated datasets can benefit the model development and future studies.\n- The studied multi-task learning on glycans demonstrates the strong correlation between different tasks.", "weaknesses": "- As a benchmark study, the GNN encoders and multi-task learning methods compared in the paper are not state-of-the-art. More advanced methods should be included in the benchmark for a comprehensive evaluation, such as [2,3,4,5].\n- Without context such as cell type or antigen, predicting immunogenicity based solely on glycan input is not practically useful since immunogenicity is heavily influenced by the biological context. Immune recognition often involves complex molecular interactions influenced by surrounding biomolecules and cellular environments.\n- One suggestion for Protein-Glycan Interaction Prediction is to conduct the splitting based on the sequence identity of binding sites instead of the whole protein sequence identity[1], which can provide a more solid evaluation protocol.\n\n[1] PoseBusters: AI-based docking methods fail to generate physically valid poses or generalise to novel sequences\n\n[2] Highly Accurate Quantum Chemical Property Prediction with Uni-Mol+, Nature Communications\n\n[3] Principal Neighbourhood Aggregation for Graph Nets, NeurIPS2020\n\n[4] Multi-Task Learning as a Bargaining Game, ICML2022\n\n[5] Conflict-Averse Gradient Descent for Multi-task Learning, NeurIPS2021", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "QtBLsIfBoY", "reviewer": "Reviewer_ym9r", "summary": "This paper first benchmark paper on ML for Glycans.", "questions": "- To my knowledge, glycans have not received much attention at ML conferences compared to proteins. Is there a specific reason for this?\n- In a glycan planar graph, nodes are monosaccharides. What feature information is used in this case?", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Introducing an important problem in the field of biology to the machine learning community and creating a new problem is very significant.", "weaknesses": "- Since biology is not my major field, there needs to be more emphasis on how biologically significant glycans are.\n- There needs to be a discussion on how well this paper fits the ICLR conference.\n- Collecting the benchmark dataset does not seem to pose significant challenges. What challenges are there?\n- All data splits are based on motif-based splits, but there is no discussion on the biological significance of motifs. From the perspective of scientific discovery, is a motif-based approach appropriate for modeling the real world? It seems that data could also be split by methods such as random, monosaccharide composition, or glycosylation type, and there needs to be a discussion on this.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)