dataesr/scientific-paragraphs-categorization huggingface dataset:
Observations and meteorological forcing This work is based on the data available from a time series of 12 hyper-resolution (5 m spatial resolution) snow depth maps collected during a single snow season over the Izas experimental catchment in the Pyrenees (55 hectares  Revuelto et al. , 2017)  (Fig. 1 ). The maps were retrieved from fixed wing drone surveys using structure from motion techniques. The drone surveys were conducted in 2020 on 14/Jan, 03/Feb, 24/Feb, 11/Mar, 29/Apr, 3/May, 12/May, 19/May, 26/May, 02/Jun, 10/Jun, 21/Jun. Several surveys were conducted before peak accumulation (estimated to be in mid March) but the late snowmelt season in May and June was sampled most densely. This dataset has been extensively validated  (Revuelto et al. , 2021a, c)  and ingested in purely temporal snow DA experiments in a previous study  (Alonso-González et al.
We used a meteorological forcing dataset that was previously generated by Alonso-González et al. (2022a) using the Mi-croMet meteorological distribution system  (Liston and Elder, 2006). MicroMet was applied to downscale hourly outputs of the ERA5 atmospheric reanalysis from their native resolution of 0. 25 •  (Hersbach et al. , 2020)  to the same 5 m grid as the drone observations. This meteorological forcing was used to drive the Multiple Snow data Assimilation system  (MuSA, Alonso-González et al. , 2022a). MuSA is an open-source snow data assimilation toolbox designed to assimilate various observations of the snowpack into an ensemble of snowpack simulations generated by the energy and mass balance model the Flexible Snow Model (FSM2  Essery, 2015). It was necessary to include several modifications in the MuSA code, as detailed in the Section 2.
Acknowledgements. This joint research effort was initiated through the  JASPER  project (no. 337515 ) funded by the  Research Council of Norway. Esteban Alonso-González  has been funded by the  CNES  postdoctoral fellowship. Kristoffer Aalstad  and  Norbert Pirk  were funded by the  Research Council of Norway  through the  Spot-On  project (no. 301552 ) and acknowledge support from the  LATICE Strategic Research Initiative at the University of Oslo. Marco Mazzolini  and  Désirée Treichler  acknowledge funding from the  Research Council of.
Code and data availability. The MuSA code can be found at https://github. com/ealonsogzl/MuSA (last access: 8 May 2023, Alonso-González (  2023 )). The complete drone surveys and meteorological forcing can be downloaded from https://doi. org/10. 5281/zenodo. 7248635 (last access: 8 May 2023, Alonso-González (  2022 )).
The literature already counts numerous works where this kind of approach is described  (Lee et al. , 2012; Li et al. , 2015; Meng et al. , 2018; Sprague et al. , 2020; Hodgson et al. , 2021; Elie et al. , 2022; Della Posta et al. , 2022). In this paper, we present the implementation, verification and validation of a new solver derived from such a coupling: the high-order LES code YALES2  (Moureau et al. , 2011)  is used to solve the flow and compute the loads acting on the blades using the ALM, and the structural response is computed by BHawC. A substantial advantage of this code is the possibility to emulate the actual controller of industrial wind turbines. To the authors' knowledge,  (Gremmo et al. , 2022)  is the only reference where a code with these capabilities has been reported. In this study, the authors investigated a row of seven industrial wind turbines, operating in several wind conditions.
Conversely, BHawC can provide the structural response of an industrial wind turbine, for a wide variety of incident flows, while also emulating real control strategies, meaning the azimuthal rotation speed is unsteady. Yet, the code does not offer any information on the resulting wake, and only one turbine can be handled at a time. In light of this, a coupling As designed, the coupling between YALES2 and BHawC only affects the blades of the wind turbine. The remaining substructures, such as the nacelle and the tower, are still represented in BHawC, but the aerodynamic loads they would normally experience are disabled. In the following, the coupling strategy is described in details, as well as the mathematical framework upon which the coupling relies to transfer data between YALES2 and BHawC.
Structural nodes Aerodynamic nodes Particles M N,P B Y 2 = M P,P B Y 2 + --→ N P P B Y 2 × F. ( 13 ) At the beginning of each of its time-steps, BHawC predicts the velocities and positions of the blade elements. The converged velocities and accelerations from the previous time-step are used for this purpose. In addition, the next orientation of the element bases is also predicted. These data are then sent to YALES2. However the corresponding variables are not updated immediately, so as to evaluate a relative velocity which accounts for the last mollification of the aerodynamic forces. Angles of attack are inferred using the element bases from the previous time-step, and the aerodynamic loads are derived subsequently in these bases. The position, velocity and basis of each element is then updated to transfer the loads into both the global frame of YALES2 and BHawC: F N,GB BH = T GB BH GB Y 2 -1 • T P B Y 2 GB Y 2 • F N,P B Y 2 , M N,GB BH = T GB BH GB Y 2 -1 • T P B Y 2 GB Y 2 • M N,P B Y 2.
Verification Before attempting real-application cases, the coupled code YALES2-BHawC has been extensively checked. Relying on appropriate unit tests, we verified both the kinematic and dynamics aspects involved in the coupling. Discussed in Section 4. 1, the kinematic verification is carried out by comparing the results from simulations run with YALES2 standalone (Y2) and YALES2-BHawC (Y2-BH) respectively. On the other hand, the verification of the dynamic aspects, reported in Section 4. 2, is supported by a comparison of the results obtained with BHawC standalone (BH) and YALES2-BHawC. In all cases we consider the SWT-6. 0-154 wind turbine model (Siemens Gamesa Renewable Energy, 2014), whose rotor diameter is D = 154 m. The rotor rotation speed is fixed to a constant value in YALES2 standalone, as it is the most straightforward approach.
In YALES2, the resulting flow is thus uniform throughout the whole domain, which allows the use of a narrow computational domain (as illustrated in Figure  7 ), along with a very coarse cartesian grid (cell size ∼ 12. 5 m). In all numerical setups, the blades are equally discretized: 75 aerodynamic nodes/particles are evenly distributed along the blades span. The look-up table used in YALES2 was built to contain the blade data BHawC would attach to its aerodynamic nodes in such a configuration. This way, blade data at the particles location in YALES2 are enforced to match those used in BHawC, as interpolation errors are avoided. Besides, the blades elasticity and shear moduli are chosen 1000 times larger than the actual ones, almost nullifying the blade deflections. Obviously, this does not apply to YALES2 standalone simulations where blades are fully rigid.
Kinematic aspects As explained in Section 3, in a coupled simulation BHawC communicates to YALES2 the position and velocity of all AL particles, as well as the 3D basis attached to them. Therefore, we need to ensure, at first, that the AL particles in YALES2-BHawC are moving as they would in YALES2 standalone. The results obtained with both codes on the configuration previously described are thus compared. However, two differences between the numerical setups must be stressed, as they cannot be removed. The first one concerns the blades: they are fully rigid in YALES2, while they almost are in YALES2-BHawC. As a result, the blades in the coupled simulation necessarily experience tiny deflections. The second difference is related to the rotor rotation speed: it is imposed constant throughout the whole simulation in YALES2, while a transient state occurred in the coupled code because of the dummy controller.
Yet, the amplitude of the deflections compares very well for all wind speeds. In Figure  18 , the comparison is continued in polar coordinates to visualize the distribution of discrepancies according to the azimuthal position. To this end, we compare the mean deflection obtained at 36 azimuthal positions, based on the data from the last N = 50 revolutions of the rotor. Overall, the results from both codes agree fairly well at all wind speeds, with discrepancies mostly lying in the range [-5%, 5%]. No clear trend appears, meaning the coupled code does not tend to either over-predict or under-predict the results from BHawC. The local gap between the results peaks in the azimuthal range [180 • , 270 • ], for the high wind speed case, where the deflection predicted by YALES2-BHawC is on average 10% higher than the one computed by BHawC.
It can be noticed that the value for the low wind speed case is almost twice the target value. While this last problem cannot be fully worked around, one way to address it would be to replace the Mann boxes in BHawC with velocity fields extracted from the YALES2-BHawC simulation, say one diameter upstream of the turbine (Houtin-Mongrolle, 2022). Considering all the aforementioned sources of discrepancies, the results given by the code YALES2-BHawC for the structural part are therefore deemed encouraging and promising. As for the prediction of the main operating parameters and performance of the turbine, the coupled code already provides very accurate results.
Computing performance Finally, we briefly report the computing performance obtained with BHawC standalone and YALES2-BHawC. All the numerical simulations previously commented on were run on the AMD Rome partition of the Joliot-Curie supercomputer from TGCC. The simulations carried out with BHawC standalone (BH) were handled with only one CPU core, as the code is fully serial, while all the coupled simulations (Y2-BH) used 255 CPU cores: one for BHawC and the remaining ones for YALES2. The return time of each simulation, given per second of computed physical time, is reported in Table  3. In all cases, the exclusivity flag of the job scheduler was deliberately activated to achieve a better computing performance. Still, one should note that all the simulations were run only once to save CPU hours.
Low speed Then the verification of the kinematic and dynamic aspects involved in the coupled code was presented. For the former, we ensured that the position, velocity and orientation of the AL particles were computed correctly. To do so, a one-to-one comparison with YALES2 standalone was conducted. For the latter, we compared the internal loads of the first blade, computed by the coupled code and BHawC standalone. For both comparisons, the discrepancies were found to be negligible. Finally, for the validation sake, we considered the case of an isolated turbine of the Westermost Rough offshore wind farm. The power output obtained numerically, as well as the rotation speed and pitch angles, were compared to field data derived from 10-minute onsite recordings. Overall, an excellent agreement was pointed out.
Difficulties in translating the actual incident wind into boundary conditions may explain the remaining discrepancies. To deepen the validation process, the deflections and internal loads of the blades, predicted both by YALES2-BHawC and BHawC, were compared. Again, the results agreed reasonably well, despite inherent differences in the numerical setups and in how the local inductions are computed. Especially, the lack of a dynamic stall model in YALES2-BHawC could explain the differences in the damage equivalent loads across a blade. Still, on average the flapwise deflections at the blade tip were shown to be only 5% apart, and their frequency spectra overlapped well. Finally it seems important to stress that, from an industrial perspective, the coupled code YALES2-BHawC remains too demanding in memory and CPU resources, and too time-consuming to be used directly in actual design cycles.
Protein language models (PLMs) based on machine learning have demonstrated impressive success in predicting protein structure and function. However, general-purpose ("foundational") PLMs have limited performance in predicting antibodies due to the latter's hypervariable regions, which do not conform to the evolutionary conservation principles that the models rely on. In this study, we propose a new transfer learning framework called AbMAP, which fine-tunes foundational models for antibody-sequence inputs by supervising on antibody structure and binding specificity examples. Our feature representations accurately predict an antibody's 3D structure, mutational effects on antigen binding, and paratope identification. AbMAP's scalability paves the way for large-scale analyses of human antibody repertoires.
AbMAP representations of repertoires reveal a remarkable overlap across individuals, transcending the limits of sequence analyses. Our findings provide compelling evidence for the hypothesis that antibody repertoires of individuals tend to converge towards comparable structural and functional coverage. We anticipate AbMAP will accelerate the efficient design and modeling of antibodies and expedite the discovery of antibody-based therapeutics. Availability: https://github. com/rs239/ablm In modern therapeutics, antibodies have been some of the most promising drug candidates  (1). This therapeutic success has been due to the remarkable structural diversity of antibodies, allowing them to recognize an extremely wide variety of potential targets. This diversity originates from their hypervariable regions which are critical to the functional specificity of antibodies.
One direction towards this has been to model the 3D structure of the entire antibody, or just its CDRs  (6, 7) , but these have had limited accuracy. They are also slow and require many minutes per antibody (or CDR) structure, making it infeasible to perform large-scale computational exploration or analyze an individual's antibody repertoire, which may contain millions of sequences. More recently, machine learning techniques used in natural language processing have been applied to generate high-dimensional protein representations  (8, 9, 10, 11). Protein language models (PLMs) capture structural features implicitly and also enable protein-property prediction. In the context of antibodies, one approach is to simply use PLMs trained on the corpus of all proteins (e. g. , ESM  (8) ). We refer to these as "foundational" PLMs, the machine learning term for broad, general-purpose models  (12).
We formulate structure prediction with AbMAP as a templatesearch task where, for a query antibody, we search a template database of AbMAP antibody embeddings. Even without template refinement, our model is able to outperform many of the state-of-the-art structure prediction techniques, both general (e. g. , AlphaFold 2 (5)) and antibody-specific (e. g. , DeepAb  (6) ). It especially excels on the prediction of individual CDR structures, which is critical for accurate design. For ddG prediction, we compare the performance of foundational PLM embeddings against their AbMAP-adjusted embeddings. Using AbMAP improves on prediction accuracy overall, achieving especially high precision in its top hits. On paratope prediction, AbMAP compares favorably with foundational PLMs as well ParaPred  (18) , despite having a smaller representation with fewer degrees of freedom.
AbMAP: a language model of antibody hypervariable regions For a given antibody sequence, we start from an embedding generated by a foundational PLM and refine it with AbMAP. There are three main steps in our refinement: identification of CDRs, augmenting the foundational PLM embeddings to focus on the CDRs, and an attention-based fine-tuning of the embedding to better capture antibody structure and function. We first apply Figure  1 : Overview of AbMAP embedding generation and its architecture. A) Given an input antibody sequence, our pipeline generates an embedding that can be applied for various downstream tasks including structure/property prediction as well as antibody repertoire analysis. B) AbMAP architecture comprises a projection module that applies contrastive augmentation and reduces the dimensionality of the input foundational PLM embedding to generate a variable length embedding, and a Transformer Encoder module that creates a {structure/function}-specific fixed-length embedding.
We then compute the augmented embedding as the difference between the embeddings of the original sequence and the average over mutants. Our augmentation aims to subtract away the sub-space of the embedding that does not correspond to the CDRs and, akin to masked language modeling, it highlights the contribution to embedding of a specific amino acid by contrasting it against potential replacements. We then optimize this augmented embedding with a Siamese neural network architecture with a single transformer layer that takes pairs of antibody sequences as inputs. Fig. 1b  provides an overview of the AbMAP architecture. We seek a final representation for each antibody where Euclidean distances capture structural and functional information. Towards this, we adopted a multi-task learning formulation, requiring that the embeddings capture both structural similarity (from SABDab  (22) ) as well as functional similarity (antigen binding specificity scores from Setliff et al.
Antibody Structure Prediction We approached structure prediction with AbMAP as a template-matching task: we searched through a database of antibody templates to find the example that we expect to be closest in structure to a query antibody. The template can later be refined (e. g. , the Evoformer module (5)), a direction we hope to explore in future work (Discussion). For benchmarking purposes, the AbMAP template quality is itself an indication of the method's ability to recapitulate structure. We constructed the template database from the set of SAbDab structures during AbMAP's training; re-using these examples allowed us to evaluate on the remaining structures in SABDab. We first applied CD-HIT  (24, 25)  to remove any template entries with greater than 70% sequence identity of test entries (our results are robust to this threshold as shown in Fig.
Compared to their respective foundational PLMs, each of the corresponding AbMAP variants performed substantially better. Overall, AbMAP-B performed better than other variants, possibly because the underlying foundation model is trained on both sequence and structure. Notably, AbMAP also improved over dedicated structure-prediction methods broadly. In particular, AlphaFold 2 performed substantially worse than others. This is consistent with reports of it underperforming on targets (like antibodies) where high-quality multiple sequence alignments are not available  (27). AbMAP was also competitive with the language model-based OmegaFold, outperforming it on the TM-score metric and being roughly equivalent with it on the RMSD metric. On both metrics, the relative performance of AbMAP was especially strong on the crucial CDR-H3 region, suggesting that our generated embeddings may contain rich information about the CDRs that contribute the most to the antibody's activity and specificity.
AbMAP Reveals Shared Landscapes Across Antibody Repertoires The scalability of our approach, along with its fidelity in capturing structure and function, enables systematic analyses of large antibody repertoires. Existing approaches are ill-suited to such analyses: while structure-prediction methods can not scale to large repertoires, purely sequence similarity-based analyses  (19)  will not be sensitive to structural/functional similarities between antibodies with different sequences. While PLMs directly address this concern, language models that learn the full antibody's representation (e. g. , AntiBERTa (  13 )) may misemphasize the framework diversity at the expense of CDR diversity, with the latter being the key determinants of antibody specificity.
LIBRA-Seq Cell Line Identification As a preliminary evaluation, we assessed AbMAP on Setliff et al. 's LIBRA-seq study that profiled B-cell receptor (BCR) binding specificity against a panel of HIV and influenza-related antigens. We wondered if AbMAP is able to recover the cell of origin of the BCR. While we had used a subset of this dataset in AbMAP's multi-task training, we note that no cell-of-origin information was provided during training. Furthermore, we evaluated on a subset of BCRs held out during training. For consistent visualization across analyses, we first standardized a 2-dimensional representation of the overall antibody space. Using the sketching algorithm Hopper (33), we selected 1,000 diverse antibodies from the SABDab training set. The farthest-first sampling approach of Hopper is motivated by the vertex k-center problem, and allows us to identify a set of antibod-ies that cover the entire space.
We then computed a 2D reduction by mapping the fixed-length AbMAP-B representations of these antibodies to the top two principal components. In all analyses that follow, AbMAP fixed-length embeddings are reduced to 2D space by this specific mapping, enabling us to visualize different datasets (e. g. , in Figures  4a-c ) on the same axes. In most plots, we mark the 1,000 "anchor" antibodies as gray dots in the background, to help contextualize the visualization. We computed AbMAP-B embeddings for 887 LIBRA-seq BCR sequences that were not part of the training set. As shown in Fig. 4a , our model representation was able to differentiate whether BCRs originated from engineered B-cell lines or real human donors. Furthermore, our model was able to discern the two different types of BCRs within the engineered cell lines: VRC01, a CD4 binding-site-directed HIV-1 bNAb (broadly neutralizing antibody), and Fe53, a bNAb recognizing the stem of group-1 influenza hemagglutinins (Fig.
The Landscape of Human Antibody Repertoires We analyzed the human antibody datasets from the Briney et al. 's study of BCR repertoires across multiple individuals. As part of this analysis, we ask two key questions: • Is the set of BCRs uniformly distributed over the embedding space, or does the distribution display "hotspots" of clustering? In case the latter is true, we also wondered if antibody drug candidates that have successfully passed pre-clinical evaluations were more likely to cluster in these hotspots. • While an extraordinary sequence diversity of BCRs has been reported across individual repertoires  (19) , multiple analyses have suggested that these diverse repertoires converge to similar structure/function  (35, 36, 37). We wondered if such similarities across individuals would be more evident in our representation space than in the raw sequence space.
We acquired 100,000 randomly-chosen BCR sequences from 9 individuals each, filtering out identical sequences from an individual. Because of assay limitations in the original study, many of the sequences were truncated on one or both ends. This would be a challenge for language models that need to consider the full antibody; however, our focus on just the CDRs enabled us to recover embeddings for the vast majority (n = 885, 422) of these BCRs. We applied AbMAP-B on these sequences and visualized their 2D reduction as described above. We found the distribution to be highly clustered, with a kernel density estimator of the distribution being unimodal (Fig. 4b ). Notably, the human cell lines from the LIBRA-seq data overlap very well with the Briney dataset. Interestingly, while the VRC01 cell line falls within the high-occupancy region, the Fe53 cell line is well out of it.