dataesr/scientific-paragraphs-categorization huggingface dataset:
05, first selecting only positive markers (higher protein levels in cluster i ), and next selecting only negative markers (lower protein levels in cluster i ). The strongest positive markers were selected by filtering out all markers which did not satisfy the criteria that 1) the highest expression of the protein was in the given NMF cluster and 2) the step ratio, defined as the NPX difference between the given NMF cluster and the second-highest expressing cluster, was at least 0. 1. A similar method filtering out markers that did not have the lowest expression in the given NMF cluster and markers with a step ratio for the second lowest cluster of at least 0. 1 was used for negative markers. Heatmaps of the protein markers per cluster were generated with the pheatmap package in R, with genes ordered according to p value.
Ligand-receptor interaction analysis A curated ligand-receptor pair database from FANTOM5 was used to search for interactions between neutrophil receptors and plasma ligands on either the basis of Severity Max or neutrophil NMF cluster  86. The database was filtered on ligand-receptor interactions identified as "literature-supported" or "putative", and was further filtered for receptors with non-zero expression in granulocytes according to the Human Protein Atlas 87. To identify neutrophil receptors associated with specific NMF clusters, differential expression was performed using DESeq2 for NMF cluster i versus all other clusters irrespective of Day. Only positive gene markers were kept with p adj < 0. 05. Differentially expressed receptors which were not unique to a single NMF cluster were excluded.
Similarly, differential expression of plasma proteins was performed using lm in R comparing NMF cluster i vs all other clusters (including Neu-Lo), and proteins were kept with p adj < 0. 05. Thus a list of potential interactions was generated using the database. To determine whether the neutrophil receptors and plasma proteins were differentially expressed within the same sample rather than the aggregated group, the percentage of samples within a given NMF cluster on a specific day which had higher than mean expression across all COVID-19+ samples of both neutrophil receptor and plasma protein were calculated. In Figures  7A, 7B  and Figure  S21 , ligands matching with multiple receptors were then colored according to the interaction which had the highest percentage of above-mean expression, and secondary interactions were indicated with reduced line width.
ACKNOWLEDGMENTS We acknowledge support from the  Ragon Institute of MGH, MIT and Harvard , the  Massachusetts Consortium on Pathogen Readiness (MassCPR) , the  NIH  ( 3R37AI080289-11S1 ,  R01AI146785 ,  U19AI42790-01 ,  U19AI135995-02 ,  U19AI42790-01 ,  1U01CA260476 -01 ,  CIVIC75N93019C00052 ,  T32 GM007592 ), the  American Lung Association , and the  MGH Executive Committee on Research. A. -C. V. acknowledges funding support from the  COVID-19 Clinical Trials Pilot  grant from the  Executive Committee on Research at MGH , a COVID-19  Chan Zuckerberg Initiative  grant ( 2020-216954 ), and funds from the  Manton Foundation  and the  Klarman Family Foundation. We thank  Arnav Mehta  for his help with the proteomics data and for the code which we adapted from Filbin et al. 2021. We also thank  Mikael Pittet  for his expertise in neutrophil biology.
Software and algorithms IntelliCyt ForeCyt (v8. 1) Sartorius https://intellicyt. co m/products/softwar e/ FlowJo (v10. 7. 1) FlowJo, LLC https://www. flowjo. com/solutions/flowj o Prism 9. 2. 0 (283) GraphPad https://www. graphp ad. com/scientific-s oftware/prism/ GTEx-TOPMed RNA-Seq pipeline Broad Institute https://github. com/ broadinstitute/gtexpipeline/ STAR v2. 5. 3a Dobin et al. 2013 https://github. com/ alexdobin/STAR/rel eases/tag/2. 5. 3a RSEM v1. 3. 0 Li et al. 2011 https://github. com/ deweylab/RSEM/re leases/tag/v1. 3. 0 RNA-SeQC 2 Graubert et al. 2021 https://github. com/ getzlab/rnaseqc.
2020 EGA accession -CaptureSelect IgA Affinity Matrix ThermoScientific CAT# 1942880005 EGAS0000100457 ○ Comparison of differential expression and plasma proteomic data 1 Protein A/G Agarose ThermoFisher CAT# 20424 ○ Ligand-receptor interaction analysis REAGENT or RESOURCE SOURCE Mouse Anti-Human IgG1-Fc PE Southern Biotech GSE76293 RRID:AB_2796628 University of Texas ARDS Neutrophil RNA-Seq Juss et al. 2016 GEO accession CAT# 9054-09 SARS-CoV-1, MERS spike proteins (S) Jason McLellan, N/A GSE149443 Antibodies hCoV-HKU1 spike protein (S) Immune Tech GSE145926 and CAT#: IT-002-025p BAL scRNA-Seq data Bost et al. 2020 GEO accession -IDENTIFIER hCoV-OC43 spike protein (S) Sino Biological 40607-V08H1 1492/ CAT#: gle_cell/study/SCP KEY RESOURCES TABLE hCoV-OC43 RBD Aaron Schmidt, Ragon Institute N/A Sepsis Neutrophil scRNA-seq data Reyes et al.
g. ,  Liu et al. , 2016; Kurth et al. , 2018; Lagerquist et al. , 2019; Chattopadhyay et al. , 2020) , weather forecasting, including hailstorm detection (e. g. ,  Grover et al. , 2016; Shi et al. , 2015; Gagne et al. , 2019) , deriving model parameterizations (e. g. ,  Jiang et al. , 2018) , and beyond. In certain applications, the targeted outcomes are the same features as the input but at a different time, e. g. , a given weather feature(s) such as temperature or pressure at a given level. The forecasting can thus proceed by using pattern-topattern correlation from a sequential training dataset with spatial information preserving full CNNs such as U-net  (Ronneberger et al. , 2015; Weyn et al. , 2020). However, this is certainly not the case for the applications where the environmental conditions associated with the targeted outcome are yet known.
Network architecture The convolutional neural network used in this study, the HazeNet  (Wang, 2020)  The network has been trained in a standard supervised learning procedure for classification, where the network takes input features to produce classification output that are then compared with known results or labels based on observations. The coefficients of the network are thereafter optimized in order to minimize the error between the prediction and the observation or label. The loss function used in optimization is cross-entropy (e. g. ,  Goodfellow et al. , 2017). Such a procedure is repeated until the performance of the network can no longer be improved. In practice, the training usually lasts about 2000 epochs (each epoch is a training cycle that uses up the entire training dataset).
Training data and methodology The labels for the training are derived using the observed daily surface visibility (hereafter referred to as "VIS") obtained from the Global Surface Summary Of the Day (GSOD) dataset consisting of daily observations of meteorological conditions from tens of thousands of airports around the globe  (Smith et al. , 2011). In the cases of Beijing and Shanghai, data are from observations in the corresponding airports of these two cities during the time from 1979 to 2019, containing 14 975 samples. For simplicity, the discussions will be mainly about the two-class training, where events with VIS ≤ the long-term mean value of the 25th percentile or p25 of VIS (6. 27 km in Beijing, 5. 95 km in Shanghai; Fig. 2b ; see also Fig. S1  in the Supplement) are defined as class 1 or severe haze and events are otherwise classified as class 0 or non-haze cases.
All input features have been normalized into a range of [-1, +1] (Fig. S2  in the Supplement). Before the training, the entire samples of labels alongside corresponding input features were randomly shuffled first then split in the following way: two-thirds of the samples went to the training set, and one-third of the samples went to the validation set, each is duly used for its designated purpose throughout the entire training process without switching. The above procedure treats each event as an independent sample. For the convenience of being able to compare performance or restart training based on a saved machine, a saved training dataset and a holdout validation dataset that has never been  The number of samples used in training HazeNet is rather limited compared to deep-learning standards.
Batch normalization  (Ioffe and Szegedy, 2015)  is an algorithm to renormalize the input distribution at certain step (e. g. , for each mini batch) to eliminate the shift of such distributions during optimization. The above approaches have effectively reduced the overfitting while overcoming the data imbalance issue, making the long training of a deep CNN possible  (Wang, 2020). Entire training events have been conducted using a NVIDIA Tesla V100-SXM2 GPU cluster, costing 25 and 17 s per epoch for the machines in Beijing and Shanghai, respectively.
Also note that, unless otherwise indicated, results shown in this Section are obtained using 16 features. In order to train a stable machine, training events with 2000 epochs or longer have been conducted instead of using certain commonly adopted skills such as early stop. As a result, the validation performance metrics of the trained machines all appeared to be stabilized by approaching the end of training (Fig. 4 ). These scores were consistent with the results of ensemble training with the same configuration but different randomly selected training and validation datasets and were also comparable among training events with different configurations. Overfitting has been clearly overcome due to such a long training procedure and the adoption of class weight and batch normalization. In a two-class classification (haze versus non-haze), trained deep HazeNet can always reach an almost perfect training accuracy (e.
Most importantly, training events using only the top nine most sensitive features have produced a performance equivalent to or even better than the same training event with 16 features (Fig. 4e ). With a reduced number of features, many further analyses can be conducted with lower workload that produce results that are easily understood. 4 Identifying and categorizing the typical regional meteorological and hydrological regimes associated with haze events A major purpose of this study is to identify the meteorological and hydrological conditions favoring the occurrence of severe hazes in the targeted cities. When using a dataset with a large number of samples, this type of analyses could be better accomplished by applying, e. g. , cluster analysis (e. g. ,  Steinhaus, 1957) , a standard unsupervised ML algorithm that groups data samples into various clusters in such a way that samples in the same cluster are more similar to each other than to those in other clusters.
Aside from this use, the trained machines can be used in many other types of machine learning and deep-learning applications, as has been partially demonstrated here. ii. The second step is to perform clustering using the new dataset with reduced size resulting from the previous step. For this purpose, it should be conducted separately for different types of samples or events, e. g. , categorizing all the samples for haze into characteristic groups with similarities and identical features, and the same for all samples of non-haze events. In order to provide additional information to further the understanding of the network's performance, the clustering was actually conducted for different prediction outcomes, by taking corresponding samples from the new dataset. In this case, TP plus FN would lead to haze events, and TN plus FP would lead to nonhaze events.
This provided a base to identify the smallest possible number of cluster centers with reasonably low inertia in actual cluster analysis. This has actually been decided by using square root of the inertia weighted by the number of samples to put the varying number of samples across various outcomes in consideration. An optimized number of clusters was chosen with a weighted inertia lower than 1/e of that of the single-cluster case. For TN, due to the large sample number, this criterion was set to be half of 1/e. As a result, the optimized number of clusters for TP, FN, FP, and TN outcomes is 4, 3, 4, and 15 for Beijing and 4, 3 3, and 10 for Shanghai, respectively. iii. The members of each cluster derived from step (ii) were recorded by the actual sample indices with the date included. Therefore, actual samples of input data grouped into various clusters can be conveniently identified with corresponding feature maps retrieved, either in a normalized or unnormalized format (i.
e. , in original quan-tity as in reanalysis dataset), and used for further analyses. In practice, cluster-averaged maps for various features were performed beforehand. Code and data availability. The Python script for network architecture, training, and validation is rather straightforward and simple, basically consisting of directly adopted function calls from Keras interface library (https://github. com/keras-team/keras, last access: 2 September 2021) with a TensorFlow-GPU (https://www. tensorflow. org, last access: 2 September 2021) as the backend or from scikit-learn library (https://scikit-learn. org/, last access: 2 September 2021). All the data used here for the analyses are publicly available: GSOD data from https://www. ncei. noaa. gov/access/ metadata/landing-page/bin/iso?id=gov.
noaa. ncdc:C00516, last access: 2 September 2021; ERA5 reanalysis data from https://climate. copernicus. eu/climate-reanalysis, last access: 2 September 2021. Supplement. The supplement related to this article is available online at: https://doi. org/10. 5194/acp-21-13149-2021-supplement. Competing interests. The author declares that there are no conflicts of interest. Disclaimer. Publisher's note: Copernicus Publications remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Acknowledgements. This study is supported by  L'Agence National de la Recherche (ANR) of France  under " Programme d'Investissements d'Avenir"  ( ANR-18-MPGA-003 EUROACE ). The author thanks the  European Centre for Medium-range Weather Forecasts  for making the ERA5 data publicly available under a license generated (and service offered) by the  Copernicus Climate Change Service  and the  National Center for Environmental Information of the US NOAA  for making the GSOD data available. All of the related computations have been accomplished using the GPU clusters of  French Grand Equipment National de Calcul Intensif (GENCI)  (project  101056 ) and the  CNRS Mesocenter of Computing of CALMIP  (project  p18025 ). Constructive comments and suggestions from two anonymous reviewers led to the improvement of the manuscript.
, 2017). Second, the SCB has one of the most spatially comprehensive and longest-running coastal observational systems in the world. Several physical and biogeochemical variables are sampled regularly and extensively, creating an ideal setting for model-data comparisons. Third, the SCB is home to one of the most densely populated coastal regions in North America, where the discharges of primary or secondary treated wastewater from a population of 20 million people are released to the coastal zone via ocean outfalls, along with the urban and agricultural runoff from 75 rivers. These nutrient sources rival natural upwelling in magnitude  (Howard et al. , 2014) , roughly doubling available nitrogen to nearshore coastal waters. Intensifying ocean acidification, oxygen loss, and harmful algal blooms have motivated California policy makers to consider reducing anthropogenic nutrients as a climate change mitigation strategy (Ocean Protection  Council, 2018) , but wastewater treatment plant upgrades and methods to increase control or reduce nonpoint sources would cost billions.
myocean. eu) for the physics, and with reconstruction of biogeochemical fields using world ocean database. We used climatological fields for organic material and relationships with density for nutrients. Full description of the boundary conditions and initialization of the parent configuration at 12-km can be found in  Deutsch et al. (2021). The 4-km configuration is run for the period 1995-2017, after a spin-up of 2 years. A 1-km simulation is initialized and forced from the 4-km model, including initial conditions and open boundary conditions, starting in October 1996 and ending in December 2007. The 0. 3-km simulation is initialized and forced at its boundaries by the 1-km simulation starting from January 1997 and ending in December 2000. The bathymetry used in this configuration comes from the Southern California Coastal Oceanic Observation System (SCCOOS) 3 Arc-Second Coastal Relief Model Development (90-m horizontal resolution).
Terrestrial and Atmospheric Forcing of Freshwater, Nutrients, and Carbon Model simulations were forced with a monthly time series of spatially explicit inputs (Figure  3 , upper), including freshwater flow, nitrogen, phosphorus, silica, iron, and organic carbon representing natural and anthropogenic sources  (Sutula et al. , 2021b). These data include POTW ocean outfalls and riverine discharges  and spatially explicit modeled estimates of atmospheric deposition. POTW effluent data were compiled from permit monitoring databases and communication with sanitary agencies. Monthly time series of surface water runoff from 75 rivers are derived from model simulations and monitoring data  (Sutula et al. , 2021b). Direct atmospheric deposition is derived from the Community Multiscale Air Quality (CMAQ) model  (Byun et al.
Ship-Based Ocean Monitoring The SCB is home to a suite of long-running monitoring programs that make it one of the best observed coastal ecosystems in the world (3, lower). Among them, the California Cooperative Oceanic Fisheries Investigations (CalCOFI) program  (McClatchie, 2016) , initiated in the 1950s, samples the SCB quarterly each year, collecting hydrographic and biogeochemical measurements in coordination with the SCCOOS. These observations are augmented nearshore by quarterly surveys of nearshore water column and benthic parameters conducted collaboratively since 1990 by POTW agencies as a part of their regulatory monitoring requirements  (Booth et al. , 2014; Howard et al. , 2014; McLaughlin et al. , 2018; Nezlin et al. , 2018). These programs provide good temporal and geographical coverage of both the offshore (CalCOFI) and nearshore (POTW) areas, coinciding with the model period, and include publicly available water quality data for targeted sites measured quarterly.
We validated model output against observed temperature, dissolved oxygen, nitrate, ammonium, chlorophyll, carbon system parameters (pH and aragonite saturation state), primary production, and nitrification. In situ measurements have inherent uncertainty, due to a combination of measurement sensitivity and sampling frequency and intensity, making them an imperfect "truth" with which to compare to model output. However, this uncertainty is not the same for all parameters. Both temperature and dissolved oxygen are collected using high-resolution probes, though the two programs used in this study incorporate slightly different calibration protocols for dissolved oxygen, lending greater confidence to data-model comparisons for these data sets. Chlorophyll is measured on discrete bottle samples in the CalCOFI program, a high-quality measurement, but inferred from in situ fluorescence measurements in the POTW monitoring program, adding uncertainty to these measurements.
Nitrate and ammonium concentrations are measured on discrete bottle samples for both programs, but the detection limits are more sensitive in the CalCOFI program. Furthermore, nutrients are not measured with the same sampling density in POTW monitoring programs as sensor data. Similarly, primary production is measured at a subset of locations in the CalCOFI program and as a short-term special study in Southern California Bight Regional Marine Monitoring Program (Bight Program). Details on measurements and sample collection protocols for the CalCOFI program can be found on their website (https://calcofi. org; McClatchie, 2016) and for the POTW monitoring programs in  Howard et al. (2014). We also use selected nutrient observations from the Santa Monica Bay Observatory (SMBO) mooring located in the Santa Monica Bay  (Leinweber et al.
Remote Sensing Observations Satellite ocean color measurements for chlorophyll were used to characterize horizontal gradients at finer scales and higher density than possible with the ship-based monitoring. We use monthly averaged surface chlorophyll concentration from the period 1997 to 2000 derived from the SeaWiFS sensor at 4-km spatial resolution. Large gaps in the data set can occur because of dense cloud cover that occurs in late spring and early summer. The products of the Vertically Generalized Production Model (VGPM) net primary production algorithm  (Behrenfeld & Falkowski, 1997)  were also considered for this validation. Despite limitations, satellite data provide a valuable representation of the spatial distribution of chlorophyll, temperature, and primary production at seasonal scales over the region.
Spatial Patterns and Seasonality of Nitrate Nitrate observations are only broadly available in the offshore CalCOFI data set, so only large-scale regional patterns in nitrate concentration can be validated. There is a clear seasonality of nitrate, where surface concentrations are higher in spring and summer and decrease in fall and winter (Figure  7 ). The model reproduces the average seasonal patterns observed in the in situ nitrate data across multiple regions. The model also captures along-shore variability in coastal nitrate concentrations, reproducing values greater than 25 mmol N m -3 off Santa Barbara, 20 mmol N m -3 off Los Angeles, and 15 mmol N m -3 off San Diego. The model also reproduces observed patterns in the depth of the nitracline  (Mantyla et al. , 2008; Nezlin et al. , 2018) , which tends to follow sloping density surfaces in the region.
The scatter plots in Figures  14e  and 14f  show a comparison of the simulated primary production between the in situ CalCOFI data and that derived from remote sensing (empirically adjusting the Behrenfeld-Falkowski VGPM). The model shows a correlation coefficient of about 0. 6 with CalCOFI, similarly to that reported by  Kahru et al. (2009)  when comparing the VGPM product with CalCOFI. The model shows a stronger correlation with VGPM data, with a correlation coefficient of the order of 0. 8. Finally, while slightly outside our model domain and simulation period, the modeled phytoplankton growth and zooplankton grazing rates were within the same order of magnitude as the measured rates from the California Current Long Term Ecological Research (CC-LTER) project (see  Landry et al. , 2009;.
Vertical Gradients and Seasonal Variability of Carbon System Parameters Together with pH, the saturation state of aragonite (Ω Ar ) is often used as a metric to identify the potential impact of ocean acidification on marine calcifiers, because it is a measure of the availability of carbonate ions for calcium carbonate precipitation  (Bednarsek et al. , 2019). Ω Ar shows similar vertical variability as dissolved oxygen  (Alin et al. , 2012; Juranek et al. , 2009). Similar to oxygen loss, reduction in pH and Ω Ar in the upper layers can be caused by coastal upwelling and local physical processes  (Feely et al. , 2018) ; over longer time scales it also reflects oceanic uptake of anthropogenic carbon from the atmosphere. We utilize sensor pH data sets to evaluate vertical profiles in the carbonate system.
Recently, the CalCOFI program has incorporated Ω Ar into its sampling design. Although the data do not line up with the model period, they are useful for evaluating seasonal variability in the model. Generally, the model reproduces seasonal and vertical variability in Ω Ar , with higher saturation states in the summer and fall, when waters are generally more stratified, and lower values in winter and spring, when upwelling brings undersaturated waters closer to the surface. Ω Ar is also much lower and more highly variable at depth. These patterns are consistent with observations throughout the SCB  (McLaughlin et al. , 2018).