dataesr/scientific-paragraphs-categorization huggingface dataset:
What do the best-performing models have in common? There were no models that performed best for all SLCF species and for all regions, highlighting the fact that it is difficult for any one model to bring together numerous complex processes and get results comparable to observations for all SLCFs. This would involve simulating aerosols and chemistry together with the right transport processes, meteorology, and clouds, which is difficult, especially for a remote region like the Arctic where parameterizations might have been built on datasets that are not always applicable there. In addition, studies like that of  Tsigaridis et al. (2014)  have shown that there was no clear change in model skill (in that case for OA) with increasing model complexity. However, several models such as CanAM5-PAM, DEHM, NorESM, and MATCH have better representation of the ver-tical distribution of BC.
The summertime evaluations of surface O 3 , BC, and OA all imply that models overestimate the amount of these pollutants coming from wildfires in the western Arctic. This could be due to uncertainties in wildfire emissions, fire plume transport, or, in the case of O 3 and secondary OA, the plume chemistry. It is also likely that wet deposition of fire pollutants is underestimated if fire aerosol size is too small and due to climate models lacking pyrocumulus clouds and precipitation. The model overestimations of SLCFs in the summer could be due to a combination of all of these uncertainties. Our analysis from ACE-FTS, TES, and the aircraft datasets show that CH 4 , O 3 , and BC model biases all imply that the modeled tropopause height is likely too low. Tropospheric species like CH 4 and BC should drop rapidly above the tropopause, but model biases increase sharply at that point.
(2008). The simulation analyzed here used a "specified dynamics" setup, wherein the model horizontal winds and temperature are nudged towards a meteorological reanalysis dataset that represents the observed historical evolution of the atmosphere. In this way the day-to-day variability of the model meteorology is much more closely aligned with the historical evolution of the atmosphere than would be possible in a free-running model. Here CMAM was nudged to 6-hourly fields from the ERA-Interim reanalysis  (Dee et al. , 2011)  on all model levels below 1 hPa and with a relaxation time constant of 24 h.
33 as used here, have been described in  Simpson et al. (2019)  and references cited therein. The main revisions were made to the parameterizations of coarse NO - 3 formation on sea salt and dust aerosols, N 2 O 5 hydrolysis on aerosols, and additional gasaerosol loss processes for O 3 , HNO - 3 , and HO 2. The EMEP model, including a user guide, is publicly available as opensource code at https://github. com/metno/emep-ctm (last access: 14 April 2022). EMEP-modeled PM 2. 5 and PM 10 include primary and secondary aerosols, both anthropogenic and natural. Secondary aerosol consists of inorganic sulfate, nitrate, ammonium, and SOA; the latter is formed from both anthropogenic and biogenic emissions using the VBS scheme detailed in  Bergström et al. (2012)  and  Simpson et al. (2012). The model also calculates sea salt aerosols and windblown dust particles from soil erosion.
wcrp-climate. org/ modelling-wgcm-mip-catalogue/modelling-wgcm-mips-2/ 240-modelling-wgcm-catalogue-amip, last access: 14 April 2022). The horizontal wind fields were nudged toward the 6-hourly Japanese 55-year Reanalysis (JRA55) data  (Kobayashi et al. , 2015)  (https://jra. kishou. go. jp/JRA-55/index_en. html, last access: 14 April 2022) in the simulation. We used the monthly anthropogenic emissions from the ECLIPSE V6B emission dataset and the monthly biomass burning emissions from the CMIP6 in the simulations. Major volcanic aerosols are given by the stratospheric aerosol dataset used in the CMIP6 experiments  (Thomason et al. , 2018). A second simulation with volcanic SO 2 emission including Holuhraun eruption was also performed for 2014-2015.
SS and LH provided Alert datasets; VV and SB provided Gruvebadet datasets; OP provided Russian cruise ship measurements; FT and YK provided the Japanese ship measurements; HS and AM provided Villum Research Station datasets; JS provided aerosol datasets and expertise. TWG and SMD wrote and developed the trace gas modelmeasurement comparison scripts. last access: 14 April 2022), and the WMO GAW network are acknowledged for Barrow and Zeppelin observational datasets. The ACE-FTS is a Canadian-led mission mainly supported by the Canadian Space Agency (CSA). IMPROVE is a collaborative association of state, tribal, and federal agencies, as well as international partners. The US Environmental Protection Agency is the primary funding source, with contracting and research support from the National Park Service.
20-55-12001 and according to the development program of the Interdisciplinary Scientific and Educational School of M. V. Lomonosov Moscow State University "Future Planet and Global Environmental Change". Development of the methodology for aethalometric data treatment was supported by RSF project no. 19-77-30004. The BC observations on R/V Mirai were supported by the Ministry of Education, Culture, Sports, Science and Technology (MEXT), Japan (Arctic Challenge for Sustainability (ArCS) project). Contributions by SMHI were funded by the Swedish Environmental Protection Agency under contract NV-03174-20 and the Swedish Climate and Clean Air Research program (SCAC) as well as partly by the Swedish National Space Board (NORD-SLCP, grant agreement ID: 94/16) and the EU Horizon 2020 project Integrated Arctic Observing System (INTAROS, grant agreement ID: 727890).
Naga Oshima was supported by the Japan Society for the Promotion of Science KAKENHI (grant nos. JP18H03363, JP18H05292, and JP21H03582), the Environment Research and Technology Development Fund (grant nos. JPMEERF20202003 and JPMEERF20205001) of the Environmental Restoration and Conservation Agency of Japan, the Arctic Challenge for Sustainability II (ArCS II) under program grant no. JPMXD1420318865, and a grant for the Global Environmental Research Coordination System from the Ministry of the Environment, Japan (MLIT1753). The research with GISS-E2. 1 has been supported by the Aarhus University Interdisciplinary Centre for Climate Change (iClimate) OH fund (no. 2020-0162731), the FREYA project funded by the Nordic Council of Ministers (grant agreement nos. MST-227-00036 and MFVM-2019-13476), and the EVAM-SLCF funded by the Danish Environmental Agency (grant agreement no.
C. H. Whaley et al. : AMAP SLCF model evaluation Code and data availability. The models' output files in NetCDF format from the simulations used in this project can be found here: http://crd-data-donnees-rdc. ec. gc. ca/CCCMA/products/AMAP/ (Canadian Centre for Climate Modelling and analysis, 2022a). Some of the models' code are available online at the following locations. CanAM5-PAM: https://gitlab. com/cccma (Canadian Centre for Climate Modelling and analysis, 2022b). CESM2: https://www. cesm. ucar. edu/models/cesm2/ (UCAR, 2022a). ECHAM-SALSA: The codes used for the ECHAM-SALSA simulations are available from the ECHAM-HAMMOZ repository under https://redmine. hammoz. ethz. ch/projects/hammoz/repository/1/ show/echam6-hammoz/branches/fmi/AMAP/AMAP_evaluation (Center for Climate Systems Modeling -C2SM at ETH Zurich, 2022), after obtaining the HAMMOZ license.
FLEXPART: https://www. flexpart. eu (GNU General Public License, 2022). GEOS-Chem: http://wiki. seas. harvard. edu/geos-chem/index. php/GEOS-Chem_12#12. 3. 2 (Harvard University, 2022). GISS-E2. 1: https://www. giss. nasa. gov/tools/modelE/ (NASA, 2022a). NorESM: https://github. com/NorESMhub/NorESM (NorESM Climate Modeling Consortium, 2022). Oslo CTM: https://github. com/NordicESMhub/OsloCTM3 (Section for Meteorology and Oceanography, 2022). The other models' code may be available upon request. The model evaluation programs can be found on GitLab here: https://gitlab. com/cynwhaley/amap-slcf-model-evaluation (Whaley.
Surface in situ SO2- 4  measurements in the major observing networks typically use ion chromatography methods, which have an approximately 3 % uncertainty range (Solomon et al. , 2014). However, SO 2- 4 measurements have been shown to have up to 20 % analytical uncertainty(AMAP,  2022). SO 2- 4 datasets were obtained from IMPROVE, EMEP, and CABM networks, often via the EBAS database. SO2-  4.
In the context of fMRI data, the All-Resolutions Inference framework introduced in [25] provides post hoc estimates of the proportion of activated voxels. However, this method relies on parametric threshold families, which results in conservative inference. In this paper, we leverage randomization methods to adapt to data characteristics and obtain tighter false discovery control. We obtain Notip, for Non-parametric True Discovery Proportion control: a powerful, non-parametric method that yields statistically valid guarantees on the proportion of activated voxels in data-derived clusters. Numerical experiments demonstrate substantial gains in number of detections compared with state-of-the-art methods on 36 fMRI datasets. The conditions under which the proposed method brings benefits are also discussed.
We perform extensive experiments on dozens of fMRI datasets to compare the number of detections obtained by this approach with that of existing methods. The paper is organized as follows. In Section 2, existing methods for the post hoc control of FDP are introduced via the notion of Joint Error Rate (JER) proposed by  [7]. Our main contribution is the Notip method presented in Section 3: a nonparametric data-driven approach that relies on the JER framework to obtain sharper post hoc FDP control. Numerical experiments and results on fMRI data reported in Sections 4 and 5 show that substantial gains in the number of detections are obtained from the proposed method, while controlling the FDP of the detected regions at a fixed level. Finally, we discuss the benefits of our proposed methodology, and outline some possible limitations.
Several other parametric templates are considered in  [1] , but the authors report that none of these attempts outperformed the Simes template. An ideal template should approximately reproduce the shape of randomized pvalues curves computed from real data. Therefore, we propose to learn a template directly from the data. A related idea has been explored in  [20]. However, since the method proposed in that paper does not distinguish between the learning and calibration steps, it suffers from circularity biases, as noted by  [7, Remark 5. 3]. Indeed, in the JER framework, the template has to be fixed a priori. In order to address this issue, we propose to learn a template from an fMRI dataset that is independent from the datasets on which inference is performed. First, we compute B randomized p-value curves on training data using Algorithm 1 and extract quantile curves t b = (t b k ) k for b = 1.
B, as shown in the left panel of Fig. 3. These quantile curves are then viewed as a set of B sorted threshold families (middle panel), which is called a learned template. Note that it is indeed a template in the sense of  [7] , that has been discretized over a set of B values. Middle panel: the resulting quantile curves are used as a template (the so-called learned template). Right panel: calibration is performed on inference data using the learned template. Notice that learned templates do not have a parametric shape (contrary e. g. to the Simes template), but follow the shape of sorted null p-values. After obtaining a learned template, calibration is performed on the inference data (i. e. any inference contrast) as would be done with a parametric template. This is shown in the right panel of Figure  3   The calibration process depends on the parameter k max , whose choice induces the following trade-off.
FMRI data To investigate the potential gain in number of detections yielded by using data-driven templates, we performed experiments on an fMRI dataset, collection 1952  [29]  of the Neurovault database (http://neurovault. org/collections/ 1952). This dataset is an aggregation of 20 different fMRI studies, consisting of statistical maps obtained at the individual level for a large set of contrasts. These images have been preprocessed using the procedure described in  [29]. In particular, they have been spatially normalized to MNI space using SPM12 software, and resampled to 3mm isotropic resolution. In the present case, the inference question concerns one-sample tests in group analyses, i. e. identifying what brain regions show a significant increase of activity for the contrast of interest, as opposed to the baseline, across participants.
Synthetic data For some of the experiments described below, we have generated simulated data using the pyrft package: https: //github. com/sjdavenport/pyrft. This package allows generates smooth noisy random fields that resemble fMRI data. In this controlled setup, the ground truth is known. An example of such simulated data can be found in Section 8. 7. The simulation setting is the following, with π 0 the proportion of null voxels: α = 0. 05, π 0 = 0. 9, FW HM = 4mm, n train = 100, n in f er = 50, q = 0. 1, B train = B in f er = 1000.
Code to reproduce the experiments Data manipulation is mostly performed through Nilearn v0. 9. 0, nibabel v. 3. 1. 1. The proposed statistical methods are implemented in the sanssouci package: https://github. com/pneuvial/sanssouci. python. The experiments presented in this section can be reproduced using the code at: https://github. com/alexblnn/Notip. This repository contains a script per experiment. The analysis we performed on this data can be divided into 6 main experiments that are detailed in the rest of this section.
Using Notip on a single dataset When learning a template on separate data is inconvenient, or to avoid the computational cost of learning the template, a natural idea is to use Notip on a single dataset. In such a setting, circularity biases may appear as in  [20]. The workaround that we propose to retain valid FDP control is to perform two independent rounds of randomization -one for training and one for inference. While this approach is formally not covered by the theoretical framework of  [7] , we have performed experiments to assess its FDP control and power on the simulated data described in Section 4. 1. 2. After learning the template on a single contrast pair (see section 4), we perform inference on all 36 pairs. For each contrast pair, we compute the largest possible region that satisfies FDP≤ q for q ∈ {0.
Notice that the number of detections is markedly higher (+ 77 %) using the learned template compared to the calibrated Simes template. We have also performed the same experiment on simulated data. In this setting, we can report the actual TPR of the methods instead of region sizes. The empirical FDP for these simulations are reported in Figure  16. Figure  6 : TPR comparison for a FDP budget q = 0. 1 at risk level α = 0. 05. We run 100 simulations and report the TPR. Notice that Notip offers substantial gains in TPR compared to both ARI (100 % on average) and calibrated Simes (50 % on average). Figure  6  illustrates the TPR gains achieved using Notip on simulated data compared to both ARI to both ARI (100 % on average) and calibrated Simes (50 % on average). Overall, simulations support the fact that Notip offers substantial performance gains compared to both ARI and calibrated Simes.
This is illustrated in Figure  9 : gains in number of detections remain consistent-yet more variable for smaller sample sizes -across datasets with different number of subjects. As noted in  [8] , high variance is unavoidable when inferring on small datasets (e. g. n in f er ≤ 25). For a single dataset comprising 17 subjects, the learned template performs substantially worse than calibrated Simes. Comparison of the number of detections between learned template and calibrated Simes for many contrast pairs with a different numbers of subjects. The gains in number of detections remain consistent across datasets with different number of subjects. However, for a single dataset comprising 17 subjects, the learned template performs substantially worse than calibrated Simes. For a fixed contrast pair "look negative cue" vs "look negative rating" and 36 different training contrast pairs, we compute the largest possible regions that ensure FDP ≤ 0.
Influence of data smoothness We have seen in Figure  10  that Notip is robust to mismatches of training and inference data across different dimensions (number of subjects, quantity of signal. ). We now examine the robustness of Notip with regards to a mismatch of the smoothing parameter between training and inference data. After learning the template on a single contrast pair (see Section 4) with smoothing full width at half maximum (FWHM) 4mm, we perform inference on all 36 pairs smoothed with FWHM 8mm. For each contrast pair, we compute the largest possible region that satisfies FDP control at level 0. 1 with risk level = 0. 05. The learned template still performs marginally better than calibrated Simes in this case, but gains are substantially lower in this regime. Figure  11  shows that the smoothing parameter of the training data and the inference data should be matched for optimal performance.
Using Notip on a single dataset To assess whether using Notip with the same dataset for training and inference controls the FDP, and whether it yields performance gains compared to ARI and calibrated Simes, we performed 1000 simulations. For each of these runs, we report the empirical FDP and TPR of all three methods. Empirical FDP for requested FDP control q = 0. 1 at level =0. 05.
Requested FDP control Figure  12 : False Discovery Proportion achieved for a FDP budget q = 0. 1 with risk level α = 0. 05 using Notip on a single dataset. We run 1000 simulations and report the empirical FDP for each one. Notice that Notip (single dataset) controls the FDP at level α = 0. 05 since FDP control is violated for 5 runs, i. e. 0. 5% < 5% of all simuations. As expected, ARI and calibrated Simes also control the FDP. Notice that as seen in Figure  12 , Notip (single dataset) indeed controls the FDP, as only 5 points are above the red line -i. e. the FDP was above the budget q = 0. 1 in 0. 5% of experiments (< α = 5%). As expected, ARI and calibrated Simes control the FDP more conservatively. for requested FDP control q = 0. 1 at level =0. 05 Empirical TPR Figure  13 : TPR comparison for an FDP budget q = 0.
1 with risk level α = 0. 05 using Notip on a single dataset. We run 1000 simulations and report the empirical TPR for each one. Notice that Notip (single dataset) offers substantial performance gains compared to both ARI (100 % on average) and calibrated Simes (50 % on average). As seen in Figure  13 , Notip (single dataset) yields sub-stantial performance gains compared to ARI and calibrated Simes: 50% on average compared to calibrated Simes, and 100% on average compared to ARI. These gains are comparable to those obtained using the classical Notip method on simulated data (see Figure  6 ).
We have introduced a data-driven approach to obtain valid post hoc FDP control, thus achieving this goal. Moreover, controlling the FDP is a substantially more informative guarantee than controlling the FDR, its expected value. We show that our procedure yields a higher number of detections than existing methods that offer the same statistical guarantees, namely ARI and calibrated Simes. We could go further by applying a step-down procedure as described in  [7] , but the gains are expected to be marginal  [10]. The gains in detections are maintained across practically all possible training sets, even in cases of poor matching between the training and inference datasets, as seen in Figure  10. training and inference data. In this case, the gain in detections obtained by using the learned template is reduced, albeit still non-negligible (30% compared to ARI and 9% compared to calibrated Simes).
We found that choosing training contrast pairs that contain a large number of subjects and low signal is optimal for performance. This is coherent with intuition since a large number of subjects and minimal signal allow a more stable and accurate estimation of the distribution of p-values under the null. Therefore, when selecting a template, it is useful to rely on a largesample dataset with small signal magnitude. Notip comes with an additional computational cost compared to classical calibration using the Simes template, since we have to learn the template before inference. Generally, this additional cost is acceptable in practice since learning a template on a contrast and inferring on a contrast have the same time complexity. If learning a template ex ante is inconvenient or simply impossible, for instance when users only have a single dataset at hand, we have shown numerically that it is possible (though not formally supported by the theory) to use Notip on a single dataset.
We have achieved the goal of obtaining valid post hoc FDP control -rather than FDR control, or even weaker guarantees on clusters -while maintaining a satisfactory number of detections. This allows users in the brain imaging community to use more reliable inference methods that provide robust guarantees, avoiding circularity biases. The efforts to build such methods appear to us as important goal for the brain imaging field. The Python code used in this paper is available at https://github. com/alexblnn/Notip. This code relies on the sanssouci package available at https://github. com/pneuvial/sanssouci. python.
As a default choice, we observe that most fMRI contrasts studied in the literature lead to less of 5% of the image domain to be declared activate, which amounts to setting |S max | = 0. 05m. Finally, a reasonable choice seems to be k max = 0. 5 * 0. 05m = 0. 025m. In the context of the experiments we described where m ≃ 50, 000, we settle for simplicity on using k max = 0. 02m = 1, 000. This is the default value of k max in the implementation we propose. To illustrate the effect of the choice of k max we display the variation of the number of detections of all three methods on 36 fMRI datasets across 9 different inference settings for varying k max in Figure  15. Except for extremely small or large values of k max Notip is at worst slightly sub-optimal and k max = 1, 000 is a safe default.
FDP control on simulated data In section 4. 2 we report the empirical TPR for experiments on simulated data, for which the ground truth is known. We also compute the FDP for each simulation run to verify that, as expected, Notip indeed controls the FDP. Empirical FDP for requested FDP control q = 0. 1 at level =0. 05 Requested FDP control Figure  16 : False Discovery Proportion achieved for a FDP budget q = 0. 1 with risk level α = 0. 05. We run 100 simulations and report the empirical FDP for each one. All three methods control the FDP, but Notip is less conservative than ARI and Calibrated Simes.
Figure3: Learning a template from training data and using this template for calibration on inference data. Left panel: quantiles of randomized p-value curves are computed on training data. Middle panel: the resulting quantile curves are used as a template (the so-called learned template). Right panel: calibration is performed on inference data using the learned template. Notice that learned templates do not have a parametric shape (contrary e. g. to the Simes template), but follow the shape of sorted null p-values.