dataesr/scientific-paragraphs-categorization huggingface dataset:
Moreover, these embedded features were proved to be capable of producing smoother results, although degradation of the effect of GGO on the smoothing of the initial classification results was seen to some extent. Our framework was validated on two ALS datasets: the ISPRS benchmark dataset and the AHN3 dataset. Here, OA achieved values of 83. 2%, and 91. 2% on the former and latter, respectively. The experimental results indicate that our strategy can provide classification results with high accuracy, especially while providing fine-grained results, even for small objects and pole-like objects such as powerlines and cars. However, our method showed weakness in identifying the boundaries of buildings and trees and classifying certain spectral and geometric similar categories, such as fence_hedge, shrubs, and trees.
NHT ¼ ∬ρc p vθ dx dz; where ρ is seawater density, c p is the specific heat capacity of seawater, v is meridional velocity, and θ is potential temperature and where the double integral is taken over the full area of the transbasin section. We use the RapidMoc package (C. D. Roberts, 2017; C. D. Roberts, Garry, & Jackson, 2013)  to calculate model quantities comparable to these observational values, with input parameters shown in Table  S2  in the supporting information. Note that some of the calculations, such as the Ekman transport, may be sensitive to the exact latitude used, as well as how the model grid aligns with these latitudes. We use Deep Mixed Volume (DMV;  Brodeau & Koenigk, 2016; Koenigk et al. , 2020)  in the Labrador Sea (70-40°W, 45-72°N) and Nordic Seas (20°W to 20°E, 65-82°N) to evaluate the deep mixing processes with implications for dense water formation.
Characteristics at 26. 5°N and Comparison With Observations 4. 2. 1. Mean State The vertical structure of observed and simulated overturning stream functions at 26. 5°N are compared in Figure  2  using RAPID-MOCHA and the hist-1950 simulations over 2004-2014. Previous work has demonstrated that the depth structure of overturning stream functions observed by the RAPID-MOCHA array can be sensitive to the choice of reference level for the geostrophic approximation (C. D. Roberts, Garry, & Jackson, 2013). To account for this potential inconsistency, all simulated profiles are calculated using the python RapidMoc package (C. D. Roberts, 2017) , which implements a RAPID-style calculation that includes the geostrophic approximation for ocean interior transports and a consistent level of no motion.
The HadGEM3-GC31-HH model is among the best models in minimizing the salinity Journal of Advances in Modeling Earth Systems bias (and the AMOC profile bias), which was also seen in the small drift shown in M. J. Roberts et al. (2019) , and perhaps this suggests an important role for explicitly representing mesoscale processes in minimizing this bias  (Griffies et al. , 2015; Small et al. , 2014). The components of the AMOC transport in the upper ocean are shown in Figure  4  for both models and observations. As for the overturning stream functions, this decomposition is performed on models using the RapidMoc python package (C. D. Roberts, 2017)  and follows the RAPID observations methodology  (McCarthy et al. , 2015). The upper AMOC limb is split into contributions from transport in the wind-driven Ekman component (ekman), the upper mid-ocean (umo) component (based on thermal wind and a mass conservation constraint), and the western boundary Florida Current (fc).
Conflict of Interest There are no real or perceived financial conflicts of interests for any author. soton. ac. uk/rapidmoc and https:// mocha. rsmas. miami. edu/mocha/ results/mocha-new/index. html websites. We acknowledge extensive use of the supercomputers at our institutes to complete the simulations and for the CEDA-JASMIN platform to enable coordinated data storage and model analysis. The authors would like to thank all the teams involved in the EU H2020 PRIMAVERA project and iHESP for the hard work in configuring and executing model simulations and making the data available for analysis. They also thank two anonymous reviewers and the Editor for their helpful feedback to improve the manuscript.
Ontology search and ranking are key building blocks to establish and reuse shared conceptualizations of domain knowledge on the Web. However, the effectiveness of proposed ontology ranking models is difficult to compare since these are often evaluated on diverse datasets that are limited by their static nature and scale. In this paper, we first introduce the LOVBench dataset as a benchmark for ontology term ranking. With inferred relevance judgments for more than 7000 queries, LOVBench is large enough to perform a comparison study using learning to rank (LTR) with complex ontology ranking models. Instead of relying on relevance judgments from a few experts, we consider implicit feedback from many actual users collected from the Linked Open Vocabularies (LOV) platform. Our approach further enables continuous updates of the benchmark, capturing the evolution of ontologies' relevance in an ever-changing data community.
Existing datasets for ontology ranking evaluation, such as the state-of-the-art benchmark CBRBench  [7] , are often built based on explicit judgments from human experts. However, the laborious approach to building such benchmarks led to relatively small datasets (in case of CBRBench, only comprising ten queries with 819 relevance judgments). Given that ontology collections typically contain hundreds of ontologies and thousands of terms, benchmarks that only comprise ten queries are not sufficient for the evaluation of ranking models in real-world settings. To overcome this issue and address the lack of ontology ranking comparison studies, we propose the ontology ranking benchmark LOVBench. It comprises both a dataset including a ground truth for ranking evaluations and an empirical comparison of state-ofthe-art ranking models.
The LOVBench dataset contains more than 180,000 inferred relevance judgments for more than 7,000 queries. LOVBench's ground truth relies on implicit, real-world user feedback in the form of queries and clicks that were collected from the Linked Open Vocabularies (LOV) platform 2  [41] , which comes with the following advantages. First, the relevance judgments for ontology terms to a query in LOVBench stem from feedback of many actual users, which is more representative than judgments from a few experts; second, collecting user feedback through search logs does not require manual effort, meaning the benchmark can be continuously updated to capture the evolution of ontologies and their relevance in the Semantic Web. The empirical evaluation is performed based on learning to rank (LTR)  [24].
Compared to combining ranking features manually, LTR allows to learn the optimal combination of the considered features using supervised machine learning techniques and, thus, allows for a fair comparison of complex ranking models. We measure the models' ranking quality using the Normalized Discounted Cumulative Gain (NDCG)  [19]  as evaluation metric and we compare ranking configurations as proposed in DWRank  [8] , AKTiveRank  [2]  and CBRBench  [7]. Our experiments confirm that all considered models outperform a straight-forward Lucene search  [28]. We further show that the consideration of features that are well-suited to the user behavior as well as the consideration of much larger feature sets (given a large enough dataset like LOVBench), can significantly improve the ranking performance.
The insights gained in this study help to understand the effectiveness of ontology ranking models and can serve as guidelines for the design of more efficient ontology ranking tools in terms of ranking quality. In summary, the main contributions of this paper are as follows: (1) We provide insights of real-world user behavior in ontology search of which some dissent dominant assumptions in the literature; (2) we introduce and publish the LOVBench dataset, a largescale ontology ranking benchmark based on actual and timely user feedback that allows for continuous updates and enables comparison studies using learning to rank; (3) we provide an experimental evaluation of ranking effectiveness of three state-of-the-art ontology ranking models (composed of 13 distinct ranking features) and propose feature variations and configurations that outperform them (using in total 33 distinct ranking features); (4) we conclude with recommendations for the design of efficient ontology search tools.
The remainder of this paper is structured as follows. In Section 2, we review previous related work. We present our approach to building an ontology ranking benchmark based on user clicks in Section 3. We evaluate the usefulness of user feedback collected through LOV in Section 4 and describe the details of the LOVBench dataset in Section 5. We report the subsequent empirical evaluation in Section 6 and discuss our findings in Section 7. Finally, we conclude the paper in Section 8.
To the best of our knowledge, CBRBench is the first published dataset with relevance labels for query-term pairs obtained from experts. It has further been used as input for LTR, such as in DWRank  [8]. However, CBR-Bench is limited as follows. First, it only contains a small number of queries and judgments (10 and 819, respectively), meaning that its application in LTR settings is limited by its size. Second, it only compares a few ranking models individually (eight in total), and not a (potentially learnt) combination of these or in configurations as they have been proposed in the literature. Moreover, it mostly compares ontology instead of term ranking features that assign the same score for all classes of the same ontology  [7]. Third, CBRBench is restricted to class ranking, meaning that it does not provide any judgments about properties, which limits its potential application to newly proposed ranking models.
Fourth, the dataset might become outdated as soon as new ontologies arise and relevance of terms changes, and because of its laborious way of obtaining the relevance labels, it cannot be easily updated. Apart from CBRBench, other datasets have been proposed for LTR in the context of ontology ranking. In TermPicker  [38] , a dataset is automatically derived from the Linked Open Data (LOD) cloud, however, relevance judgments only indicate whether a certain triple pattern appears in the LOD cloud or not (binary relevance) and it does not contain any judgments for keyword-based queries. Kolbe et al. [21]  propose a dataset derived from scholarly data, which, however, only considers ontology and no term ranking, is also limited by its size (1,028) and focuses on IoT ontologies. With LOVBench, we aim to overcome these limitations.
An overview of existing benchmarks and LOVBench is presented in Table  1. In contrast to the related work, LOVBench satisfies all the following: it relies on keyword-based queries, ranks classes as well as properties of ontologies from various domains, graded relevance labels are mined cost-efficiently from actual, real-world user feedback (views and clicks), and it provides sufficient judgments for complex models in LTR settings (7,395 queries and 184,224 relevance judgments). Moreover, our comparison of state-of-the-art ranking models considers best combinations of originally proposed along with newly designed features (in total 33), and we make the dataset as well as the feature implementation publicly available.
LOVBENCH APPROACH In this section, we present our approach to benchmarking ontology ranking models. As illustrated in Figure  1 , our approach comprises four main steps: (1) analyzing the collected LOV search logs (containing users' queries and clicks), (2) inferring and evaluating relevance labels, (3) creating the LOVBench dataset, and (4) performing the empirical evaluation of ontology ranking models. In the following, we present an overview of each step and respective outcomes including key features for the design of LOVBench. Analyzing LOV search logs. In the first step (cf. Section 4. 1), we analyze the LOV search logs to confirm whether the logs fulfill basic requirements to generalize user behavior for ontology search. Moreover, we extract fundamental insights concluded from the observed real-world user actions in the context of literature assumptions on ontology ranking design, which guide our later feature design and discussions on experimental results.
In general, an ontology benchmark is composed of two main components: first, a set of sampled queryterm pairs with relevance judgments (the ground truth) and second, the extracted scores of selected ranking features for these queryterm pairs (used as input for LTR experiments). In the third step of our LOVBench approach (cf. Section 5), we first sample terms for the ground truth based on common practices (similar to the one followed by LETOR  [31] ). Second, our strategy for the selection of ranking features for LOVBench is guided by related literature surveys  [14, 22, 45] , and respectively discovered features are selected based on applicability constraints. In summary, we build the LOVBench dataset as follows. (1) For each query contained in the search log, we sample relevant and non-relevant terms following common practices, (2) we infer the relevance labels based on the best-performing user click model of the previous step, (3) we select ranking features from the literature and propose variations, and (4) we extract corresponding feature scores for each query-term pair of the sample.
As a result, we derive a ground truth file for ontology evaluations and a dataset that can be used for LTR experiments. Evaluating ranking models. The last step of our benchmark (cf. Section 6) is concerned with the comparison of the ranking models' performance based on the LOVBench dataset. We follow a standard LTR experimental methodology  [24]  and use standard evaluation metrics for information retrieval (NDCG). In addition to feature configurations as proposed in the literature, we propose novel variations that are motivated by the observed user behavior from the LOV search logs.
The results of the relevance prediction evaluation are also summarized in Table  2 , showing the correlation r for each model with respect to the CBRBench judgments. On the one hand, this evaluation step confirms the previous results that the UBM should be preferred over DCTR and DBN. Furthermore, UBM demonstrates a a strong correlation with CBRBench's relevance judgments (0. 613), meaning that the relevance predictions of the UBM model learnt from the LOV logs are close to the expert judgments from CBR-Bench. We explain the better performance of UBM compared to DCTR and DBN due to its consideration of previous clicks and their position in the same session, based on the experimental findings and intuition presented in  [10] , where UBM also performs best. Thus, we consider the relevance labels inferred from the UBM model in the LOVBench dataset.
• The feature must not depend on metadata information that cannot be derived from the ontology collection itself. Such features are often considered in ontology ranking by formulating scores involving user feedback such as ratings and clicks  [26] , ontology and term usage in LOD  [38] , etc. While these features can be very useful for ranking, applying these features to different ontology collections can be problematic because the respective metadata is often not available or difficult to obtain for other ontology collections  [21]. Since ranking models evaluated by LOVBench should be applicable to any ontology collection, we choose to exclude them. • The feature must form a significant distinction to already considered features. • Priority is given to features for which the complete ranking configuration as proposed in the literature can be replicated.
Other considerations for a fair comparison of ranking models include the constraints that define whether a term matches a query or not. We choose to harmonize the query match for all features independent from the respective feature's original approach. This gives a more accurate query match for all features by considering the meta vocabularies used in the LOV collection. Lastly, some ranking features depend on hyperparameters that need to be set by experts. In LOVBench, we stick to the hyperparameters as suggested in the original source of the feature. However, our large ground truth for the application of LTR further allows us to decompose some of these features (e. g. , in case of weighted sums) and implicitly learn the parameters from the data instead.
Term Sampling When building a ground truth, it is usually impractical to judge and extract features for all terms in the repository  [24]. In the context of LTR, the chosen strategy of sampling some documents for each query for learning will impact the observed effectiveness of the learnt ranking model, referred to as sample selection bias  [25, 29]. Given the discussions of document sampling's impact  [25] , our goal is to ensure that the LOVBench dataset contains a similar distribution of relevance labels and average sample size compared to well-known state-of-the-art datasets for conventional ad-hoc Web retrieval tasks. We follow a similar strategy to the one presented in LETOR  [31] , by (i) considering all query-term pairs without judgment as non-relevant and (ii) selecting samples from an existing ranking, i.
e. , the term search results of LOV. For each query of the LOV search logs, the relevance of the first ten terms is inferred from the best-performing click model using UBM (see Section 4. 2), and an additional amount of non-relevant terms is randomly sampled from the remaining query match. As a popular convention, we map the relevance inferred from the click model, which is measured in terms of satisfaction probability (0 ≤ Rel Q,t ≤ 1), to relevance labels l Q,t on a scale from 0-4, and further consider a junk grade (-2). We denote the set of labelled data for query Q and term t by L = {(Q, t, l Q,t )}, the standard input for supervised LTR settings, and refer to L with all relevance predictions as ground truth. As shown in Table  4 , this approach results in a dataset with a similar distribution of relevance judgments compared to established LTR datasets, such as TREC Web 2013 & 2014  [11, 12]  and a selection of ClueWeb12 7 entries as presented in  [40].
Final LOVBench Dataset In summary, the LOVBench dataset contains: (i) extracted ranking features (Section 5. 1) and (ii) inferred relevance judgments (Section 5. 2) for 184,224 query-term pairs. We provide a single csv-file containing the query-term pairs and their relevance judgments which can be used as ground truth for evaluation purposes. Moreover, the full dataset (including extracted features) is provided in a format for LTR experiments. It is randomly split by query into five equal-sized partitions, which in turn are used to derive five folds with three partitions for training and the remaining two for validation and testing.
Experimental Setup The experiments are based on the standard framework for LTR evaluation  [24]. In the following, we describe the considered feature configurations, baselines and evaluation metrics. We rely on the previously introduced LOVBench dataset for the evaluation. Configurations. We experiment with feature configurations that are associated with three different groups: baseline, literature, and variations. An overview of all configurations subject to the experiments is given in Table  5. The purpose of our baseline (a standard Lucene search on terms' rdfs:label properties) is to provide the performance of a straight-forward search approach as a reference point. We then compare several ranking models as they have been proposed in the literature with the baseline using the feature sets from AKTiveRank  [2] , DWRank  [8] , and CBRBench  [7].
Third, CBRBench is, similar to AKTiveRank, mostly composed of ontology features, and further of many complex features with predefined hyperparameters, not allowing the LTR algorithms to learn these from the data. As shown by our experiments, adapting the ranking features to these requirements significantly improved the performance. Our results also demonstrate that increasing the number of ranking features, given a large enough dataset like LOVBench, allows to increase ranking performance. However, in practice it is often necessary to form a reasonable trade-off between effectiveness and computation cost. This especially concerns query-dependent features that cannot be pre-computed and have to be extracted at run-time. As previously mentioned, the experimental results need to be interpreted in the context of the biases imposed by the current LOV ranking from which the relevance labels for the ground truth are derived.
The LOV search relies on a Lucene-based match with property boost and a popularity score measured in term of usages in LOD datasets  [41]. Thus, it is reasonable that relevance features and qualitative importance features such as PageRank perform better on our dataset than graph-structural criteria such as density. In conclusion, the following considerations should be taken into account when designing ontology ranking models, which were also followed for the LOVBench light configuration: (1) mixing and prioritizing of ranking granularities depending on the platform (ontologies/terms and classes/properties), (2) diverse coverage of all feature categories (query match, repository graph, ontology graph and RDF graph analysis), (3) preferring decomposed features when possible, (4) being aware of the user behavior of targeted platform, (5) the number of query match features should be minimized.
CONCLUSION In this paper, we address the problem of ontology ranking evaluations and comparisons in LTR settings. We analyze logged user interactions of a real-world ontology search platform (LOV) and use this implicit user feedback infer relevance judgments for queryterm pairs. Our evaluation shows that inferred relevance judgments are close to those made by human experts. We create the LOVBench dataset that comprises 184,224 relevance judgments for 7,395 queries and considers 33 different ontology ranking features. We then evaluate and compare three state-of-the-art ranking models from the literature based on this ground truth. We explain the results in the context of the observed user behavior in LOV and propose variations and configurations that outperform the baseline. The published dataset can be applied for future ontology ranking evaluations and LTR experiments.
This work focuses on the Ventura basin, southern California, USA to characterize the three-dimensional geometry of an active fault system using subsurface well-log data and then to investigate how static stress may be transferred along the fault network for different rupture scenarios. Several authors have suggested that large-magnitude (Mw 7. 5-8. 0) multi-fault earthquakes may occur between the Pitas Point, Ventura, and San Cayetano faults (Fig. 1 ) based on large coseismic uplift events inferred from paleoseismic studies, which imply a high degree of assumed structural connectivity between these faults at depth  (Hubbard et al. , 2014; McAuliffe et al. , 2015; Rockwell et al. , 2016). For example, four uplift events between 7-12 m are interpreted to have occurred on the Pitas Point fault at 6.