dataesr/scientific-paragraphs-categorization huggingface dataset:
ELG could function as a means to keep interested stakeholders informed by serving as an information source and matchmaker for buyers and suppliers alike (marketplace approach, see Chapter 12). Interoperability: 1. Throughout Europe, there is a sizable number of other relevant platform and infrastructure initiatives including, among others, Gaia-X 11  , the European AI-on-demand platform 12  , EOSC 13  and NFDI 14. The SMEs mentioned their expectation that ELG becomes part of this larger ecosystem of platforms around Artificial Intelligence, data economy, research data management and Open Science, i. e. , that ELG should ideally be fully interoperable with these other infrastructures, eventually opening up additional markets  (Rehm et al. 2020a ). 2. Furthermore, providers of LT need to understand what the requirements are to participate in ELG and why it is beneficial for them.
ELG needs to be compatible with existing businesses and should not duplicate existing systems. Since various companies already operate their own or managed cloud platforms, platform interoperability should be ensured so that ELG complements existing or emerging clouds rather than appearing like competition. ELG should avoid creating the impression of being yet another collection of data and tools but rather emphasise the ability to combine services and resources from different companies. 3. For this, however, full interoperability on the level of the actual tools and services, i. e. , on the level of APIs, annotations, semantic descriptions, closed vocabularies etc. needs to be achieved (also see Strategy and collaboration above).
Equally important are Language Resource and data providers that own existing data sets and repositories. These two key partners contribute to the thriving of the ELG platform. Their motivation is not (or not only) to use available services and resources, but they offer their own services and resources and create value or profit for their own organisations. Another key partner is the wider ELG community, including the ELG consortium, the 32 National Competence Centres, the national language communities, and all running EU projects and initiatives in the field of LT (includ-ing ELE). This community consists of academic and research partners as well as a number of companies that need multilingual datasets and services for their research. The most important resource is the ELG platform itself with all its functionalities and included services, corpora and additional information.
ELG can be regarded as a set of seed technologies, tools and components that are extended over time. Customer feedback can be seen as a useful resource as well. It can come in many different forms such as evaluation from market data or helpdesk and user support feedback. Equally important is a dedicated ELG team, committed to not only maintaining existing technology, but growing it and promoting the importance of ELG on an international level. To achieve this, a wide international network is a key resource. The consortium combines vast experience and expertise, good knowledge of ongoing trends and access to numerous European networks in academia and industry. Segment: Value Proposition "Which customer needs are being satisfied? What core value is delivered to the customer?" ELG is envisioned to become the primary LT platform for Europe and to function as a one-stop-shop, offering a rich portfolio of LT services, tools and datasets.
Segment: Channels "Through which channels do customers want to be reached? Which channels work best? How much do they cost? How can they be integrated into customers' routines?" Customers will be reached through a variety of channels. Events, both established and new ones, will play an important role, for example, events targeted at stakeholders in a specific industry domain. Dedicated networking sessions, conferences and presentations are also foreseen. Online advertising campaigns will accompany all events. Since ELG builds on an existing network of stakeholders, email marketing and social media campaigns have proven to be successful means of reaching out. Presence on social media channels such as Twitter or LinkedIn helps to promote events and maintain customer relationships. ELG itself is a channel through which customers can retrieve information, not only about services and datasets, but also about the community and events.
Cloud platforms that are either currently being developed in other EU or national projects as well as exist-ing commercial platforms can also act as channels to point potential customers to ELG. SEO can also help promoting ELG since users trust search engines. Segment: Customer Segments "For which segment is value being created? Who is the most important customer?" The ELG platform offers value to different customer segments. LT providers, both commercial and academic ones, can use ELG to offer their services and datasets. Research organisations can benefit immensely from the wide offer. Customers from industry that demand LT (including large enterprises, SMEs, startups etc. ) represent an essential customer segment that contributes to turning ELG into a flourishing marketplace. The European Union, public administrations and NGOs can also integrate ELG services into their current solutions.
Segment: Revenue Streams "For what value are customers willing to pay?" Part of the overall revenue will be generated through different products including usage or subscription fees, brokerage fees (marketplace approach), commission fees and products such as LT as a Service (LTaaS; hosting of services, models, datasets), LT Platform as a Service (PaaS; combining ELG services into workflows) and Repository as a Service (RaaS; hosting service for whole repositories). Advertisements can, for instance, showcase companies, services, conferences etc. Sponsored content, services, data sets, companies etc. present another revenue stream as well as commission fees. Paid training events, tutorials, webinars etc. can be offered to commercial stakeholders. Conferences (event registration fees; sponsorship packages for companies) are also an opportunity to generate income as well as general consulting services around ELG and languagecentric AI.
Product Category: Data-as-a-Service Data-as-a-Service (for commercial users) This product is very similar to LT-asa-Service but instead of focusing upon running services or tools, it only allows making datasets or other (static) resources available on ELG, again, with guaranteed availability. Like LT-as-a-Service, this is an entry level product and, thus, only allows hosting a limited number of datasets (or up to a certain amount of data) on ELG. In case of more demand on the side of the customer, a different type of product needs to be purchased (see Section 3. 5. 6). This product needs to be priced lower than the LT-as-a-Service product. Data-as-a-Service (for academic users) Technically, this product is like the previous one but it targets academic users. The monthly or annual fee will be significantly lower than the fee of the Data-as-a-Service product for commercial users.
1. Passive acoustic monitoring can be an effective method for monitoring species, allowing the assembly of large audio datasets, removing logistical constraints in data collection and reducing anthropogenic monitoring disturbances. However, the analysis of large acoustic datasets is challenging and fully automated machine learning processes are rarely developed or implemented in ecological field studies. One of the greatest uncertainties hindering the development of these methods is spatial generalisability-can an algorithm trained on data from one place be used elsewhere? 2. We demonstrate that heterogeneity of error across space is a problem that could go undetected using common classification accuracy metrics. Second, we develop a method to assess the extent of heterogeneity of error in a random forest classification model for six Amazonian bird species.
| INTRODUC TI ON Passive acoustic monitoring (PAM) is an increasingly common ecological survey tool. PAM has many advantages over traditional survey methods, facilitating sampling across larger spatiotemporal scales, and in places where access is logistically challenging, thus increasing costefficiency  (Darras et al. , 2018; Gibb et al. , 2019). Given this capacity to record audio data autonomously, PAM can accrue very large datasets, often too large to analyse manually. Automated classification presents a solution to this challenge. A variety of approaches have been trialled including template matching, machine learning techniques such as clustering and random forests along with deep-learning algorithms such as convolutional neural networks  (Priyadarshani et al. , 2018). However, outside of chiropterology, few studies have used fully automated classification to answer applied ecological questions in terrestrial landscapes, and especially for the challenge of multi-species classification across large audio datasets from tropical forests.
, 2014) , plus other sources of biophony, geophony or anthropophony; and environmental factors that can impact sound propagation, such as weather conditions and the density of surrounding vegetation  (Yip et al. , 2017). There are also potential sources of audio variation that fit between the two categories-for example when a source extrinsic to the target animal causes an intrinsic change to the vocalisation of the target species. Responses to predators or competitors, such as duetting or lekking birds can fit in this category  (Mennill & Vehrencamp, 2008) , as do broadband sources of noise such as cicadas or vehicle engines that may cause a change in the frequency at which calls are made. In comparison to intrinsic variation, these sources may be more challenging to represent well in a training dataset, as they are likely to be both more variable and less predictable.
In particular, the use of online libraries such as Macaulay Library (https://www. macau layli brary. org), xeno-canto (https://www. xeno-canto. org) or AmphibiaWeb (https://amphi biaweb. org) could cause training datasets to be less representative, as recordings made with directional microphones of single species have high signal-to-noise ratios  (Priyadarshani et al. , 2018 , Towsey et al. , 2012) , and are unrepresentative of external sources of acoustic variation. Errors associated with variation in noise could be resolved by applying noise reduction techniques (e. g. Priyadarshani et al. , 2020). While this approach is undoubtedly effective in some or even many, cases, it is a difficult approach when dealing with a multi-species classifier. Here, one target species call is 'signal' when considering its own classification, but can also be considered as 'noise', and impact classification accuracy, for all the other species included in the classifier.
Therefore, even removing all of the sound that could be considered 'noise' in all circumstances still leaves a considerable amount of variable sonotypes in the training data. Furthermore, noise reduction is not a universally agreed approach to improve classification accuracy, for instance the Open SOundScape package  (Lapp et al. , 2022)  offers many approaches to augment and increase noise in training data-for instance overlaying training samples on top of each other. Intrinsic and extrinsic variation in audio data make obtaining truly representative datasets extremely difficult if the classifier is intended to operate across large spatial extents, long periods of time or across heterogeneous habitats  (Zhong et al. , 2020). In these cases, providing representative data labels at local scales would require huge increases in labelling effort  (LeBien et al.
K E Y W O R D S automated signal recognition, autonomous recording unit, bioacoustics, ecoacoustics, machine-learning shortfalls in obtaining representative datasets mean that classification accuracies obtained on test datasets may not translate to field conditions  (Stowell, Wood, et al. , 2019) -a common problem in supervised learning fields termed covariate shift  (Shimodaira, 2000). Many classification algorithms may exhibit biases that will lead to heterogeneous error structure when exposed to these variations. Heterogeneity of error could be especially problematic if the covariate responsible for shifting classification accuracy is the same as that being studied for ecological purposes. This may occur, for example, when error varies; spatially in a space-for-time swap experimental design, temporally in a phenological study or at ecotones where replacement species with similar vocalisations may overlap when examining habitat preference.
Automated classification models are typically assessed by deriving a range of accuracy metrics from a manually labelled test dataset that has been randomly subsampled from the training data, or independently subsampled from the dataset on which the algorithm is to be applied  (Knight et al. , 2017; Priyadarshani et al. , 2018). Following  Knight et al. (2017) , precision, recall, F-score and area under the curve (AUC) have been widely adopted to determine classification algorithm performance (Table  1 ). However, these methods fail to explicitly test the generalisability of the algorithm across the gradient of a shifting covariate. Consequently, using only these metrics to assess classification performance risks masking high variability in false-positive error and subsequent confounding of results if error covaries with a variable of ecological interest.
Although both heterogeneity in false-positive and false-negative errors can be detrimental, we focus here on false-positive errors, and consequently the precision metric, in keeping with other studies highlighting precision as important for acoustic surveys (e. g. Juodakis et al. , 2021). Unlike false negatives, false positives cannot be mitigated by summarising presence across longer time periods or spatial extents  (Metcalf et al. , 2019) , violate the assumptions of many standard methods for modelling detection probability and can lead to poor model inference  (Royle & Link, 2006). While research has been conducted into the overall reduction in false-positive error in ecoacoustic datasets (e. g. Balantic & Donovan, 2020; Clare et al. , 2021; Knight et al. , 2020) , and in reducing the variation in error when analysing the ecological variables through the use of occupancy models (e.
g. Chambert et al. , 2018; Rempel et al. , 2019) , there has been limited research into methods to reduce variability of falsepositive error at the classification stage. We present a case study using automated classification of an Amazonian PAM dataset to highlight the challenges in detecting heterogeneity of precision error. First, by incorporating a measure of heterogeneity into the metric used to set a threshold for classification confidence score, so that thresholding goes beyond a twoway trade-off between precision and recall, and instead becomes a three-way trade-off between precision, recall and heterogeneity of (precision) error. Second, by incorporating a secondary classification that accounts for the context in which the classification is made, through the use of neighbouring classification scores for the target and other species, environmental and temporal data.
F-Score The harmonic mean of precision and recall = 1 ( 2 + 1) × Precision × Recall 2 × Precision + Recall F 0. 5 -Score The harmonic mean of precision and recall, weighting precision as twice as important as recall = 0. 5 ( 2 + 1) × Precision × Recall We collected two sets of acoustic data. The first was used exclusively for the purpose of training the classification algorithm (hereafter 'Acoustic Dataset 1') between 20 November and 30 December 2017 from two of our survey points, each with three recorders 150 m apart. The second set of acoustic data (hereafter 'Study Dataset') was collected from a single recording location at each of the 29 survey points between 12 June and 16 August 2018. Recordings for the Study Dataset were made over 1-2 recording periods at each point, with recording period varying in length between 3 and 22 days.
This gave as optimal a coverage of nocturnal species as logistical limitations would allow (nocturnal species vocalisation rate may be impacted by the lunar cycle). A minimum of 13 days were surveyed at each location. Both datasets were collected using Frontier Labs Bioacoustic Recorders  (Frontier Labs, 2015). Recordings were continuous across the diel cycle, and were filtered afterwards to only include astronomical night, measured using the Suncalc package in R  (Thieurmel & Elmarhraoui, 2019)  using the location of our field station (coordinates given above). Full details of recording periods, equipment and protocols for each location are given in SOM Appendix 1. This work was conducted under SISBIO permit number 53271-13.
| Automated classification and verification datasets Tadarida is an open-source toolbox for detecting, labelling and classifying sounds  (Bas et al. , 2017)  and shown to be effective at classifying various European species of insects and mammals  (Barré et al. , 2019; Newson et al. , 2017). We used Tadarida to build a classifier in R (R Core Team, 2020) for seven nocturnal bird species-four owl species; Southern Tawny-bellied Screech-owl Megascops usta, Crested Owl Lophostrix cristata, Spectacled Owl Pulsatrix perspicillata, Amazonian Pygmy-owl Glaucidium hardyi and three nightjar species; Ocellated Poorwill Nyctiphrynus ocellatus, Silky-tailed Nightjar Antrostomus sericocaudatus and Common Pauraque Nyctidromus albicollis. Tadarida first identifies sound events using a hysteresis function; the sound event starts when a high amplitude threshold is passed and ends when the signal-to-noise ratio drops below a second lower threshold.
The program extracts 269 acoustic features (e. g. minimum and maximum frequency, peak frequency, duration) from each detected sound event and facilitates feature labelling for use as training data in a random forest classifier (see  Bas et al. , 2017  for full details). As multiple detected sound events may be identified from a single animal vocalisation, Tadarida uses simple rules to group events and makes classification predictions. Consequently, Tadarida works best over short-duration sound files, so we split all the recordings into 15 s files for all further processes. We limited all detections to those with the point of highest amplitude between 0. 2 and 4. 2 kHz which includes most terrestrial nocturnal vertebrates in the region. To create training data for the classification algorithm, we undertook manual labelling of sound events detected by Tadarida (this labelled dataset hereafter referred to as 'Training Dataset 1') (Figure  1 ).
We identified each sonotype to species level where appropriate and possible. Where identification was not apparent, online resources such as the Macaulay Library, xeno-canto and AmphibiaWeb were consulted, and some call types were shared with relevant regional experts. If identification was still not possible, the sound type was left unidentified. To obtain training data, we systematically searched for discrete sound types in our recording datasets. First, we labelled data from a subset of Acoustic Dataset 1. This subset consisted of 3 hr of recording per night-1 hr up to 30 min before sunrise, 1 hr commencing 30 min after sunset and 00:00-01:00, every 3rd night from each of the three recording units deployed, totalling 96 hr of data. While each sound file was searched systematically, training data were added based on the labeller's discretion so that not all calls in an extended vocalisation bout were necessarily included, especially for common sound types.
As this data only came from three survey points, we additionally labelled data from eight other survey points in the Study Dataset, to increase spatial coverage and representation of forest disturbance, which can impact species abundance and composition. As the systematic search method adopted for Acoustic Dataset 1 was extremely time-consuming, we adopted a more targeted approach to labelling the additional data from the Study Dataset, choosing data from periods of time and places which we knew were most likely to contain vocalisations of species that were known to be present in the region  (Lees et al. , 2013) , but were thus far under-represented in Training Set 1. Finally, we supplemented labels generated from our own audio data using recordings from online archives (Macaulay Library, xenocanto), also split into 15 s.
wav files. We augmented all recordings by adding noise at six amplitude levels by combining each labelled file with three files identified in our own recordings as containing only heavy rainfall, and manipulating each by increasing the amplitude, giving six 'rain' files. Each rain file was then combined with each labelled file, increasing six-fold the number of labels in the training dataset. For full details of the Tadarida labelling and data augmentation process, see SOM Appendix 2.
| Assessing classifier performance To assess classifier performance, we followed  Knight et al. (2017)  in using precision, recall, F-score and the area under the precision/ recall curve (hereafter 'PR-AUC'). We calculated these scores from two test subsets of the Study Dataset by comparing Tadarida predictions against manual assessment of the same audio files. We detect the presence or absence of a species at 15 s resolution. Tadarida gives a confidence score for each class included in the classification algorithm (n = 59) for each group of detected sound events (Tadarida made classification predictions on n = 28,030,710 grouped sound events in the Study Dataset), and we considered the species with the highest score in each grouped sound event to be the species predicted as present by Tadarida.
We then summarised the predictions to the 15-s file level, taking the maximum score for each predicted species per file so that a species only had one score per file. All predictions for non-target species were discounted. Manual assessment of target species present within 15 s files was conducted using Raven Pro (Center for Conservation Bioacoustics, 2019). When creating a test dataset for manual labelling, it is vital to use a sample large enough to be representative of the original dataset to accurately assess classification performance  (Knight et al. , 2017). However, test sets of 10% or even 1% may not be practical for very large datasets, as the subsample would take too long for manual assessment, as is the case here (n = 1,081,780 fifteen second files in the Study Dataset). Instead, we made a subjective decision on test set sample size based on the trade-off between manual labelling effort and representativeness.
Consequently, the first test set (Test Set 1) consisted of 2900 15 s files stratified such that 100 files were taken from each survey point, just under 0. 3% of the total number of unique 15 s files in the study dataset. Test Set 1 was randomly subsampled from the study dataset prior to classification following  Knight et al. (2017) , ensuring independence from the training dataset. A 15 s file was considered to be a true positive when Tadarida predicted a species presence within that file and the species was also detected in the file during manual assessment. A 15 s file was considered a false positive when Tadarida predicted a species presence within that file, but the species could not be detected in the file during manual assessment, and so on, respectively, for true negative and false negatives.