dataesr/scientific-paragraphs-categorization huggingface dataset:
The authors would like to thank the following for their contributions to the available data sets: Bjarni Bessasson (Iceland geology), Susana Vilanova (V S30 data for Portugal), Pauline Kruvier (V S30 model for Groningen), Marco Pilz (V S30 model for Cologne), Donat Fäh and Valerio Poggi (V S30 data for Switzerland). The development of the site amplification model has benefitted from discussions with Kyriazis Pitilakis, Evi Riga, Vitor Silva, Venetia Despotaki, Pierre-Yves Bard, Donat Fäh, Céline Beauval, Myriam Belvaud, Afifa Imtiaz and Roser Hoste Colomer. The manuscript has been improved following the comments from two anonymous reviewers. Topography and bathymetry data are taken from General Bathymetric Chart of the Oceans (https:// www. gebco. net/) and strong motion data from the Engineering Strong Motion database and flatfile (http:// esm.
Data availability The data sets generated within this work are available from the EFEHR seismic risk services (http:// risk. efehr. org/ site-model/). The site response model is implemented as part of the OpenQuake open-source software for seismic hazard and risk analysis, which is freely available for download from https:// github. com/ gem/ oq-engine/. An additional software to allow users to construct site input files for their own seismic hazard and risk applications using the ESRM20 site model described here is also available for download from https:// gitlab. seismo. ethz. ch/ efehr/ esrm20_ sitem odel.
Acknowledgements The authors would like to thank the following for their contributions to the available data sets: Bjarni Bessasson (Iceland geology), Susana Vilanova (V S30 data for Portugal), Pauline Kruvier (V S30 model for Groningen), Marco Pilz (V S30 model for Cologne), Donat Fäh and Valerio Poggi (V S30 data for Switzerland). The development of the site amplification model has benefitted from discussions with Kyriazis Pitilakis, Evi Riga, Vitor Silva, Venetia Despotaki, Pierre-Yves Bard, Donat Fäh, Céline Beauval, Myriam Belvaud, Afifa Imtiaz and Roser Hoste Colomer. The manuscript has been improved following the comments from two anonymous reviewers. Topography and bathymetry data are taken from General Bathymetric Chart of the Oceans (https:// www. gebco. net/) and strong motion data from the Engineering Strong Motion database and flatfile (http:// esm.
Determining which species are at greatest risk, where they are most vulnerable, and what are the trajectories of their communities and populations is critical for conservation and management. Globally distributed, wide-ranging whales and dolphins present a particular challenge in data collection because no single research team can record data over biologically meaningful areas. Flukebook. org is an open-source web platform that addresses these gaps by providing researchers with the latest computational tools. It integrates photo-identification algorithms with data management, sharing, and privacy infrastructure for whale and dolphin research, enabling the global collaborative study of these global species. With seven automatic identification algorithms trained for 15 different species, resulting in 37 species-specific identification pipelines, Flukebook is an extensible foundation that continually incorporates emerging AI techniques and applies them to cetacean photo identification through continued collaboration between computer vision researchers, software engineers, and biologists.
With over 2. 0 million photos of over 52,000 identified individual animals submitted by over 250 researchers, the platform enables a comprehensive understanding of cetacean populations, fostering international and cross-institutional collaboration while respecting data ownership and privacy. We outline the technology stack and architecture of Flukebook, its performance on real-world cetacean imagery, and its development as an example of scalable, extensible, and reusable open-source conservation software. Flukebook is a step change in our ability to conduct large-scale research on cetaceans across biologically meaningful geographic ranges, to rapidly iterate population assessments and abundance trajectories, and engage the public in actions to protect them.
Flukebook (https:// www. fluke book. org) is a non-profit, open-source cetacean data archiving, analysis, and photoidentification tool developed as part of the larger Wildbook platform (https:// wildme. org/#/ wildb ook). By bringing together software developers, machine learning researchers, and biologists, Flukebook bridges the gap between modern computing techniques and ecological field research to provide the latest computer vision technology to cetacean researchers around the world. Users upload data through a web interface, and the platform automatically detects and draws boxes around identifiable cetacean features in those photos, and identifies the individual animals therein using the latest algorithms for cetacean photographic ID. The platform also facilitates the management of these catalogs of photos and observations that are necessary to support these studies.
Flukebook's server resources can be easily scaled-up by the developers at Wild Me (https:// www. wildme. org), for example adding more processing power or database storage as the platform grows, and this server is automatically backed-up nightly ensuring catalogs cannot be lost by technological accidents. Fixes and improvements to the platform are instantly available to all users without any effort on their part. The platform is free for approved users, and the Wild Me nonprofit organization which develops and manages Flukebook has no plans to charge for its use at any point in the future. The nonprofit's operating costs including Flukebook servers are covered primarily by grants (see this paper's funding statement), and to a lesser extent service contracts where the nonprofit is paid to develop and make available new features for the platform.
The most important organizational feature of Flukebook is its position as an open-source project in the larger family of Wildbook platforms. As of June 2021, there are 21 Wildbooks maintained by Wild Me, supporting the study of over 50 species. Each Wildbook instance runs a branch of the open-source Wildbook codebase (https:// github. com/ WildM eOrg/ Wildb ook), and can be understood as a computer vision-supported catalog for different species, such as sharks (https:// www. shark book. ai/), giraffe (https:// giraf fespo tter. org/), or giant sea bass (https:// spott inggi antse abass. msi. ucsb. edu/), with Flukebook being the Wildbook for cetaceans. The open-source nature of Wildbook allows new features or algorithms developed for one platform to be easily deployed on all of them at virtually no cost.
The architecture of Flukebook and other Wildbook platforms is bipartite in nature. These two parts can be understood as the data management server and the image analysis server (Fig. 1 ). The data management server hosts the frontend web application on Flukebook. org, where users go to log in; inspect, manage, and upload data; start matching jobs; etc. This frontend contains all of the user interface (UI) components and interactions facing the users. The backend computer vision server, running a separate code repository on a separate machine, is called Wildbook image analysis (WBIA). These servers communicate via http requests, with images and computer vision queries sent from Flukebook to WBIA, and computer vision results sent back from WBIA to Flukebook. Both servers run SQL databases containing the data necessary for that server's function, which are synchronized by these calls and other nightly integrity scripts.
Data management and curation Though perhaps not as exciting as the computer vision algorithms, the data management web app's frontend of flukebook. org is equally if not more important in making the platform a valuable scientific tool. Tasks such as uploading data, inspecting photos, querying, reviewing, and confirming match results, bulk processing and exporting data, sharing data and collaborating, and producing a resolved catalog of unique individuals are all tasks separate from the computer vision processing that takes place on the image analysis server. In essence, this data management platform can be understood as a user interface on top of a database of ecological observations, with a suite of data processing and inspection tools. The database structure was designed with mark-recapture population studies in mind.
As of June 2021, Flukebook hosts over 2 million photos spanning 266,038 sightings of 52,638 identified individual cetaceans. The user interface is available in four languages (English, French, Spanish and Italian). The Flukebook frontend and data management server is an instance of Wildbook, specified in the 'flukebook' branch of the open-source Wildbook repository available on GitHub (https:// github. com/ WildM eOrg/ Wildb ook/ tree/ fluke book). Except for user interface themes and cetacean-specific algorithm configurations, Flukebook's code is largely identical to that of the master-branch Wildbook, a trait that all other Wildbook 7. x platforms share. Wildbook is a Java web application built on the Apache Tomcat framework (http:// tomcat. apache. org/). Table  1  provides an overview of the core high level data objects on Flukebook.
New identifications made with the aid of matching algorithms on Flukebook also must be confirmed by the users, ensuring that data owners maintain ultimate control over data integrity and the Marked Individual objects, which contain the consolidated photos and observations of a single animal over time. The third major object is the Sighting, representing a sighting of a group of animals, containing observational fields which are not specific to a single individual such as group size, group behavior, and weather conditions. The main objects have unique web endpoints, such that any Encounter, Marked Individual, and Sighting has a static URL that can be referenced and linked, where all data fields can be inspected and edited by approved users. These three objects are linked to each other in both the database and user interface (UI), so that one can navigate in a single click from a Marked Individual to one of its Encounters, then to the Sighting where one can find the other animals that were co-occurring during that observation, and so on.
Standardizing data options such as these at the Organization level, rather than Flukebook-wide, is crucial for supporting the broad range of data collection standards used by cetacean researchers while still enforcing consistency within projects. For example, the metrics used to quantify photo quality or distinctiveness of an individual, both which are important to photo-identification work and the analyses they support, vary both between researchers studying the same species as well as between species; and often show observer bias between researchers using the same metric (see for example  Urian et al. 2015  who found a remarkable degree of variation between experienced researchers' photo quality scores, even when using the same scale, and in addition they disagreed significantly on the total number of individuals in a controlled dataset).
When a collaboration request is sent, an email is sent to the recipient which includes an optional message from the requester and importantly, the requester's email address (user emails are otherwise hidden from other Flukebook users, and the requester does not see the recipient's email unless the recipient emails them). Sharing this contact information allows the two parties to establish communication directly and agree on how each other's data can be used; some users choose to establish, over email, formal memoranda of understanding regarding these agreements. Within Flukebook, a notification is sent to the user's account page where Collaborations can be managed, approved, and revoked. Within this data ownership system Flukebook strongly encourages collaboration. While complete data records are only visible as prescribed by the Collaborations, summary information is just visible enough to other users, so that those users can determine if they would like to request access to those data (with the exception of certain lockeddown datasets described in the next paragraph).
Search tools Robust search and analysis tools are useful not only for navigating the catalogs on Flukebook but for generating useful data products. The Flukebook database can be searched by Encounters, Marked Individuals, and Sightings. Search forms translate a user's input on a web page into SQL queries of the database backend. Most data fields can be used as search filters, for example species, date, location, submitter, observed behavior, life stage, photographed features such as fluke or dorsal fin, whether a biological sample was taken, whether a satellite tag was deployed during the observation, and so on. Search results are presented in a tabular format where each row is one database record. Clicking that row takes the user to the record's web page; the table also includes hyperlinks to related data objects such as the Marked Individual page for a given Encounter.
The SQL query which was derived from the search input is shown below the results, providing added clarity for technical users and developers. Since search results effectively represent a dataset on Flukebook, for example "male sperm whales sighted by me and my collaborators in Dominica in 2018," search results are where many of Flukebook's data analysis tools are located. In addition to the default tabular representation of these results, users can see them mapped geographically, distributed temporally on a calendar, or as a gallery of all matching images. There is a further 'Analysis' tab available on Encounter and Marked Individual search results that shows various computed analyses of that dataset, such as a discovery curve (Fig. 3 ), an annual seasonality histogram showing the number of Encounters per calendar week (Fig.
Data exports In addition to the analysis and inspection tools provided in search results, users have the option of exporting the data represented by a search in a number of different formats for analysis using other third-party tools. These exports are also subjected to the Flukebook data security model, ensuring no user can export data they do not have permission to access. The most basic format is a standard Excel sheet with one row per database object and one column per database field. This is the "complete" Flukebook export that shows all the data fields stored for a given object type and search query. This is the "Flukebook Standard Format" used by the Bulk Uploader described in the following section. This complete export allows for advanced filtering that may not be implemented in the user interface: for example, the Marked Individual page shows every confirmed Encounter of that individual, but a user might want to analyze only those Encounters that meet certain data quality or photo quality criteria; this is possible with search and export filtering.
There are also exports formatted specifically for compatibility with other databases, in particular the OBIS Ocean Biodiversity Information System. Other exports include a GIS shapefile of Encounter locations; a KML export that is compatible with Google Earth; and a configurable mark-recapture export compatible with RMark (Laake 2013), useful for using Flukebook data to generate abundance and trend estimates. A benefit of being open source is that it is easy for developers and researchers outside of the Wild Me organization (which manages Flukebook) to produce tools to export or analyze Flukebook data. The most significant example of this is the RWildbook R package  (Bonner and Huang 2018) , which allows a user to pull data records directly from Flukebook into their R environment after providing their login credentials.
If multiple individuals were seen in a group, that co-occurrence information is recorded with optional Sighting columns. Many more detailed columns are included in the standard format, the use of which are optional. Automatic data quality checks are performed, such as ensuring that data fields are provided in the correct type and format, and the user reviews their submission before it is committed to the database. Ensuring data quality and integrity in this way has been found to be very important when allowing users to upload hundreds or thousands of records at once. To avoid issues related to network connectivity, a user is allowed to upload photos in multiple web sessions, and all prior-uploaded photos are available for reference from subsequent Excel sheet uploads. To avoid over-taxing the server, a limit of 1000 rows per Excel file is encouraged.
For new users with exceptionally large historical catalogs that may be tens or hundreds of gigabytes in size, researchers may contact the developers at Wild Me to onboard these datasets on a contracted basis without going through the Bulk Uploading process themselves. 4  Flukebook species-specific image processing pipelines from automated detection (yellow diamonds) of body parts (orange parallelograms: FL fluke, DF dorsal fin, HD head, LT lateral side of body), through identification through multiple algorithms which have been trained for specific species (blue, rounded rectangles: HS HotSpotter, DTW dynamic time warping, CR CurvRank, PIE pose-invariant embeddings, K7 Kaggle7, FF FinFindR, DSAI Deepsense. ai), and finally to defining Marked Individuals (green ovals). Fifteen of the 37 species-specific pipelines are new since the beginning of 2021, with more under active development Importantly, Flukebook does not require photos to be associated with each Encounter, which allows users to bulk upload entire historical sightings catalogs on a purely presence-based basis.
Computer vision: Wildbook image analysis Access to cutting edge computer vision algorithms is the novel feature that draws in most users to Flukebook. The goal of this system is to decrease the amount of research time and effort needed to perform individual identification. This core computer vision functionality operates on a separate server than the frontend data management web application, on an instance of the Wildbook image analysis server (WBIA, available at https:// github. com/ WildM eOrg/ wildb ook-ia). This image analysis server is a Python application (https:// www. python. org) that uses the Flask framework (https:// flask. palle tspro jects. com) to operate as a web server in communication with the frontend. The computer vision server operates a database containing primarily images and information algorithmically derived from those images.
This image database structure is summarized in Table  2. The image analysis server contains a large number of tools for image manipulation and processing, supporting the two primary phases of the image analysis pipeline: (1) detection, where the system takes an unprocessed image and draws bounding boxes around every animal in that image, each box and additional metadata forming an Annotation, and (2) identification, where one or more matching algorithms are run on each Annotation to identify the individual animal within it.
2018) , and a humpback fluke matcher we have deemed "Kaggle7". The wide range of these algorithms' origins demonstrates the flexibility of the platform to rapidly integrate new methods. Algorithm development alone is not useful to biologists and managers Fig. 5  Using an interface for curating training data, this right whale image was manually processed by an experienced user to generate input for machine learning models that can then repeat the detection task automatically without human review. This process is required for any new species. Note the dotted line indicating the front of the whales, which is manually labeled in the training data so that orientation can be extracted automatically once operational on the platform. Inset: Once trained, this sperm whale fluke was detected automatically in Flukebook, allowing downstream machine learning identification tasks to focus on only areas of interest in a photo without a software interface to host data, query the system, and to store and export results.
The architecture of WBIA's identification pipeline is modular, allowing these algorithms to be used interchangeably and further algorithms to be integrated in a standardized way. An internal identification API uses the image analysis server's Chips and Annotations as inputs and produces a standardized match-score output format. One challenge in the development and integration of novel algorithms is that, like much in academic computer science, computer vision and machine learning research often take the form of developing cutting-edge algorithms based on the latest theory and technology in order to publish in leading field-specific journals. This often leads to "academic software", which as a result of systemic constraints rather than any failure on the researcher's part, are hard to scale to production and can have relatively little focus on the usability of the tool for conservation.
One of the underlying principles of Flukebook is to bridge this gap between academic algorithm development and applied conservation utility by bringing professional software design and development expertise together to package and deliver the latest computer vision algorithms as useful tools to researchers. In a real sense, Flukebook becomes the long-term steward of these algorithms, which ensures their curation, access, and source code legacy while also supporting their utility through software updates, re-training with new data, and changes to protocols. Below, we broadly describe each identification algorithm available in Flukebook and maintained by the Flukebook developers.
Kaggle7 Another Kaggle competition solution, dubbed "Kaggle7", was selected for its relatively straightforward implementation and ability to train quickly (original GitHub repository at https:// github. com/ ducha-aiki/ whale-ident ifica tion-2018). It uses components of the DTW algorithms to predict the end points of the fluke and the notch, which help to precondition the image prior to training and ID. Like the Deepsense. ai algorithm (and unlike other algorithms listed here), this algorithm is a fixed classifier, meaning it is only able to match those individuals that were in its training set. In the case of Kaggle7, this training set consists of pacific humpback whales uploaded by the Cascadia Research Collective (CRC), a set of 1701 individuals. Despite training only on CRC data, all users have access to this matcher.
Pose-invariant embeddings An algorithm dubbed PIE developed by Olga Moskvyak and colleagues  (Moskvyak et al. 2019 ) uses triplet loss methods  (Schroff et al. 2015)  to train a deep learning encoding method that maps an image into a feature vector. Query and catalog image are matched based on distances between their associated feature vectors. This allows PIE to accommodate new individuals as they are added to the database. PIE is trained on a per-species basis, and originally two models were developed for manta rays and humpback whale flukes (two separately trained models). It has now been trained by both Flukebook developers on killer whales, sperm whales, and North Atlantic right whales (this model is applied on both North Atlantic and southern right whales), and by the original algorithm developer (O.
Performance It should be noted that a computer vision algorithm does not have an inherent level of accuracy because its accuracy is always dependent on the data that are provided. A top performing algorithm will not be as accurate on blurry images as on perfectly framed and in-focus exemplar images. This is why in the broader field of computer vision, there are established reference datasets such as the MNIST dataset of hand-written digits  (LeCun et al. 1998 , dataset available at http:// yann. lecun. com/ exdb/ mnist) which are used to compare algorithms against each other. These do not yet exist in conservation-focused computer vision yet; and as such, the reported accuracy numbers in Table  3  reflect both the algorithms being presented as well as the datasets described in the cited sources.