dataesr/scientific-paragraphs-categorization huggingface dataset:
Presently, $DESIGN in NONMEM can only be used for models with continuous data as the FIM is more complex to evaluate for discrete or time to event models. We have used the Nelder method of search for these examples, but there is a risk of reaching local minima, and other search algorithms such as random search or stochastic gradient search are available in NONMEM's $DESIGN feature which can also be used, sometimes in combination with the Nelder search method. We illustrated in the current tutorial seven examples of basic/standard optimal design, but more specifications are available for design evaluation and optimization in the documentation. However, other specific tools, such as PopED and PFIM, may have more advanced methods if needed.
The Optimized Results, for optdesign2, example 5Note: The -log(det(FIM)) and SE's are obtained from the raw output file(. ext) or the report file, shrinkage is obtained from the. shk or the report file, and sample times are obtained from the $TABLE file (. tab, optimization) or data file (. csv, evaluation). Notice that some of the RSE's are high. RSE's reduce by a factor of sqrt(number of subjects).
8 (EVBSHRINKVR) %Shrinkage(V) 26. 2 38. 8 14. 5 14. 5 15. 3 %Shrinkage(KA) 7. 14 16. 7 2. 48 2. 48 2. 6 Sample Time TSTRAT = 1 1 1. 55 0. 13 0. 13 0. 13 Sample Time TSTRAT = 2 4 3. 69 7. 01 7. 109 (3x) 4. 0 Sample Time TSTRAT = 3 8 -- 7. 05 -- 7. 0 Sample Time TSTRAT = 4 -- -- 7. 11 -- 12. 0 Sample Time TSTRAT = 5 -- 186. 8 159. 3 159. 9 160 T A B L E 3Note: The -log(det(FIM)) and SE's are obtained from the raw output file(. ext) or the report file, shrinkage is obtained from the. shk or the report file, and sample times are obtained from the $TABLE file (. tab, optimization) or data file (. csv, evaluation).
The Optimized Results, for optdesign2, example 5Note: The -log(det(FIM)) and SE's are obtained from the raw output file(. ext) or the report file, shrinkage is obtained from the. shk or the report file, and sample times are obtained from the $TABLE file (. tab, optimization) or data file (. csv, evaluation). Notice that some of the RSE's are high. RSE's reduce by a factor of sqrt(number of subjects). UNINT on Omegas (optdesign2, No UNINT, optdesign2c, FIXED on Omegas Item OFVTYPE = 6) OFVTYPE = 1) (optdesign2d, OFVTYPE = 1) Objective function -42. 182 -103. 694 -42. 327 %RSE(CL) per subject 1. 01 1. 04 1. 04 %RSE(V1) 1. 34 1. 34 1. 34 %RSE(Q) 3. 30 3. 77 3. 49 %RSE(V2) 1. 24 1. 13 1. 30 %RSE(var(CL)) U/F 17. 6 17. 5 - %RSE(var(V1)) U/F 26. 2 28. 0 - %RSE(var(Q)) U/F 37. 6 46. 7 - %RSE(var(V2)) U/F 94.
Note:The -log(det(FIM)) and SE's are obtained from the raw output file(. ext) or the report file, shrinkage is obtained from the. shk or the report file, and sample times are obtained from the $TABLE file (. tab, optimization) or data file (. csv, evaluation). Bayes Objective function -43. 335 -42. 893 %RSE(VC) (per subject) 26. 8 26. 7 %RSE(K10) 29. 4 29. 1 %RSE(K12) 35. 5 35. 5 %RSE(K21) 36. 7 35. 8 %RSE(VM) 26. 2 27. 1 %RSE(KMC) 37. 2 33. 7 %RSE(K03) 29. 1 28. 5 %RSE(K30) 30. 9 30. 9 %SHK(VC) 8. 66 8. 50 %SHK(K10) 20. 9 20. 0 %SHK(K12) 37. 1 37. 5 %SHK(K21) 39. 8 38. 5 %SHK(VM) 8. 08 13. 3 %SHK(KMC) 50. 8 41. 2 %SHK(K03) 18. 1 16. 4 %SHK(K30) 28. 7 30. 6 Optimal Dose 5. 334 mg/kg 5. 189 mg/kg Fixed Sample Time 0 0 predose (CMT = 3) Sample Time (CMT = 3) 0. 01 (redundant --- with fixed predose) Sample Time (CMT = 1) 0.
In the field of time series, many machine learning methods such as support vector regression (SVR), neural networks, classification and regression tree (CART), and k-nearest neighbor (kNN) regression were proven able to model and forecast time series as well  [19, 20]. There are some discussions on comparing the performance of different forecasting approaches. Ahmed et al. [21]  performed an empirical comparison of eight machine learning models over the 1045 monthly series involved in the M3 Competition, but only onestep-ahead forecasting was considered. Makrirdakis et al. did some similar works  [22] , comparing statistical and machine learning methods, but without any decomposition method being introduced as a preprocessing step. Using the M1 Competition dataset, Theodosieu  [23]  compared a new STL-based method with some common benchmarks but without combining STL with them, and only up to an 18-month forecasting was considered.
Dataset We selected 332 monthly series from the industry category which contains the highest number of points per series from the M3 Competition dataset. We set 84 as the length of the historical data and tested five different forecast horizons, i. e. , 1, 6, 12, 18, and 24 months. Thus, the total length required for an appropriate series is 108. The two series N2011 (78 points) and N2118 (104 points) were thus removed from the original 334-series dataset.
Deep Learning Methods The two deep learning models were constructed in Python 3. 8. 5 with the Keras-2. 4. 0 framework  [35]  under TensorFlow-2. 3. 1  [36]. The hyperparameters were empirically tuned. For CNN-LSTM, we stacked one CNN layer by two LSTM layers and three dense layers. The CNN uses a ReLU activation function and has 16 filters, where each filter has a kernel size of 5. Each LSTM layer has 128 units, and the two following dense layers have 32 and 16 units, respectively. The number of units of the last dense layer is identical to the forecast horizons. For ConvLSTM, we stacked two ConvLSTM layers, followed by three dense layers. Each ConvLSTM layer has 128 filters where each filter has a shape of  [1, 2]. The three dense layers are identical to those of CNN-LSTM.
The recommendations specifically address principle 7 on Specificity and Verifiability of the Joint Declaration of Data Citation Principles  [7]  and aim specifically at providing an automated, machine-provided and machine-actionable solution in addition to being human-readable. The recommendations center around the principle of ensuring evolving data is timestamped and versioned, with subsets being identified via time-stamped queries that are stored. By assigning a persistent identifier (PID, e. g. a Digital Object Identifier -DOI) to these queries they become resolvable and can be re-executed transparently against the time-stamped database to recreate the exact same subset that was initially selected. This eliminates the need for pre-defined subsets that are frozen at pre-defined intervals, avoids data duplication and is transparent to the researcher, while at the same time being applicable to virtually all types of data, such as databases, spreadsheets, collections of files or an individual image.
Similarly, queries can come in any form, ranging from SQL database queries to marking an area on an image. After a one-year intensive evaluation, both conceptually, as well as via reference implementation and actual operational deployments, these recommendations were officially endorsed by the RDA in September 2016 and are currently listed to become a European Technical Specification. They are further being referred to in an increasing number of standards and reference guidelines such as ISO 690:2010, Information and documentation -Guidelines for bibliographic references and citations to information resources  [19]  and the ESIP Data Citation Guidelines for Earth Science Data Version 2  [11]. Given the experiences gained from several further implementations and deployments, as well as the discussions taking place at a variety of workshops and meetings of the Working Group, it seems timely to review the various approaches that different institutions have taken to implement a data identification and citation service, compiling the lessons learned in order to provide guidance and good-practice examples to other repositories wishing to offer such services to their customers.
The remainder of this paper is structured as follows. Section 2 provides a brief review of the recommendations, followed by a descriptions of several proof-of-concept reference implementations in Section 3 that show the feasibility of the approach in a range of different data settings. Section 4 describes several operational implementations of the recommendations from a range of disciplines (e. g. Medicine, Earth Sciences, Astrophysics), and a range of different data types (relational databases, NetCDF files, repositories of image data) that vary in size and data dynamics. Section 5 discusses the lessons learned and identifies open issues given the current state of the recommendations and their deployment. Finally, Section 6 summarizes the current state of affairs and provides an outlook onto future development.
Compute a checksum of the normalized query to efficiently detect identical queries. -R5 -Stable Sorting: Ensure that the sorting of the records in the data set is unambiguous and reproducible. -R6 -Result Set Verification: Compute fixity information (checksum) of the query result set to enable verification of the correctness of a result upon re-execution. -R7 -Query Timestamping: Assign a timestamp to the query based on the last update to the entire database (or the last update to the selection of data affected by the query or the query execution time). This allows retrieving the data as it existed at the time a user issued a query. -R8 -Query PID: Assign a new PID to the query if either the query is new or if the result set returned from an earlier identical query is different due to the changes in the data.
• Upon modifications to the Data Infrastructure -R13 -Technology Migration: When data is migrated to a new representation (e. g. new database system, a new schema or a completely different technology), migrate also the queries and associated fixity information. -R14 -Migration Verification: Verify successful data and query migration, ensuring that queries can be re-executed correctly. The recommendations are applicable for all types of data representation, be it individual images or commaseparated value (CSV) files, collections of files such as PDF documents, source code or NetCDF files, structured data in relational databases, semi-structured data in XML files, linked data in triple stores, and more. The recommendations also apply for all kinds of data characteristics, be they massive repositories of satellite imagery or small collections of aggregated statistical information, highly dynamic data and continuous sensor data streams or entirely static data that never changes.
Proof-of-Concept Implementations We developed four proof-of-concept, reference implementations for different data storage scenarios. These cover (1) relational databases in a solution relying on MySQL  [31, 30] , (2) a Git based approach to deal with file-based data, e. g. CSV files  [30] , (3) two different approaches to XML databases  [17] , and (4) an approach integrating a NoSQL database into the CKAN repository system. We review these briefly to provide some context for the operational pilot deployments described thereafter, including pointers to source code.
Relational databases Relational database management systems (RDBMS) are a well-established and well-understood technology based on the strong theoretical foundation of set theory. The four fundamental operations known as CRUD allow basic data manipulation by offering functions for creating (INSERT), reading (SELECT), updating (UPDATE) and deleting (DELETE) records. These operations are essential for interacting with the structured data which is stored in tables inside databases. RDBMS use transactions for ensuring the Atomicity, Consistency, Isolation, Durability (ACID) principle. While this powerful feature ensures that changes are visible to other transactions in a consistent way, it does not provide any temporal information that would allow reconstructing the state of a data set without additional metadata.
The SQL 2011 standard  [13]  introduces features for temporal data support, automatically creating history tables and adding time periods of record validity, thus taking care of versioning and time-stamping "out-of-thebox". As support for these temporal tables is still limited, we describe how a data citation mechanism can be implemented regardless of the SQL standard compliance and independent from vendor implementations. [31]  To establish data citation as an accepted practice, the implementation of a data citation model should be as convenient and non-invasive and as possible, but often database queries are deeply coupled with the application code base or external tools. Thus changing queries and interfaces is cumbersome and often not feasible. Here we describe the benefits and limitations of three approaches towards addressing these challenges and handling the metadata required for data citation: (1) integrated, (2) history table, and (3) hybrid.
This approach requires a major redesign of the database, as well as the interfaces of programs accessing it. As an advantage, the retrieval of the earlier data state then requires a simple query that selects the latest version per record and omits deleted records. From a storage perspective this approach only produces low overhead as records only append timing and versioning data. Dedicated history table : The second approach implements a full history using a dedicated history table. Records that are inserted into the source table are immediately copied into the history table which is extended by timestamp-added and timestamp-deleted columns. Deleted records are removed from the original table and only marked as deleted in the history table. An advantage of this approach is that the original tables remain unchanged, incurring no changes to their interfaces.
There is also no impact on the database performance as the source table only store the live data. All requests for data citation are handled by the history table. Hybrid: For the hybrid approach, a history table has to be added for all tables. The history table, extended again by timestamp-added and timestamp-deleted columns, is used for storing all records that are updated or deleted from the source table, thus keeping only the latest version in the source tables. The original table always reflects the latest version, whereas the history table records the evolution of the data. The advantage of this approach is a minimal storage overhead, especially for append-only scenarios. A disadvantage is a more complex query structure for retrieving historical data because the system needs to check whether updates exist and then retrieve the records either from the original source or history tables.
Yet, in settings where the re-execution of historic queries is a low-frequency event, this might be the preferred solution. This is thus also the solution specified in the SQL2011 standard and rolled out as temporal tables by several wide-used database engines including DB2, PostgreSQL, Teradata, Microsoft SQL Server, Oracle and others. A crucial aspect of result sets is their sorting. The results returned by a database query are commonly fed into subsequent processes in their order of appearance in the result set. In situations where the order of processing has some effect on the outcome (e. g. in machine learning processes), we need to ensure consistent sorting of the result sets is provided. Hence, the order in which records are returned from the database needs to be maintained upon a query's re-execution.
This is challenging as relational databases are inherently set based. According to the SQL standard, if no ORDER BY clause is specified, the sorting is implementation specific. Even if an ORDER BY clause is provided, the exact ordering may not be defined if there are several rows having the same values for the specified sorting criteria. For this reason, queries need to be re-written to include a sort on the primary key prior to applying any user-defined sort. The reference implementation was developed for the MySQL database engine 2  but will work for any relational database management system supporting the SQL standard. The Posgres database engine natively features support for temporal tables.
File-based data via Git Comma separated value (CSV) files are present both in small and big data scenarios, but there is no common approach to identifying versions or subsets of the data within the files. All steps required to create citable subsets can be fully automated, storing queries or scripts that created the subsets. In order to shift the identification mechanism from the data set level to the query level all three establish a link between versioned data sets at a specific time and the query as it was executed at that point in time. The solution relies on an underlying Git repository  3. Git is a distributed version control system for textbased artifacts, like source code. In the last decade Git evolved to the de-facto standard for managing source code, allowing for a variety of contribution processes  [14].
The first approach offers a simple solution to storing reproducible data sets within Git repositories. However, the repository has to ensure that the scripting language used to identify the subset is maintained. In addition to any script-like interfaces, support for SQL-like query languages can be provided as well. Since both CSV and SQL are based on a tabular view of data, CSV data can be easily mapped into relational database tables and accessed via library functions supporting SQL-like queries on CSV files. Hence the translation process of a CSV subset selection process can be mapped to an SQL query. Thus we are able to provide support for sub-setting using an SQL-like query language that can be executed on CSV files with the help of additional libraries like CSV2JDBC 4. On top, a wide range of tools and libraries exist for manipulating CSV data.
XML databases A challenge when dealing with XML data is its inherent tree-like structure. Thus,  [17]  identified an approach to make XML data sets stored in native XML databases citable. This method also makes use of timestamps, versioning, PIDs and query stores similar to the methods previously described. The reference implementation works with two different open source native XML database engines, namely (1) BaseX 7  , and eXist-db 8  , both supporting XPath and XQuery but differing in their syntax and behavior. Furthermore, two different approaches to versioning and timestamping have been realized and evaluated, one relying on branch copying, the other on capturing the versions via parent-child relationships. Both approaches suffer from the complexity induced by the fact that a range of different scenarios need to be covered to support operations both on the hierarchy (element nodes) as well as attribute levels.
All other operations (insert, deletes and replacements) of both nodes and attributes are handled accordingly. For the second approach, only insert timestamp attributes are added for the top-level node that is added to the hierarchy. Delete attributes are only added to the element node being deleted, marking the entire branch as invalid. Attributes and text nodes are versioned via versioning blocks which are children of their respective element nodes. The existence of such a block indicates that edits happened to any of its elements. While the first approach is conceptually simpler to implement it obviously carries a massive storage overhead, especially when the XML data set represents deep hierarchies. Also, the query execution times are lower for the second approach, mostly due to the inefficiency of copying large branches in deep hierarchies, that make up for the more complex processing required for the individual updates, with eXist-db showing significantly lower performance than BaseX (9-15 sec for updates as opposed to 0.
NoSQL based data citation support added to CKAN This reference implementation demonstrates a solution for NoSQL databases, in this case MongoDB 10 , a popular open-source NoSQL database engine. It is integrated into CKAN, a prominent Open Source data portal platform 11 , but also integrates links to a source code repository to connect and serve both code and data. By also integrating a handle service it provides a full solution for supporting dynamic data citation within a data portal. It consists of four independent applications, a CKAN instance, a CKAN datapusher service and a Source Code Repository and a Handle registry service. Furthermore, there are four different storage containers. The CKAN instance requires a storage to store all uploaded files and a database to maintain application-specific data (e.
g. credentials) and meta data of resources. These two containers are required by a default CKAN installation, the remaining two storage containers, data and query store, are specific to this solution. The CKAN extension consists of two plugins, the mongodatastore and the reclinecitationview plugin. The first one is a novel DataStore implementation which aligns with the RDA Recommendations for Data Citation. The second plugin was needed to additionally add the citation feature to the view were data sets can be queried. Data in MongoDB is organized in collections that consist of several JSON objects which are referred to as documents. Using the terminology of relational databases, the document corresponds to one row of a database table and each attribute represents a column and its associated value.
Every document has an id attribute by default, which is of type ObjectId and maintained by the database. ObjectIds are 12-byte values consisting of a 4-byte timestamp, a 5-byte random value and a 3-byte incrementing counter. In order to maintain older versions of a data set, the data store must not overwrite data records once they are updated. Therefore, updates are added as new documents to the collection while the older version of the record remains unchanged. In other words, one document in the collection represents the state of a data record for a certain period of time. This requires another id in order to define the relation between data records and its states. In order to define which point of time a certain document was valid, two additional attributes, created and valid to, are added to the documents.
The first attribute indicates when the document was added to the collection and the second attribute indicates until when it was valid. The VersionedDataStoreController encapsulates access to the MongoDB in a way that all changes (inserts, updates and deletions) to the data set are tracked. MongoDB queries are represented as JSON documents. Each attribute defines a filter that is applied on all documents. A query extension approach was chosen to restore a historic state of the data set on querytime. The idea is that for each query conditions are added that only apply to the records that where valid at a certain point of interest. In order to detect semantically equal queries, they have to be transformed to a normalized form. As for the queries the order of their attributes has no impact on the query result, they are sorted alphabetically by key before they are processed by the database.
In order to re-execute queries that were submitted to the data store in the past, all queries including their metadata are stored into a relational database. It contains all data that is required to re-execute a historic query and subsequently validate the result set, i. e. hash values for the query, result set and assigned record fields to identify query duplicates. As different scientific communities may require different information for citing data sets, the information on record fields is stored as key-value entries in a separate table. The entire software package, including the plug-ins as well as test data and evaluation scripts, is available as OpenSource software on Github 12.