dataesr/scientific-paragraphs-categorization huggingface dataset:
If all cows demonstrated comparable levels of consistency in their daily time budgets, then reducing the effective sample size of our simulated data sets through a subsampling routine would increase the ensemble variance estimates. This, in turn, would make our approximation of measurement error hyper-conservative, but this increase would be uniform across all cows. If, instead, some cows were less consistent in their time budgets across days, then the sampling error imposed by the subsampling routine would be greater, resulting in a larger ensemble variance estimate. Thus, we would expect a stronger penalty to be applied to cows who demonstrated greater plasticity in their behavioral responses to both transient and persistent changes in the production environment. For small datasets with a limited number of replications, the number of subsamples could be set quite close to the size of the complete sample, and would thus emulate a jackknife approach to variance estimation  [9, 10, 28].
For larger datasets, however, the subsample size could be set smaller, to make the resulting ensemble variance estimates progressively more sensitive to the uncertainty in the underlying behavioral signal. To evaluate the empirical performance of these dissimilarity estimators, distance matrices were calculated for the 177 cows with complete CowManager time budget records. Euclidean distance and KL Distance were calculated using base R utilities, with speed up options utilizing the Rfast package  [23, 29]. An ensemble-weighted dissimilarity matrix was first calculated using simulated values accounting only for measurement error using the most conservative joint Dirichlet-multinomial sampling scheme, hereafter referred to as noise-penalized distance. A second ensemble-weighted dissimilarity matrix was then calculated using the same sampling scheme for measurement noise but aggregated over a 14-day subsample to account for behavioral plasticity in daily time budgets, hereafter referred to as plasticity-penalized distance.
The LIT package provides users a clustering visualization utility, which converts dissimilarity matrices into a dendrogram using the hclust utility in base R with default Ward D2 linkage  [23] , and the generates heatmap visualizations of the resulting clustering results using the pheatmap package  [30]. Heatmaps were generated on a grid of cluster values from k = 1. 10 for each of the four dissimilarity estimators, with complete results provided in Supplementary Materials, the results for k = 10 clusters are provided. The LIT package also provides users with a plotting utility to visually contrast the broader patterns between behavioral encodings. Outputs from the clustering utility are passed in to create a contingency matrix generated using ggplot2 with cells colored by their corresponding cell count  [31].
Improving Tree Pruning Decisions through Simulation An optimal encoding strategy seeks to minimize the loss of relevant information by retaining as much of the underlying deterministic signal as possible, while hemorrhaging only noise  [26]. In a hierarchical clustering framework, this is achieved by pruning the dendrogram built from the dissimilarity matrix at the point where the branches cease to represent differences in the underlying signal. Standard pruning strategies allow users to either: (1) provide a dissimilarity cutoff, below which value all further branches are Sensors 2022, 22, 1 grouped into the same bin, or (2) extract the first K branches of the tree  [9, 10]. As with the default Euclidean distance dissimilarity estimator, this approach may be appropriate for datasets with relatively homogenous variance structures.
For data drawn from intrinsically heterogenous distributions, however, the branch lengths cannot be directly compared across the domain of support, making globally-defined pruning rules a suboptimal strategy for analysis of time budget data. More fundamentally, a homogenous pruning strategy may be too simplistic for many PLF sensor datasets, for which the underlying signal often represents a complex composite of behavioral mechanisms that operate at multiple scales. Although some environmental factors might be expected to have an impact on cattle behaviors that are uniform across the herd, other factors might elicit responses that differ in magnitude for different subgroups within the larger population, or even become isolated within smaller social cliques. For example, we might expect the number of times cows are moved each day for milking will place similar constraints on the time left to lie down across all animals, but overstocking with respect to stall spaces might have a much larger magnitude of impact on the lying patterns of subordinate heifers than the more dominant older cows  [33].
In such a complex system, we would expect the heterogeneity imposed by the underlying biological signal to differ in scale across the dataset. Subsequently, in attempting to employ a global cutoff decision to encode information for such a dataset, we would always be faced with the difficult decision to either ignore the subtler behavioral patterns present in some branches of the tree, or else allow noise to contaminate our encoding of other branches with intrinsically coarser behavioral patterns. Although all the components that contribute to the signal in a complex livestock system might be difficult to anticipate a priori, we propose that a more dynamic pruning algorithm might still be achieved, by again employing flexible simulation-based approaches to emulate the comparably simpler sources of uncertainty.
HClustering is then performed independently on each data mimicry in either ensemble, and the first k branches are extracted to create an ensemble of discrete encodings. Under the alternative hypothesis, a strong signal should produce a robust tree structure such that, even after the addition of simulated noise, the resulting encoding would still closely mirror that of the original observed data. As the stochastic component of a dataset becomes stronger relative to the signal, these bifurcation points will become progressively less stable, and the subsequent encodings less reliably aligned with the original data. When the signal falls below the resolution of the data, the tree structures of the simulated data would then seldom match that of the original data, and so would become poorly distin-Sensors 2022, 22, 1 guished from encodings generated under the null, with no signal component.
We propose that mutual information, which can be calculated without any additional distributional assumptions, can be used to quantify the similarity between the observed data and each mimicked dataset, and subsequently used to determine if simulations under the alternative are distinguishable from the null  [12]. In our study, a bifurcation was determined to be significant if less than 5% of the MI values calculated for data simulated under the alternative hypothesis fell below the 95th quantile of MI values calculated for data simulated under the null. If a bifurcation was instead deemed insignificant, the branch was pruned and all cows within it assigned to the same cluster in subsequent encodings. In evaluating the significance of a bifurcation, it seems intuitive that a k = 2 binary encoding should be utilized.
Under the alternative hypothesis, a strong signal should produce a robust tree structure such that, even after the addition of simulated noise, the resulting encoding would still closely mirror that of the original observed data. As the stochastic component of a dataset becomes stronger relative to the signal, these bifurcation points will become progressively less stable, and the subsequent encodings less reliably aligned with the original data. When the signal falls below the resolution of the data, the tree structures of the simulated data would then seldom match that of the original data, and so would become poorly distinguished from encodings generated under the null, with no signal component. We propose that mutual information, which can be calculated without any additional distributional assumptions, can be used to quantify the similarity between the observed data and each mimicked dataset, and subsequently used to determine if simulations under the alternative are distinguishable from the null  [12].
Given that the joint probability distribution under the null, which is randomly permuted to intentionally remove any nonrandom features in the sample, can be expected to have a more uniform distribution of probability than the observed dataset, we can anticipate that the magnitude of the bias may differ between these two distributions as the sample becomes more granular, preventing MI estimates from being directly comparable. To overcome this issue, the LIT package by default provides entropy estimators based on the Maximum Likelihood frequency estimates, but allows users to select from a range of bias-corrected frequency estimates available in the entropy package  [35]. Based on the simulation work by  Hausser and Strimmer (2009) , the JS "shrink" estimator was used in our study to conduct bias-corrected mutual information permutation tests  [36].
Some systematic heterogeneity was also revealed across the herd in the high activity axis, despite its lower magnitude. Unfortunately, the KL distance also appears to have over-stratified cows whose time budgets lie at the extremes. In particular, the cows with extremely low time spent eating (clusters 6-8) were divided into clusters that are likely too small and narrowly defined to facilitate cross-sensor inferences in downstream analyses, and thus may obscure important behavioral dynamics in this dataset. The KL distance heatmap is also annotated on the row axis with the variance in observed daily time budgets for each behavioral axis. Given that time budgets have been normalized here and expressed as proportions, the resulting variance terms were quite small in magnitude (less than zero), and so have been re-expressed on a log-scale, where an increasingly negative value represents a smaller relative magnitude of variation.
This result is not necessarily surprising, given the extended observational period over which sensor records were recorded. With over 1500 min of observation for each cow, even in using a relatively conservative simulation strategy that very likely overestimated the noise intrinsic to this sensor, we should expect by the CLT that the standard error attributable to measurement error would ultimately be quite small after averaging over so many sampled timepoints. Subsequently, these results reinforce that the sensors themselves should impose few limitations on downstream inferences for this dataset, and that inconsistencies in the environment and the animals themselves should be the true limiting factor for the resolution of this encoding. The extremely fine encoding with 38 stochastically validated clusters demonstrates that, with so many recorded observations over this extended observation window, the accuracy of the sensor itself should impose few constraints on our behavioral inferences.
Supplementary Materials: The following are available online at https://www. mdpi. com/article/ 10. 3390/s22010001/s1: A zip file containing an RMarkdown document with full documentation of all code development and files containing all rendered data visualizations. The most current version of the LIT package can also be downloaded at https://github. com/cgmcvey/LIT, accessed on 16 December 2021.
Sensors 2021,  21, 8060  The labels used for data annotation are presented in Tables  4  and 5. These two sets of labels distinguish the two datasets introduced before, namely general behaviour and standing/lying behaviour. Considering both label sets, approximately 1. 777 h of cow behaviour have been annotated by two experienced observers (working part-time in morning and afternoon shifts). The distribution of the annotated actions is presented in Figure  4  for general behaviour actions and in Figure  5  for standing/lying behaviour. around the time of calving (one and four days, respectively). This approach however is not practical for application where monitoring is less frequent, such as extensive livestock farms where large herds are held. Therefore, our research was focused on the long-term monitoring of the pregnant animals, which extended up to two months after calving.
Using this approach, we could analyse the individual behavioural change during different stages of pregnancy, which previous research has proved can differ greatly between individuals  [21]. The labels used for data annotation are presented in Tables  4  and 5. These two sets of labels distinguish the two datasets introduced before, namely general behaviour and standing/lying behaviour. Considering both label sets, approximately 1. 777 h of cow behaviour have been annotated by two experienced observers (working part-time in morning and afternoon shifts) The distribution of the annotated actions is presented in Figure  4  for general behaviour actions and in Figure  5  for standing/lying behaviour. General behaviour (Table  4 ) action prediction  [33]  has been explored during the study as additional input for calving prediction.
g. , walking-standing), but other may occur both standing and lying. For this reason, a second set of labels (Table  5 ) only accounting for lying and standing postures has been used to ensure a correct classification of these two postures when needed due to the importance of lying bouts detection for calving detection (as discussed in the introduction). The annotated data from human observations show a small delay between the change of action of the cow and the annotation of this new action due to the response time of the observer and their need to operate the annotation software. For this reason, the datapoints recorded two minutes before an action change were ignored. This time period was chosen as a balance between loss of data and minimizing mislabelled data in the dataset. As cows did not change actions with high frequency in the recorded labels, two minutes resulted in enough certainty without losing a significative volume of data for each action.
• Standing/lying behaviour annotated data (Table  5 ). This dataset, although smaller compared to b, serves for statistical learning tasks, as well as semi-supervised learning techniques. The experiments and algorithms have been developed using Python 3. 7  [34]  along several libraries, mainly: Pandas  [35] , Keras  [36] , Numpy  [37] , and SciPy  [38]. Figures have been plotted using the Seaborn [39] library.
A very important aspect of the developed calving detection algorithm is its simplicity. This algorithm must be programmed in the microcontroller housed in the collar placed on the cow's neck, which represents a significant limitation in the microcontroller's computing and memory fields. Furthermore, a coarse estimate of the calving date in the receiving end of the calving prediction is also useful to use in conjunction with the algorithm, as it ensures the rejection of any naturally inviable false positive from the algorithm (i. e. , calving detection one month before and after mount). The dataset used in previous studies  [27, 40] , usually includes data in a small temporal window around calving (1-5 days). The data collected in our study enable the back test of algorithms in a much more extensive temporal window, something essential to validate any algorithm that would run in a real-world environment, such as the proposed sensorized wearable device.
Furthermore, a coarse estimate of the calving date in the receiving end of the calving prediction is also useful to use in conjunction with the algorithm, as it ensures the rejection of any naturally inviable false positive from the algorithm (i. e. , calving detection one month before and after mount). The dataset used in previous studies  [27, 40] , usually includes data in a small temporal window around calving (1-5 days). The data collected in our study enable the back test of algorithms in a much more extensive temporal window, something essential to validate any algorithm that would run in a real-world environment, such as the proposed sensorized wearable device. Although the proposed algorithms of this study are focused on calving prediction, the developed sensorized wearable device and the collected data enable the development of different algorithms that could be of great help to farmers both in extensive and intensive livestock.
One hurdle which can impact extraction of useful phenotypic information is the processing and analysis of these images which can be time consuming and therefore costly, especially if there is a need to re-analyse historic databases to measure newly emerging phenotypes. Machine learning and artificial intelligence have been successfully implemented to increase phenotype detection speed and accuracy within many different medical areas including brain cancer detection, COVID status in lungs and classification of organ deformities  [15] [16] [17]. Recently the same technology has been applied to areas of agricultural science such as detection of bovine tuberculosis status based upon milk spectral data  [18, 19]. Briefly put, these networks work by passing data such as images, or segments thereof, through a series of layers containing artificial neurones which determine the likelihood of visually similar animals such as pigs, sheep or horses on a scale of 0 (absent) to 1 (present).
GAN Training The GAN network trained in this study is an implementation of AUTOMAP  [37]  and Pix2Pix  [25]  which has been optimised for use with paired image datasets  [38]. This particular GAN was chosen for this study as it was designed from the ground up to process paired sets of images, such as those commonly found in the medical field where an image can be altered to produce a "before" and "after" whilst maintaining the same subject ID and type, e. g. , sheep-sheep, human-human, in a conditional synthesis process. This is in contrast to other popular GANs, such as CycleGAN and DCGAN, which perform unconditional synthesis by capturing key style concepts, from large batches of example images to translate images between two highly different abstract style concepts such as horse-to-zebra, photograph-to-Van Gogh or sketch-to-cat  [39, 40].
A dataset containing 126 raw and ground truth image pairs of mixed breed ovine CT scans taken from 2019-2020 were used for GAN training (Supplementary File S1). DICOM pairs were first split into training (n = 101) and validation (n = 25) datasets (80% and 20%, respectively). The raw and ground truth pairs of DICOM filename IDs were first given a suffix of "_0" or "_1", respectively, to act as identifiers. All file extensions were then Sensors 2021, 21, 7268 modified to ensure compatibility with the DICOM processing libraries used in this study. The script used to train the GAN, along with the full list of GAN settings used for this study, is available within Supplementary File S2. Key settings for training the GAN were as follows: random translation = 0, epochs = 100, weight for L1 reconstruction loss = 0, weight for L2 reconstruction loss = 10.
Computing Hardware and Software The training of machine learning models can be an intensive computational task which typically requires powerful graphics processing units (GPUs). As such, all computation was performed on an NVIDIA DGX Station workgroup server  [45]. The DGX workstation provided supercomputing performance with one out of a total of four TESLA V100 GPUs being used for computations underpinned by an Ubuntu operating system. All code was run within a Compute Unified Device Architecture (CUDA) 10. 1 docker container which allows parallelisation of general-purpose processing to be applied to the powerful GPUs. Within this container, the open source learning framework Chainer was used to accelerate creation of the neural networks  [46]. The GAN trained in this study is an implementation of AUTOMAP  [37]  and Pix2Pix  [25]  which has been optimised for use with paired image datasets  [38].
Images Produced from Trained Model The trained model was able to perform the major structural alterations within the image dataset needed to transform the raw CT scans into something which, by eye, strongly resembled the ground truth images as shown below in Figure  3. Image IDs 1732, 9638 and 8353 were chosen to illustrate this transformation since, on visual inspection, they contained the largest area of features needing to be removed (large testes and a large scanning cradle). The trained model was able to perform the major structural alterations within the image dataset needed to transform the raw CT scans into something which, by eye, strongly resembled the ground truth images as shown below in Figure  3. Image IDs 1732, 9638 and 8353 were chosen to illustrate this transformation since, on visual inspection, they contained the largest area of features needing to be removed (large testes and a large scanning cradle).
Image Similarity Metrics Confirm a High Degree of Similarity Just as raw and ground truth image histograms were compared previously, likewise the ground truth and predicted images were compared in a similar fashion which revealed two histograms, highly similar, showing a large proportion of overlap and a high degree of similarity from visual inspection. The likeness of the raw, ground truth and predicted image sets (n= 32) was compared pairwise using MSE and SSI. Both raw vs. ground truth and raw vs. predicted showed the lowest image similarity values with an average MSE of 58,674 ± 17,766 and 58,008 ± 17,319 and with average SSIs of 0. 49 ± 0. 025 and 0. 48 ± 0. 024, respectively, indicating a high degree of image dissimilarity. On the other hand, comparing images in the ground truth and predicted datasets showed a much lower average MSE (1028 ± 1201) and a far higher average SSI of 0.
Leg Tissue Composition Tissue abundance and distribution of fat, muscle and bone, within the single 2D image analysed, were calculated by counting pixels which fell within experimentally determined tissue pixel intensity windows compared to the total tissue area. Binary visualisation of these tissue value windows allowed rapid profiling of tissue distribution as seen below in Figure  5. Using this method, tissue abundances were calculated for each medical image in terms of both area and percentage composition (Figure  6 ). On average, the area of bone, muscle and fat across the dataset was 6488 ± 533, 44,274 ± 4051 and 5712 ± 1377 mm 2. Carcass tissue composition percentage-wise for bone, muscle and fat was 11. 52 ± 0. 78, 78. 41 ± 1. 90 and 10. 07 ± 2. 03%.
Phenotype Extraction Accuracy Phenotypes from both predicted and ground truth datasets were extracted using the computer vision pipeline and compared to determine the suitability of predicted images for phenotype determination as seen below in Figure  8. Across all phenotypes, the average values were on average 101. 44% that of the ground truth value with a standard deviation of 12. 90% (n = 32). Muscle % was the most accurate predicted phenotype with estimated values between 93. 67 and 106. 65%. On the other hand, calculated fat area was the least accurate predicted phenotype with estimated values between 42. 50 and 156. 18% (following incomplete ovine testes removal from image ID 8346, fat-related phenotypes were not included in accuracy calculations as testes are calculated as fatty tissue.
All other phenotypes for this image were recorded normally such as muscle area, bone area and gigot geometry). Sensors 2021, 21, 7268 age analysed, were calculated by counting pixels which fell within experimentally determined tissue pixel intensity windows compared to the total tissue area. Binary visualisation of these tissue value windows allowed rapid profiling of tissue distribution as seen below in Figure  5. Using this method, tissue abundances were calculated for each medical image in terms of both area and percentage composition (Figure  6 ). On average, the area of bone, muscle and fat across the dataset was 6488 ± 533, 44,274 ± 4051 and 5712 ± 1377 mm 2. Carcass tissue composition percentage-wise for bone, muscle and fat was 11. 52 ± 0. 78, 78. 41 ± 1. 90 and 10. 07 ± 2. 03 %.