dataesr/scientific-paragraphs-categorization huggingface dataset:
Deep Model Training An inherent drawback of encoder-decoder networks and deep learning, in general, is that parameters of the neural networks must be learnt, and this requires very large datasets on which to train. Furthermore, the training times can take many hours or even days. For the demonstrated example, the training time was under 6 h using an 11GB NVIDIA 1080Ti graphics card. The encoder part of the network gradually reduces the spatial dimension with pooling layers. While this serves to aggregate the contextual information (i. e. , the key features which reveal the scene composition), the precise location information of objects in the scene is eroded with each spatial down-sizing. However, semantic segmentation requires the exact alignment of class maps and thus, the location information needs to be preserved.
The hyperplane found in the high dimensional feature space corresponds to a decision boundary in the input space. In SVM, the classifier hyperplane is generated based on training datasets. For the proposed approach, the training data is the pixel colour information within the background and biofouling regions, as detected by SegNet in the preceding phase. Although some of the training data is likely to be incorrect since SegNet will rarely achieve 100% accuracy, it is expected that the vast majority of the training data will be appropriately labelled, and this will outweigh the relatively small number of errant training instances. Given a training dataset of l points in the form   1.
( , ) l h h h u v  where h denotes the h th vector in the dataset, uh is a real f-dimensional input vector containing the mean and kurtosis values associated with each region f h u  , and vh is an instance label vector   ( 1 , 1 ) l h v   ; for this example, a value of +1 indicates biofouling and -1 indicates the background, although SVMs can readily be extended to classify multiple classes using multi-class SVMs. To identify the maximum-margin hyperplane in the feature space, the SVM requires the solution of the optimization problem: It is evident from Figure  6b  that while SegNet is effective at identifying the general location of biofouling regions in an image, it does not produce clean segmentations. A popular way to refine the segmentation is to apply a post-processing stage using a conditional random field (CRF).
The linear SVM is used for linearly separable data using a (f -1) dimensional hyperplane in f dimensional feature space  [23] [24] [25]. This hyperplane is called a maximum-margin hyperplane which ensures maximized distance from the hyperplane to the nearest data points on either side in a transformed space. The linear kernel function is the dot product between the data points and the normal vector to the hyper-plane. The kernel function concept is used to simplify the identification of the hyperplane by transforming the feature space into a high dimensional space. The hyperplane found in the high dimensional feature space corresponds to a decision boundary in the input space. In SVM, the classifier hyperplane is generated based on training datasets. For the proposed approach, the training data is the pixel colour information within the background and biofouling regions, as detected by SegNet in the preceding phase.
Although some of the training data is likely to be incorrect since SegNet will rarely achieve 100% accuracy, it is expected that the vast majority of the training data will be appropriately labelled, and this will outweigh the relatively small number of errant training instances. Given a training dataset of l points in the form {(u h , v h )} l h=1 where h denotes the hth vector in the dataset, u h is a real f -dimensional input vector containing the mean and kurtosis values associated with each region u h ∈ f and v h is an instance label vector (v h ∈ {1, -1} l ); for this example, a value of +1 indicates biofouling and -1 indicates the background, although SVMs can readily be extended to classify multiple classes using multi-class SVMs. To identify the maximum-margin hyperplane in the feature space, the SVM requires the solution of the optimization problem: {w, e} = argmin w,b,ξ 1 2 w T w + C l ∑ h=1 ξ h ; C > 0 subject to v h w T ϕ(u h ) + e ≥ 1 -ξ h ; ξ h ≥ 0 (1) The function φ maps the training vectors u h into a higher dimensional space.
Method Mean Accuracy Mean Intersection over Union (IoU) Proposed method 94% 87% REMPS  [24]  88% 75% Texture analysis  [25]  87% 72%   With regards to the last point, while it would be possible to have many different classes for several marine growth species (such as some of those shown in Figure  1 ), it may be better to group similar classes together. In practice, having many distinct classes is often more desirable and produces a segmentation technique of potentially greater utility than a segmentation technique that can only segment into two broader classes. However, having more classes can be problematic if there is high inter-class similarity within the training dataset. As an example, having separate classes for various types of marine growth species will be challenging in cases where there is a high degree of visual similarity between different marine species.
Extension to Other Applications The same concept of using virtual images to train an encoder-decoder for real-world application can be extended to other tasks, some of which are depicted in Figure  10. This is facilitated by the fact that deep learning models are highly repurposable. For instance, a model that is trained on a large dataset of marine growth images can be reused as a crack detection model by simply training the model on crack images instead, such as the study by Maeda et al. 2018  [28]. Very minor changes to the architecture of the model, if any, are necessary. With regards to the last point, while it would be possible to have many different classes for several marine growth species (such as some of those shown in Figure  1 ), it may be better to group similar classes together.
In practice, having many distinct classes is often more desirable and produces a segmentation technique of potentially greater utility than a segmentation technique that can only segment into two broader classes. However, having more classes can be problematic if there is high inter-class similarity within the training dataset. As an example, having separate classes for various types of marine growth species will be challenging in cases where there is a high degree of visual similarity between different marine species. In such cases, a better strategy may be to group similar classes together, i. e. , have a 'soft-fouling' class that includes non-calcareous fouling organisms such as seaweeds, hydroids, and algae, and have another 'hard-fouling' class that includes calcareous fouling organisms such as barnacles, coral, and mussels.
Extension to Other Applications The same concept of using virtual images to train an encoder-decoder for real-world application can be extended to other tasks, some of which are depicted in Figure  10. This is facilitated by the fact that deep learning models are highly repurposable. For instance, a model that is trained on a large dataset of marine growth images can be reused as a crack detection model by simply training the model on crack images instead, such as the study by Maeda et al. 2018  [28]. Very minor changes to the architecture of the model, if any, are necessary. Additionally, marine biologists are often interested in detecting fish and estimating their size using camera systems. Being able to accurately detect fish, even in the presence of challenges such as overlapping fish and poor visibility, is of high value.
This puts an emphasis on devising robust and effective techniques that can interpret scenes with credibility. Deep learning algorithms have already attracted significant interest in other fields owing to the high performances that they can achieve; however, they have not had a significant impact in the domain of underwater imaging as of yet, largely due to the lack of available training data. This study presents a framework for generating large datasets of synthetic images, from which deep neural networks can be trained and applied to tackle real-world problems. To demonstrate this concept, this paper looks at the specific problem of biofouling detection on marine structures. A contemporary deep encoder-decoder network, termed SegNet, is trained using 2500 annotated synthetic images of size 960 × 540 pixels.
Once trained, SegNet is applied to new images and tasked with segmenting the images into one of two classes: soft fouling marine growth and the uncolonized background. The initial segmentation is then refined using an iterative support vector machine (SVM) based algorithm. The proposed technique is validated on a host of real-world underwater inspection images. The results demonstrate that using annotated synthetic images is an effective way to train deep learning based techniques and have particular value for applications where large annotated datasets do not exist. Additionally, marine biologists are often interested in detecting fish and estimating their size using camera systems. Being able to accurately detect fish, even in the presence of challenges such as overlapping fish and poor visibility, is of high value.
Once trained, SegNet is applied to new images and tasked with segmenting the images into one of two classes: soft fouling marine growth and the uncolonized background. The initial segmentation is then refined using an iterative support vector machine (SVM) based algorithm. The proposed technique is validated on a host of real-world underwater inspection images. The results demonstrate that using annotated synthetic images is an effective way to train deep learning based techniques and have particular value for applications where large annotated datasets do not exist.
The parameters controlling the SET (resp. RESET) operation are: SET ( RESET ) the nominal rate, Ea SET (Ea RESET ) the activation energy, SET ( RESET ) barrier-lowering coefficient. The FORMING operation is also perfectly taken into account thanks to a second state variable (r CFmax ) determined with equation 2: *&/ 3. +-( *&/ !" &!" 0 !" , % ')) ( $ (2) where Ea FORM is the activation energy for Electroforming and FORM the nominal forming rate. After model card extraction, the model matches quasi-static and dynamic experimental data measured on HfO 2 -based memory elements. To study reliability issues at circuit level, the device to device (D2D) spread and the cycle to cycle (C2C) variability of OxRAM cells are included in the model.
timetree. org/) [37]. Supportive evidence for HTT was also obtained from the TE dating analysis within each genome. A very similar intragenomic TE dating was found in parasitic nematode and bird genomes: i) 25/23 Mya in the ancestor of Brugia spp. and Wuchereria bancrofti and 4 bird species belonging to Psittacidae, Bucerotidae, Trochilidae and Tinamidae bird orders; ii) 22. 2/17. 7 Mya in the ancestor of Loa loa and other 3 bird species which belong to Suboscines, Trogonidae and Mesitornithidae orders. AviRTE elements have a patchy distribution on the avian tree, being found in bird orders that diverged from 79 Mya (Bucerotidae and Trogonidae) up to 110 Mya (Tinamidae and Bucerotidae orders). Such data suggests 2 retrotransposon HT waves between birds from different orders and filarial nematodes occurred: one in the Oligocene period (25 and 23.
Moreover, such small elements can be easily inserted into viral genomes, active circular forms could be packed into VLPs or even be self-transmissible considering that active episomes can be ingested frequently in a predator/ prey or parasitism relationship  [60] [61] [62] [63] [64]. Altogether, this new data shows that DNA transposons (mainly Tc1-mariner and Helitron) are tuned for both short and long range HTT events and that viral-like TEs do transfers more frequently among close related species probably due to infective constraints imposed by cell receptor hosts species. Such large dataset should also allow further insight when ecological data becomes available for the analyzed species such as ecological relationships and their food chain position in the food web. It could help us to decipher the most important ecological factors which impact HTT frequency and distribution.
Hence, the best approach would be to create a method that accounts for the variation of codon bias and its impact on the dS estimates. With this goal in mind, Wallau and collaborators 2015  [50]  developed VHICA (Vertical and Horizontal Transfer Consistency Analysis), an R package with the implementation of a new method accounting for codon bias. This new method extracts both synonymous substitutions (dS) and codon usage bias (CUB) estimates from codon alignment of vertically transferred host genes and TEs. In a second step, it performs correlation of these values in each species-pair and estimates the expected regression line from the resulting correlation (Fig. 1 ). In the third step, it computes the residuals of VT genes to the expected regression line. TEs residuals from the expected regression line are then calculated and compared with reference genes residual distribution (Fig.
VHICA can be used to analyze several hundred genomes, however, it is important to point out two practical limitation for such analysis: I -VHICA is not a completely functional pipeline in the sense that one need to provide a curated TE and gene datasets input, that is, a multiple codon alignment for each TE family and gene. The challenge here is that codon alignment is not a trivial task especially when considering TEs which are predominantly found fragmented and degraded in the genomes demanding manual curation and/or reconstruction of potential coding region from the most complete copy; II -VHICA was built to analyze each TE family separately, which could be, a time consuming task when TEs from several genomes are analyzed, nevertheless, it is relatively easy to make a loop in the VHICA function to analyze and print the output for all TEs at once.
Databasing and data availability The unprecedented amount of data about HTT is allowing a quantitative assessment of several questions as reported earlier. However, it also poses new challenges such as: I -new findings should be checked carefully against previous findings to remove information duplication; II -in order to make inferences about ecological factors a good databasing is necessary to keep track of all information and connectivity between them; III -the need for benchmarking and performance comparison of the different detection methods; and IV -wide application of standard methods in several taxa. With those new challenges in mind we call all researchers working with TEs and HTT to make available as much data as possible to incorporate into HTT database, thus ensuring that future studies can use the information to check against previous detected transfer and benchmarking new methods and pipelines.
A fallowed field may be confused with cropped fields due to climatic and soil conditions  [20] , cropping techniques  [19] , crop failure  [16] , or may be confused with surrounding ecosystems due to its natural regeneration. Despite these difficulties, examples of methods for fallow characterization using remote sensing data exist in the literature. This characterization is generally performed in three steps: (1) cropland mapping, (2) annual/seasonal fallow mapping inside the cropland mask using satellite time series, and (3) multi-annual analysis of the fallow maps to determine the fallowing frequency, also known as the fallow age. Xie et al. [21]  showed that masking the non-agricultural areas blended with agricultural fields was a critical pre-processing step for distinguishing active crops and fallow lands.
In Niger, Tong et al. [19]  developed a classification approach based on the different phenological characteristics of the vegetation, including Normalized Difference Vegetation Index (NDVI), amplitude, and rate of decrease rate, as monitored by a MODerate-resolution Imaging Spectroradiometer (MODIS) 250 m resolution data. They showed that fallowed fields generally have a higher NDVI and a more rapid decrease than unmanured cropped fields (Figure  2 ). In the Central Valley of California, Melton et al. [18]  tracked land fallowing using a NDVI time series based on Landsat data completed with MODIS data, and monthly ground data collections. They used a decision-tree algorithm to evaluate the information about a set of phenological metrics extracted from the NDVI time series, and the land-use changes relative to previous years, to assign each field to a land-use class.
No accurate discrimination of tree crop types has been achieved without a significant input of field knowledge. Most remote sensing studies that addressed tree crop planting patterns focused on vineyards, and less on olive and nut groves. Even if some studies showed the feasibility of separating different cropping structures, such as simple row vs. pergola (continuous trelliswork) in large vineyards using decametric resolution images  [38]  with ASTER (Advanced Spaceborne Thermal Emission and Reflection Radiometer) data at a spatial resolution of 15 m, most authors highlighted the need for high spatial resolution images, with metric and sub-metric resolution, to detect intra-field cropping structure. The best imagery was acquired from Unmanned Aerial Vehicles (UAV) or UtraLight Aircrafts (ULA).
Torres et al. [47]  developed software (CLUstering Assessment or CLUAS) to provide some quantitative agronomical parameters and environmental indicators in olive groves, such as the number of olive trees, the size of each tree, the fraction of soil covered by trees, or the potential productivity based on density calculations. However, hyperspectral and/or UAV technologies are needed for good accuracy in tree detection  [48]. Stereo imagery at very high spatial resolution, which obtains surface and elevation models, is also promising but only in situations with good contrast between the tree crown and its immediate environment to allow correct matching of the stereo pairs.
These studies were then upscaled to regional-scale maps of these systems over the entire state of Mato Grosso  [63] [64] [65]. In these studies, Vegetation Index (VI) time series are generally pre-processed to limit misclassification due to noisy data. Although the MODIS VI data are made of composite values, low noise VI values may remain in tropical areas with an intense rainy season. To correct these values, various smoothing algorithms can be used  [66] , as well as wavelet analysis  [62]. Most of the studies then train supervised classifications with reference data, usually in the form of ground observations  [49, 63, 64]. Unsupervised classification approaches are somewhat less common but have achieved good results when adopting hyperclustering methods [17,  59, 67]. Clustering approaches do not require ground data, which are time-and resource-consuming to collect, especially when data must be gathered twice during the growing season over large areas.
However, labeling the final classes In these studies, Vegetation Index (VI) time series are generally pre-processed to limit misclassification due to noisy data. Although the MODIS VI data are made of composite values, low noise VI values may remain in tropical areas with an intense rainy season. To correct these values, various smoothing algorithms can be used  [66] , as well as wavelet analysis  [62]. Most of the studies then train supervised classifications with reference data, usually in the form of ground observations  [49, 63, 64]. Unsupervised classification approaches are somewhat less common but have achieved good results when adopting hyperclustering methods  [17, 59, 67]. Clustering approaches do not require ground data, which are time-and resource-consuming to collect, especially when data must be gathered twice during the growing season over large areas.
The local scale is the most documented, as methods can be tested in a more controlled environment with less variability in irrigation practices, with using a considerable amount of ground data and expertise, and uses costly high or very high-resolution imagery without a necessary operational objective. Historically, the first methods principally used photointerpretation, with Landsat 1 imagery alone  [98, 99] , or combined with field boundary information  [100]. These methods are maybe the most accurate, but are the least efficient in terms of time and human resources. Consequently, automated classification approaches were developed, including unsupervised classification  [101] , single or multi-stage supervised classification  [102] , decision tree [103], and supervised learning models such as Random Forest or Support Vector Machines  [104, 105].
Zheng et al. [150]  developed a multitemporal approach to minimize the confounding effects of green vegetation. Using this approach, the authors reported a R 2 of 0. 89 between the minNDTI (the minimum NDTI from a NDTI time series) and crop residue cover in central Indiana, U. S. Moreover, the increasing availability of multiband or multifrequency SAR data allows independent assessment of soil surface moisture, which could then be used to interpret optically-based tillage mapping. Specifically for SAR, Quad-Pol configurations help separate roughness from other factors, such as soil moisture  [151]. With the exception of Radarsat-2, current (like Sentinel-1) and future radar missions (like Radarsat-constellation) do not consider Quad-Pol, or their configuration is infrequent (e. g. , ALOS-2 and TerraSAR-X).
Detection of Harvested Area The detection of harvested crops that are cultivated year round or over a long time period (several months) is a key piece of information for the transformation and commercial sectors. For example, in smallholder sugarcane production regions, updated information on the status of the cane fields or blocks for the whole area serviced by the mill may help to better estimate the effective yield, and better assess the standing cane areas to be harvested, thus optimizing the cutter deployment and transport operations. Only examples on sugarcane harvest detection were found in the remote sensing literature and are summarized hereafter. In 2007, using a SPOT image time series and a sugarcane crop mask, Lebourgeois et al. [152]  mapped the sugarcane harvest on Reunion Island using the ISODATA algorithm with an overall accuracy of 97.
Mello et al. [159]  proposed an alternative automated classification procedure, based on the Spectral-Temporal Analysis by Response Surface (STARS) that synthesizes the full information content of a multitemporal/multispectral remote sensing image dataset, to represent the spectral variation of features on the earth's surface over time. Applied to sugarcane, they obtained high Kappa values (>0. 9 in many cases) for harvest detection and harvest mode mapping. In conclusion, harvest date and mode detection requires the use of a time series, mainly optical, since radar remains ambiguous with soil and plant water status. For crop residue management in Taiwan, Liu et al. [163]  used FORMOSAT-2 satellite images and ground-truth data to conduct a supervised classification of the visible bands only, and calculated the extent of the straw burning areas.