DOI: 10.5281/zenodo.18139538
title: Datasets for the development of a model for automatic trimming of transposable element sequences

description: Datasets for the Development of an Automatic Transposable Element Trimming Model

This repository provides a collection of datasets used for the training, validation, and testing of a deep learning model designed for the automatic identification and trimming of transposable elements (TEs) in genomic sequences of 15,000 base pairs.

Dataset design and case composition

The datasets were constructed to represent a diverse set of biologically realistic scenarios involving transposable elements embedded within genomic DNA. Each sequence belongs to one of the following cases:





Case 1: DNA fragment + first TE + DNA fragment + second TE + DNA fragment




Case 2: DNA fragment + first TE + DNA fragment + second TE + DNA fragment + first TE + DNA fragment




Case 3: Tandemly repeated microsatellites + first TE + tandemly repeated microsatellites




Case 4: DNA fragment + first TE + DNA fragment



All transposable elements included in these datasets were extracted from a curated TE library (r.1.5_all.fasta), comprising sequences derived from 903 unique species.

Training datasets

Two training datasets were generated to evaluate model robustness and reproducibility with increasing data volume:





dataset_9218.fasta:The initial training dataset, containing 9,218 sequences.




dataset_27165.fasta:An expanded dataset containing 27,165 sequences, created to assess whether model performance and learned representations were reproducible at larger scales.



Testing datasets

Model performance was evaluated using independent testing datasets:





dataset_testing.fasta:A testing dataset containing 2,143 sequences, with a uniform number of sequences per case. Model predictions were compared against the true TE positions to compute the R² score.




Species-independent evaluation dataset:A subdataset derived from dataset_27165.fasta, containing only sequences from species not present in the 9,218-sequence training dataset. This dataset was used to evaluate the model’s ability to generalize to previously unseen species.



Sequence identifiers

Each sequence identifier encodes detailed information about the simulated scenario and the embedded TE. Identifiers include:





Case number




Dfam TE identifier




Initial TE position within the sequence




TE length




Species name



All fields are separated by underscores (_).Example:







 



>Caso1_DR003540075_s_1#CLASSI/LTR/GYPSY_6028_3762_Lupinus_angustifolius



Dataset structure

For each dataset, the processed data are provided as four NumPy arrays:





features_data.npy:Numerical representations of the four TE-Aid plots generated for each sequence.




labels_data.npy:Normalized start and end positions (values between 0 and 1) of each transposable element.




case_labels.npy:Header information for each sequence (up to the # symbol).




species_labels.npy:Species identifiers associated with each sequence.





Trained models

Two trained models are provided alongside the datasets. Each model consists of:





Four convolutional neural network (CNN) branches, one per TE-Aid plot, with a ResNet18-like architecture.




A fully connected neural network (FNN) for final prediction.



The available models are:





model_trained_27165: Trained using the 27,165-sequence dataset.




model_trained_9218: Trained using the 9,218-sequence dataset.





Reuse and applications

These datasets and models are intended to support:





development and benchmarking of automatic TE trimming methods,




evaluation of model generalization across species,




and reproducible research in transposable element annotation and deep learning–based genomic analysis.