Package: rrequested-demultiplex
rrequested-demultiplex-General purpose and applicabilityrrequested-demultiplex is the python package distribution of the modular shellscript tool RREQUESTED (Raw REads QUality Extraction, Size Trimming and Ex-novo Demultiplexing). It is aimed at preprocessing (quality filtering, size selection and demultiplexing) of raw basecalled reads, especially produced by Oxford Nanopore, a Third Generation Sequencing technology.Even though the program has been tested only on Nanopore sequencing results, the program could be cautiously applied also to products from NGS or other ThirdGen technologies.-InstallationThe basic requirment for this installation is to have python 3.10 installed on your machine.To install the package, just run:pip install rrequested-demultiplexYou can retrieve general information on it by running:pip show rrequested-demultiplex-Functions and usageModuleClassMethodDescriptionfile_systemFileSystemmakedir_orchangeMethod returns the provided path after attempting to create the directory, handling the case where the directory already exists.file_systemFileSystemget_base_dirMethod extracts information such as the base directory, base name, and the base directory of the base directory from the provided file path, using the os.path.splitext function and string manipulation.quality_filterQuality_Filterload_dataMethod reads data from the input file, handling both gzipped and non-gzipped fastq formats.quality_filterQuality_Filterascii_conv_and_meanMethod converts ASCII characters to Phred quality scores and calculates the mean.quality_filterQuality_FilterfilterMethod performs the filtering based on the quality threshold and updates the input file with filtered reads, providing a summary of the process.size_filterSize_Filterload_dataMethod reads data from the input file, handling both gzipped and non-gzipped fasta/fastq formats.size_filterSize_Filtersize_filterMethod performs the size-based filtering and writes the selected sequences to a new file, providing a summary of the process.unref_demultiplexUnreferenced_demultiplexingfind_the_numMethod finds highly divergent sequences from the input file.unref_demultiplexUnreferenced_demultiplexingfind_the_num_listMethod finds highly divergent sequences from a given list of sequences.unref_demultiplexUnreferenced_demultiplexingdemultiplexMethod performs the demultiplexing process and writes the demultiplexed sequences into separate files.Here are some example usages of the package:Size_Filter# Import necessary modulesfromrrequested_demultiplex.size_filterimportSize_Filter# Specify input file, minimum size, and maximum sizeinput_file="your_input_file.fastq"min_size=20max_size=200# Create an instance of Size_Filtersize_filter_instance=Size_Filter(infile=input_file,minsize=min_size,maxsize=max_size)# Load data from the input filedata=size_filter_instance.load_data()# Perform size filteringfiltered_data=size_filter_instance.size_filter()Quality_Filter# Import necessary modulesfromrrequested_demultiplex.quality_filterimportQuality_Filter# Specify input file and quality thresholdinput_file="your_input_file.fastq"quality_threshold=20# Create an instance of Quality_Filterquality_filter_instance=Quality_Filter(infile=input_file,quality=quality_threshold)# Load data from the input filedata=quality_filter_instance.load_data()# Perform quality filteringquality_filter_instance.filter()FileSystem# Import necessary modulesfromrrequested_demultiplex.file_systemimportFileSystem# Create an instance of FileSystemfile_system_instance=FileSystem()# Specify the path where you want to create a directorydirectory_path="/path/to/your/directory"# Create a new directory or change to an existing onebase_directory=file_system_instance.makedir_orchange(directory_path)# Get information about the base directorybase_dir,base_name,base_basedir=file_system_instance.get_base_dir(base_directory)# Display the directory informationprint("Base Directory:",base_dir)print("Base Name (without extension):",base_name)print("Base Directory of the Base Directory:",base_basedir)Unreferenced_Demultiplexing# Import necessary modulesfromrrequested_demultiplex.unreferenced_demultiplexingimportUnreferenced_Demultiplexing# Specify input fileinput_file="input_file.fasta"# Create an instance of Unreferenced_Demultiplexingdemultiplex_instance=Unreferenced_Demultiplexing(infile=input_file)# Find highly divergent sequenceshighly_divergent_seqs=demultiplex_instance.find_the_num()# Demultiplex the sequencesdemultiplex_instance.demultiplex()# The demultiplexed sequences will be written to individual files in a new folder# The folder will be named based on the input file, in this example: "input_file-fa-demultiplexed"-How does it work?The quality filtering method is based on the easiest implementation one could think of: for every read, the filtering algorithms takes the mean quality and discards the reads that are under a given value (default is 7, so this step will take place nevertheless if the file is fastq/fastq.gz)The size filtering method is also based on the easiest implementation one could think of: for every read, the filtering algorithms takes the length and, if this is below the minimum or above the maximum allowed, the read gets discarded.The demultiplexing method is based on super-fast global alignment and it is divided into two main parts: in the first, the demultiplexer identifies unique (higly divergent) reads, that are the ones which score less than 50% in similarity with all the other sequences. After that, it globaly aligns all the raw reads against the "self-made reference", grouping the ones that share more than 70% of their code. After having demultiplexed this way, the program checks the leftovers, to see wether there are worthy-to-save data or not. This brings to a five-round cycle that identifies higly divergent sequences in the non-grouped ones and clusters the "nogroup" reads against them. If there are still ungrouped remainders after this step, they get clustered together (3). Only groups encompassing more than 1% of the total reads will be written as demultiplexed fasta files, named N.fasta (where N is a number) or nogroup.fasta if they belong to the unclustered reads: they could be found in the folder basefilename-extensionabbreviation-demultiplexed (an example could be: if you are demultiplexing a file named coleoptera.fastq, the folder will be coleoptera-fq-demultiplexed)-Final considerationsAs a practical suggestion, we strongly advise to be cautious while using rrequested-demultiplex with files containing reads from multiple individuals, especially if the quality of the data is low: not because there is the risk that you will miss something, but because it can produce more groups than needed.Moreover, please note that rrequested-demultiplex is still experimental and may contain errors, may fail/take really long while performing large analyses with limited computational power (e.g. on a normal laptop) and may output not-100%-reliable results, so always check them and pull issues whenever you feel it to be the case, we'll be on your back as soon as possible to fix/implement/enhance whatever you suggest!-License and rights of usageThe code is distributed under the MIT license.The MIT License is a straightforward and permissive license that encourages collaboration and widespread use. It allows developers the freedom to build upon and share code while providing clear guidelines for responsible and respectful use. Whether you are an individual or a corporation, the MIT License promotes an open and collaborative approach to software development.