Package: seq-pigeon
PigeonIntroductionTool for pipeing inputs and outputs of multiple cli tools.
Pigeon takes in only a config file as input. Everything required to run the pipeline are specified in config file. The config file is specified according topython configparser.Quick InstallLinux&Macsudo pip3 install --index-urlhttps://test.pypi.org/simple/pigeonWindowspip install --index-urlhttps://test.pypi.org/simple/pigeonResources for NGSNone of the tools or data files are supplemented by pigeon so they need to be downloaded.
For example configuration file, exome sequencing pipeline,ToolsFastQCBWAPicardGATKAnnotation ToolReference FilesReference genome and known SNP&INDELShg19 orhg38Bed fileSee website of capture kit used in sequencingHow to useCreate yourself a configuration filepigeon createconfigModify for your analysis. (See below.)pigeon -c my_config.conf -dIf everything looks alright run for real.pigeon -c my_config.confConfig FileConfig file consists of three parts.GeneralPipelineIndividual tool blocksGeneralArea used to define project name, output directory, input files, and resource files like reference genome or target file. Following variables are necessary for run.Required:project_name : name of your projectoutput_dir : where to write output filesinput_files : input files for analysis, space separated, pairs should be next to each other. e.g.input_files = A.txt B.txt C.txtor for pairedinput_files = A1.txt A2.txt B1.txt B2.txt C1.txt C2.txtOptional variable can also be decleared here. Based on your or tool requirements. Later these variables can be called in the config file using ${GENERAL:optional_variable}.Optional(example):reference_genome = /path/to/my/reference_genome.fabed_file = /path/to/my/target.bedknown_snp = /path/to/my/snp.vcfmy_database = /path/to/my/favorite.dbPipelineThis area should contain paths to tools that is understanble by your shell. As well as the run order of tools. e.g.pipeline = job1 job2 job3A = path/to/AB = path/to/BC = path/to/CTool BlocksName of the block should be same as inpipeline. By continuing example above;[job1][job2][job3]Arguments that can be used in these blocks as follows:Run Argstool: tool variable from pipeline block. e.g.tool = ${PIPELINE:A}sub_tool: if tool has a sub tool like 'bwa mem'. e.g.sub_tool = memargs: arguments of the toolsjava: if tool is a jar file add java -jar before it.pass: if True it won't run the block. But the block still be part of the pipeline. This option is helpful for resuming interrupted pipeline.Input Argsinput_from: Name of the block that that's output is this jobs input. First jobs input_from should be input_files.input_multi: can be 'paired' or 'all'. Paired option splits input files stream into groups of two. All option uses all of the input files.input_flag_repeat: If tool requires input flag for each input this command will add given flag before each input.secondary_in_placeholderOutput Argssuffix: add suffix to output file nameext: file extension of the outputdump_dir: creates a directory and outputs there.paired_output: this option will pair the input and the output of the tool.secondary_out_placeholdersecondary_suffixsecondary_extsecondary_dump_dirPlaceholdersThese are joker words that can be used inargs.input_placeholdersecondary_input_placeholderoutput_placeholdersecondary_output_placeholderExample Config[[GENERAL]]project_name = my_projectoutput_dir = /path/to/output_directoryinput_files = A.txt B.txt C.txtmy_db = /path/to/my.db[[PIPELINE]]pipeline = job1 job2 job3A = /path/to/AB = /path/to/BC = /path/to/C[job1]tool = Ainput_from = input_filesargs = -i input_placeholder -o output_placeholdersuffix = job1_Aext = txt[job2]tool = Binput_from = job1args = -i input_placeholder -o output_placeholdersuffix = job2_Bext = txt[job3]tool = Cinput_from = job2args = -i input_placeholder -o output_placeholdersuffix = job3_Cext = txt