Package: witch-msa
Developer:Chengze ShenTable of contentsNewsTODO listMethod OverviewNote and AcknowledgementInstallationInstall with PyPI (pip)Install from the source fileRequirementsInstallation Stepsmain.configUsageExamplesScenario AScenario BScenario CScenario D - additional optionsNews(NEW)Added an option-y/--bypass-setupto avoid being asked where to put the config file when running WITCH for the first time. Usage:witch.py-y[...additionalparameters]. You only need to use this option once and you are all set!Now support PyPI installation! Please install the latest release withpip installwitch-msa.Automatically infer data type if None is specified (use--moleculeto specify).Checkpoint system set up for most steps except HMMSearch jobs (ongoing).Added progress bar (python packagetqdm) to visualize the alignment progress at various stages.ImplementedWITCH-ng’s way to align each query sequence with additional tweaks. Now the alignment process for query sequences isfast and memory-efficient, particularly for short/fragmentary sequences.TODO listNone for now.Method OverviewWITCH is a new multiple sequence alignment (MSA) tool that combines techniques fromUPPandMAGUS.
It aims to solve alignment problems, particularly when input sequences contain fragments. The whole pipeline can be described as follows:Given a set of unaligned sequencesS, pick at most 1,000 “full-length” sequences to form abackbone alignmentBand abackbone treeT(Full-length sequences refer to sequences of lengths that are within 25% of the median length).Create an ensemble of HMMs (eHMM, seeUPPfor more details) fromBandT.For each remaining unaligned sequence, align it to high-ranked HMMs to obtain a set of weighted support alignments; then, merge the support alignments using Graph Clustering Merger
(GCM, an alignment merger technique introduced in MAGUS). 4. Transitively add the merged alignment of  each query toB, and report the final alignment onS.For a more detailed explanation of the WITCH algorithm, please refer to the publication below:PublicationShen,
Chengze, Minhyuk Park, and
Tandy Warnow. “WITCH:
Improved Multiple Sequence
Alignment Through Weighted
Consensus Hidden Markov
Model Alignment.” Journal
of Computational Biology,
May 17, 2022.https://doi.org/10.1089/cmb.2021.0585.Note and AcknowledgementWITCH includes and uses:MAGUS(we use the Github version updated on April 5th 2021).HMMER suites(v3.1b2 - hmmbuild, hmmsearch, hmmalign).UPP(v4.5.1; we use only partial functionalities).FastTreeMP(v2.1).MAFFT(macOS v7.490).MCL(linux version from MAGUS; macOS version 21-257).InstallationThis section lays out the necessary steps to run WITCH. WITCH was tested and passed builds
onPython 3.7 to 3.11.Now, the program fully supports Linux and macOS systems.
We provide necessary binary executables for both systems,
but you can supplement your own by changing the paths
in themain.configfile. In cases of conflicting installations
(e.g., different versions of MAFFT), please supplement with the version
on your system. If you experience any difficulty running WITCH, please
contact Chengze Shen (chengze5@illinois.edu).For the macOS system on the latest chips (e.g., M1/M2), you may need to compile and supply your own binaries for WITCH to run successfully.
That is, change the paths of binaries inmain.configto the ones on your system.Install with PyPI (pip)The easiest way to install WITCH is to use the PyPI distribution.# 1. Install with pip (--user if no root access)pip3installwitch-msa[--user]# 2. After installation, users can run WITCH with either "witch-msa" or "witch.py" anywhere in the system
#    (Optional) Include "-y" or "--bypass-setup" to avoid being asked where to put the WITCH config file.
#               Using this option will default to use "~/.witch_msa" as the config directory. You only
#               need to use this option once.witch-msa[-h][-y]# or,witch.py[-h][-y]Install from the source fileRequirementspython>=3.7
cython>=0.29
configparser>=5.0.0
DendroPy>=4.4.0,<4.6.0
numpy>=1.15
psutil>=5.0
tqdm>=4.0.0Installation Steps# 1. Install via GitHub repogitclonehttps://github.com/c5shen/WITCH.git# 2. Install all requirements
# If you do not have root access, use "pip3 install -r requirements.txt --user"cdWITCHpip3install-rrequirements.txt# 3. (Optional) Run setup.py to set up main.config. Please refer to "witch_msa/default.config"
#    Additionally, software binaries available in the user's environment will be prioritized for usage.
#    Use "-c" if want to install to WITCH/.witch_msa/main.config
#    Default is to ~/.witch_msa/main.configpython3setup.pyconfig[-c]# 4. Execute the WITCH python script with -h to see allowed commandline parameter settings
#    When running WITCH normally, if step 3 is not run, you will be prompted to generate
#    "main.config" when running WITCH for the first time.
#    (Optional) Include "-y" or "--bypass-setup" to avoid being asked where to put the
#               WITCH config file. Using this option will default to use "~/.witch_msa"
#               as the config directory. You only need to use this option once.python3witch.py[-h][-y]main.configmain.configfile will be created after running WITCH for the first time or created withpython setup.py config[-c].
If it is not found, you will be prompted to choose where to create the file (default:~/.witch_msa/main.config).
As mentioned above, you can use-yor--bypass-setupto bypass this prompt by defaulting to~/.witch_msa/main.config.UsageThe general command to run WITCH:python3witch.py-i[unalignedsequencefile]-d[outputdirectory]-o[outputfilename]Default behavior: WITCH will pick at most 1,000 sequences from the input around the median length as the backbone sequences.
Then, it uses MAGUS to align the backbone sequences and FastTree2 to estimate a tree. It uses UPP decomposition strategy to generate an eHMM,
and uses HMMSearch to calculate bit scores between HMMs and unaligned sequences. Bit scores are used to calculate weights, and each unaligned
sequence is aligned to topk=10HMMs ranked by weights.ExamplesAll the following examples can be found in theexamples/run.shbash
script.Scenario AUnaligned sequences only.Running WITCH for the first time and bypassing the prompt for setting up the configuration file(-y).python3witch.py-y-iexamples/data/unaligned_all.txt\-dscenarioA_output-oaligned.txtScenario BUnaligned sequences only; using bit scores (instead of the default weighted bit scores); using 10 HMMs to align a sequence.python3witch.py-iexamples/data/unaligned_all.txt\-dscenarioB_output-oaligned.txt-w0-k10Scenario CBackbone alignment available; backbone tree missing; query sequences available.python3witch.py-bexamples/data/backbone.aln.fasta\-qexamples/data/unaligned_frag.txt-dscenarioC_output\-oaligned.txtScenario D - additional optionsBackbone alignment available; backbone tree available; query sequences available; saving weights to local; saving decomposition results for future usage (e.g., faster rerun).python3witch.py-bexamples/data/backbone.aln.fasta\-eexamples/data/backbone.tre-qexamples/data/unaligned_frag.txt\-dscenarioD_output-oaligned.txt\--save-weight1--keep-decomposition1