Package: reHCstar
byYuri PirolaStarted: September 27, 2010Current release:2.1.5(March 21, 2016)IntroductionThis program is based on a reduction of theHaplotype Configuration
with Recombinations and Errorsproblem toBoolean Satisfiability,
which is then solved by a SAT solver. A haplotype configuration is
finally recovered from the satisfying assignment.The algorithm is described in the following papers:Yuri Pirola, Gianluca Della Vedova, Stefano Biffani, Alessandra Stella,
and Paola Bonizzoni.A fast and practical approach to genotype phasing
and imputation on a pedigree with erroneous and incomplete information.
IEEE/ACM Transactions on Computational Biology and Bioinformatics
(2012).LinkYuri Pirola, Gianluca Della Vedova, Stefano Biffani, Alessandra Stella,
and Paola Bonizzoni.A fast and practical approach to genotype phasing
and imputation on a pedigree with erroneous and incomplete information.
In: Proc. of IEEE 2nd International Conference on Computational Advances
in Bio and Medical Sciences, ICCABS 2012.LinkDownload and InstallationreHC-* is currently distributed only on source form. It has been
developed on Ubuntu Linux machines (10.04 and later) and has been tested
on both 32 and 64 bit. The program should work on (or should be easily
ported to) on MacOS X but has not been tested and it is not supported on
this operating system.DependenciesPython (>= 2.7)CMake (>= 2.8)GNU makeBoost FileSystem, System, DateTime, ProgramOptions, IOStreams, and
other include-only libraries (tested with 1.42)Apache Log4cxx (tested with 0.10.0)Automatic Download and InstallationThe easiest way for having reHC-* correctly installed on your machine
is throughPyPIwith the command:$ pip install -v reHCstar(Please be patient, because it can take some time to build the package.)Ifpipis not available on your system (and you cannot install it
followingthese
instructions, you can
manually download the reHC-* package fromhttps://github.com/yp/reHCstar/tarball/master, unpack it in a directory
of your choice, and then build it with the command:$ python setup.py installAlternatively, you can proceed with the manual download, compilation,
and installation as detailed below.Manual DownloadreHC-* is developed on theyp/reHCstarGit repository hosted by
GitHub. The repository can be explored using the GitHub web interface athttps://github.com/yp/reHCstar.The latest stable version of reHC-* can be downloaded in either.tar.gzor in.zipformat. Previous
stable releases can be downloaded fromhttps://github.com/yp/reHCstar/archives/master.It is also possible to clone the entire repository using the following
command:$ git clone git://github.com/yp/reHCstar.gitOr, if you have a GitHub account, you can fork the project from therepository web page.Manual CompilationThe program can be compiled by issuing the command at the command
prompt:$ make STATUS=ReleaseThe program can be compiled in three different variants:withan integrated SAT solver andwithoutthe ability to
invoke an external SAT solver. This is the default variant, and
should be the most efficient on both time and memory.withan integrated SAT solver andwiththe ability to invoke
an external SAT solver. This is the most flexible variant, but it is
also the variant that uses more time and memory than the others.withoutan integrated SAT solver butwiththe ability to
invoke an external SAT solver. This variant should be preferred when
one wants to use an external SAT solver, since it is more memory- and
time-efficient than the previous one.Currently, the SAT solvers that can be directly integrated into reHC-*
areCryptoMiniSat v2.9.1by Mate
Soos andMiniSat v2.2.0by
Niklas Een and Niklas Sorensson. reHC-* can interact (in the 2nd and
3rd variants) with any SAT solver that follow the standardrequirementsof the last SAT competitions. The variant can be specified by modifying
the fileCMakeOptions.txtin the root directory. In particular, two
options have to be used to specify the variant:INTEGRATE_SAT_SOLVER, which specifies if the SAT solver should be
integrated into reHC-*;DISABLE_EXTERNAL_SAT_SOLVERS, which specifies if the invocation
of external SAT solvers is allowed.The following combinations are allowed:INTEGRATE_SAT_SOLVER=ONandDISABLE_EXTERNAL_SAT_SOLVERS=ON,
(default), which corresponds to thefirstvariant;INTEGRATE_SAT_SOLVER=ONandDISABLE_EXTERNAL_SAT_SOLVERS=OFF,
which corresponds to thesecondvariant;INTEGRATE_SAT_SOLVER=OFFandDISABLE_EXTERNAL_SAT_SOLVERS=OFF, which corresponds to thethirdvariant.The SAT solver that will be integrated (ifINTEGRATE_SAT_SOLVERisON) can be specified by settingUSE_CRYPTOMINISATorUSE_MINISATtoONin the fileCMakeOptions.txt.By default, reHC-* generates SAT instances in a augmented CNF format
where the “extended clauses” can also be XORs of literals. The SAT
instance is generally smaller if XORs are allowed, represents better the
“internal structure” of the Boolean formula, andshould be usedif it
is supported by the SAT solver. The SAT solver embedded by default,
CryptoMiniSat, supports this functionality, while MiniSat does not. If
MiniSat is chosen instead of CryptoMiniSat by editing the fileCMakeOptions.txt, then XOR-clauses are automatically disabled. To
disable the augmented CNF format, reHC-*must be rebuiltspecifying
the preprocessor symbolAVOID_XOR_CLAUSESin theCXXFLAGS. For
example, if the program is compiled in a bash shell in Linux, it
suffices the following command to enable the “pure” CNF format:$ CXXFLAGS="-DAVOID_XOR_CLAUSES" make STATUS=ReleaseUsageThe program takes as input a genotyped pedigree (with missing genotypes)
and returns (if possible) a complete haplotype configuration with at
mostrrecombinations andeerrors. (The file formats are described
below.) Depending on the variant that has been compiled, the program
works in four different modes that have to be specified on the command
line as program parameter:--create(short form-1), that, given a genotyped pedigree,
creates the associated SAT instance. (Available only on variants2and3.)--read(short form-2), that, given a genotyped pedigree,
reads a satisfying model of the associated SAT instance (if such a
model exists) and computes the associated haplotype configuration.
(Available only on variants2and3.)--create-read(short form-3), that, given a genotyped
pedigree, creates the associated SAT instance, invokes the external
SAT solver, reads a satisfying model of the SAT instance (if such a
model exists), and computes the associated haplotype configuration.
This mode essentially combines the previous two modes by
automatically invoking the external SAT solver. (Available only on
variants2and3.)--solve-internal(short form-4), that, given a genotyped
pedigree, creates the associated SAT instance, uses the integrated
SAT solver for solving the instance, and, if the SAT instance is
satisfiable, computes the associated haplotype configuration.
(Available only on variant1.)The following options are used to specify the input/output files:--pedigree(short form-p), that specifies the file
containing the genotyped pedigree (input file);--sat(short form-s), that specifies the file containing the
SAT instance associated with the genotyped pedigree (output file);--result(short form-r), that specifies the file containing
the results computed by the external SAT solver for the SAT instance
associated with the genotyped pedigree (input file);--haplotypes(short form-h), that specifies the file that
will contain the haplotype configuration of the genotyped pedigree
computed by reHC-* (output file);--assumptions(short form-a), that specifies anoptionalfile that contains additional assumptions thatmustbe satisfied by
the resulting haplotype configuration. Assumptions are specified one
for each row with the following syntax:<variable kind> <individual id> <locus> <value>Where<variable kind>is one ofsp(paternal source),sm(maternal source),p(paternal allele),m(maternal allele),rp(paternal recombination),rm(maternal recombination), ande(genotyping error),<individual id>is the numerical
identifier of the individual (1-based),<locus>is the genotype
locus, and<value>is the boolean value (0/1) that the variable must
have. Please note that biallelic and multi-allelic loci are treated
differently, thus they have different set of variables.For the--create-readmode, the command-line that has to be used to
invoke the external SAT must be specified by using the--sat-cmdline(short form-c) program option. The strings%%INPUT%%and%%OUTPUT%%are placeholders for, respectively, the input and the
output files of the SAT solver. If the SAT solver can read the SAT
instance from its standard input, then it is possible to write the SAT
instance to the solver’s standard input by specifying the option--pipe. In this case, the placeholder%%INPUT%%willnotbe
used.Options for Recombinations and ErrorsBy default, reHC-* search for a haplotype configuration with zero
recombinations and zero errors. To enable recombinations in the
haplotyping process, the program options--global-recomband either--global-recomb-rate=XXor--global-recomb-number=YYmust be
specified. HereXXis a number between0.0and1.0that
represents the maximum number of recombinationsras a fraction of the
total number of possible recombination loci, whileYYis (directly)
the maximum number of recombinationsr. Moreover, if option--global-recombis enabled and--global-recomb-numberis used,
it is also possible to search for a haplotype configuration with a given
minimum number of recombinations by specifying the option--global-recomb-min-number=ZZ, whereZZis the sought lower
bound. This option should only be used to specify a lower bound that has
been already proved since the resulting haplotype configuration could
induce unnecessary recombination in order to satisfy the given lower
bound.Similarly, to enable genotyping errors in the computed haplotype
configuration, the program options--global-errorand either--global-error-rate=XXor--global-error-number=YYmust be
specified. As before,XXis a number between0.0and1.0that represents the maximum number of errorseas a fraction of the
number of non-missing genotypes, whileYYis (directly) the maximum
number of errorse.Other program options allow a finer control over the distribution of
recombinations and errors. Please refer to the help of the program (that
can be obtained by specifying the--helpprogram option) for their
presentation and explanation.Other OptionsreHC-* can also read and write files compressed by GZip. The GZip
compression allows to save some space and, especially for large
instances and when an external SAT solver is used, it could reduce the
running time, since it greatly reduces to time spent for I/O operations.
It is disabled by default since not all the SAT solvers support it.
Three options regulates the GZip compression:--compress-input, which enables the GZip compression of some
files that are read by reHC-* (currently only the--pedigreefile);--compress-output, which enables the GZip compression of some
files that are written by reHC-* (currently the--satand--haplotypesfiles);--compress(short form-z), which is equivalent to specify
both--compress-inputand--compress-output;--compress-sat, which enables the GZip compression only for the
file that contains the computed SAT instance.Temporary files of the--create-readmode are automatically removed
by default. To keep them (for example, for manual inspection), the
program option--keep(short form-k) has to be specified.A summary of the available program options can be printed by invoking
reHC-* with the--help(short form-?) option.ExampleFor example, if the genotyped pedigree is described in filegenotyped-pedigree.txt, the following commands perform the complete
haplotype inference process (saving the resulting haplotype
configuration in filehaplotype-configuration.txt).Using the integrated SAT solver (variant1or2):$ ./bin/reHCstar -4  \
      -p genotyped-pedigree.txt  \
      -h haplotype-configuration.txtUsing an external SAT solver (variant2or3) withmanualinvocation of the SAT solver:$ ./bin/reHCstar -1  \
      -p genotyped-pedigree.txt  \
      -s instance.cnf
# ...execution of the external SAT solver, assuming that
#    it writes the results in file sat-result.txt
$ ./bin/reHCstar -2  \
      -p genotyped-pedigree.txt  \
      -r sat-result.txt  \
      -h haplotype-configuration.txtUsing an external SAT solver (variant2or3) withautomaticinvocation of the SAT solver:$ ./bin/reHCstar -3  \
      -p genotyped-pedigree.txt  \
      -h haplotype-configuration.txt  \
      -c "./external-sat-solver %%INPUT%% %%OUTPUT%%"Or, if the SAT solver reads the SAT instance from its standard input:$ ./bin/reHCstar -3  \
      -p genotyped-pedigree.txt  \
      -h haplotype-configuration.txt  \
      --pipe  \
      -c "./external-sat-solver %%OUTPUT%%"Optimization VersionreHC-* also includes a program that uses the basicreHCstarexecutable in order to achieve two different aims:finding (by a bisect-like search) the haplotype configuration that
induces the minimum number of recombinations;splitting long input genotypes into smaller overlapping blocks on
which a partial haplotype configuration is computed independently and
then used to reconstruct the complete haplotype configuration.Please notice that the optimality of the solution (in term of number of
recombinations) is guaranteed if the genotypes arenotsplit into
smaller blocks.These functionalities are provided by the programreHCstar-mgrwritten inPythonversion 3 and later.reHCstar-mgrrequires two parameters,-pand-r, that
specify, respectively, the file containing the input genotyped pedigree
and the file on which the computed haplotype configuration will be
saved.By default,reHCstar-mgrinvokes thereHCstarexecutable in the
current directory using the internal SAT solver mode (option--solve-internaldescribed above). To change the default, the
complete command line must be provided as argument of the program option--cmdand must contain the following three placeholders{pedigree},{haplotypes}, and{assumptions}that will be
replaced, respectively, with the input pedigree file, the output
haplotype configuration file, and the input additional assumption file.For example, the default value of the--cmdoption (i.e. the default
command line) is:./reHCstar -4 -p "{pedigree}" -h "{haplotypes}" -a "{assumptions}"The command line used to invoke thereHCstarexecutable is composed
by concatenating the argument of the previous option with the arguments
of two other options:--cmd-recand--cmd-time. The first one,--cmd-rec, specifies the options (ofreHCstar) that regulates
the maximum (and, possibly, minimum) number of recombinations. In
particular, the argument must include the placeholder{number}which
will be replaced before invocation with the actual maximum number of
recombinations. Moreover, the argument may include the placeholder{min_number}which will be replaced before invocation with the
largest lower bound on the number of recombinations computed so far.For example, the default value of the--cmd-recoption is:--global-recomb --global-recomb-number "{number}" --global-recomb-min-number "{min_number}"The last option that regulates the final command line ofreHCstaris--cmd-timeand, if specified, must include the placeholder{time}which will be replaced before invocation with the maximum CPU
time of thereHCstarexecution (in seconds). An empty argument
disables the running time limit control (albeit it could be enforced
anyway via OS services).For example, the default value of the--cmd-timeoption is:--time-limit {time}The following sections present the other main features ofreHCstar-mgrwhile the full list of its options is available in the
integrated help (option-h).Automatic Genotype PartitionThe subdivision of the input genotypes in (smaller) overlapping blocks
is regulated by the following two options:--block-length(short
form-l, default50) and--lookahead-length(short form-a, default0). The first option specifies the non-overlapping
(maximum) length of each block which the genotypes are divided into,
while the second option specifies the number of loci (in addition to a
single fixed locus) which two consecutive blocks overlap on. In other
words, a single block can be considered as composed by three parts: the
first part spansblock-lengthloci, the second is composed by a
single locus, and the third (optional) part spanslookahead-lengthloci. (Hence, the total length of a block isblock-length+1+lookahead-length.) The second part of a block always overlaps with
the first locus of the first part of the next genotype block. Moreover
the haplotype configuration computed on this locus during the solution
of the “current” block is used as assumptions during the solution of the
next block (thus coincide). The third part of a block, the “look-ahead”
part, if it is present overlaps with the next block starting from its
second locus. This part is used to compute a haplotype configuration of
the “current” block, but the solution is then discarded when the next
block is considered (thus it may not coincide). Its purpose is to
provide a hint of the structure of the next block and it should be
particularly useful when the proportion of missing genotypes is
relevant, since when the overlapping locus has many missing genotypes,
the solution of the current block could impute the genotypes in a way
that is locally optimal, but globally sub-optimal.Please notice thatreHCstar-mgrfinds a solution that requires the
minimum number of recombinations only if the genotypes arenotdivided
into blocks.Initial Bounds on the Number of RecombinationsInitial lower and upper bounds on the number of recombinations may be
specified with the options--initial-recomb-lb=XXand--initial-recomb-ub=YY, respectively. The options’ arguments,XXandYY, are non-negative numbers such that a haplotype configuration
withXXrecombinations does not exist and a haplotype configuration
withYYrecombinations certainly exists. The default value of both
of them is-1which means that no bound is known/provided. Moreover
it is possible to specify a file containing an initial haplotype
configuration thatreHCstar-mgrtries to improve (in terms of number
of recombinations). In this case, the initial haplotype configuration is
read and the number of recombinations that it induces is used as initial
upper bound. If not better solution is found (for example, due to time
limits), thenreHCstar-mgroutputs the initial haplotype
configuration. The file containing the initial haplotype configuration
is specified as argument of the--initial-haplotype-configurationprogram option. Please notice that options--initial-haplotype-configurationand--initial-recomb-ubcannot
be used together. These options could help to speed-up the process of
searching the solution with the minimum number of recombinations since
they provide the initial interval which the bisect-like search is
performed on.If an initial upper bound is known but an initial lower bound is not, it
is possible to enable abootstrapphase that attempts to quickly
identify an initial lower bound and then the execution continues by
bisecting the interval so determined. The bootstrap phase can be
activated by specifying the--bootstrapswitch, while the maximum
CPU time spent in the bootstrap phase can be specified with the--bootstrap-time-limit=XXparameter, whereXXis the time limit
expressed in seconds.Running Time ManagementreHCstar-mgrprovides basic tools for limiting its total running
time (CPU time). In particular, option--time-limit=SSspecifies the
maximum running time of the program (SSseconds). For the proper
functioning of this feature, the option--cmd-timemust be valid. If
the program execution exceeds the given time limit, thenreHCstar-mgrtries to save the solution computed so far in a file
whose name is the name specified by the option--resultsconcatenated with the (fixed) extension.part. The saved solution
could bepartial(if the original instance has been partitioned in
blocks) and/orsuboptimal(if the minimum number of recombinations has
not been computed within the time limit). The status of the solution is
saved as a comment line in the same file of the solution. We suggest to
enable the verbose mode (with-vor-vv) for getting additional
information.File FormatsInput: Genotyped PedigreesGenotyped pedigrees are described by a single file with the standard PED
format used inplink.
In particular, each line of the pedigree file fully describes a single
individual and it is composed by at leastsixwhitespace-separated
fields. The first (mandatory) six fields are:Family ID(numeric only)Individual ID(numeric only, greater than0)Paternal ID(the ID of the father,0if unknown/not present)Maternal ID(the ID of the mother,0if unknown/not present)Sex(1= male,2= female)Phenotype(ignored, could be any string not containing a
whitespace)Remark:reHC-* currently works only on single-family pedigrees,
thus theFamily IDmust bethe same for all the individuals.The remaining fields (field 7 onwards) represent the genotype of the
individual, where each field represents a single allele of a single SNPbialleliclocus. Both the alleles of each locusmust bespecified
(they can be missing alleles), thus the total number of fields of each
rowmust beeven. Major and minor alleles are encoded by the
characters1and2. Missing genotypes are encoded by the pair0 0(i.e. by two fields containing the missing allele0). The
pairs composed by a valid allele (1or2) and a missing allele
(0)are not valid. Since reHC-* 2.0.0, there could also bemulti-allelicloci. Alleles are encoded by a number greater than0(which is always considered the missing allele code).Rows starting with the character#are considered as comments and
ignored.Remark:The order of the two alleles on each locus is meaningless
(i.e., the pair2 1is considered the same as the pair1 2).A simple single-family pedigree composed by 5 individuals genotyped over
5 biallelic loci is as follows.0 1 0 0 1 phenotype 1 1 2 2 2 2 2 2 1 1
0 2 0 0 2 phenotype 2 2 1 1 1 1 1 1 1 1
0 3 1 2 2 phenotype 1 2 0 0 1 2 1 2 1 1
0 4 0 0 1 phenotype 1 2 1 2 1 1 1 1 0 0
0 5 4 3 1 phenotype 1 2 1 2 0 0 1 1 1 2Output: Haplotype ConfigurationThe haplotype configuration computed by reHC-* is represented in a
PED-like format. In particular, the first six fields are equal to the
PED format. The remaining fields represent the computed haplotype pair
of the individual, whereeachfield represents the two alleles on a
single locus (separated by the character|). In this case, the order
of the two alleles is important and represents thephaseof each
locus. The first allele in each pair is the paternal allele, while the
second one is the maternal allele.For the example, a zero-recombinant haplotype configuration for the
previous genotyped pedigree is as follows.0 1 0 0 1 phenotype 1|1 2|2 2|2 2|2 1|1
0 2 0 0 2 phenotype 2|2 1|1 1|1 1|1 1|1
0 3 1 2 2 phenotype 1|2 2|1 2|1 2|1 1|1
0 4 0 0 1 phenotype 2|1 1|2 1|1 1|1 2|2
0 5 4 3 1 phenotype 1|2 2|1 1|1 1|1 2|1where the two (multi-locus) haplotypes of individual5are12112(paternal haplotype) and21111(maternal haplotype).LicensereHC-* is released under the terms of the GNU General Public License
(GPL) as published by the Free Software Foundation, either version 3 of
the License, or (at your option) any later version.reHC-* is distributed in the hope that it will be useful, but WITHOUT
ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
more details.Please refer to fileCOPYINGor to theGNU
websitefor a copy of the GNU General
Public License.AcknowledgmentsThe template of reHC-* is based on thecpp-project-templateby Michael Aaron Safyan.reHC-* incorporates the following SAT solvers:CryptoMiniSatversion 2.9.1
(commit e819ab3236e, date 26/May/2011) by Mate Soos, which is
distributed under the GNU General Public License version 3;MiniSatversion 2.2.0 by
Niklas Een and Niklas Sorensson, which is distributed under the MIT
license.For extracting source version information from git repository tags,
reHC-* usesautorevisionby dak180
and others, which is distributed under the MIT license.We would like to thank Gianluca Della Vedova for useful discussions.ContactsPlease contactYuri Pirolafor additional information.E-mail:yuri.pirola@gmail.comWeb page:http://algolab.eu/pirola