Package: goldilocks
Locating genomic regions that are “just right”.Documentation:http://goldilocks.readthedocs.org.What is it?Goldilocksis a Python package providing functionality for locating ‘interesting’
genomic regions for some definition of ‘interesting’. You can import it to your
scripts, pass it sequence data and search for subsequences that match some criteria
across one or more samples.Goldilocks was developed to support our work in the investigation of quality
control for genetic sequencing. It was used to quickly locate
regions on the human genome that expressed a desired level of variability,
which were “just right” for later variant calling and comparison.The package has since been made more flexible and can be used to find regions
of interest based on other criteria such as GC-content, density of target k-mers,
defined confidence metrics and missing nucleotides.What can I use it for?Given some genetic sequences (from one or more samples, comprising of one or more
chromosomes), Goldilocks will shard each chromosome in to subsequences of a
desired size which may or may not overlap as required. For each chromosome from
each sample, each subsequence or ‘region’ is passed to the user’s chosen strategy.The strategy simply defines what is of interest to the user in a language that
Goldilocks can understand. Goldilocks is currently packaged with the following
strategies:StrategyCensus DescriptionGCRatioStrategyCalculate GC-ratio for subregions across the
genome.NucleotideCounterStrategyCount given nucleotides for subregions across
the genome.MotifCounterStrategySearch for one or more particular motifs of
interest of any and varying size in subregions
across the genome.ReferenceConsensusStrategyCalculate the (dis)similarity to a given
reference across the genome.PositionCounterStrategyGiven a list of base locations, calculate
density of those locations over subregions
across the genome.Once all regions have been ‘censused’, the results may be sorted by one of four
mathematical operations:max,min,medianandmean. So you may be interested
in subregions of your sequence(s) that feature the most missing nucleotides, or
subregions that contain the mean or median number of SNPs or the lowest GC-ratio.Why should I use it?Goldilocks is hardly the first tool capable of calculating GC-content across a
genome, or to find k-mers of interest, or SNP density, so why should you use it
as part of your bioinformatics pipeline?Whilst not the first program to be able to conduct these tasks, it is the first
to be capable of doing them all together, sharing the same interfaces. Every strategy
can quickly be swapped with another by changing one line of your code. Every strategy
returns regions in the same format and so you need not waste time munging data to
fit the rest of your pipeline.Strategies are also customisable and extendable, those even vaguely familiar with
Python should be able to construct a strategy to meet their requirements.Goldilocks is maintained, documented and tested, rather than that hacky perl
script that you inherited years ago from somebody who has now left your lab.RequirementsTo use;numpymatplotlib (for plotting)To test;toxpytestFor coverage;nosepython-coverallsInstallation$ pip install goldilocksCitationPlease cite us so we can continue to make useful software!Nicholls, S. M., Clare, A., & Randall, J. C. (2016). Goldilocks: a tool for identifying genomic regions that are "just right." Bioinformatics (2016) 32 (13): 2047-2049. doi:10.1093/bioinformatics/btw116@article{Nicholls01072016,
    author = {Nicholls, Samuel M. and Clare, Amanda and Randall, Joshua C.},
    title = {Goldilocks: a tool for identifying genomic regions that are ‘just right’},
    volume = {32},
    number = {13},
    pages = {2047-2049},
    year = {2016},
    doi = {10.1093/bioinformatics/btw116},
    URL = {http://bioinformatics.oxfordjournals.org/content/32/13/2047.abstract},
    eprint = {http://bioinformatics.oxfordjournals.org/content/32/13/2047.full.pdf+html},
    journal = {Bioinformatics}
}LicenseGoldilocks is distributed under the MIT license, see LICENSE.History0.1.1 (2016-07-07)Updated citation.Please cite us! <3[PR:ar0ch] Add lowercase matching in GCRatioStrategyFixes ‘feature’ where lowercase letters are ignored by GCRatioStrategy.0.1.0 (2016-03-08)Goldilocks is published software!0.0.83-beta-land-sCLI arguments and correspondinglengthandstrideparameters
toGoldilocksconstructor now support SI suffixes:K,M,G,T.utilmodule containsparse_si_bpused to parse option strings and return
the number of bases for length and stride.Add length and stride to x-axis label of plots.Addignore_queryoption toplotto override new default behaviour of plot
that only plots points for regions remaining after a call toquery.Removeprofilefunction, useplotwithbins=Ninstead.Add binning toplotto reduce code duplication.Addchromkwarg toplotto allow plotting of a single chromosome across
multiple input genomes.Fix support for plotting data from multiple contigs or chromosomes of a single
input genome when provided as a FASTA.Addignore_querykwarg toplotfor ignoring the results of a query on
theGoldilocksobject when performing a plot afterwards.Bins no longer have to be specified manually, usebins=N, this will create
N+1 bins (a special 0 bin is reserved) between 0 and the largest observed
value unlessbin_maxis also provided.Bins may have a hard upper limit set withbin_max. This will override the
default of the largest observed value regardless of whetherbin_maxis smaller.Plots can now be plotted proportionally withprop=True.Improve labels for plotting.Reduce duplication of plotting code insideplot.Share Y axis across plot panels to prevent potentially misleading default plots.Reduce duplication of code used for outputting metadata:Addfmtkwarg toexport_metathat permits one of:bedBED format (compulsory fields only)circosA format compatible with the circos plotting toolmeltA format that will suit import to an R dataframe without the need
for additional munging with reshape2tableA plain tabular format that will suit for quick outputs with
some mungingRemoveprint_melt, useexport_metawithfmt=melt.Addis_pos_filekwarg to Goldilocks, allows user to specify position based
variants in the formatCHRtPOSorCHR:POSin a newline delimited file.Changed requiredidxkey tofilein sequence dictionaries.Added custom strategy and plotting examples to the documentation.TheGoldilocksclass is now imported asfrom goldilocks import Goldilocks.Thetextwrap.wrapfunction is used to write out FASTA more cleanly.A serious regression in the parsing of FASTA files introduced by v0.0.80 has
been closed.Improved plotting functionality for co-plotting groups, tracks of chromosome
has been introduced. Tracks can now be plotted together on the same panel by
providing their names as a list to thetrackskeyword.reset_candidatesallows users to “reset” the Goldilocks object after a
query or sort has been performed on the regions.0.0.82 (2016-01-29)Changed example to useMotifCounterStrategyover removedKMerCounterStrategy.Fix runtimeNameErrorpreventingPositionCounterStrategyfrom executing correctly.Fix runtimeNameErrorpreventingReferenceConsensusStrategyfrom executing correctly.Add defaultcounttrack toPositionCounterStrategyto prevent accidental
multiple counting issue encountered when couting with thedefaulttrack.Add LICENSEPaper accepted for press!0.0.81 (2016-01-29)Fix versioning error.0.0.80 (2015-08-10)Added multiprocessing capabilities during census step.Added a simple command line interface.Removed prepare-evaluate paradigm from strategies and now perform counts
directly on input data in one step.Skip slides (and set all counts to 0) if theirend_posfalls outside of
the region on that particular genome’s chromosome/contig.RenameKMerCounterStrategytoMotifCounterStrategyFixed bug causinguse_andto not work as expected for chromosomes not
explicitly listed in theexceptionsdict when also usinguse_chrom.Support use of FASTA files which must be supplied with asamtools faidxstyle index.Stopped supporting Python 3 due to incompatability withbufferandmemoryview.Preventqueryfrom deep copying itself on return. Note this means that a query
will alter the original Goldilocks object.Now using a 3D numpy matrix to store counters with memory shared to
support multiprocessing during census.RemovedStrategyValueas these cannot be stored in shared memory. This makes
ratio-based strategies a bit of a hack currently (but still work…)tldr; Goldilocks is at least 2-4x faster than previously, even without multiprocessing0.0.71 (2015-07-11)Officially add MIT license to repository.Deprecate_filter.Update and tidyexamples.py.is_seqargument to initialisation removed and replaced withis_pos.Useis_posto indicate the expected input is positional, not sequence.Force use ofPositionCounterStrategywhenis_posis True.Sequence data now read in to 0-indexed arrays to avoid the overhead of stringre-allocation by having to append a padding character to the beginning of very
long strings.Region metadata continues to use 1-indexed positions for user output.VariantCounterStrategynowPositionCounterStrategy.PositionCounterStrategyexpects 1-indexed lists of positions;preparepopulates the listed locations with 1 and thenevaluatereturns the sum as before.test_regression2updated to account for converting 1-index to 0-index whenmanually handling the sequence for expected results.queryacceptsgmaxandgminarguments to filter candidate regions by
the group-track value.CandidateListremoved and replaced with simply returning a newGoldilocks.0.0.6 (2015-06-23)Goldilocks.sorted_regionsstores a list of region ids to represent the result
of a sorting operation following a call toquery.Regions inGoldilocks.regionsnow always have a copy of their “id” as a key.__check_exclusionsnow accepts agroupandtrackfor more complex
exclusion-based operations.region_group_lteandregion_group_gteadded to usable exclusion fields to
remove regions where the value of the desired group/track combination is
less/greater than or equal to the value of the group/track set by the
currentquery.querynow returns a newGoldilocksinstance, rather than aCandidateList.Goldilocks.candidatesproperty now allows access to regions, this property
will maintain the order ofsorted_regionsif it has one.export_metanow allowsgroup=NoneCandidateListclass deleted.Test data that is no longer used has been deleted.Scripts for generating test data added totest_gen/directory.Tests updated to reflect the factCandidateListlists are no longer returned
byquery._filteris to be deprecated in favour ofqueryby 0.0.7Beta (2014-10-08)Massively updated! Compatability with previous versions very broken.Software retrofitted to be much more flexible to support a wider range of problems.0.0.2 (2014-08-18)Remove incompatible use ofprint0.0.1 (2014-08-18)Initial package