Indexing a reference sequence for oligomer sequence mapping

Generating an index includes receiving a reference sequence and applying one or more key patterns to the reference sequence to obtain a plurality of keys in the index. Each of the one or more key patterns is derived based on a corresponding set of oligomer sequence relationships of a plurality of oligomer sequences that are expected to be generated from the reference, and the keys correspond to a plurality of candidate and/or validated locations in the reference sequence.

BACKGROUND OF THE INVENTION

Genetic studies have seen rapid advances in recent years. The entire genomes of specific organisms, including some individual human beings, have been sequenced and become available as references. In genetic research, genetic testing, personalized medicine, and many other applications, it is often useful to obtain a sample of genetic material, determine a sequence of that sample, and to map that sample sequence to a location on an available reference. Once the mapping is done, a comparison can be made to a reference in order to identify polymorphisms or mutations or obtain other useful information.

Existing approaches typically map long, contiguous sample sequences to locations in a reference. However, some techniques used for obtaining sample sequences yield data sets comprising short sequences (sometimes referred to as oligomers) with predicted spatial relationships. Such ‘polyoligomer data sets’ consist of multiple oligomers that have variable but constrained amounts of spacing or overlap (referred to as separation distance) between oligomers. Where individual oligomers are too short to identify one or a small number of possible locations on a reference sequence, and the spacing between oligomers is variable, existing approaches are not adequate.

It would be useful to have indexes with the ability to accurately map relatively short oligomer sequences with variable separation distances to a reference. It would also be desirable to create indexes that would allow such mapping to be efficient both in terms of computational speed and cost

SUMMARY OF THE INVENTION

The present invention provides methods of producing searchable indexes for analysis of polyoligomer data sets. The indexes are created by generating keys from a reference corresponding to predicted sequence relationships within the data sets. The index keys are generated using key patterns that correspond to predicted data sets based on the reference. The index allows external data sets to be mapped via keys within the index via the use of keys generated from the data sets; mapping of the keys to the index allows identification of potential locations of the data set within the reference.

In one implementation, the invention provides a method of generating an index comprising application of one or more key patterns to a reference to generate a plurality of keys. Such key patterns are based on predicted reference data sets having at least one variable separation distance. The keys are provided in a searchable medium, and are preferably stored in a computer readable medium.

In another implementation, the invention provides a system for generating an index for oligomer sequence analysis, comprising an interface configured to receive a reference, and a processor coupled to the interface, with the processor configured to apply one or more key patterns to the reference to obtain a plurality of keys in the index. Each of the key patterns is derived from predicted sequence relationships of data sets, and each data set has at least one area of variable separation distance. The plurality of keys in the index corresponds to a plurality of possible locations in the reference.

In yet another implementation, the invention provides a computer program product for generating an index, the computer program product being embodied in a computer readable medium and comprising computer instructions for receiving a reference and applying one or more key patterns to the reference to obtain a plurality of keys in an index. The key patterns are derived from predicted data sets based on the reference, and each data set has at least one area of variable separation distance. The plurality of keys in the index corresponds to a plurality of possible locations in the reference.

DETAILED DESCRIPTION

Definitions

The term “polyoligomer data set” (also, simply “polyoligomer” or “data set”) refers to a collection of two or more determined oligomer sequences (e.g. using a biochemical process), and whose separations from one another are, or for computational purposes are assumed to be, restricted to certain known or estimated values.

The term “reference” refers to a known sequence of nucleotides. It may be an entire genome sequence of a reference organism, a portion of a reference genome, a consensus sequence of many reference organisms, a compilation sequence based on different components of different organisms, or any other appropriate sequence. It may also include information regarding variations of the reference known to be found in a population of organisms.

The term “reference index” refers to an index relating keys consisting of sequences of bases to locations in the reference that is created by application of one or more key patterns to a reference.

The term “sequence relationship” refers to a known, inferred or hypothesized specification of separation distances among two or more oligomers, e.g. defined as the number of (unknown) bases between two oligomers. The sequence relationship may be a separation distance, an overlap distance, or the two oligos may be directly adjacent to one another in the reference.

The term “instantiation” refers to the derivation of a sequence of contiguous bases, some known, some possibly unknown, by applying a specific set of sequence relationships to a polyoligomer data set. The term may also refer to the derived sequence itself, which may otherwise be called an “instantiated polyoligomer”.

The term “mapping” refers to a process which relates a polyoligomer to zero, one or more locations in the reference to which the polyoligomer is similar, e.g., by matching the instantiated polyoligomer to one or more keys within an index of the invention corresponding to a location within a reference.

The term “candidate locations” refers to potential locations in a reference of oligomers of a data set identified based on the generation of keys from the data set and the mapping of these keys using a reference index. Since a key derived from a given polyoligomer does not necessarily include all bases of the polyoligomer, the polyoligomer may or may not be a perfect match with a given candidate location in a reference.

The phrase “perfect match” means an exact data match between one possible instantiation and the reference. In certain implementations, an unknown base in a data set may be considered to be a perfect match to any base.

The term “validated locations” means candidate locations which have been further confirmed to be locations within a reference that are compatible with a polyoligomer.

The phrase “degree of conservation” as used herein refers to the likelihood that one or more of the bases in the oligomer sequences of a data set will be in a given position in the key when the key pattern is applied to the correct instantiation of the data set.

The term “sample analysis” means any use of the information obtained through use of the keys and indexes of the invention, including but not limited to genomic analysis (including sequence assembly), polymorphism analysis, mutation analysis, phylogenetic analysis and the like.

The term “key pattern” refers to a predetermined spatial relationship that is used to derive one or more keys from a reference and/or an instantiated polyoligomer.

The Invention in General

Mapping of polyoligomer data sets to locations in a reference is disclosed. The oligomer sequences are obtained from a sample of genetic material (such as DNA or RNA molecules from an organism), e.g., by subjecting the sample or a fragment thereof to a biochemical process. The oligomer sequences are mapped to one or more possible locations in a reference and those locations are output to facilitate further sample analysis.

In some embodiments, an index is generated for mapping key sequences to locations in the reference. The key sequences in the index are generated based on the reference and key patterns that are derived from expected oligomer sequence relationships. Examples of such oligomer sequence relationships include oligomer sequence length, the likely amount of spacing or overlap between oligomer sequences (also referred to as “separation distance”), the statistical distribution of these separation distances (the “distance variations”), and the statistical distribution of possible combinations of distance variations for sets of related oligomer sequences. The relationships may be determined based on existing knowledge about the biochemical process used to generate the oligomer sequences (i.e., based on oligomer sequences that would be expected to be obtained if the biochemical process were applied to a sample), empirical estimates based on preliminary analysis of oligomer sequences, estimation by experts, or other appropriate techniques.

The oligomer sequence relationships depend at least in part on the biochemical process used to generate the oligomers. Numerous processes can be used to generate oligomer data sets for use with the present invention. These include, but are not limited to hybridization methods as disclosed in U.S. Pat. Nos. 6,864,052; 6,309,824; 6,401,267; sequencing-by-synthesis methods as disclosed in U.S. Pat. Nos. 6,210,891; 6,828,100, 6,833,246; 6,911,345; 7,329,496 and Margulies, et al. (2005),Nature437:376-380 and Ronaghi, et al. (1996),Anal. Biochem.242:84-89; ligation-based methods as disclosed in U.S. Pat. No. 6,306,597, WO2006073504, WO2007120208, all of which are incorporated by reference in their entirety. In a specific implementation, a Combinatorial Probe Anchor Ligation (cPAL) process is used in some embodiments (see U.S. Ser. No. 11/679,124, filed Feb. 24, 2007, which is incorporated herein by reference in its entirety).

The processes used to generate related oligomers in data sets may result in oligomers of various sizes, including different sizes within single data sets. For purposes of example only, and not to be limiting in scope, the oligomer sequences discussed in detail below for purposes of illustration are described as having a length of 6 bases, and a distance variation of +/−2 or +/−3 bases. It will be apparent to one skilled in the art upon reading the present disclosure that other sequence lengths and distance variations (e.g., +/−1 or +/−4) can be used in the described implementations.

The key generator also receives as one of its inputs one or more key patterns. The key patterns are derived from one or more oligomer sequence relationships. In some embodiments, various key patterns for different oligomer relationships are derived prior to the mapping process, stored, and retrieved as needed.

FIG. 1Ais a block diagram illustrating an implementation of a system for mapping oligomer sequences onto a reference. In this example, the system100includes a communication interface110configured to receive inputs. Examples of a communication interface include without limitation external connections, such as a port, cable, wire line or wireless network interface card, etc., and internal connections such as a communication bus. The interface110is coupled to an oligomer sequence mapper component112, which includes a key generator102and a reference index search engine104. The key generator receives as one of its inputs a set of related oligomer sequences, e.g., a polyoligomer data set, and receives as another of its inputs one or more key patterns. When mapped to the reference, the related oligomer sequences are expected to be located with respect to each other within a small range of expected separation distance. The oligomer sequence relationships may include the separation distance or other relationships or expected correlations between oligomers.

Based on its inputs, the key generator generates one or more keys, which are strings of base sequences, compressed representations of base sequences or any other appropriate representations of base sequences that are suitable for computer processing. Using the keys, a reference index search engine104queries an index106to determine possible locations of the oligomer sequences in a reference108. In the examples discussed below, keys within index106include strings that are permutations of bases. The index maps keys to their respective possible locations in the reference. These candidate locations are then examined to confirm the match between a data set and a given location, by comparison to the reference108. The validated locations may be output and used for further sample analysis, which can have applications in genetic research, genetic testing, personalized medicine including diagnosis, prognosis and identification of predispositions, and the like.

An oligomer sequence mapper112with key generator102, reference search engine104, and index106are shown as separate logical components in the diagram. In some embodiments the components are separate components such as separate processors or separate processes operating on one processor, and in some embodiments some of the components may be combined and implemented on the same device, as a single integrated circuit, and/or as parts in the same process operating on a processor.

FIG. 1Bis a block diagram illustrating an implementation of a system for generating an index. System160may also include a general purpose computer and/or specialized hardware. In this example, system160includes an interface152, which is configured to receive inputs such as the reference and one or more key patterns. The interface is coupled to an index generator154that includes a processor configured to generate keys based on the inputs. Keys and the locations from which they are derived are maintained in index106, which may be stored on a computer readable storage medium such as a computer data storage device (e.g., random access memory), disk storage, and/or any other appropriate storage device. Such storage devices are optionally coupled with a central processing unit.

The keys for a specific data set are generated by applying pre-defined key patterns to the data set, and the obtained keys can then be compared to keys within a reference index. In certain implementations, the key patterns are applied directly to the data set without any further manipulation of the initial data obtained. In other implementations, the sequences of the data set may be further manipulated, e.g., re-ordered, or homopolymer-compressed; the key pattern is then applied to such derived sequences, and the resulting keys are compared to a reference index whose keys have been similarly manipulated

In specific aspects, the keys obtained by applying the key pattern to a data set or a portion of a data set can be reverse-complemented. Preferably, the data set may be reverse complemented and the key pattern applied to obtain the keys. In other instances, the data set may be reverse complemented after the key pattern is applied but before comparison of the keys to a reference index This technique, in which the both the obtained keys and the complement keys (based on the key pattern) are searched within a reference index allows simultaneous searching for candidate locations in both strands of a sequence in a double-stranded molecule, e.g., DNA. Using such a technique effectively allows interrogation of both the forward strand sequence and the reverse strand sequence in a single process step using an index that is roughly half the size of an index containing all possible keys for both strand sequences.

In other specific aspects, the degree of conservation of specific positions within a key pattern can be used to re-order the key to prioritize positions with the highest degree of conservation (and thus likelihood that the position will be included in the key pattern). This is especially useful if there are potentially gaps in the data of the polyoligomer data sets, as the missing data will not then prevent analysis of the available data.

In some implementations, the use of multiple indexes is employed. When the relationships amongst the oligomers of a data set are sufficiently constrained, a single index will often suffice. If relationships amongst the oligomers of a data set are more variable, however, it may be advantageous to use multiple indexes instead of a single index. One example of this would include the use of multiple indexes where each is created using a single key pattern. This enables, for instance, a large range of separation distances while retaining key length/specificity, as discussed further below. Another example is to use multiple subsets of a reference based on key characteristics, e.g., separate indexes for keys beginning with each of the possible nucleotides.

Index Generation

FIG. 2is a flowchart illustrating an embodiment of a process for generating the index. Process200may be implemented using an index generator such as154ofFIG. 1B. A reference202and potential data set sequence relationships208are received by the reference key generator212. At204, one or more key patterns are derived, as described further below. At206, the derived key patterns are applied to the reference to obtain keys in the index for mapping sequence strings to possible locations in the reference. In some embodiments, multiple indexes are generated from multiple key patterns.

In some embodiments, once a key pattern is obtained, keys can be generated by repeatedly comparing the key pattern to the reference, each time sliding the key pattern forward by 1 base to a new location on the reference. The key-location mapping data is stored in an index.FIG. 3Aexemplifies key generation for use in an index. Assuming that key pattern314is initially applied to reference302starting at location350akey316is produced for this location. This key can reflect the direct order of sequence within the reference316, or the bases obtained from the application of the key pattern to the reference may be reordered318based on confidence levels to improve mapping performance as described in more detail herein. Advancing along sequence302by one base, the key pattern314is applied again starting at the next location348. A new key is produced for the new location, again having either a direct sequence338or a re-ordered sequence340. By repeating the process for the entire length of the reference, each potential location within the reference is mapped to a key.

Different key indexes can be generated from a single reference by varying the bases included within the key pattern. For example, inFIG. 3Bthe key pattern342used includes four bases excluded in the key pattern314ofFIG. 3A. The resulting keys344,346are longer than they keys generated inFIG. 3A, which may be useful for certain mapping implementations. InFIG. 3C, a shorter key pattern366is applied to the reference to include only the bases with the highest match confidence to create keys (such as key368) that can be used when using generated data sets with more variable spacing. Shorter keys, however, generally lead to identification of more potential locations in the reference, so the length of the keys should be balanced by the distribution of separation distances so as not to miss too many of the true mappings without producing an unwieldy number of low-specificity candidate matches.

FIG. 4Ais an illustration of an index. In this example, each key-location pair is stored as an entry in index400. The key strings are in lexical order. It is possible to have keys that correspond to more than one reference location as well as keys that do not correspond to any reference locations (the latter are omitted from the table in some embodiments). To look up a key in the index, a search (such as a linear, binary, or hashed search depending on implementation) based on the key string is performed.

FIG. 4Billustrates another example of an index consisting of two levels or stages designed to increase lookup speed and reduce memory footprint. In this example, each key is split into two portions. The first portion is referred to as the prefix and the second portion is referred to as the suffix. The prefixes are stored according to their lexical order in a first sub-index402, also referred to as a prefix table. The length of the prefix is chosen based on statistical probabilities, i.e., short enough so that nearly all sequence combinations of that length occur in the reference, and long enough so that the average number of occurrences for each sequence combination meets desired criteria. In this example, the prefix includes 12 bases. Given a prefix length, the length of the suffix depends on the pattern used to generate the key. For example, the suffix length is 4 for reordered keys generated using key pattern314(FIG. 3A), and 10 for key pattern320(FIG. 3B). The example shows a suffix length of 8. In the prefix table, each entry maps a prefix to an offset of a set of suffixes in a second sub-index404, also referred to as the suffix table. Each set of suffixes include permutations of bases for the given suffix length. The offset is a value that represents or can be used to derive the base address of the set of suffixes relative to the address at the beginning of the suffix table. In the suffix table, each suffix maps to one or more corresponding reference locations that would generate the key that is the prefix/suffix combination. To find a location in the reference for a given key string, the prefix portion of the key string is identified in the prefix table to obtain the offset. The corresponding set of suffixes can be quickly located in the suffix table using the suffix table base address and the offset. The reference location can be found by looking up the suffix in the suffix set, e.g., by binary search. The suffix lookup is constrained not to exceed the starting location of the next set of suffixes. For example, given prefix406, the search is restricted to a range between x and y.

Key Pattern Design and Derivation

The design of a key pattern can influence the efficiency, speed and/or computational cost of its use in index generation and mapping. In certain aspects, the oligomer sequences within a data set are individually too short to be used as effective keys on their own, as they will result in information on location that is not sufficiently specific. In other aspects, a longer key may be unnecessarily specific; this may result in an index that doesn't fit in available memory, or requires an excessive computational cost to avoid unacceptable sensitivity to sequencing error or variation between the reference and the sample being sequenced. The shorter the key used the less specific the sequence will be in a reference, and thus the more candidate locations that will be generated by the key. As a general rule, since these candidate locations need to be validated, having a greater number of initial candidate locations will require additional effort to be expended in the validation step. An acceptable key design is thus one in which an appropriate number of candidate locations are determined to allow for validation of these locations at a desired computational cost.

A key pattern may be derived from a corresponding set of oligomer sequence relationships. Examples of such oligomer sequence relationships include expected length(s) of the oligomers, the variation in separation or overlap distance (referred to as distance variation), the statistical distribution of the distance variations amongst oligomer sequences, and the statistical distribution of possible combinations of distance variations within sets of related oligomer sequences.

In some embodiments, the key patterns are derived by comparing possible arrangements of oligomer sequence sets according to the oligomer relationships, and finding conserved positions in the sequences. As will be shown in greater detail in the example below, bases located at the conserved positions can be determined with a high probability for any instantiation of a given set of oligomer sequence relationships. In what follows, we first consider positions which are absolutely conserved and then explain how this can be extended to handle various degrees of conservation. The probability of conservation of a base within a data set may be largely due to the nature of the biochemical process used to obtain the data set. Such information can inform the ordering of the key, as described in more detail herein.

FIG. 3Ais a diagram illustrating an example of a key pattern derived from a set of oligomer sequence relationships. In this example, an arbitrary portion of a sample represented by sequence302is used to illustrate the key pattern derivation process described in204of process200(FIG. 2). The oligomer generation process divides the contiguous sequence302into a first portion of bases TGCTATTCATTAGCC, and a second portion of bases GTGTGGGTCAAGGTA. The two portions are separated by a fixed distance, in this case a distance of +2 (which are the two bases AA). For purposes of example, the following specification discusses in detail oligomer sequences that are six bases long (referred to as hexamers), although as discussed other oligomer sequence lengths are possible and in some instances preferable. In this example, the oligomer generation process yields two hexamers for each portion, giving a total of four hexamers, with the inner hexamers of each portion assumed to be positioned at the inner ends of the portions, hence separated by two bases (AA). The data generation process is such that the pair of adjacent hexamers obtained from the same sequence portion may have a variable separation or overlap distance, referred to as a distance variation. In this example, the two adjacent hexamers within each portion of302are said to have a distance variation of +/−2, which means that the hexamers may be separated by 1 or 2 bases (which corresponds to a distance of 1 or 2, respectively), overlap by 1 or 2 bases (which corresponds to a distance of −1 or −2, respectively), or abut each other without separation or overlap (which corresponds to a distance of 0). In other embodiments, the amount of distance variation may be different.

304-312are various four-hexamer data sets illustrating the distance variations that may be generated by the biochemical process. As shown, in304, hexamers TTCATT and TTAGCC have a distance of −2, and hexamers GTGTCG and GGTCAA have a distance of −1. In306, hexamers ATTCAT and TTAGCC have a distance of −1, while GTGTGG and TCAAGG have a distance of +1. In308, TATTCA and TTAGCC have a distance of 0 and GTGTGG and GGGTCA have a distance of −2. In310, CTATTC and TTAGCC have a distance of +1, and GTGTGG and GTCAAG have a distance of 0. In312, GCTATT and TTAGCC have a distance of 2, as do GTGTGG and CAAGGT.

A set of conserved positions that would make a reasonable key pattern can be determined by a comparison of these arrangements. Certain base positions in sequence302are included in the hexamers in any polyoligomer derived from the sequence with the inner two hexamers separated by the two bases AA and a variable data set distance variation of +/−2, regardless of which of the possible separation distance sets is utilized by a given polyoligomer. These conserved positions are shown as shaded regions in sequence302. In this example, the conserved positions correspond to sequence portions326,322,324, and328(TT, TTAGCC, GTGTGG, and CA), which appear in all the hexamer sets304-312. For a given set of oligomer relationships, the conserved positions correspond to positions in a sequence that are determinable despite different distance variation between sequences. In other words, given a set of oligomer sequences generated from an unknown sequence using a specific (but variable) generation process, the bases of the sequence at the conserved positions will be determined with high probability.

One key pattern that is useful for such oligomer relationships is shown in314, where the 16 conserved positions of the polyoligomer data set are shown as boxes containing a dash and the non-conserved positions (i.e., positions that cannot be easily determined based on a set of oligomer sequences) are shown as “x”. Applying the key pattern to the portion of the reference shown at302, beginning at position350, produces key316, which includes all the bases corresponding to the conserved positions, arranged in the same order as they appear in sequence302. These bases can be rearranged based on additional probability of conservation to place the bases with the lowest probability of being unspecified in the data set at the beginning of the key string, as shown in key318.

FIG. 3Cis a diagram illustrating another example of a key pattern derived from another set of oligomer sequence relationships. In this example, the adjacent hexamers are known to have a distance variation of +/−3, which means that the hexamers may have a separation or an overlap of up to 3 bases.352-364show various possible arrangements of hexamers resulting from sequence302, including an arrangement in which two adjacent hexamers overlap by 3 bases as shown in352, and an arrangement in which two adjacent hexamers are separated by 3 bases as shown in364. A comparison of these arrangements shows that while sequence portions322and324are still conserved (i.e., appearing in all hexamer arrangements) just as inFIG. 3A, sequence portions326and328are no longer conserved since the bases in these sequence portions do not appear in all the hexamer sets. The derived key pattern is shown as364, where the conserved positions are again shown as boxes containing a dash and the non-conserved positions are shown as “x”. In this example, key pattern364includes 12 conserved positions and is shorter than key pattern314ofFIG. 3A.

Some tradeoffs exist among possible key patterns. Pattern364ofFIG. 3Cis more inclusive of the possible distance variations than pattern314as the former takes into account distances of +3 and −3. Pattern364is shorter than pattern314since fewer bases are conserved. An index generated based on a shorter pattern such as364would allow mapping of oligomer sequence sets with a distance of +3 or −3 to specific locations in the reference, while an index based on pattern314would not generally allow mapping in the naïve implementation (though see below). Thus, for a given collection of polyoligomer data sets, a shorter pattern accounting for a greater distance range and having fewer conserved bases leads to a higher portion of oligomer sequence sets being usable. In other words, a higher percentage of the oligomer sequence sets can be mapped to locations in the reference. On the other hand, pattern364includes fewer conserved positions than pattern314and results in shorter keys that are more likely to be mapped to multiple locations in the reference than a longer key. The identification of more candidate locations will require additional validation on an increased number of locations. Further processing is required in some embodiments to more accurately determine which of the multiple locations can be validated for a given data set, increasing computational cost. Thus, an effectively designed index takes into account these tradeoffs, and employs one or more key patterns designed to be both sufficiently long to avoid generating too many possible locations in the reference and sufficiently succinct to not rule out appropriate sequence relationships.

In some applications, the number of positions that is conserved in all possible instantiations of a polyoligomer dataset may not be enough to provide an acceptably-specific key. Thus, in certain implementations, the requirements for conserved positions are relaxed to obtain a longer key pattern such as key pattern342ofFIG. 3B. In this example, included in key pattern342are bases at positions330,332,334, and336(C, A, G, T, respectively), which only occur some of the time in various sets of oligomers. The corresponding sequence portions obtained by applying key pattern342to sequence302can be provided in the sequential base order of the reference, as in key344, or attached towards the end of the key as in346. In the naïve implementation, keys344and346allow some oligomer sets (such as304and308) to be mapped but not others (such as306,310, and312). It is also more likely for key342to map to a unique position in the reference than a shorter key such as316or318. Related issues are discussed further in the following paragraphs.

In some embodiments, certain possible instantiations are simply excluded from the key pattern design as well as from mapping processes. Where statistical information regarding the frequency of various instantiations is available, it may be possible to determine a key pattern of acceptable specificity (length) at the expense of not covering an acceptably small fraction of datasets. For example, it may be that the possible distance variations between two oligomers are −3, −2, −1, 0, +1, +2, and +3. Without knowledge of the corresponding probabilities of occurrence, key pattern364shown inFIG. 3C, based on the maximum distance variation of +/−3 bases, might be selected. However, if the corresponding probabilities of occurrence are determined (by empirical measurements, by estimation, or by other techniques) to be 0.05%, 9.95%, 20%, 10%, 35%, and 24.95%, and 0.05%, possible sequence arrangements that have a distance of +3 or −3 might be disregarded, permitting a key pattern based on distance variation of +/−2. Thus, a key pattern314shown inFIG. 3Amay be used for index generation and distance variations of +/−2 may be used for generation of keys from polyoligomers. In this case, those uncommon polyoligomer sets with a true distance of +3 or −3 will not be correctly instantiated and will fail to map or possibly be mapped incorrectly.

In other embodiments, the longer pattern is again used but instantiations resulting in ambiguous bases are accommodated by generating multiple keys, as described in more detail herein. This may be desirable if the use of the longer pattern without such accommodation results in too large a fraction of data sets not mapping. For those instantiations of a given polyoligomer which leave positions in the key pattern unspecified, bases may be selected based on data quality and/or statistical probability of base conservation, while in other implementations all possible completions of the pattern can be considered, as discussed herein. Examples of “ambiguous bases” includes bases in the data set which are not clearly one of the four expected bases for the data set, e.g., G, A, T, and C for DNA or G, A, U, and C for RNA. An ambiguous position includes an unidentified base, a base that is identified as two or more different bases for a single position in a data set, and the like.

Expanding ambiguous positions in this fashion can result in an exponential increase in compute cost. For this reason as well as others, in some embodiments the bases of a key pattern may be reordered in constructing a key. Some of the possible motivations are as follows.

In the situation just discussed, where the allowed separation distances are such that some portions of a key pattern may be unknown, if the bases selected by the key pattern are reordered to form the final key so that the unknown bases are the last bases of the key (or the least significant bits in terms of index lookup), and the index lookup mechanism is appropriateit is possible to avoid explicit enumeration of the possible completions. In specific situations, the key is effectively shorter (less specific), but only for those instantiations that result in incomplete specification of the key. Even if the goal of placing the unknown bases at the end of the key is not fully realized, a design in which the missing bases affect later positions (lower-order bits) of the key results in better memory locality (improved cache behavior). These situations can be illustrated byFIG. 3A, if we assume that distances of +3 and −3 bases will be considered in generating keys from polyoligomers, but are disregarded when deriving the key pattern. Pattern314ofFIG. 3Ais chosen in this example. Sequence groups322(TTAGCC) and324(GTGTGG) are more likely to appear in the oligomers since their locations are fixed and do not depend on the distance variation between oligomers. In comparison, sequence groups326(TT) and328(CA) are less likely to appear because these sequence groups appear in oligomer sequence sets if the distance variation is within +/−2 bases but do not always appear if the distance is +3 or −3 bases. Thus, key318is generated, with sequence groups322and324placed at the beginning of the key, and sequence groups326and328appended after portions322and324in the key since groups322and324are more likely to appear in keys derived from sequence sets than326and328. When using the resulting index to map a polyoligomer with separation distances allowed to range over +/−3, some bases of the key will be unspecified in some instantiations of the polyoligomer. For certain instantiations (in the example, separation distance within +/−2 on the left, and separation distance −3 on the right), the unspecified base is the last base of the key, so we do not need to enumerate the possible values. In others, one or two of the final four bases of the key may be unspecified. Due to reordering, the first twelve bases of the key will always be determined by the inner hexamers, hence retaining better specificity in the former approach and yielding better locality of multiple index lookups in the latter.

In some embodiments, reordering the bases of the key may have additional advantages even when all positions within the key are specified in all allowed separation relationships. A plurality of different keys for a single polyoligomer set may be obtained from a single key pattern, corresponding to instantiation with different possible separation relationships. Even where all bases of the key pattern are specified in all keys, certain bases of the pattern are modified less than others. If the bases of the key pattern are reordered to place the more-constant bases first in the key, better locality (cache performance) can be achieved across the set of index lookups for a given data set. This can be illustrated byFIG. 3A. Due to the requirement that the inner two hexamers be separated by precisely two bases, the same twelve bases can be made to occupy the same positions in all keys derived from a single polyoligomer. Reordering the bases of key pattern314such that segments322and324are placed at the beginning of the key, so that sequence302gives rise to key318, is an illustration of this concept. In some embodiments, this is taken further; keys are generated from a single data set and key pattern by examining sequence relationships in an order that improves locality among index lookups. By enumerating sequence relationship possibilities in certain orders, the more significant bits of the keys change less often. Referring toFIG. 3A, if the final order of bases in the key for sequence302is322+324+326+328, giving rise to keys such as318, e.g., it will be preferable to enumerate the possible separation relationships for a polyoligomer data set by considering all possible separations of the oligomers in the right portion together for each possible separation within the left portion; this will lead to considering all keys sharing the first 14 bases in a group. Whereas, inverting the process to consider all possible separations of the left portion together for each possible separation of the right portion will lead to considering keys sharing the first 12 and the last 2 bases in a group. For some embodiments of the reference index, the former will give better locality.

Oligomer Sequence Mapping

Once an index is generated, it can be used to map oligomer sequences obtained from samples using the key patterns used to produce the index.FIG. 5is a flowchart illustrating an implementation of a process for mapping oligomer sequences. Process500may be implemented on an oligomer mapping system such as that illustrated inFIG. 1at100. At502, a data set of related oligomer sequences (510) are received. At504, one or more key patterns (512) derived from a set of oligomer relationships are applied to a polyoligomer data set to obtain one or more keys that are consistent with the data set. The one or more keys are located (506) in a reference index (514) configured to map possible keys to their respective candidate locations in the reference. The reference index may be generated as illustrated inFIG. 2. The identified candidate locations are optionally validated (508), either using the index (514) as shown, or through other means. For example, the candidate locations can be validated by comparing the sequence at a candidate location of the reference to possible instantiations of the complete sequences of the polyoligomer. The successfully mapped location(s) are optionally output and may be used for genetic analysis, subsequent software processing, and many other applications. For example, the location(s) are used in some embodiments to assemble a new sequence/genome assembly. In some embodiments, the locations are used to detect novel sequences.

FIG. 6is a flowchart illustrating an implementation of a process for obtaining keys that are consistent with a given set of related oligomer sequences. In some embodiments, process600is used to implement process500(FIG. 5). At602, one or more distance values that could have existed between at least some of the given set of oligomer sequences are selected. At604, these possible distance values are used to instantiate the data set. One or more key patterns, based on the inputs of the sequence set610and the reference608, are applied to these possible instantiated sequences with the selected distance values to obtain one or more corresponding keys. These keys can be compared to those in the appropriate index606to identify possible locations.

FIG. 7is a sequence diagram illustrating an example of obtaining keys that are consistent with a given set of related oligomer sequences, using a process similar to600. For purposes of illustration, the example shows a partial key pattern700and a partial set of two adjacent, related hexamers702and704. A complete key pattern and a full set of related oligomers are used in practice. Here, key pattern700includes 2 conserved positions, followed by 2 non-conserved positions, followed by 6 conserved positions. Furthermore, the last base of the pattern is constrained to correspond to the last base of the second hexamer. Selective distance values and possible sequences are evaluated to determine whether hexamers702and704can be generated from such sequences.706shows the two hexamers overlapping by two bases. Since the end of hexamers702and the beginning of704(TC and TT, respectively) do not match perfectly when overlapped by 2, it is impossible for the distance between the hexamers to be −2, unless there is an error in the data set in one of the two conflicting bases. Had the end of hexamer702been TT instead of TC, a distance of −2 would have been feasible.708shows that it is also impossible for the distance to be −1 due to the mismatched end bases (C for702and T for704). It is possible, however, for the hexamers to have a distance of 0, 1, 2, or 3 and forming sequences710,712,718, and722, respectively. The instantiations from the data set can then be reordered to place the bases with the lower quality scores at the back of the key, forming reordered keys712,716,720, and724. Due to the fact that sequence722has one unknown position, there are four potential instantiations for724, one with each of the four bases at the unknown position. Thus, this instantiation is either disallowed or the missing position is filled in with all possible values to create substituted keys, as shown in724A-D. Locations in the reference can be looked up in the index using these keys.

In some embodiments, statistical distribution information about distance combinations within a set of related oligomer sequences can be used to limit computation. The information is used to include or exclude certain instantiations of a data set. For example, if it is known that when there is a distance of +1 between the first pair of oligomers, there is a 99.9% chance that there is a distance of 0 between the second pair of oligomers, sequence arrangements assigning a distance of +1 between the first pair and a non-zero distance to the second pair could therefore be omitted for the purpose of deriving keys from each data set.

In other embodiments, obtained polyoligomer data sets corresponding to the same mapping region may have one or more positions in the oligomer sets that are not clear data points, referred to herein as “ambiguous” bases or positions, such as those illustrated inFIGS. 8A through 8E. InFIG. 8A, a polyoligomer data set having sets802,804,806,808and812, which are all individually complete oligomers. Although these bases should have consistently conserved positions when key814is applied, as inFIG. 3A, here there is a discrepancy between the data sets, with sets804,806and810having a “T” at position848in the reference, and sets808and812having an “A” at position848. Instead of searching the index for a key with a base missing, as illustrated in sequential key816and reordered key818, the mapping operation can be carried out for each of the two potential variations, A and T, as shown in substituted sequential keys816A and816B and substituted reordered keys818A and818B.

In other examples, an unidentified position may be substituted with each of the potential bases, resulting in four possible keys for each unidentified position in a polyoligomer data set.FIG. 8Billustrates a scenario in which position848in the obtained polyoligomer data sets is completely unidentified, and therefore may be any of the four bases. The key814applied to these data sets results in a key of 16 bases, with 14 contiguous bases and one unknown base, illustrated as sequential key828and reordered key830. Rather than use only the 14 contiguous bases to map the data in the index, the index may be searched in four separate operations using the keys828A-D or830A-D, which respectively correspond to substituted keys having 16 contiguous bases that represent all of the possible combinations of the unidentified base in the initial key.

The use of multiple predicted keys generated from a polyoligomer data set to map potential locations in a reference index can be used to map data sets having two or more ambiguous or unidentified bases within a single data set.FIGS. 8C and 8Dillustrate the use of multiple keys that may be generated from polyoligomer data sets having two unidentified bases when key pattern842is applied to the data set (890,892,894and812). In this example, application of key pattern842results in two unidentified bases at positions832and834in reference802. When key pattern842is applied to the polyoligomer data set, the initial keys that are generated have two unidentified positions, as shown in sequential key896and reordered key898.

With multiple unidentified positions in a key, there are 4ndifferent potential keys that could be used to map the data set in an index, with n= the number of unidentified positions. The use of a large number of instantiations will need to be balanced against computer cost, likelihood for ambiguous or incorrect mapping data or technical cost for re-obtaining the data sets. In certain circumstances, it may be preferable to search the multiple keys, andFIG. 8Cillustrates the sequential keys896A-P that would be generated for two unidentified positions using key pattern842, with each of the variable positions highlighted in bold.FIG. 8Dillustrates the potential permutations if an additional third position were ambiguous, with a total of three unidentified bases in the initial key. This would result in 43or 64 possible substituted keys, which are set forth in 8D with the variable positions highlighted in bold.