Temporal memory adapted for single-shot learning and disambiguation of multiple predictions

Single-shot learning and disambiguation of multiple predictions in hierarchical temporal memory is provided. In various embodiments an input sequence is read. The sequence comprises first, second, and third time-ordered components. Each of the time-ordered components is encoded in a sparse distributed representation. The sparse distributed representation of the first time-ordered component is inputted into a first portion of a hierarchical temporal memory. The sparse distributed representation of the second time-ordered component is inputted into a second portion of the hierarchical temporal memory. The second portion is connected to the first portion by a first plurality of synapses. A plurality of predictions as to the third time-ordered component is generated within a third portion of the hierarchical temporal memory. The third portion is connected to the second portion by a second plurality of synapses. Based on the plurality of predictions, additional synaptic connections are added between the first portion and the second portion.

BACKGROUND

Embodiments of the present invention relate to hierarchical temporal memory systems, and more specifically, to using sparse distributed representations to learn and process temporal sequences in hierarchical temporal memory systems.

BRIEF SUMMARY

According to embodiments of the present disclosure, methods of, and computer program products for, operating a temporal memory are provided. An input sequence is read. The sequence comprises first, second, and third time-ordered components. Each of the first, second, and third time-ordered components is encoded in a sparse distributed representation. The sparse distributed representation of the first time-ordered component is inputted into a first portion of a hierarchical temporal memory. The sparse distributed representation of the second time-ordered component is inputted into a second portion of the hierarchical temporal memory. The second portion is connected to the first portion by a first plurality of synapses. A plurality of predictions as to the third time-ordered component is generated within a third portion of the hierarchical temporal memory. The third portion is connected to the second portion by a second plurality of synapses. Based on the plurality of predictions, additional synaptic connections are added between the first portion and the second portion.

According to other embodiments of the present disclosure, a system comprising temporal memory is provided. The temporal memory comprises first, second, and third portions. Each portion comprises a plurality of columns. Each column comprises a plurality of cells. The second portion is connected to the first portion by a first plurality of synapses. The third portion is connected to the second portion by a first plurality of synapses. A plurality of cells is active in the first portion, encoding a sparse distributed representation of a first time-ordered component. A plurality of cells is active in the second portion, encoding a sparse distributed representation of a plurality of instances of the second time-ordered component. The temporal memory is adapted to generate a plurality of predictions as to a third time-ordered component within the third portion, and based on the plurality of predictions, add additional synaptic connections between the first portion and the second portion.

According to other embodiments of the of the present disclosure, methods of, and computer program products for, operating a temporal memory are provided. A first time-ordered component of an input sequence is read. The first time-ordered component is encoded in a sparse distributed representation. The sparse distributed representation of the first time-ordered component is input into a region of a hierarchical temporal memory. The region is interconnected by a plurality of synapses. A second time-ordered component of the input sequence is read. The second time-ordered component is encoded in a sparse distributed representation. The sparse distributed representation of the second time-ordered component is input into the region of the hierarchical temporal memory. A plurality of predictions is generated as to a third time-ordered component of the input sequence within the region of the hierarchical temporal memory. Based on the plurality of predictions, additional synaptic connections are added in the region.

DETAILED DESCRIPTION

Hierarchical Temporal Memory (HTM) systems represent a new approach to machine intelligence. In an HTM system, training data comprising temporal sequences and/or spatial patterns are presented to a network of nodes. The HTM network then builds a model of the statistical structure inherent to the spatial patterns and temporal sequences in the training data, and thereby learns the underlying causes of the temporal sequences of patterns and sequences in the training data. The hierarchical structures of the HTM system allow them to build models of very high dimensional input spaces using reasonable amounts of memory and processing capacity.

The training process of an HTM system generally comprises a form of unsupervised machine learning. During a training process, one or more processing nodes of the HTM system form relationships between temporal sequences and/or spatial patterns present in training input and their associated causes or events. During the learning process, indexes indicative of the cause or events corresponding to the training input may be presented to the HTM system to allow the HTM system to associate particular categories, causes or events with the training input.

Once an HTM system has built a model of a particular input space, it can perform inference or prediction. To perform inference or prediction, novel input including temporal sequences or spatial patterns are presented to the HTM system. During the inference stage, each node in the HTM system produces an output that is more invariant and temporally stable than its input. That is, the output from a node in the HTM system is more abstract and invariant compared to its input. At its highest node, the HTM system will generate an output indicative of the underlying cause or event associated with the novel input.

Some HTM systems include both a spatial pooler and a sequence memory. The spatial pooler converts sequences of input data into sequences of sparse distributed representations (SDRs), or a long binary vector in which most of the bits are 0 and only a few are 1. This SDR is then used to activate the “columns” of a sequence memory.

Some HTM systems further include a processing node for learning and storing temporal sequences of spatial patterns in an input signal. The processing node may learn and store relationships between spatial patterns or temporal sequences of spatial patterns. The learning and storing of relationships or temporal sequences are performed autonomously in a manner that is robust against noise in the input signal. Based on the stored relationships, the processing node may process a subsequent input signal and generate an output that may represent prediction, identity of sequences of spatial patterns or other useful information.

The processing node includes a sequence processor receiving and processing the signal from the spatial pooler to learn, recognize and predict temporal sequences in the input signal. The sequence processor includes one or more columns, each column including one or more cells. A subset of columns may be selected by the spatial pooler signal, causing one or more cells in these columns to activate. When a cell activates, activation states of some other cells in the same node and/or level are detected and stored. By collectively storing the cell activation states in different cells, the sequence processor may store temporal sequences in the input signal.

Each cell includes one or more temporal memory segments. Different temporal memory segments in the cell store different cell activation states at different times, or points within a learned sequence. The sequence processor may activate a cell when the activation states of other cells correspond to cell activation states stored in a temporal memory segment of the cell.

There are a number of drawbacks to the sequence memory approach described above. Every time a cell fails to be predicted (is surprised), synaptic weights (sometimes described as permanences) are decreased, all of the cells within the column are activated (thus potentially leading to numerous subsequent surprised cells in the next timestep), and one cell is randomly chosen as a learning cell which is connected to cells activated in the next timestep. If this random choice turns out to be inappropriate, repeated learning over a number of exposures to the same sequence can eventually correct such a problem. This has the advantage of favoring the learning of repeating features and suppresses any noise in the sequence. However, the associated increasing and decreasing of permanence values while this learning occurs destroys any correlation between permanence strength and the number of times a sequence has been observed, and also steadily degrades the stored memory of previously learned sequences. In addition, when two different sequences that share a section have been learned and the system reaches this branch point and must supply a prediction, the system is incapable of differentiating between the predicted columns that correspond to the first learned sequence and the second learned sequence. Instead, the union of all such columns is predicted, without any information even as to how many unique SDRs might be participating in this union, nor how many times each SDR has been previously observed in this context (coming next after the just-observed partial sequence).

Thus, the present disclosure provides for improved temporal memory that can learn to recognize temporal sequences after a single learning cycle. Upon later exposure to a learned sequence, embodiments of the present disclosure correctly disambiguate in the case of multiple predictions. In particular, each relevant predicted SDR is provided together with the number of times that SDR has been previously observed as the next step in the temporal sequence.

In particular, the present disclosure provides an improved processing node for a temporal memory system. An improved sequence processor, receiving a sequence of sparse distributed representations from some stage lower in the hierarchical system, learns to recognize temporal sequences after a single learning cycle. Upon later exposure to a learned sequence, systems according to the present disclosure can correctly disambiguate in the case of multiple predictions, providing each relevant predicted sparse distributed representation distinctly together with the number of times that sparse distributed representation has been previously observed as the next step in the temporal sequence.

In various embodiments, temporal memory is provided in which synaptic permanences are never decreased, multiple cells are never activated within a column, and the activated cell (representing current bottom-up input to the overall system) in a current sparse distributed representation depends on predicted activations in a future (or next) SDR. Inclusion in the set of potentially active cells can either be restrictive, in which at least one segment must contain a threshold number of connections back to activated cells in the past (or previous) SDR, or permissive, in which at least one segment must contain a threshold number of connections back to any cell in an active column in the previous SDR.

In some embodiments, the assessment of which cell within the current cell-SDR to activate is based on predictions into the future column-SDR and is also identified by a stored iteration number. If the next SDR is not predicted by any cell within a column, a new instance is generated with a random pattern of cell activations and is assigned the next unused instance number. An unused instance number is identified with respect to the current context—the particular current column-SDR followed by that particular future column-SDR.

In some embodiments, once the current cell SDR is identified using future SDR information, then connections from that current SDR back to a past (or previous) SDR are forged, by linking multiple synaptic connections into a single segment which contains connections selected from across the previous SDR. This selection process is mostly random, but is designed to suppress connections from one column to itself, to ensure that all columns of the previous SDR are represented in at least one segment at the current cell-SDR, and to favor connections to columns that are activated infrequently across recently observed sequences. In addition, the relevant instance number is stored with the segment. If no new connections are necessary, then the correctly predicting synaptic weights are incremented, thus tabulating the number of times this particular last-to-current transition has been observed in this unique context.

For prediction, all columns in the next SDRs predicted by any cell in the current SDR are predicted. Unique SDRs are identified by the instance number recorded with the predicting cell segment.

With reference now toFIG. 1, an exemplary hierarchical temporal memory node100is depicted. Memory103includes columns104, each containing cells105. Each cell is laterally connected by a plurality of synapses106aggregated through segments107. Each HTM region consists of a number of highly interconnected columns. In some embodiments, cortical columns tend to inhibit neighboring columns, thus creating a sparse activation of columns. A cortical column is understood as a group of cells that have the same receptive field. Each column has a number of cells that are able to remember several previous states. A cell can be in one of three states: active, inactive and predictive.

When a cell becomes active, it gradually forms connections to nearby cells that tend to be active during several previous time steps. Thus, a cell learns to recognize a known sequence by checking whether the connected cells are active. If a large number of connected cells are active, this cell switches to the predictive state in anticipation of one of the few next inputs of the sequence. The output of a region includes columns in both active and predictive states. Thus, columns are active over longer periods of time, which leads to greater temporal stability seen by the parent region.

Cortical learning algorithms are able to learn continuously from each new input pattern. During inference, HTM tries to match a stream of inputs to fragments of previously learned sequences. This allows each HTM region to be constantly predicting the likely continuation of the recognized sequences. The index of the predicted sequence is the output of the region. Since predictions tend to change less frequently than the input patterns, this leads to increasing temporal stability of the output in higher hierarchy levels. Prediction also helps to fill in missing patterns in the sequence and to interpret ambiguous data by biasing the system to infer what it predicted.

During training, a node100receives a temporal sequence of spatial patterns as its input. Input data101are encoded into input neurons. A spatial pooler maps each input excitation to an appropriate SDR of constant sparsity102in memory103. A temporal pooler generates invariant representations (SDR)108of each recognized sequence. Once trained, the temporal memory can predict the next SDR in the sequence given the current SDR. Each cell can be predicted by lateral synaptic excitation from other cells, aggregated through segments.

In various embodiments described below, the temporal memory SDRs are considered to include one column vector for each of 3 successive time steps; was, is, and will-be.

Referring now toFIG. 2, exemplary column SDRs within a temporal memory are depicted (each column extending horizontally instead of vertically for ease of illustration). Column SDRs201,202,203correspond to was, is, and will-be, respectively. Cells within column SDR201are connected to cells in column SDR202by synapses211. Synapses are aggregated by segments212, and a threshold number of incoming synaptic activations is required to activate each segment. Cells in column SDR202are in turn connected to cells in column SDR203by synapses213through segments214. In conventional training, when a column is surprised, all of its cells are activated. Synapses that predict correctly get their permanence increased. Incorrect predictions or failing to assist in prediction have permanence decreased. In this case, an exemplary phrase “In the . . . ” corresponds to the was and is column SDRs201,202. In this example, all cells in column215are activated, indicating that the column was surprised. This burst on surprise compromises the ability of a network to learn a pattern on a single exposure, because the excess activations lead to formation of connections that must later be unlearned, damaging the network as a whole.

Referring toFIGS. 3-4, two exemplary phrases, “in the first” and “in the second” are illustrated for comparison. Each phrase is distributed across column SDRs201,202,203/401corresponding to sequential time steps. As shown, in both permutations, “the” has the same column SDR202. However, the permutations have two distinct cell SDRs, as illustrated by the different pattern of cell activations. The predictions end up being the union of all SDRs seen when at this point before.

Referring toFIGS. 3 and 5, two exemplary phrases, “in the first” and “by the first” are illustrated for comparison. Each phrase is distributed across column SDRs201/501,202,203corresponding to sequential time steps. As shown, in both permutations, “the” has the same column SDR202. However, the permutations are activated by different segments as illustrated by the distinct column SRDs201,501. It will be apparent that the same SDR (e.g.,202) can thus be differentiated by both outgoing and incoming context.

Referring toFIG. 6, a composite of the exemplary phrases ofFIGS. 3-4are illustrated. In this case, column SDR202, corresponding to the is state, includes the cell SDRs of each occurrence (SDR union), in this case given as “the” and “the′”. Column SDR601, corresponding to the will-be state, includes all possible next SDRs (SDR union) together with observed-frequency-of-occurrence, thereby enabling prediction of all next values. In this example, possible completions of the sample phrase, “first,” “second,” “third,” or “penultimate” are depicted by sets of activated cells602,603,604,605.

As set forth below, a single-shot temporal memory according to embodiments of the present disclosure provides differentiation of the same SDR by both outgoing and incoming context. Single-shot learning allows sequences to be memorized the first time they are shown. This allows prioritized assignment of finite resources, such that with limited resources, the most infrequent permutations are preferentially forgotten.

Referring toFIGS. 7-11, a learning process according to embodiments of the present disclosure is illustrated. Column SDRs701,702,703correspond to sequential time steps as described above.

Referring toFIG. 7, initially information is posted into the learning queue from the was cell SDR701. Given the current bottom-up inputs711, all relevant cells712within the is column SDR702are activated. In restrictive-mode, cells in the is cell SDR702are activated if and only if there exists one or more segment (e.g.,704) with a threshold number of synaptic connections (e.g.,705) back to the active cells711in the was cell SDR701. In permissive-mode, cells in the is cell SDR702are activated if there exists one or more segment (e.g.,704) with a threshold number of synaptic connections (e.g.,705) back to any cell within the active was column SDR701.

Referring toFIG. 8, given the saturated is cell SDR702, all cells of any possible (and not yet known) will-be column-SDR are passed through, marking all predicted segments. In this example, cells are shown in column SDR703in groups801. . .804, corresponding to each possible prediction. For each instance that is observed, the will-be cell SDR703it is passed through and all associated columns & cells are tagged for both this will-be SDR and the is SDR702(representing1disambiguated prediction, here shown as groups801. . .804). In example, tags are shown for exemplary instance numbers #7 and #8. The largest associated permanence values represent the number of times each instance is observed.

Referring toFIG. 9, given this saturated is cell-SDR, all cells of the now known will-be column SDR703are passed through, tabulating the number of times that cells in the saturated is cell-SDR702will successfully be able to predict a will-be segment. In this example, cells901have cardinality of 4 and cells902have cardinality of 1. At the same time, the instance numbers are noted. If a sufficiently large number of will-be columns is predicted, then the responsible instance number is identified. In this example, only cells803,804corresponding to those two instances meet the threshold. Adopting the example from above, these cells may be associated with the instance numbers corresponding to “the first” and “the second.”

Referring toFIG. 10, if will-be is predicted, all columns of the is column SDR702are traversed, and only the most-responsible cell of each column of the is column SDR is activated (in this example, cells901). If will-be is not predicted, all columns of the is column SDR702are traversed, and a cell is activated to form a new instance by going through all columns of the is column SDR. In some embodiments, a new cell is chosen randomly, while in some an already active cell is kept. Unlike in convention learning methods, there is no penalty imposed for failed predictions. Instead, synapses are added or supplemented after looking ahead to the will-be column SDRs.

Referring toFIG. 11, if the is cell-SDR (in SDR702) represents a new instance or if the previous was to is (701to702) connection failed to be predicted or recognized, all cells of the is cell-SDR (in SDR702) are passed through. At this point, SDR702is no longer saturated, having just one active cell901per column. A new segment1101is added, full of synaptic connections1102back to the was cell-SDR701, using the information posted into the learning queue.

Referring toFIG. 12, results are illustrated after training single shot hierarchical temporal memory according to the present disclosure. The horizontal scale corresponds to the number of words presented. A sequence of length3000, comprising the same 1000 Sparse Distributed Representations repeated three times is presented. Each SDR encodes one English word. Once the 1000-word sequence has been observed once, the system can fully predict the sequence without error and does no subsequent learning, only incrementing permanences as appropriate to record the multiple observations.

Graph1201illustrates the unpredicted columns during the presentation of 1000 words. During initial interval1211, most columns are unpredicted. In contrast, presentation of the same 1000 words in intervals1212and1213result in no unpredicted columns. Graph1202illustrates the new segments during the presentation of 1000 words. During initial interval1211, most many segments are new. In contrast, presentation of the same 1000 words in intervals1212and1213result in no new segments. Graph1203illustrates the synapses pulled from the learning queue during the presentation of 1000 words. During initial interval1211, many synapses are pulled. In contrast, presentation of the same 1000 words in intervals1212and1213result in no synapses pulled. Graph1204illustrates the permanence increments during the presentation of 1000 words. During initial interval1211, few permanences are incremented. In contrast, presentation of the same 1000 words in intervals1212and1213results in many permanence increments.

Referring toFIG. 13, results are illustrated after training single shot hierarchical temporal memory according to the present disclosure. The horizontal scale corresponds to the number of words presented. Five variations on a 14 word sentence are presented. Once the sentence has been observed once, the system can accurately predict the sequence notwithstanding its variations.

Graph1301illustrates the unpredicted columns during the presentation of the 14 word sentence. During initial interval1311, most columns are unpredicted. In contrast, presentation of variations on the 14 word sentence in intervals1312. . .1315result in few unpredicted columns. Graph1302illustrates the new segments during the 14 word sentence. During initial interval1311, most segments are new. In contrast, presentation of variations on the 14 word sentence in intervals1312. . .1315result in few new segments. Graph1303illustrates the synapses pulled from the learning queue during the presentation of the 14 word sentence. During initial interval1311, many synapses are pulled. In contrast, presentation of variations on the 14 word sentence in intervals1312. . .1315result in few synapses pulled. Graph1304illustrates the permanence increments during the presentation of the 14 word sentence. During initial interval1311, no permanences are incremented. In contrast, presentation of variations on the 14 word sentence in intervals1312. . .1315results in many permanence increments.

Referring toFIG. 14, an exemplary method according to embodiments of the present disclosure is illustrated. At1401, an input sequence is read. In some embodiments, the sequence comprises first, second, and third time-ordered components. At1402, each component of the input sequence is encoded in a sparse distributed representation. In some embodiments, each of the first, second, and third time-ordered components is encoded in a sparse distributed representation. At1403, the sparse distributed representation of the first time-ordered component of the input sequence is inputted into a first portion of a hierarchical temporal memory (HTM). At1404, the sparse distributed representation of the second time-ordered component of the input sequence is inputted into a second portion of the hierarchical temporal memory. In some embodiments, the second portion is connected to the first portion by a first plurality of synapses. At1405, a plurality of predictions as to the third time-ordered component is generated within a third portion of the hierarchical temporal memory. In some embodiments, the third portion is connected to the second portion by a second plurality of synapses. At1406, based on the plurality of predictions, additional synaptic connections are added between the first portion and the second portion.

It will be appreciated that each portion of the hierarchical temporal memory referred to above may share a common region of the hierarchical temporal memory. In particular, in various embodiments, the first, second and third portions may correspond to SDRs within the temporal memory. Thus, in some embodiments, the various portions are intermingled within the same region of the temporal memory, and the first and second time-ordered components are input at different times. The input of the first and second time-ordered components may or may not share cells and columns within a region, depending on the sparse distributed representation. Accordingly, it will be appreciated that the multiple SDR examples discussed above may represent the state of the same region of a hierarchical temporal memory at different times.