Patent Description:
<CIT> relates to stream processing which can be performed using a pull-based, anchor-based methodology that guarantees once and only once processing and repeatability of the creation of output with no additional communication overhead during normal processing. Each node (computing device) in the graph (representing interconnected computing devices) establishes a system of anchors. An anchor describes a point in the output stream of the node, so that every event in the stream is either before or after any given anchor.

<CIT> relates to a method for facilitating anchor shortening across streaming nodes in an event stream processing system which includes receiving a full anchor at an upstream marshaller. The full anchor is associated with a data batch that corresponds to one or more event streams. The full anchor includes an indication of an input point for the one or more event streams. The full anchor is received from an upstream compute processor. The method also includes mapping the full anchor to an index anchor and passing the index anchor to a downstream marshaller.

<CIT> relates to a system and method for startup and/or recovery for stream processing. During a startup phase: start anchor request(s), each identifying a particular time, are accumulated until request(s) are pending from downstream nodes. A minimum time of the accumulated start anchor request(s) is determined. If the processing system is an input node, an anchor associated with the determined minimum time is generated. Otherwise, the start anchor request is provided to an upstream node identifying the determined minimum time. Once the anchor associated with the determined minimum time is received (or generated), the anchor is provided in response to o polled start anchor request anchor for the determined minimum from a downstream node. Asynchronous requests for batches of data bounded by two specific anchors are performed in accordance with information stored in an ordered collection of anchors during a recovery phase.

The object of the invention is to provide enhanced journaling of streaming data anchors.

Described herein is a system for journaling of a streaming anchor resource, comprising: a processing system comprising a processor and a memory having computer-executable instructions stored thereupon which, when executed by the processor, cause the processing system to: by an input node of a streaming data processing system: in response to a determination to generate an anchor by the input node, store a value of a property associated with the streaming data in a persistent indexed data structure; generate the anchor that describes a particular point in time in a data stream, the anchor comprising an index into the persistent indexed data structure of the stored value of the property associated with the streaming data; and provide the generated anchor and streaming data to the downstream node.

Various technologies pertaining to journaling of streaming anchor resource(s) are now described with reference to the drawings, wherein like reference numerals are used to refer to like elements throughout. In other instances, well-known structures and devices are shown in block diagram form in order to facilitate describing one or more aspects. Further, it is to be understood that functionality that is described as being carried out by certain system components may be performed by multiple components. Similarly, for instance, a component may be configured to perform functionality that is described as being carried out by multiple components.

The subject disclosure supports various products and processes that perform, or are configured to perform, various actions regarding journaling of streaming anchor resource(s). What follows are one or more exemplary systems and methods.

Aspects of the subject disclosure pertain to the technical problem of recovery for stream processing using anchors. The technical features associated with addressing this problem involve by an input node of a streaming data processing system: in response to determination to generate an anchor by the input node, storing a value of a property associated with the streaming data in a persistent indexed data structure; generating the anchor that describes a particular point in time in a data stream, the anchor comprising an index into the persistent indexed data structure of the stored value of the property associated with the streaming data; and providing the generated anchor and streaming data to the downstream node. Accordingly, aspects of these technical features exhibit technical effects of more efficiently and effectively storing information (e.g., property(ies) for recovery of computing nodes and/or output node(s) of a stream processing system), for example, reducing consumption of network bandwidth and/or computing node processing resource(s).

As used herein, the terms "component" and "system," as well as various forms thereof (e.g., components, systems, sub-systems, etc.) are intended to refer to a computer-related entity, either hardware, a combination of hardware and software, software, or software in execution. For example, a component may be, but is not limited to being, a process running on a processor, a processor, an object, an instance, an executable, a thread of execution, a program, and/or a computer. By way of illustration, both an application running on a computer and the computer can be a component. One or more components may reside within a process and/or thread of execution and a component may be localized on one computer and/or distributed between two or more computers. Further, as used herein, the term "exemplary" is intended to mean serving as an illustration or example of something, and is not intended to indicate a preference.

In some embodiments, the technologies disclosed herein are used in conjunction with pull-based stream processing employing anchor-based technology that facilitates once and only once processing by output node(s). An "anchor" is generated by an input node (e.g., along with data of an output stream) and describes a point in the output stream of the input node, so that other unit(s) of data in the output stream are either before or after any given anchor. For purposes of discussion, data "flows" downstream from an input node through any intermediate computing node(s) to an output node. A request for data is sent upstream from an output node through any intermediate computing node(s) to an input node.

During normal operation, down-stream nodes receive an anchor along with a batch of data. The anchor, generated by the input node that generated the batch of data, is a pointer in the output stream reflecting the last portion of data included in the batch of data. Once a downstream node (e.g., intermediate node and/or output node) has received and processed the batch of data, the downstream node can request another batch of data beginning with the anchor of batch of data last received/processed (e.g., indicative of the downstream node having processed data up to the anchor).

In some embodiments, non-input nodes (e.g., intermediate computing nodes and output nodes) do not need to understand anchors or how the anchors were generated by input nodes. The non-input nodes are responsible for being able to determine comparability of anchors. That is, whether a particular anchor indicates an event or point in the output stream occurring before another anchor, whether the particular anchor is equal to another anchor, or whether the particular anchor indicates an event or point in the output stream occurring after another anchor.

Described herein is a system and method for using journal(s) to persistently store anchor information (e.g., textual value(s)). In response to a determination to generate an anchor by an input node, the input node can generate the anchor to be utilized by downstream node(s) to process streaming data provided in response to the request. As discussed, below requisite comparability of anchors can be maintained.

Instead of storing one or more property(ies) associated with the streaming data (e.g., textual property(ies)) directly in the anchor, the input node can store value(s) of one or more property(ies) associated with the streaming data being provided in response to the request in one or more persistently stored journals. The anchor generated and provided to downstream node(s) includes an index (or indices) into the journal(s). During recovery, a downstream node provides the anchor with the index (or indices) into the journal(s), which is used by the input node, to retrieve the value(s) of property(ies) from the journal. A batch of streaming data is provided to the downstream node based, at least in part, upon the retrieved value(s) of property(ies).

Referring to <FIG>, a system for journaling of a streaming anchor resource <NUM> is illustrated. The system <NUM> includes a streaming data processing system <NUM> having one or more input nodes <NUM> and one or more output nodes <NUM>. Optionally, the streaming data processing system <NUM> can further include one or more compute nodes <NUM>. The system <NUM> further includes one or more persistent journal stores <NUM>.

The system <NUM> can be used with pull-based stream processing employing anchor-based technology that facilitates once and only once processing by output node(s). A request for data is sent upstream from the output node <NUM> through any intermediate compute node(s) <NUM> to the input node <NUM>.

An anchor, generated by the input node <NUM> that generated the batch of data, is a pointer in the data stream reflecting the last portion of data included in the batch of data. Once a downstream node (e.g., compute node <NUM> and/or output node <NUM>) has received and processed the batch of data, the downstream node can request another batch of data beginning with the anchor of batch of data last received/processed (e.g., indicative of the downstream node having processed data up to the anchor).

Streaming anchors are thus pointers in the data stream that carry the necessary information to resume processing from a specific point across a plurality of inputs. Additionally, anchors generated by a specific entity must have full order ability among themselves - that is, the anchors must be temporally comparable.

In order to achieve these features, an anchor can carry one or more attributes with some attributes being, for example, numeric (e.g., timestamp, offset, etc.) and some attributes being textual (e.g., resource identifier, paths, a name of an event hub, a full path of a blob, etc.).

In some embodiments, textual attribute(s) can generate memory load on the streaming data processing system <NUM>, since each batch generated by any entity in the system <NUM> carries the entire tree of anchors spanned by the entity in the topology graph. In some embodiments, each such anchor carries the values mentioned above (e.g., usually twice - once for the ability to compare - binary format and once for the ability to resume - textual/numeric).

To reduce this memory load on the streaming data processing system <NUM>, the input node <NUM> uses data structure(s) (e.g., journal(s) or table(s)) stored in the persistent journal store <NUM> to persistently store anchor information (e.g., textual value(s)). Instead of storing one or more property(ies) associated with the streaming data (e.g., textual property(ies)) directly in the anchor, the input node stores value(s) of one or more property(ies) associated with the streaming data being provided in one or more persistently stored data structures (e.g., journals, tables) stored in the persistent journal store <NUM>. The anchor generated and provided to downstream node(s) includes an index (or indices) into the data structure(s) (e.g., journal(s), table(s)). In some embodiments, entry(ies) are only added to the data structure(s) for property(ies) (e.g., pointing to a resource) that have not been previously included in the data structure(s).

The input node <NUM> determines to generate the anchor in response to a request for streaming data from a downstream node. In some embodiments, the input node <NUM> determines to generate the anchor in response to other trigger(s) within the streaming data processing system <NUM>.

In this manner, the memory load on the streaming data processing system <NUM> can be significantly reduced without affecting the ability of the input node <NUM> to facilitate recovery of a downstream node. In some embodiments, the memory load can be reduced by approximately two orders of magnitude with respect to metadata carried by the streaming data processing system <NUM>.

During recovery, a downstream node provides the anchor with the index (or indices) into the data structure(s) (e.g., journal(s), table(s)), which is used by the input node <NUM>, to retrieve the value(s) of property(ies) from the data structure(s) stored in the persistent journal store <NUM>.

In some embodiments, the persistent journal store <NUM> is stored physically separate from the streaming data processing system <NUM>. In this manner, failure(s) of input node(s) <NUM>, output node(s) <NUM>, compute node(s) <NUM>, and/or the streaming data processing system <NUM> itself does not affect the data stored in the persistent journal store <NUM>.

Referring to <FIG>, an input node <NUM> can include an anchor generation component <NUM> and a node recovery component <NUM>. During normal operation, in response to a request for streaming data from a downstream node, the input node <NUM> can obtain streaming data with an anchor generation component <NUM> generating an anchor to be provided to the downstream node(s) along with the obtained streaming data.

The anchor generation component <NUM> can store one or more value(s) of property(ies) (e.g., attribute(s)) associated with the streaming data in an append only ordered collection (e.g., data structure such as a journal, table, etc.) in the persistent journal store <NUM>. In some embodiments, only value(s) for specific property(ies) are stored in the persistent journal store <NUM> with other value(s) for other specific property(ies) being stored directly in the anchor. In some embodiments, values for each specific property are stored in separate data structures in the persistent journal store <NUM>, that is, each specific property has a separate data structure. In some embodiments, the ordered collection comprises a table <NUM> having an index <NUM> and a stored value <NUM>.

In some embodiments, indices (e.g., a numeric value) into the ordered collection and anchors are generated in a strictly growing order, for each input node <NUM>. Accordingly, a new anchor and index into the ordered collection (data structure) generated by an input node <NUM> is greater than (e.g., represents a later point in a stream) any anchor and index previously generated by the particular input node <NUM>. This can allow for the system to store value(s) (e.g., attribute(s)) for anchors in the ordered collection (e.g., data structure such as a journal, table) for use during recovery.

Thus, the anchor generated by the anchor generation component <NUM> describes a specific point in time in the data stream and include an index (e.g., eight byte numeric value) into the persistent indexed data structure(s) of stored value(s) of property(ies) (e.g., attribute(s)) associated with the streaming data. The input node <NUM> can then provide the generated anchor and streaming data to the downstream node.

In some embodiment, during normal operation, the persistent indexed data structure(s) (e.g., journal(s)) are not read, and the only additional overhead is a write operation, which is also done only when there is change in the entity being handled (e.g., such as path, resource identifier, etc.), that is, a change in property(ies)/attribute(s). In some embodiments, these operation(s) are not usually required for every batch.

During recovery of downstream node(s), the node recovery component <NUM> can utilize information stored in the ordered collection (e.g., data structure, journal, table, etc.) stored in the persistent journal store <NUM>. In response to receiving a request for streaming data commencing with a particular anchor, the node recovery component <NUM> can retrieve value(s) of property(ies) associated with the particular anchor from the ordered collection (e.g., data structure, journal, table, etc.) stored in the persistent journal store <NUM>. Using the property value(s) retrieved, the input component <NUM> can provide a batch of streaming data to the downstream node commencing with the anchor.

<FIG> illustrate exemplary methodologies relating to journaling of streaming anchor resource(s). While the methodologies are shown and described as being a series of acts that are performed in a sequence, it is to be understood and appreciated that the methodologies are not limited by the order of the sequence. For example, some acts can occur in a different order than what is described herein. In addition, an act can occur concurrently with another act. Further, in some instances, not all acts may be required to implement a methodology described herein.

Moreover, the acts described herein may be computer-executable instructions that can be implemented by one or more processors and/or stored on a computer-readable medium or media. The computer-executable instructions can include a routine, a sub-routine, programs, a thread of execution, and/or the like. Still further, results of acts of the methodologies can be stored in a computer-readable medium, displayed on a display device, and/or the like.

Referring to <FIG>, a method of generating a streaming data anchor <NUM> is illustrated. In some embodiments, the method <NUM> is performed by the input node <NUM> during normal operation (e.g., non-recovery of input node(s) <NUM>, output node(s) <NUM>, and/or compute node(s) <NUM>) of the streaming data processing system <NUM>.

At <NUM>, in response to a determination to generate an anchor by an input node, a value of a property associated with the streaming data is stored in a persistent indexed data structure. At <NUM>, the anchor that describes a specific point in time in a data stream is generated. The anchor includes an index into the persistent indexed data structure of the stored value of the property associated with the streaming data. At <NUM>, providing the generated anchor and streaming data are provided to the downstream node.

Referring to <FIG>, a method of utilizing a streaming data anchor <NUM> is illustrated. The method <NUM> is performed by the input node <NUM>. In some embodiments, the method <NUM> is performed by the input node <NUM> during recovery of output node(s) <NUM>, and/or compute node(s) <NUM> of the streaming data processing system <NUM>.

At <NUM>, a request for streaming data starting with a particular point in time in a data stream referenced by the anchor is received. At <NUM>, a value of a property associated with the streaming data is retrieved from the persistent indexed data structure based, at least in part, upon the received anchor. At <NUM>, a batch of streaming data to the downstream node is provided in response to the request for streaming data. The batch of streaming data starts with the particular point in time referenced by the anchor. The batch is based, at least in part, upon the retrieved the value of the property.

Next, referring to <FIG> and <FIG>, a method of journaling of streaming anchor resource(s) <NUM> is illustrated. In some embodiments, the method <NUM> is performed by the input node <NUM>. In some embodiments, acts <NUM>-<NUM> occur during normal operation of the streaming data processing system <NUM>, and acts <NUM>-<NUM> occur during recovery of the streaming data processing system <NUM>.

At <NUM>, in response to a determination to generate an anchor by an input node, value(s) of property(ies) associated with the streaming data is stored in one or more persistent indexed data structure(s). At <NUM>, the anchor that describes a specific point in time in a data stream is generated. The anchor includes index/indices into the persistent indexed data structure(s) of the stored value(s) of the property(ies) associated with the streaming data. At <NUM>, the generated anchor and streaming data are provided to the downstream node.

At <NUM>, a request for streaming data starting with a particular point in time in a data stream referenced by the anchor is received. At <NUM>, using the received anchor, value(s) of property(ies) associated with the streaming data are received from the persistent indexed data structure(s). At <NUM>, provide a batch of streaming data is provided to the downstream node. The batch of streaming data starts with the particular point in time referenced by the anchor. The batch is based, at least in part, upon the retrieved the value of the property.

Described herein is a system for journaling of a streaming anchor resource, comprising: a processing system comprising a processor and a memory having computer-executable instructions stored thereupon which, when executed by the processor, cause the processing system to: by an input node of a streaming data processing system: in response to a determination to generate an anchor by the input node, store a value of a property associated with streaming data in a persistent indexed data structure; generate the anchor that describes a particular point in time in a data stream, the anchor comprising an index into the persistent indexed data structure of the stored value of the property associated with the streaming data; and provide the generated anchor and streaming data to the downstream node.

The system can include the memory having further computer-executable instructions stored thereupon which, when executed by the processor, cause the processing system to: receive a request for streaming data starting with a particular point in time in a data stream referenced by the anchor; retrieve a value of a property associated with the streaming data from the persistent indexed data structure based, at least in part, upon the received anchor; and provide a batch of streaming data to the downstream node, wherein the batch of streaming data starts with the particular point in time referenced by the anchor, and, the batch of streaming data is based, at least in part, upon the retrieved the value of the property.

The system can further include wherein a plurality of values of a particular property are stored in the persistent indexed data structure. The system can further include wherein the plurality of values are stored in an append only ordered collection. The system can further include wherein the property comprises a textual attribute associated with the streaming data and comprises a name of an event hub, or a full path of a blob.

The system can further include wherein during execution of a streaming job, a downstream node is configured to determine whether the anchor describes a point in an output stream before, after, or equal to another anchor. The system can further include wherein the persistent indexed data structure is stored physically separate from the streaming data processing system.

Described herein is a method of journaling of streaming an anchor resource, comprising: by an input node of a streaming data processing system: in response to a determination to generate an anchor by the input node, storing a value of a property associated with the streaming data in a persistent indexed data structure; generating the anchor that describes a particular point in time in a data stream, the anchor comprising an index into the persistent indexed data structure of the stored value of the property associated with the streaming data; and providing the generated anchor and streaming data to the downstream node.

The method can further include receiving a request for streaming data starting with a particular point in time in a data stream referenced by the anchor; retrieving a value of a property associated with the streaming data from the persistent indexed data structure based, at least in part, upon the received anchor; and providing a batch of streaming data to the downstream node, wherein the batch of streaming data starts with the particular point in time referenced by the anchor, and, the batch of streaming data is based, at least in part, upon the retrieved the value of the property.

The method can further include wherein a plurality of values of a particular property are stored in the persistent indexed data structure. The method can further include wherein the plurality of values are stored in an append only ordered collection. The method can further include wherein the property comprises a textual attribute associated with the streaming data.

The method can further include wherein the property comprises a name of an event hub, or a full path of a blob. The method can further include wherein the persistent indexed data structure is stored physically separate from the streaming data processing system, and wherein during execution of a streaming job, a downstream node is configured to determine whether the anchor describes a point in an output stream before, after, or equal to another anchor.

Described herein is a computer storage media storing computer-readable instructions that when executed cause a computing device to: in response to a determination to generate an anchor by an input node, store a value of a property associated with the streaming data in a persistent indexed data structure; generate the anchor that describes a particular point in time in a data stream, the anchor comprising an index into the persistent indexed data structure of the stored value of the property associated with the streaming data; and provide the generated anchor and streaming data to the downstream node.

The computer storage media can store further computer-readable instructions that when executed cause a computing device to: receive a request for streaming data starting with a particular point in time in a data stream referenced by the anchor; retrieve a value of a property associated with the streaming data from the persistent indexed data structure based, at least in part, upon the received anchor; and provide a batch of streaming data to the downstream node, wherein the batch of streaming data starts with the particular point in time referenced by the anchor, and, the batch of streaming data is based, at least in part, upon the retrieved the value of the property.

The computer storage media can further include wherein a plurality of values of a particular property are stored in the persistent indexed data structure. The computer storage media can further include wherein the plurality of values are stored in an append only ordered collection. The computer storage media can further include wherein the property comprises a textual attribute associated with the streaming data. The computer storage media can further include wherein the property comprises at a name of an event hub, or a full path of a blob.

With reference to <FIG>, illustrated is an example general-purpose computer, processing system, or computing device <NUM> (e.g., mobile phone, desktop, laptop, tablet, watch, server, hand-held, programmable consumer or industrial electronics, set-top box, game system, compute node, etc.). For instance, the computing device <NUM> may be used in a system for journaling of a streaming anchor resource <NUM>.

The computer <NUM> includes one or more processor(s) <NUM>, memory <NUM>, system bus <NUM>, mass storage device(s) <NUM>, and one or more interface components <NUM>. The system bus <NUM> communicatively couples at least the above system constituents. However, it is to be appreciated that in its simplest form the computer <NUM> can include one or more processors <NUM> coupled to memory <NUM> that execute various computer executable actions, instructions, and or components stored in memory <NUM>. The instructions may be, for instance, instructions for implementing functionality described as being carried out by one or more components discussed above or instructions for implementing one or more of the methods described above.

The processor(s) <NUM> can be implemented with a general purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. The processor(s) <NUM> may also be implemented as a combination of computing devices, for example a combination of a DSP and a microprocessor, a plurality of microprocessors, multi-core processors, one or more microprocessors in conjunction with a DSP core, or any other such configuration. In one embodiment, the processor(s) <NUM> can be a graphics processor.

The computer <NUM> can include or otherwise interact with a variety of computer-readable media to facilitate control of the computer <NUM> to implement one or more aspects of the claimed subject matter. The computer-readable media can be any available media that can be accessed by the computer <NUM> and includes volatile and nonvolatile media, and removable and non-removable media. Computer-readable media can comprise two distinct and mutually exclusive types, namely computer storage media and communication media.

Computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer-readable instructions, data structures, program modules, or other data. Computer storage media includes storage devices such as memory devices (e.g., random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), etc.), magnetic storage devices (e.g., hard disk, floppy disk, cassettes, tape, etc.), optical disks (e.g., compact disk (CD), digital versatile disk (DVD), etc.), and solid state devices (e.g., solid state drive (SSD), flash memory drive (e.g., card, stick, key drive) etc.), or any other like mediums that store, as opposed to transmit or communicate, the desired information accessible by the computer <NUM>. Accordingly, computer storage media excludes modulated data signals as well as that described with respect to communication media.

Communication media embodies computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media. By way of example, and not limitation, communication media includes wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared and other wireless media.

Memory <NUM> and mass storage device(s) <NUM> are examples of computer-readable storage media. Depending on the exact configuration and type of computing device, memory <NUM> may be volatile (e.g., RAM), non-volatile (e.g., ROM, flash memory, etc.) or some combination of the two. By way of example, the basic input/output system (BIOS), including basic routines to transfer information between elements within the computer <NUM>, such as during start-up, can be stored in nonvolatile memory, while volatile memory can act as external cache memory to facilitate processing by the processor(s) <NUM>, among other things.

Mass storage device(s) <NUM> includes removable/non-removable, volatile/non-volatile computer storage media for storage of large amounts of data relative to the memory <NUM>. For example, mass storage device(s) <NUM> includes, but is not limited to, one or more devices such as a magnetic or optical disk drive, floppy disk drive, flash memory, solid-state drive, or memory stick.

Memory <NUM> and mass storage device(s) <NUM> can include, or have stored therein, operating system <NUM>, one or more applications <NUM>, one or more program modules <NUM>, and data <NUM>. The operating system <NUM> acts to control and allocate resources of the computer <NUM>. Applications <NUM> include one or both of system and application software and can exploit management of resources by the operating system <NUM> through program modules <NUM> and data <NUM> stored in memory <NUM> and/or mass storage device (s) <NUM> to perform one or more actions. Accordingly, applications <NUM> can turn a general-purpose computer <NUM> into a specialized machine in accordance with the logic provided thereby.

All or portions of the claimed subject matter can be implemented using standard programming and/or engineering techniques to produce software, firmware, hardware, or any combination thereof to control a computer to realize the disclosed functionality. By way of example and not limitation, system <NUM> or portions thereof, can be, or form part, of an application <NUM>, and include one or more modules <NUM> and data <NUM> stored in memory and/or mass storage device(s) <NUM> whose functionality can be realized when executed by one or more processor(s) <NUM>.

In some embodiments, the processor(s) <NUM> can correspond to a system on a chip (SOC) or like architecture including, or in other words integrating, both hardware and software on a single integrated circuit substrate. Here, the processor(s) <NUM> can include one or more processors as well as memory at least similar to processor(s) <NUM> and memory <NUM>, among other things. Conventional processors include a minimal amount of hardware and software and rely extensively on external hardware and software. By contrast, an SOC implementation of processor is more powerful, as it embeds hardware and software therein that enable particular functionality with minimal or no reliance on external hardware and software. For example, the system <NUM> and/or associated functionality can be embedded within hardware in a SOC architecture.

The computer <NUM> also includes one or more interface components <NUM> that are communicatively coupled to the system bus <NUM> and facilitate interaction with the computer <NUM>. By way of example, the interface component <NUM> can be a port (e.g., serial, parallel, PCMCIA, USB, FireWire, etc.) or an interface card (e.g., sound, video, etc.) or the like. In one example implementation, the interface component <NUM> can be embodied as a user input/output interface to enable a user to enter commands and information into the computer <NUM>, for instance by way of one or more gestures or voice input, through one or more input devices (e.g., pointing device such as a mouse, trackball, stylus, touch pad, keyboard, microphone, joystick, game pad, satellite dish, scanner, camera, other computer, etc.). In another example implementation, the interface component <NUM> can be embodied as an output peripheral interface to supply output to displays (e.g., LCD, LED, plasma, etc.), speakers, printers, and/or other computers, among other things. Still further yet, the interface component <NUM> can be embodied as a network interface to enable communication with other computing devices (not shown), such as over a wired or wireless communications link.

Claim 1:
A system for generating and utilizing a streaming data anchor, comprising:
a processing system (<NUM>) comprising a processor (<NUM>) and a memory (<NUM>) having computer-executable instructions stored thereupon which, when executed by the processor, cause the processing system to:
by an input node (<NUM>) during normal operation of a streaming data processing system (<NUM>):
in response to a request for streaming data from a downstream node (<NUM>, <NUM>), determine to generate an anchor;
in response to the determination to generate the anchor, instead of storing a property associated with the streaming data directly in the anchor, store (<NUM>) a value of the property associated with the streaming data in a persistent indexed data structure, wherein entries are only added to the persistent indexed data structure for properties that have not been previously included in the data structure;
generate (<NUM>) the anchor that describes a particular point in time in a data stream, the anchor comprising an index into the persistent indexed data structure of the stored value of the property associated with the streaming data; and
provide (<NUM>) the generated anchor and the streaming data to the downstream node; and
by the input node during recovery of the downstream node:
receive (<NUM>) a request for streaming data starting with the particular point in time in the data stream referenced by the anchor, including receiving the anchor with the index into the persistent indexed data structure;
retrieve (<NUM>) the value of the property associated with the streaming data from the persistent indexed data structure based upon the received anchor with the index into the persistent indexed data structure; and
provide (<NUM>) a batch of streaming data to the downstream node, wherein the batch of streaming data starts with the particular point in time referenced by the anchor, and, the batch of streaming data is based upon the retrieved value of the property.