Suppressing stream functionality to expedite preferred data

A controller analyzes a tuple in an operator graph. The controller determines that the tuple includes one or more selected characteristics. These characteristics signify preferred data. The controller determines operations of the operator graph which can be suppressed. The controller suppresses the one or more operations. The controller suppresses those operations in response to the tuple including one or more of the selected characteristics.

BACKGROUND

This disclosure generally relates to stream computing, and in particular, to computing applications that receive streaming data and process the data as it is received.

In traditional data processing, a controller will run queries against static data sources, resulting in generally static results. Alternatively, stream computing allows a controller to execute an effectively continuous query (e.g., a query on a stream). In this way, results may be regularly updated as data sources are updated and added to the stream.

SUMMARY

Embodiments of the disclosure provide a method, system, and computer program product for processing data. The method, system, and computer program product receive two or more tuples to be processed by a plurality of processing elements operating on one or more computer processors.

Aspects of the disclosure are directed towards suppressing streams functionality to expedite processing of preferred data. A controller may analyze a tuple in an operator graph. The controller may determine that the tuple includes one or more selected characteristics. These characteristics may signify preferred data. The controller may determine operations of the operator graph which may be suppressed. The operations may include an operator or processing element which will act upon the tuple or peripheral activities such as dynamic connections or extraneous logging which will facilitate processing the tuple. The controller may then suppress the one or more operations. The controller may both determine the operations to suppress and also suppress those operations in response to the tuple including one or more of the selected characteristics. These operations may be suppressed only in the execution of the tuple, leaving other tuples to be executed using the operations as applicable. Alternatively, the operations may be suppressed entirely in all instances until the tuple has passed through the operator graph.

DETAILED DESCRIPTION

Aspects of the present disclosure relate to data streams, more particular aspects relate to bypassing stream functionality when certain data is present. When a streaming application processes data, some of the data may be relatively more important than other portions of the data from the perspective of an end user. A stream may therefore be monitored to identify which tuples hold these relatively more important portions of data. The stream may be monitored by a controller. The controller may determine that a tuple contains important data. As a result, the controller may suppress some streaming operations to speed up the processing of the tuple and accomplish goals of the application quicker. While the present disclosure is not necessarily limited to such applications, various aspects of the disclosure may be appreciated through a discussion of various examples using this context.

In stream computing environments, data moves from one mini-process (e.g., operator) to another. When an operator receives data, the operator may “wake up” to perform its process. These processes can include analyzing, sorting, or manipulating data in predetermined ways. Following completion of its process, the operator may transmit the data to another operator. Operators are configured in specific orders, where their mini-processes can combine to create a more complex process (e.g., processing element), which can itself be combined to create a closed process (e.g., graph, or operator graph) which would often have an end goal for how the data will be analyzed, sorted, or manipulated. Operator graphs may be configured to take data and use a number of operators/processing elements to transform the data from one form/location to another form/location, depending upon the contents of the data. For some streams, certain operators are configured only for some types of data, so a certain type of data may pass through one or more operators unchanged if those operators are not configured for that certain type. Additionally, certain operators may not be necessary but are only helpful, such as operators that clean up data, remove noise from data, or augment data into a more useful state.

New data may trigger work in a streaming system. Data may move through the streaming system as tuples. These tuples may be processed sequentially in a first-in-first-out format. The tuples may be processed by the streaming system for different applications. For applications that prioritize certain queries/conditions over others, the tuples related to those queries/conditions may be more important than others tuples. If the streaming system is working with either a long process or a large backlog of data when an “important” tuple is received, the application may not gain the benefit of processing this important tuple until the backlog is cleared and/or the long process is complete. In operator graphs with a long process, the final benefit of the tuple may be delayed by processing the tuple in ways which are not relevant or necessary to deriving the final benefit. In instances where a query is time sensitive, some benefit of an important tuple may be lost while searching for an answer which currently exists stuck in such a long process or backlog.

Aspects of the disclosure relate to monitoring tuples in an operator graph. The tuples may be analyzed to determine whether or not the tuples contain any of the data which is deemed important. Data may be deemed important because of hard coded values, reactions of the system to the data, or learned values of the system. The tuples could be analyzed for this important data by many means known in the art. Once a controller determines that a tuple contains important data, the functionality of the streaming process may be altered for the tuple. Specifically, a controller may suppress some functionality of the streaming process for the tuple. By suppressing some functionality, the controller may accelerate processing of the tuple, resulting in performance benefits for the streaming system.

Stream-based computing and stream-based database computing are emerging as a developing technology for database systems. Products are available which allow users to create applications that process and query streaming data before it reaches a database file. With this emerging technology, users can specify processing logic to apply to inbound data records while they are “in flight,” with the results available in a very short amount of time, often in fractions of a second. Constructing an application using this type of processing has opened up a new programming paradigm that will allow for development of a broad variety of innovative applications, systems, and processes, as well as present new challenges for application programmers and database developers.

In stream computing applications, stream operators are connected to one another such that data flows from one stream operator to the next (e.g., over a TCP/IP socket). When a stream operator receives data, it may perform operations, such as analysis logic, which may change the tuple by adding or subtracting attributes, or updating the values of existing attributes within the tuple. When the analysis logic is complete, a new tuple is then sent to the next stream operator. Scalability is achieved by distributing an application across nodes by creating executables (i.e., processing elements), as well as replicating processing elements on multiple nodes and load balancing among them. Stream operators in a stream computing application can be fused together to form a processing element that is executable. Doing so allows processing elements to share a common process space, resulting in much faster communication between stream operators than is available using inter-process communication techniques (e.g., using a TCP/IP socket). Further, processing elements can be inserted or removed dynamically from an operator graph representing the flow of data through the stream computing application. In addition, stream operators in the same operator graph may be hosted on different nodes (e.g., on different compute nodes or on different cores of a compute node).

Data flows from one stream operator to another in the form of a “tuple.” A tuple is a sequence of one or more attributes associated with an entity. Attributes may be any of a variety of different types (e.g., integer, float, Boolean, string, etc.). The attributes may be ordered. In addition to attributes associated with an entity, a tuple may include metadata (i.e., data about the tuple). A tuple may be extended by adding one or more additional attributes or metadata to it. As used herein, “stream” or “data stream” refers to a sequence of tuples. Generally, a stream may be considered a pseudo-infinite sequence of tuples.

Tuples are received and output by stream operators and processing elements. An input tuple corresponding with a particular entity that is received by a stream operator or processing element, however, is generally not considered to be the same tuple that is output by the stream operator or processing element, even if the output tuple corresponds with the same entity or data as the input tuple. An output tuple need not be changed in some way from the input tuple.

Nonetheless, an output tuple may be changed in some way by a stream operator or processing element. An attribute or metadata may be added, deleted, or modified. For example, a tuple will often have two or more attributes. A stream operator or processing element may receive the tuple having multiple attributes and output a tuple corresponding with the input tuple. The stream operator or processing element may only change one of the attributes so that all of the attributes of the output tuple except one are the same as the attributes of the input tuple.

Generally, a particular tuple output by a stream operator or processing element may not be considered to be the same tuple as a corresponding input tuple even if the input tuple is not changed by the processing element. However, to simplify the present description and the claims, an output tuple that has the same data attributes or is associated with the same entity as a corresponding input tuple will be referred to herein as the same tuple unless the context or an express statement indicates otherwise.

Specifically, as further described below, a break point path in an operator graph may be a point in which the output of a stream operator may be sent to one or more of a plurality of stream operators, depending upon qualities of the output. For example, a first stream operator provides its output to a second stream operator, the second stream operator provides its output to a third stream operator, and so on. The first, second, third, and additional operators can define a break point path. When a particular tuple “A” is received by the first stream operator, the corresponding tuple processed by the first stream operator is referred to herein as the same tuple A. After the tuple A is processed by the first stream operator and received by the second stream operator, the corresponding tuple processed by the second stream operator is referred to herein as the same tuple A. More generally, a tuple received by a stream operator at the head of the break point path may be referred to as the same tuple at the input and output of each subsequent stream operator in the path.

FIG. 1illustrates one example of a computing infrastructure100that may be configured to execute a stream computing application, according to some embodiments. The computing infrastructure100includes a management system105and two or more compute nodes110A-110D (i.e., hosts) which are communicatively coupled to each other using one or more communications networks120. The communications network120may include one or more servers, networks, or databases, and may use a particular communication protocol to transfer data between the compute nodes110.

The communications network120may include a variety of types of physical communication channels or “links.” The links may be wired, wireless, optical, or any other suitable media. In addition, the communications network120may include a variety of network hardware and software elements for performing routing, switching, and other functions, such as routers, switches, or bridges. The communications network120may be dedicated for use by a stream computing application or shared with other applications and users. The communications network120may be any suitable size. For example, the communications network120may include a single local area network or a wide area network spanning a large geographical area, such as the Internet. The links may provide different levels of bandwidth or capacity to transfer data at a particular rate. The bandwidth that a particular link provides may vary depending on a variety of factors, including the type of communication media and whether particular network hardware or software is functioning correctly or at full capacity. In addition, the bandwidth that a particular link provides to a stream computing application may vary if the link is shared with other applications and users. The available bandwidth may vary depending on the load placed on the link by the other applications and users. The bandwidth that a particular link provides may also vary depending on a temporal factor, such as a time of a day, day of a week, day of a month, or a season.

The stream manager134may be configured to monitor a stream computing application running on compute nodes (e.g., compute nodes110), as well as to change the deployment of an operator graph (e.g., operator graph132). The stream manager134may move processing elements from one compute node110to another, performing such actions as managing the processing loads of the compute nodes110in the computing infrastructure100. Further, stream manager134may control the stream computing application by inserting, removing, fusing, un-fusing, or otherwise modifying the processing elements and stream operators (or what tuples flow to the processing elements) running on the compute nodes110.

The bypass manager140may determine that some data is important. When the bypass manager determines that some data is important, it may determine what streams functionality to suppress/bypass to expedite processing of the important data. The bypass manager may enact the functionality discussed inFIG. 4.

FIG. 2is a view of a computing system200. In some embodiments, the computer system200may be the management system105or one or more of the compute nodes110ofFIG. 1. InFIG. 2, the dotted lines may be indicative of portions ofFIG. 2which are optional, or which occur in some embodiments but not others. For example, the bypass manager140can be included in embodiments where the computer system200is implemented as the management system105, but omitted in embodiments where the computer system200is implemented as a computer node110. The computing system200may include one or more processors (central processing units (CPUs))205, a network interface215, an interconnect220, a memory225, and storage230. The computing system200may also include an I/O device interface210used to connect I/O devices212(e.g., keyboard, display, or mouse devices) to the compute node110. In some embodiments one or more of these components may be optional.

A CPU205retrieves and executes programming instructions stored in the memory225or storage230. Similarly, the CPU205stores and retrieves application data residing in the memory225. The interconnect220is used to transmit programming instructions and application data between each CPU205, I/O device interface210, storage230, network interface215, and memory225. The interconnect220can be implemented using one or more busses. The CPUs205may be a single CPU, multiple CPUs, or a single CPU having multiple processing cores in various embodiments. In one embodiment, a processor205may be a digital signal processor (DSP).

The memory225is generally included to be representative of a random access memory (e.g., Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), or Flash). The storage230may also include non-volatile memory, such as a hard disk drive, solid state device (SSD), removable memory cards, optical storage, flash memory devices, network attached storage (NAS), connections to storage area network (SAN) devices, or other non-volatile storage devices. The network interface215is configured to transmit data via the communications network120.

The memory225may store one or more processing elements235. A processing element may include one or more stream operators240as described herein. In some embodiments, a processing element235can include multiple stream operators240. Specifically, a stream computing application may include one or more stream operators240that may be compiled into a processing element container235. In one embodiment, a processing element235is assigned to a single CPU205and is therein executed by that CPU205, although in other embodiments the stream operators240of a processing element235may include one or more threads that are executed on two or more CPUs205.

The memory225may include two or more processing elements235, each processing element having one or more stream operators240. Each stream operator240may include a portion of code that processes tuples flowing into a processing element and outputs tuples to other stream operators240in the same processing element, in other processing elements, or in both the same and other processing elements in a stream computing application. Processing elements235may pass tuples to other processing elements that are on the same computing system200(e.g., the same compute node110fromFIG. 1) or on other computing systems (e.g., compute nodes) that are accessible via communications network120. For example, a processing element235on a compute node110A fromFIG. 1may output tuples to a processing element235on the compute node110B fromFIG. 1. The memory may include the bypass manager140fromFIG. 1. The bypass manager140may enact the functionality discussed inFIG. 4. The bypass manager140may suppress one or more processing elements235or stream operators240.

Processing elements235may be configured to receive or output tuples in various formats (e.g., the processing elements or stream operators could exchange data marked up as XML documents). Furthermore, each stream operator240within a processing element235may be configured to carry out data processing functions on received tuples, such as writing to database tables or performing database operations such as data joins, splits, or reads, as well as performing other data analytic functions or operations.

The storage230may include a buffer260. Although shown as being in storage, the buffer260may alternatively be located in the memory225of the computing system or in a combination of both locations. Moreover, storage230may include storage space that is external to the computing system200, such as in a cloud. The buffer260may be used when moving data to or through processing elements235and stream operators240on a compute node computing system200.

The computing system200may include one or more operating systems262. An operating system262may be stored partially in memory225and partially in storage230. Alternatively, an operating system may be stored entirely in memory225or entirely in storage230. The operating system provides an interface between various hardware resources, including the CPU205, and processing elements and other components of the stream computing application. In addition, an operating system provides common services for application programs, such as providing a time function.

The memory225may store a stream manager134. Additionally, the storage230may store an operator graph132. The operator graph132may define how tuples are routed to processing elements235for processing. The memory225may store a compiler. The compiler compiles modules, which include source code or statements, into the object code, which includes machine instructions that execute on a processor. The compiler may also provide the application administrator with the ability to optimize performance through profile-driven fusion optimization. The output of the compiler may be represented by an operator graph132.

FIG. 3illustrates an example of an operator graph300for a stream computing application beginning from one or more sources302through to one or more sinks304,306, according to some embodiments. This flow from source to sink may also be generally referred to herein as an execution path. In addition, a flow from one processing element to another may be referred to as an execution path in various contexts. AlthoughFIG. 3is abstracted to show connected processing elements PE1-PE10, the operator graph300may include data flows between stream operators240fromFIG. 2within the same or different processing elements. Typically, processing elements, such as processing element235fromFIG. 2, both receive tuples from the stream and transmit tuples into the stream. Exceptions to this include a sink (e.g., where the stream terminates) and a source (e.g., where the stream begins). While the operator graph300includes a relatively small number of components, an operator graph may be much more complex and numerous operator graphs may be linked statically or dynamically together.

The operator graph shown inFIG. 3includes ten processing elements (labeled as PE1-PE10) running on the compute nodes110A-110D. A processing element may include one or more stream operators fused together to form an independently running process with a unique process identifier (PID) and memory space. In cases where two or more processing elements are running independently, inter-process communication may occur using a transport (e.g., a network socket, a TCP/IP socket, or shared memory). However, when stream operators are fused together, the fused stream operators can use faster communication techniques for passing tuples among stream operators in processing elements.

The operator graph300begins at a source302and ends at a sink304,306. Compute node110A includes the processing elements PE1, PE2, and PE3. Source302flows into the processing element PE1, which in turn outputs tuples that are received by PE2and PE3. For example, PE1may split data attributes received in a tuple and pass some data attributes in a new tuple to PE2, while passing other data attributes in another new tuple to PE3. As a second example, PE1may pass some received tuples to PE2while passing other tuples to PE3. Tuples that flow to PE2are processed by the stream operators contained in PE2, and the resulting tuples may then be output to PE4on compute node110B. Likewise, the tuples output by PE4may flow to operator sink PE6304. Similarly, tuples flowing from PE3to PE5also reach the operators in sink PE6304. Thus, in addition to being a sink for this example operator graph, PE6could be configured to perform a joint operation, combining tuples received from PE4and PE5. This example operator graph also shows tuples flowing from PE3to PE7on compute node110C, which itself shows tuples flowing to PE8and looping back to PE7. Tuples output from PE8flow to PE9on compute node110D, which in turn outputs tuples to be processed by operators in a sink processing element, which here is PE10306.

Because a processing element may be a collection of fused stream operators, an operator graph may have one or more execution paths between specific stream operators, which may include execution paths to different stream operators within the same processing element.FIG. 3illustrates execution paths between processing elements for the sake of clarity.

FIG. 4is a flowchart illustrating a method400for suppressing streams functionality for preferred data. The stream functionality may be suppressed when data (e.g., tuples) within the data stream is identified as preferred (e.g., important). In some embodiments, tuples may be portions of data which are processed for a software application in a data stream. The visual arrangement of blocks in the flowchart ofFIG. 4is not to be construed as limiting the order in which the individual acts/operations may be performed, as certain embodiments may perform the operations ofFIG. 4in alternative orders. Blocks which are depicted with dashed lines are to be construed as optional operations.

At block410a controller may determine what is the selected (e.g., important) data. The controller may be a component of the streams manager. In certain embodiments, the controller could be part of the streams runtime. In the discussion ofFIG. 4, important data and selected data may be used interchangeably. In some embodiments, important data may be an identifiable type of data, or data containing one or more selected or identifiable characteristics, which has been thusly identified as warranting expedited processing in relation to other data. In some embodiments, it may not be necessary to determine the important data. It may be unnecessary if data may be specified as important through hard-carded values, as a program does not need to determine something which is already hard-coded in. For example, for a program which searches through data to help locate missing children, a controller may be hard-coded to identify any data relating to “child” as important. However, a controller may determine additional important data by deriving values from this hard-coded value of “child.’ Determining these derivations may include analyzing word meanings or roots (e.g., a controller may include synonyms of hard-coded values as triggers of important data). For example, a controller may determine that related words “children” and “kid” are also important and should be included in the important data.

A controller may also receive triggers from external programs to include new portions of important data. For example, using the missing children example, an external program may tabulate the names of missing children. The external program could provide this data to the controller, wherein the list of missing children could supplement the list of important data.

A controller could also determine what constitutes important data by monitoring how data behaves in the system. Specifically, a controller could identify a subset of data which meets certain conditions which correlate to important data, and add that subset of data to a list of known important data. A condition may mean that a certain type of data is more likely to end at a conspicuous conclusion (e.g., a sink which often leads to important data) in the processing chain, more likely to be deposited in a conspicuous repository (e.g., database file which often includes important data), or more likely to be queued for a conspicuous outward message (e.g., placed in a java message service (JMS) queue to a recipient which typically receives important data). For example, a controller may determine that data which includes a birthday may occasionally be matched within the system to the birthday of the missing child, which results in storing this information in a file of “found children” while preparing an outward message to authorities. The controller may match this system behavior (filing to “found children” while preparing message) corresponds to important information, and may therein add “matching birthday” to the list of important data.

At block420a controller may analyze tuples for selected/important data. A controller may check the tuples in response to the tuples entering an operator graph and/or the controller may check tuples after a processing element/operator has transformed said tuples. In some embodiments, transformation of a tuple may include changing the values within the tuple. In certain embodiments, the controller may analyze a tuple for important data when the tuple has not been analyzed for important data while in the current form of the tuple. Put differently, tuples may be analyzed upon both entrance into an operator graph and transformation within an operator graph. In some embodiments, an operator graph is a self-contained collection of processes which collectively receives data, performs processes upon the data, outputs the data into predetermined formats, and transmits that output data to specific locations depending upon the values of the data. In some embodiments, an operator graph may transmit more than one output, and may handle more than one source of data.

At block430a tuple is determined to have selected/important data. The controller may determine that the tuple has important data. A tuple could be checked for important data by a comparison of data in the tuple to data in the important data records. For example, a banking application may identify data with negative values as important data with the goal of identifying possible overdrafts. If a controller analyzes a tuple with a value of −$18.54, the controller may determine this tuple as including important data. A controller may also determine that a tuple has important data following transformation of the tuple via a transformative operator (e.g., rather than determining if a tuple is important at a source of the operator graph, the controller may determine if a tuple is important following transformations while the tuple progresses through the operator graph). For example, an incoming tuple could have a value of “withdraw $25.50 from account 1234” when entering a stream for the banking application above. Upon checking, the controller may correctly identify that this tuple has no important data. The tuple may then go to an operator A which subtracts 25.50 from the current balance 20.00 in account 1234. The operator may then transform the tuple, changing the value of the tuple to “$25.50 withdrawn from account 1234 for a balance of −$5.50.” Upon analyzing this new value, the controller may determine the tuple to include important data of a negative amount.

In some embodiments, the controller may determine a tuple to include important data through conditions met by the tuple. For example, the banking application may closely track accounts when the balance falls below certain thresholds. Falling below a threshold may therefore indicate important data. Different accounts may have different thresholds, so a tuple which only includes a balance may not be sufficient to determine if the data is important. Instead, the controller may determine if a tuple is important as related to these thresholds by analyzing the behavior of relevant processes in the operator graph (e.g., if a first processing element which determines thresholds sends a tuple to a second processing element which handles compliant accounts, the tuple may be important).

For example, account 1234 may have a threshold of $100, and account 5678 may have a threshold of $75. In such embodiments, the controller may be unable to determine if a tuple is important data based only on a value of $80 in said tuple. Instead a controller may determine that the tuple includes important data when, for example, a processing element independently determines that the tuple is for account 1234 and sends the tuple to a location (e.g., a sink, processing element, database file, or JMS queue) which typically or exclusively handles data regarding accounts below their thresholds. In this way a controller may determine that a tuple includes important data by analyzing how a tuple is routed through an operator graph and/or what operations the tuple activates when passing through an operator graph.

In some embodiments, aspects of the disclosure may relate to determining that the tuple has important data by other means. These other means may be known to those skilled in the art or otherwise obvious to those skilled in the art.

At block440, aspects of the disclosure may relate to determining streaming operations to suppress for the tuple. In some embodiments, a controller may determine the operations which can be suppressed. The operations may be determined for a tuple by a controller in response to the controller determining that the tuple included important/selected data. In some embodiments, these operations may include actions taken to process a tuple or actions taken in response to processing a tuple which can be suppressed while still deriving an anticipated result of the tuple in the operator graph. Put differently, when a tuple includes important data a controller may suppress operations and or procedures not necessary to successfully process the tuple and achieve the result which makes the tuple important.

In embodiments, operations which may be suppressed include stream operators (e.g.,240fromFIG. 2), processing elements (e.g.,235fromFIG. 2and PE1-10fromFIG. 3), extraneous logging (e.g., records of processes, actions, performance, etc.), dynamic connections (e.g., ability to import or export mid-stream to other applications or other application instances), language cleanup functions (e.g., java garbage collection, accounting collections, optional metric collections, etc.), or other routines within an operator graph. Where a tuple would typically be transmitted to two different locations for two different purposes at the same time, a controller could suppress the transmittance of the tuple to the location which did not have the purpose which made the selected data important. Alternatively, if both purposes were equally important, the controller could allow the dual transmittance. In some embodiments, a controller could also skip the tuple ahead of other tuples in the stream, breaking the typical first-in-first-out methodology of the stream, to accelerate processing of the tuple.

The controller may know which operations to suppress by stored data related to the important data. This data could be in the form of hard-coded values or metadata on the operations. For example, a controller may determine that, when important data “A” is identified in a tuple, operations 1-6 are suppressed. In certain embodiments, a controller may suppress the same operations for any instance of important data. Alternatively, a controller may suppress different operations for different varieties of important data.

For example, an energy utility may use an energy application to process real time data regarding where energy is being consumed and generated to balance loads across a service area. The application may handle a great magnitude of data showing many loads at many locations at many times. Some of this data, such as tuples indicating unexpected spikes of electricity loads, may be more important than other portions of data, such as tuples indicating predicted loads. Some of this important data, such as when the spike is not only unexpected but also large, may be relatively more important than other important data. Within these confines, the controller may detect a small unexpected demand spike, and may determine processing elements 2 and 4 as suppressible in order to expedite processing of the small spike. Soon after this determination the controller may detect a large unexpected demand spike, and may determine processing elements 2-5, the last operator in processing element 1, and dynamic connections as suppressible for the large spike. The controller may also determine that all tuples not currently being processed may be skipped by the tuple related to the large spike to expedite processing. In this way a controller may determine which operations may be suppressed as soon as important data is detected. A controller may alternatively determine if an operation may be suppressed in response to the operation attempting to act.

In some embodiments, aspects of the disclosure may relate to determining a new route for the tuple. The new route may be different than the route the tuple would take if the tuple did not include important data. The controller may determine the new route. For example, looking atFIG. 3, the controller may determine that PE1and PE3are suppressible. Rather than spending the time to route the tuple to these processing elements and then determine a next location, the controller may determine a new route directly from the source302to PE2. Such alternate routes for tuples containing important data may be hard-coded into the streaming application prior to the detection of important data.

At block450, aspects of the disclosure may relate to suppressing operations for the tuple. The operations determined at block440may be suppressed in response to determining the tuple including important data at block430. A controller may suppress the operations. Other tuples within the operator graph which do not have important data may not have the operations suppressed, and may therein undergo the operations.

For example, a national security organization may be using a streaming application to process high volumes of data searching for a small number of high interest individuals. A portion of the data may be useful to other organizations, so dynamic connections may be enabled so that mid-stream data can be shared. At the same time, there may only be a few tuples which include data relating to the high interest individuals. The tuples may also contain time-sensitive information. Due to the high volume of data, the streaming application may also employ numerous cleanup functions and maintenance routines to ensure the stream does not collapse under the volume of data.

In this example, a controller within the streaming application may flag data relating to the high interest individuals as selected/important data. Numerous operators and processing elements may have metadata indicating if said operators and processing elements can be skipped when important data is present in a tuple. The cleanup functions and maintenance routines may all be flagged as suppressible for all important data, and dynamic connections may be suppressible for suspect A of the high interest individuals.

To further the example, the controller may be analyzing tuples in the operating graph, and may determine that a tuple exiting an operator which translates aliases into names has information concerning suspect A. The controller may determine that this tuple, due to its inclusion of suspect A, has important data. At this point a tax collection application may attempt to use dynamic connections to reference the tuple as it exited the alias operator. The controller may suppress this attempt to expedite processing of the tuple. However, the controller may allow the tax collection application to export a tuple immediately following the tuple regarding suspect A. At the same time, a maintenance routine may attempt java garbage collection, and in response the controller may suppress the action across the operator graph, therein suppressing all such maintenance until the tuple has concluded processing on the operator graph. The tuple may then be routed from operator to operator, with the controller suppressing numerous operations along the route as metadata allows. When a processing element has metadata indicating that all operators are suppressible, the controller may suppress the entire processing element. If the tuple comes to a hard-coded route to skip operations due to the important data, the controller may route the tuple along this hard-coded path. In this way the controller may react to important data in tuples by suppressing streams functionality to realized performance benefits.

Characteristics are as Follows:

Service Models are as Follows:

Deployment Models are as Follows:

Workloads layer90provides examples of functionality for which the cloud computing environment may be utilized. Examples of workloads and functions which may be provided from this layer include: mapping and navigation91; software development and lifecycle management92; virtual classroom education delivery93; data analytics processing94; transaction processing95; and stream functionality suppression. Stream functionality suppression may detect that important data (e.g., data which is relatively more important than other data) is present somewhere in the data stream of the cloud computing environment. In response to detecting this important data, certain stream functionalities of the cloud computing environment may be suppressed. By suppressing certain stream functionalities of the cloud computing environment, the important data may be processed more quickly, allowing components of the cloud computing environment to receive the benefits of this important data more expediently.