Centralized chromatic pluralizing of internet of things (IOT) communication

Assigning chromatic numbers based on a pre-assigned scheme to a stream of sensor data from sensors of Internet connected devices to “color” the message which can then be processed by chromatic number to obtain a pluralized stream of data.

BACKGROUND

The present invention relates to communications from Internet of Things (IoT) devices, and more specifically to centralized, chromatic pluralizing of IoT communication.

The “Internet of Things” (commonly abbreviated as “IoT”) is the networking of physical devices (also referred to as “connected devices” and “smart devices”), buildings, and other items. These “things” have embedded electronics, software, sensors, actuators, and network connectivity that enable these objects to collect and exchange data across a global network. Where the things are sensors, the sensors can create a continual or near continual stream of data for processing.

The data produced by these sensors in multiple devices is massive in volume and can be in different formats. This massive volume of data needs to be processed within a very short amount of time, for example, a span of milliseconds, to maximize the benefit of such data and execute any corrective action which may be necessary. Due to the volume of data being collected, it can be a problem for the data captured by these devices and their associated sensors to be stored and fully managed within central databases, as the data needs to be aggregated and analyzed within a very short time.

As shown in prior artFIG. 8, the sensor data from the IoT devices is sent in packets35a-35neach of which consists of a header, subdivided into a mandatory fixed header30and optional extension headers31a-31n, having control information for addressing and routing, and a payload consisting of an upper-layer header32and the sensor data33.

The extension headers31a-31nmay carry Internet Layer information, and are placed between the fixed header30and the upper-layer protocol header32. The headers form a chain, and are typically updated and added as the packet35a-35nprogresses through the Internet from its origin to its destination.

The prior art does not have a mechanism which uses chromatic numbers to represent sensor data for faster processing via an extension header of the sensor data.

SUMMARY

According to one embodiment of the present invention, a method of pluralizing an incoming stream of data from Internet connected devices for processing by a centralized database is disclosed. The method comprising the steps of: a computer receiving a sensor data stream from the Internet connected devices; the computer obtaining a chromatic number to be used for each sensor data point within the sensor data stream; the computer generating a data packet with each sensor data point and the chromatic number; and the computer sending the data packet to a framework for pluralization and output of pluralized data based on the chromatic number.

According to another embodiment of the present invention, a computer program product for pluralizing an incoming stream of data from Internet connected devices for processing by a centralized database by a computer is disclosed. The computer comprising at least one processor, one or more memories, one or more computer readable storage media, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by the computer to perform a method comprising: receiving, by the computer, a sensor data stream from the Internet connected devices; obtaining, by the computer, a chromatic number to be used for each sensor data point within the sensor data stream; generating, by the computer, a data packet with each sensor data point and the chromatic number; and sending, by the computer, the data packet to a framework for pluralization and output of pluralized data based on the chromatic number.

According to another embodiment of the present invention, a computer system for pluralizing an incoming stream of data from Internet connected devices for processing by a centralized database is disclosed. The computer system comprising a computer comprising at least one processor, one or more memories, one or more computer readable storage media having program instructions executable by the computer to perform the program instructions comprising: receiving, by the computer, a sensor data stream from the Internet connected devices; obtaining, by the computer, a chromatic number to be used for each sensor data point within the sensor data stream; generating, by the computer, a data packet with each sensor data point and the chromatic number; and sending, by the computer, the data packet to a framework for pluralization and output of pluralized data based on the chromatic number.

According to another embodiment of the present invention, a method of pluralizing data packets of a stream of data from Internet connected devices is disclosed. The method comprising the steps of: a computer receiving a plurality of incoming streams of data packets each containing a sensor data point from the Internet connected devices and a chromatic number; the computer parsing the incoming steam of data packets to extract events based on the chromatic number of the data packet; the computer sorting the extracted events based on the chromatic number within a period of time; and the computer analyzing and outputting data based on the period of time to create a new data packet based on the chromatic number.

According to another embodiment of the present invention, a computer program product for pluralizing data packets of a stream of data from Internet connected devices by a computer is disclosed. The computer comprising at least one processor, one or more memories, one or more computer readable storage media, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by the computer to perform a method comprising: receiving, by the computer, a plurality of incoming streams of data packets each containing a sensor data point from the Internet connected devices and a chromatic number; parsing, by the computer, the incoming steam of data packets to extract events based on the chromatic number of the data packet; sorting, by the computer, the extracted events based on the chromatic number within a period of time; and analyzing and outputting, by the computer, data based on the period of time to create a new data packet based on the chromatic number.

According to another embodiment of the present invention, a computer system for pluralizing data packets of a stream of data from Internet connected devices is disclosed. The computer system comprising a computer comprising at least one processor, one or more memories, one or more computer readable storage media having program instructions executable by the computer to perform the program instructions comprising: receiving, by the computer, a plurality of incoming streams of data packets each containing a sensor data point from the Internet connected devices and a chromatic number; parsing, by the computer, the incoming steam of data packets to extract events based on the chromatic number of the data packet; sorting, by the computer, the extracted events based on the chromatic number within a period of time; and analyzing and outputting, by the computer, data based on the period of time to create a new data packet based on the chromatic number.

DETAILED DESCRIPTION

In an embodiment of the present invention, a framework to manage the data collected by the IoT devices is disclosed, such that analytics on a central database of such data can be performed in less time. The framework increases the efficiency of the network and the computer performing analytics on the central database. The framework uses a chromatic number to represent sensor data for faster processing. Since chromatic numbers are generated during a particular period for a sensor and are processed by multiple nodes, the analysis speed is increased. Furthermore, the need of processing of IoT data for interpreting values is removed, improving the performance of the IoT devices.

FIG. 1is an exemplary diagram of a possible data processing environment provided in which illustrative embodiments may be implemented. It should be appreciated thatFIG. 1is only exemplary and is not intended to assert or imply any limitation with regard to the environments in which different embodiments may be implemented. Many modifications to the depicted environments may be made.

Referring toFIG. 1, network data processing system51is a network of computers in which illustrative embodiments may be implemented. Network data processing system51contains network50, which is the medium used to provide communication links between various devices and computers connected together within network data processing system51. Network50may include connections, such as wire, wireless communication links, or fiber optic cables.

The Internet connected device (ICD)56a-56nmay be a mobile device, computer, a personal device or any other device with a sensor for capturing data regarding a variable and is connected to the Internet. The Internet connected device (ICD)56a-56nmay contain an interface55, which may accept commands and data entry from a user. The interface55can be, for example, a command line interface, a graphical user interface (GUI), a natural user interface (NUI) or a touch user interface (TUI). The Internet connected device56a-56nincludes a set of internal components800aand a set of external components900a, further illustrated inFIG. 7. The Internet connected device (ICD)56a-56ncan transmit raw data collected by a sensor to other components on network50as well as other networks. The raw data collected may include, but not limited to data regarding temperature, motion, lighting, entertainment, consumption, use, surveillance, resource usage, etc. . . . and preferably includes a time stamp.

The raw data from the ICD devices56a-56nis sent to the network50via a chromatic processor54. Chromatic processor54includes a set of internal components800band a set of external components900billustrated inFIG. 7. In the depicted example, chromatic processor54provides a chromatic number to be added to the raw data received from the Internet connected devices56a-56n. The chromatic processor54preferably includes a chromatic number program58for generating a data packet with a header which is padded with a chromatic number and includes the raw data. The chromatic number is obtained from a repository53, which preferably includes mapping of chromatic numbers and range value, as well as condition based business rules or logic business rules. The repository53may additionally include a pre-configured, customizable database which will store the chromatic number relative to different sets of business rules to be applied on sensor data. These business rules can support range values, simple and/or logic business rules. For example, chromatic number “1” (Green) may be set for temperature range 14 and 22, chromatic number “2” (which depicts amber) for temperature range between 23 and 30, and chromatic number “3” (depicts red) for temperature greater than 30. In this same example, the business rules may be defined based on multiple variable-like combination of temperature and humidity with to combination of and/or conditions etc.

The data packet with the assigned chromatic number is then sent to a High-Level Data Link (HDLC) framework57through the network50from the chromatic processor54. The HDLC framework57is a framework which receives the data packets which include the chromatic number. The HDLC framework57forwards the data packets to the chromatic pluralization framework52for further processing. The HDLC framework57also adds a framework flag (a fixed bit) to represent the start and end of a message. The framework flag is used to apply a standard for bit padding to add a chromatic number in the message.

The HDLC framework57includes a set of internal components800band a set of external components900billustrated inFIG. 7.

The chromatic pluralization framework52is a framework which receives the data packets with the chromatic number in different data streams in real time. The data packet preferably includes the sensor data, chromatic number in at least one extension header, and framework flag. The chromatic pluralization framework52may include resilient distributed datasets (RDD) across multiple nodes which can be utilized in parallel. The chromatic pluralization framework52additionally conducts sort and parse operations which may be executed by a chromatic pluralization program65. The chromatic pluralization framework52includes a set of internal components800band a set of external components900billustrated inFIG. 7.

In the depicted example, Internet connected device (ICD)56a-56nvia the chromatic processor54with a repository53, a chromatic pluralization framework52, and a High-Level Data Link (HDLC) framework57are connect to network50. In other exemplary embodiments, network data processing system51may include additional client or device computers, storage devices or repositories, server computers, and other devices not shown.

Program code and programs such as chromatic number program58and chromatic pluralization program65may be stored on at least one of one or more computer-readable tangible storage devices830shown inFIG. 7, on at least one of one or more portable computer-readable tangible storage devices936as shown inFIG. 7, or in repository53connected to network50, or may be downloaded to an Internet connected device56a-56n, HDLC framework57, chromatic pluralization framework52, or chromatic processor computer54, for use. For example, program code and programs such as chromatic number program58and chromatic pluralization program65may be stored on at least one of one or more storage devices830on chromatic processor54and downloaded to an Internet connected device56a-56n, HDLC framework57, or chromatic pluralization framework52over network50for use. Alternatively, chromatic processor54can be a web server, and the program code, and programs such as chromatic number program58and chromatic pluralization program65may be stored on at least one of the one or more storage devices830on chromatic processor54and accessed by an Internet connected device56a-56n, HDLC framework57, or chromatic pluralization framework52. In other exemplary embodiments, the program code, and programs such as chromatic number program58and chromatic pluralization program65may be stored on at least one of one or more computer-readable storage devices830on an Internet connected device56a-56n, HDLC framework57, chromatic pluralization framework52or distributed between two or more servers.

Chromatic numbers are numbers which are assigned to data, grouping the data by “colors” such that the minimum number of “colors” are used which are needed to color the vertices of a graph such that no two adjacent vertices have the same “color”. The chromatic numbers do not necessarily represent real spectrum colors, but may be referred to by those names—red, orange, yellow, etc.—for explanatory purposes.

FIG. 2shows an example of how such chromatic numbers might be assigned to a stream400of sensor data402from sensors of Internet connected devices56a-56n. In the example, the sensors of the Internet connected devices56a-56nare temperature sensors, each measuring a temperature and reporting the temperature once a second. The sample data stream402from the Internet connected devices sensors56a-56nconsists of a sensor identification, a temperature reading, and a time stamp. For example purposes, data values for only two sensors, coded as S1and S2, are provided, although it will be understood that data from any number of sensors56a-56nmight be included in the stream400, and the data might include any sort of information in addition to that shown.

The raw data stream402is input to the chromatic processor54, which assigns chromatic numbers to each of the data packets based on a pre-assigned scheme. These chromatic numbers will be derived from a configuration database, for example repository53, based upon range values and condition based business rules. The chromatic numbers are mapped with a range of values.

An IoT (Internet of Things) or Internet connected devices device56a-56nis calibrated to provide a metric (e.g. temperature). This metric follows a range (e.g.20to120). Conventionally, the data of the message includes just the metric reading (e.g. temperature reading) with a time stamp as to when the metric was collected, which leads to message flooding at an integrator of a centralized database, which has to do an additional pattern matching. In contrast, an embodiment of the present invention allows for fast track parsing to decrease the message flooding at the integrator. In an embodiment of the present invention, the chromatic number, which symbolically ‘colors’ the message from the IoT device, is added to the metric and transmitted to the integrator of a chromatic pluralization framework52. For example, if the metric were temperature, then the temperature range in which data was being collected would be calibrated into divisions of 10, starting from 20, with each division being represented by a chromatic number.

In the example, the sensors of the Internet connected devices56a-56nwill be sensing temperatures in the range of 0° C. to 59° C., so the chromatic numbers might be assigned as follows:

The chromatic processor54thus assigns chromatic numbers to the raw data402, resulting in a coded data stream404. The chromatic numbers can be added to the packet header—for example in the extension header, or, if desired, anywhere else in the packet header. Alternatively, the chromatic numbers can be stored by reserving bytes in the data part of the packet33.

The sensor data with the chromatic numbers in the extension header are then sent to a chromatic pluralization framework52for additional processing, resulting in a pluralized data stream406. Prior to the coded data stream404being sent to the chromatic pluralization framework52, the coded data stream404may be sent to a HDLC framework57, which then sends the coded data stream404to the chromatic pluralization framework52.

As can be seen inFIG. 2, the amount of data in the coded data stream404is reduced significantly by this pluralization. The pluralized data stream406can then be sent to a central processor69, as in the prior art, for whatever processing the system performs.

The chromatic pluralization framework52can be implemented as a multi-node model such as RDD (Resilient Distributed Datasets) which uses multi-node processing language such as SPARK RDD to parse a message from different streams of incoming data from sensors. These RDD's are processed by multi-nodes in parallel to parse the messages rapidly and check the messages for sorted events. Alternatively, if the data could be processed without the use of parallel processing using multiple nodes.

A sort operation is conducted on the events extracted from these messages based upon chromatic numbers in the header. The chromatic pluralization framework52receives data in real time for further analyzation. Each event is tagged to be in a period of time. Data analysis for a particular period is then conducted based on the chromatic numbering. Since all of the RDD's are stored in memory, the analysis speed is increased significantly in comparison to sending all data to a centralized database as it occurs in real time.

By using chromatic numbering as described above and applied to sensor data402, fewer bytes with actual sensor data are passed for further processing69in the pluralized stream406. These bytes will be used to depict pre-configured chromatic numbers which represent the different state/range of actual data associated with them, enabling the integrator of the chromatic pluralization framework to parse the data quickly.

FIG. 3shows a flow diagram of a method of chromatic numbering padding of raw data received by the

In a first step, sensor data streams are received from the Internet connected devices56a-56n(step202) by the chromatic processor54.

The chromatic processor54obtains a chromatic number to be used for each current sensor data point to be sent to the chromatic pluralization framework (step204), for example by the chromatic number program57.

The chromatic processor54then generates a packet with sensor data and chromatic number (step206). The chromatic number is preferably added to the packet header—for example in the extension header, or, if desired, anywhere else in the packet header.

Prior to generating the padded packet with the chromatic number, a signal-to-noise ratio (difference in decibels between the received signal and the background noise level) may be measured from the Internet connected devices56a-56nto compare to a threshold to determine whether the data stream is good for further processing. If the signal-to-noise ratio is less than a threshold, a data packet containing the data from the internet connected devices56a-56nis not sent to the chromatic pluralization framework52for pluralization. Alternatively, the signal-to-noise ratio may be compared to a threshold as soon as the raw data is received by the chromatic processor52and prior to obtaining a chromatic number to be associated with the raw data from the Internet connected devices56a-56n.

The padded packet is then sent to the HDLC framework57by the chromatic processor54(step208). The HDLC framework57preferably adds framework flags to the data packet.

The packet is then sent by the HDLC framework to the chromatic pluralization framework52(step210) and the chromatic pluralization framework52outputs pluralized data to for additional processing and the method ends.

FIG. 4show s a schematic of the process of chromatic pluralization by the chromatic pluralization framework andFIG. 6an associated flow diagram of a method of processing data using parallel processing using multiple nodes. The chromatic pluralization framework52can be implemented as a multi-node model such as RDD (Resilient Distributed Datasets) which uses multi-node processing language such as SPARK RDD to parse a message from different streams of incoming data packets from sensors. These RDD's are processed by multi-nodes in parallel to parse the messages rapidly and check the messages for sorted events. Alternatively, if the data could be processed without the use of parallel processing using multiple nodes as discussed inFIG. 5.

The chromatic pluralization framework52receives a plurality of incoming stream of packets404from the HDLC framework57. The incoming stream of data packets404is then split300by the framework52and assigned to incoming RDDs303.

The incoming RDD's303are processed by multi-nodes in parallel to parse304the data packets rapidly into incoming events305based on the chromatic number present in the extension header of the data packets.

A sort operation306is conducted on the incoming events305extracted from the data packets based upon chromatic numbers in the header to obtain sorted events307within a time period. Since each data packet includes a time stamp, each event can be tagged or assigned into a time period. The period can be any length or unit of time as defined by an administrator. For example, a period can be a specific number or range of microseconds or minutes. Data analysis for a particular period is then conducted based on the chromatic numbering to output a data packet with a state associated with the time period based on the sensor data in state RDDs308, which can then be sent on for additional processing by a central database. Since all of the RDD's are stored in memory, the analysis speed is increased significantly in comparison to sending all data to a centralized database as it occurs in real time. Preferably, data with a chromatic number processed during a first period will be processed for additional analysis during a next, subsequent period.

Referring toFIG. 6, the chromatic pluralization program65of the chromatic pluralization framework52receives a plurality of incoming streams of data packets (step260). The incoming data packets are split, for example by the chromatic pluralization program65and are assigned to incoming RDDs (step261). The incoming data packets are then parsed to extract events based on chromatic number (step262).

If the period associated with the data packets has not ended (step264), the method returns to step260of receiving the incoming stream of data packets.

If the period associated with the data packets has ended (step264), the method continues to steps266of the chromatic pluralization program65of sorting events based on chromatic number within a period (step266).

The sorted events within a period based on chromatic number are then analyzed and data is outputted based on period in a state RDD to create a packet based on chromatic number (step268) and the method ends. After step268, the packet may then be sent to a centralized database69for additional processing.

FIG. 5shows a flow diagram of a method of chromatic pluralization without using RDDs.

The chromatic pluralization program65of the chromatic pluralization framework52receives a plurality of incoming streams of data packets (step250). The incoming data packets are then parsed to extract events based on chromatic number (step252).

If the period associated with the data packets has not ended (step254), the method returns to step250of receiving the incoming stream of data packets.

If the period associated with the data packets has ended (step254), the method continues to steps256of the chromatic pluralization program65of sorting events based on chromatic number within a period (step256).

The sorted events within a period based on chromatic number are then analyzed and data is outputted based on period to create a packet based on chromatic number (step258) and the method ends. After step258, the packet may then be sent to a centralized database69for additional processing.

FIG. 7illustrates internal and external components of components of the network, IoT devices56a-56n, server computer54, HDLC framework57and chromatic pluralization framework52in which illustrative embodiments may be implemented. InFIG. 7, IoT devices56a-56n, server computer54, HDLC framework57and chromatic pluralization framework52include respective sets of internal components800a,800b, and external components900a,900b. Each of the sets of internal components800a,800bincludes one or more processors820, one or more computer-readable RAMs822and one or more computer-readable ROMs824on one or more buses826, and one or more operating systems828and one or more computer-readable tangible storage devices830. The one or more operating systems828, chromatic number program58and chromatic pluralization program65are stored on one or more of the computer-readable tangible storage devices830for execution by one or more of the processors820via one or more of the RAMs822(which typically include cache memory). In the embodiment illustrated inFIG. 4, each of the computer-readable tangible storage devices830is a magnetic disk storage device of an internal hard drive. Alternatively, each of the computer-readable tangible storage devices830is a semiconductor storage device such as ROM824, EPROM, flash memory or any other computer-readable tangible storage device that can store a computer program and digital information.

Each set of internal components800a,800bcalso includes a R/W drive or interface832to read from and write to one or more portable computer-readable tangible storage devices936such as a CD-ROM, DVD, memory stick, magnetic tape, magnetic disk, optical disk or semiconductor storage device. Chromatic number program58and chromatic pluralization program65can be stored on one or more of the portable computer-readable tangible storage devices936, read via R/W drive or interface832and loaded into hard drive830.

Each set of internal components800a,800balso includes a network adapter or interface836such as a TCP/IP adapter card. Chromatic number program58and chromatic pluralization program65can be downloaded to the device computer52, server computer54, and ICD computer56from an external computer via a network (for example, the Internet, a local area network or other, wide area network) and network adapter or interface836. From the network adapter or interface836, chromatic number program58and chromatic pluralization program65can be loaded into hard drive830. Chromatic number program58and chromatic pluralization program65can be downloaded to the server computer54from an external computer via a network (for example, the Internet, a local area network or other, wide area network) and network adapter or interface836. From the network adapter or interface836, chromatic number program58and chromatic pluralization program65can be loaded into hard drive830. The network may comprise copper wires, optical fibers, wireless transmission, routers, firewalls, switches, gateway computers and/or edge servers.

Each of the sets of external components900a,900bincludes a computer display monitor920, a keyboard930, and a computer mouse934. Each of the sets of internal components800a,800balso includes device drivers840to interface to computer display monitor920, keyboard930and computer mouse934. The device drivers840, R/W drive or interface832and network adapter or interface836comprise hardware and software (stored in storage device830and/or ROM824).

Chromatic number program58and chromatic pluralization program65can be written in various programming languages including low-level, high-level, object-oriented or non object-oriented languages. Alternatively, the functions of a chromatic number program58and chromatic pluralization program65can be implemented in whole or in part by computer circuits and other hardware (not shown).