Patent ID: 12260760

DETAILED DESCRIPTION

The present disclosure describes systems and methods for remotely accessing vehicle sensors, such as exterior vehicle cameras. For example, a service subscriber may request access to available parking spots in a particular area. Publishing vehicles that agree to share vehicle sensor data may transmit indications of available parking spots based on their vehicle sensor data and location. In another scenario, a service subscriber may have parked their vehicle in an isolated location. Before walking to their vehicle, the subscriber may request access to camera images from publisher vehicles parked next to or in the vicinity of the subscriber's vehicle to monitor the environment near the subscriber's vehicle.

Throughout the description reference is made toFIGS.1-5. When referring to the figures, like structures and elements shown throughout are indicated with like reference numerals.

In one exemplary configuration, a system for accessing vehicle sensor data is disclosed. The system includes a server with a server processor and a server memory. The server memory stores server instructions executable by the server processor to receive the vehicle sensor data and a vehicle location associated with the vehicle sensor data from a publisher vehicle. Another receiving instruction causes the server processor to receive, from a service subscriber, a subscriber request for remote data. The subscriber request includes a request location. A determining instruction determines whether there is a match condition. The match condition may include a vehicle location match between the request location and the vehicle location. A transmitting instruction causes transmission of the vehicle sensor data to the service subscriber upon the occurrence of the match condition.

In one configuration, the vehicle sensor data includes camera images of a physical area adjacent the publisher location. The vehicle sensor data may include an indication of an available parking location adjacent the publisher vehicle. The vehicle sensor data may include a published data time and the subscriber request may include a request time period. The match condition may include a time match between the request time period and the published data time.

In one implementation, the vehicle sensor data includes a publisher data type and the subscriber request includes a subscriber data type. The condition may include a type match between the publisher data type and the subscriber data type. The server instructions can include server instructions to receive an indication of an allowed region for the vehicle sensor data and the match condition may include a region match between the vehicle location and the allowed region. Similarly, the vehicle sensor data may include a published data time and the server instructions may include server instructions to receive an indication of an allowed time for the vehicle sensor data. The match condition may include a time match between the allowed time and the published data time.

The server instructions may include instructions to train a machine learning model to determine recommended data sources based on historical subscriber requests from the service subscriber. In another configuration, the server instructions include server instructions to train a machine learning model to determine recommended data sources based on historical subscriber requests from a plurality of service subscribers. The plurality of subscribers can include subscribers in a particular geographical region (such as downtown Chicago), time (such as 11 PM to 12 PM), and/or other selection basis. The instructions may include instructions to provide the recommended data sources to the service subscriber.

The server instructions may include server instructions to receive stationary sensor data associated with a stationary location and to transmit the stationary sensor data to the service subscriber upon the occurrence of the match condition. The match condition may include stationary location match between the request location and the stationary location.

The system may include a subscriber device including a device processor and device memory. The device memory may include device instructions executable by the device processor to send the subscriber request for the remote data to the server and receive the vehicle sensor data from the server. The subscriber device may be, for example, a smartphone, a computer carried by a subscriber vehicle, or any internet enabled smart device. The device instructions may include instructions to display the vehicle sensor data on a vehicle display at the subscriber vehicle.

Another exemplary configuration is a method for accessing remote sensor data. The method may include steps of receiving vehicle sensor data and a vehicle location associated with the vehicle sensor data from a publisher vehicle, receiving, from a service subscriber, a subscriber request for remote data, the subscriber request including a request location, determining whether there is a match condition, and transmitting the vehicle sensor data to the service subscriber upon the occurrence of the match condition. The match condition may include a vehicle location match between the request location and the vehicle location.

The vehicle sensor data may include camera images of a physical area adjacent the publisher location. The vehicle sensor data may include an indication of an available parking location adjacent the publisher vehicle. In one configuration, the vehicle sensor data includes a published data time and the subscriber request includes a request time period. The match condition may include a time match between the request time period and the published data time.

The method may include receiving an indication of an allowed region for the vehicle sensor data. The match condition may include a region match between the vehicle location and the allowed region.

Yet another implementation may include a computer program product for accessing vehicle sensor data. The computer program product includes computer readable program code configured to receive the vehicle sensor data and a vehicle location associated with the vehicle sensor data from a publisher vehicle, to receive, from a service subscriber, a subscriber request for remote data, the subscriber request including a request location, to determine whether there is a match condition, and to transmit the vehicle sensor data to the service subscriber upon the occurrence of the match condition. The match condition may include a vehicle location match between the request location and the vehicle location.

FIG.1shows an example system102for remotely accessing vehicle sensors. The system102includes a publisher vehicle104in communication with a server106via a communication network121. The publisher vehicle104includes a plurality of vehicle sensors108, such as exterior facing cameras, LIDAR sensors, ultrasonic sensors, radar sensors, proximity sensors, and location sensors. Thus, the publisher vehicle104can generate vehicle sensor data and can associate the vehicle's location with the vehicle sensor data.

The vehicle sensor data may include camera images of a physical area115adjacent the publisher vehicle's location. As used herein, adjacent means within a sensor's field of view. For example, the vehicle sensor data may include an indication of an available parking location114adjacent the publisher vehicle104. The vehicle sensor data may include a video image of a parked vehicle105adjacent the publisher vehicle104.

The system102may also include a publisher station110in communication with the server106via the communication network121. The publisher station110can be positioned at a stationary geographic location and may include one or more stationary sensors112that generate stationary sensor data associated with stationary location. For example, the publisher station110can be a computer mounted in a parking lot location with a camera as a stationary sensor112.

The system102may further include a service subscriber116in communication with the server106via the communication network121. The service subscriber116may be, for example, a portable computing device, such as a smartphone, a smart display, or a portable computer. In one configuration, the portable computing device is a computer on a subscriber vehicle118. The subscriber vehicle118may include one or more vehicle displays120. The subscriber vehicle118can include a plurality of vehicle sensors, such as, but not limited to, exterior facing cameras, LIDAR sensors, ultrasonic sensors, radar sensors, proximity sensors, audio sensors, and location sensors.

The server106, the service subscriber116, publisher vehicle104, publisher station110, and subscriber vehicle118can be programmed to communicate with one or more remote sites, such as the server106, via the communication network121. The network121can be one or more of various wired or wireless communication mechanisms, including any desired combination of wired (e.g., cable and fiber) and/or wireless (e.g., cellular, wireless, satellite, microwave, and radio frequency) communication mechanisms and any desired network topology (or topologies when multiple communication mechanisms are utilized). Exemplary communication networks include wireless communication networks using, for example, Bluetooth®, Bluetooth® Low Energy (BLE), IEEE 802.11, vehicle-to-vehicle (V2V), such as Dedicated Short Range Communications (DSRC), local area networks (LAN) and/or wide area networks (WAN), including the Internet, to provide data communication services.

FIG.2shows an example configuration of the server106. The server106may be a general-purpose server or a special-purpose server, and may be a virtual server or a cloud server. The server106includes a server processor202and a server memory204. The server processor202is configured to execute computer instructions stored in the server memory204. The server processor202can be multiple computer processors coupled together. Various types of memory technologies may be utilized in the server memory204, such as random access memory (RAM), read only memory (ROM), and Flash memory.

The server memory204may include instructions to cause the server processor202to receive vehicle sensor data and a vehicle location associated with the vehicle sensor data from the publisher vehicle104. The server106may also receive stationary sensor data from the publisher station110. The server106may receive additional meta data about the sensor data, such as a timestamp or a published data time, i.e., a time at which the sensor reports having detected and/or the server106reports having transmitted the data. The meta data may include a publisher data type that indicates, for example, the type of data transmitted by the publisher to the server and a publisher type that indicates, for example, whether the publisher is a publisher vehicle104or a publisher station110. Herein, when referring to sensor data, a data type means a type of sensor or sensing medium (e.g., video data, LIDAR data, etc.); a publisher type refers to a category to which a publisher belongs and typically includes at least a vehicle and station publisher type.

The server106may process sensor data to generate additional data types. For example, the server106may use sensors and weather data to alert a service subscriber116to an amount of snow in the vicinity of his/her vehicle. In another example, the server106may perform image processing of vehicle and/or stationary cameras, combined with location mapping and LIDAR data, to provide available parking data indicating whether a parking spot114is available. In this document, a “location” means a physical location on the surface of the earth, and can be specified according to some coordinate system, such as according to geo-coordinates used in a global coordinate system, e.g., by the Global Positioning System (GPS) or some other system using the Global navigation satellite system (GNSS).

The server106may receive an indication of an allowed region for the vehicle sensor data. That is, the publisher vehicle104may specify one or more locations where access to the vehicle sensor data is permitted or denied. For example, the publisher vehicle104may permit access to the vehicle sensor data on public streets but may deny access to the vehicle sensor data while on private roads and driveways. An allowed region in the present context is an area defined by a set of locations (e.g., according to a geo-fence or the like) where access to the vehicle sensor data is permitted.

The server106may further receive an indication of an allowed time for the vehicle sensor data. The allowed time specifies when access to the vehicle sensor data is allowed or denied. For example, the publisher vehicle104or the publisher station110may permit access to the vehicle sensor data during certain specified times of the day or certain specified days of week, and can deny access to sensor data at other specified times.

The server memory204may include instructions to cause the server processor202to receive a subscriber request from the service subscriber116. The subscriber request may include a request for remote data from a particular request location. The subscriber request may contain a request time period and a request data type.

For example, a subscriber in a subscriber vehicle118may be interested in locating an available parking location114every morning before work. The subscriber may issue a subscriber request for available parking data (request data type) at a particular area (request location) in real time from 8:15 AM to 8:45 AM on work days (request time).

In another example, a subscriber may want to monitor the physical area115around their parked vehicle105upon walking from an office building to the parked vehicle105. The subscriber may issue a subscriber request for video data (request data type) at a parking area (request location) in real time (request time) to be displayed on the service subscriber116.

The instructions stored in server memory204may cause the server processor202to determine whether there is a match condition between the subscriber request and available sensor data. The match condition can include a vehicle location match between the request location in the subscriber request and the vehicle location associated with the vehicle sensor data and/or the location associated with the stationary sensor data. The match condition may include a time match between the request timer period and the published data time of the vehicle sensor data. The match condition may include a type match between the publisher data type and the subscriber data type.

As discussed above, the publisher may prefer to limit access to the vehicle sensor data. The match condition may include a region match between the vehicle location and the allowed region associated with the vehicle sensor data. In this context, a region match means that the vehicle location overlaps an allowed region defined by a set of locations (e.g., according to a geo-fence or the like) where access to the vehicle sensor data is permitted. The match condition may include a time match between the allowed time period and the published data time associated with the sensor data. As used herein, a time match means the published data time falls within the allowed time period.

Once a match condition is found, the instructions may cause the server processor202to transmit the vehicle sensor data and/or stationary sensor data to the service subscriber. The server106can transmit and receive information, including sensor data and subscriber requests, via wired and wireless communication technologies. For example, the server106may communicate with the publisher vehicle104and/or the service subscriber through the communication network121.

The server memory204may also include instructions to cause the server processor202to provide a financial incentive for transmission of vehicle sensor data by the publisher vehicle104. For example, the publisher vehicle104may be given access to sensor data from other publishers in return for access to its sensor data. The server106may provide payment to the publisher vehicle104based on the number of subscribers to the sensor data. Thus, payment to the publisher vehicle104may increase based on the popularity of the vehicle's or station's sensor data.

In one configuration, the server106can include algorithms that can learn from data, including artificial intelligence (AI), getting computers to act without being explicitly programmed, automated reasoning, automated suggestion, automated adaptation, automated decision making, automated learning, or combination thereof. For example, the server106may utilize a machine learning model to automatically suggest to a service subscriber116a subscription to a data feed from a publisher vehicle104and/or a publisher station110.

A machine learning model can provide machine learning including prediction, recommendation, filtering such as data source filtering, machine learning processes, machine learning functions, or a combination thereof. For example, a suitable machine learning (ML) program such as a deep neural network (DNN) may be trained and then used to output, filter and/or recommend data sources. A DNN can be a software program that can be loaded in memory and executed by a processor included in a computer, such as a server106, and/or the service subscriber116, for example. In an example implementation, the DNN can include, but is not limited to, a convolutional neural network CNN, R-CNN Region-based CNN, Fast R-CNN, and Faster R-CNN. The DNN includes multiple nodes or neurons. The neurons are arranged so that the DNN includes an input layer, one or more hidden layers, and an output layer. The input and output layers may also include more than one node.

As one example, the DNN can be trained with ground truth data, i.e., data about a real-world condition or state. For example, the DNN can be trained with ground truth data and/or updated with additional data. Weights can be initialized by using a Gaussian distribution, for example, and a bias for each node can be set to zero. Training the DNN can include updating weights and biases via suitable techniques such as back-propagation with optimizations. Ground truth data means data deemed to represent a real-world environment, e.g., conditions and/or objects in the environment. Thus, ground truth data can include sensor data depicting an environment, e.g., a speed, location, etc., along with a label or labels describing the environment, e.g., a label describing the data, e.g., an available parking location, a prior user(s) subscriptions, etc.

In the present context, ground truth data used to train a DNN could include, for example, available parking locations, prior user subscriptions, day/time blocks, locations, and/or weather conditions. Then, the DNN could be included in the server106and/or a vehicle computer to output a prediction or recommendation for a data subscription.

In some configurations, a prediction output from a DNN or the like can recommend a data source or data sources, and can filter data sources. The server memory204may include instructions to train a machine learning model to determine recommended data sources based on historical subscriber requests from the service subscriber116. The server memory204may include instructions to train a machine learning model to determine recommended data sources based on historical subscriber requests from a plurality of other service subscribers. Additionally, the instructions may provide the recommended data sources to the service subscriber116.

FIG.3shows an example configuration of the service subscriber116. The service subscriber116may include a device processor302and device memory304. Depending on the exact configuration and type of service subscriber116, device memory304may be volatile, such as RAM, non-volatile, such as ROM and flash memory, or some combination of the two. Additionally, the service subscriber116may also have mass storage (removable and/or non-removable), such as a magnetic or optical disks or tape. Similarly, the service subscriber116may also have input devices, such as a keyboard and/or a mouse. The service subscriber116may include a display306, such as a vehicle display120, to output information. Other aspects of the service subscriber116may include network connections to other devices, computers, networks, servers, and the like using either wired or wireless media.

The device memory304may include device instructions executable by the device processor302to send the subscriber request for the remote data to the server106. As discussed above, the subscriber request may include a request for remote data from a particular request location, and may contain a request time period and a request data type.

The device memory304may further include instructions executable by the device processor302to receive the vehicle sensor data from the server106and to display the vehicle sensor data on the device display306. For example, the service subscriber116may a smartphone carried by someone walking to their parked vehicle105and the device display306shows a video image from a publisher vehicle104adjacent to the parked vehicle105. In another example, the service subscriber116is a computer carried by a subscriber vehicle118and the device display306is the vehicle display120showing available parking spots114on a displayed driving map.

FIGS.4A and4Bshow an example method for sensor access performed by the server106. The method includes receiving operation402to receive vehicle sensor data and a vehicle location associated with the vehicle sensor data by a server106from a publisher vehicle104. The receiving operation402may include receiving stationary sensor data associated with a stationary location from a publisher station110. It is contemplated that many sensor data publishers may communicate with the server106and create a pool of sensor data, with each publisher configuring their sensor data feeds to their preferences.

The sensor data may be generated from vehicle sensors108mounted at the publisher vehicle104and/or stationary sensors112at a publisher station110. For example, the vehicle sensor data may be camera images of a physical area115adjacent a publisher vehicle104and/or a publisher station110. Other sensors may be used by the method, such as LIDAR sensors and ultrasonic sensors. For example, LIDAR sensors may be used to indicate an available parking location114adjacent the publisher vehicle104. It is contemplated that the sensor data may include other metadata, such as a published data time and a publisher data type. After receiving operation402, control passes to receiving operation404.

At receiving operation404, the server106receives an indication of an allowed region and/or allowed time for the vehicle sensor data. In this manner, the publisher vehicle104can opt-in to share vehicle sensor data pursuant to their preferences. For example, publisher vehicle104can be configured by its user to share sensor data only at some locations and at only some days and/or times. The publisher vehicle104may opt to share certain data types, such as a live video feed from specific cameras and LIDAR sensor data but not audio sensor data. After receiving operation404, control passes to receiving operation406.

At receiving operation406, the server106receives, from a service subscriber116, a subscriber request for remote data. The subscriber request may include a request location, a request time period, and a subscriber data type. For example, a service subscriber116may operate a parked vehicle105that is not equipped with a video camera. The service subscriber may send a subscriber request to access video feeds from vehicle sensors108of publisher vehicles104located next to the parked vehicle105. The video feed can be used to monitor the physical area115next to the parked vehicle105. The service subscriber116may receive push notifications when the vehicle sensors108detect movement near the parked vehicle105. In this way, the service subscriber116may benefit and experience remote sensor features without their vehicle being equipped with a video camera or other sensors that monitor the surrounding environment.

In another scenario, a service subscriber116may plan to visit a particular destination location, such as camping location. The service subscriber116may send a subscriber request for video feed to the server106of the camping location to evaluate the conditions of the camping location. The subscriber request may be tailored to a specific time of day the subscriber is planning to arrive at the camping location to determine, for instance, the lighting conditions or parking availability at the camping location.

In a different scenario, a subscriber may be driving in a subscriber vehicle118and looking for an available parking spot114in a particular area. The subscriber vehicle118may request sensor data of available parking spots near its location from the server106to save time searching for a place to park. After receiving operation406, control passes to determining operation408.

At determining operation408, the server determines whether there is a match condition between the requested sensor data by service subscriber116and the available sensor data from the publisher vehicles104and/or publisher stations110. A match condition between a first datum and a second datum is met when the respective data are the same, e.g., matching strings, matching numeric values, when the first datum falls within a value range of the second datum, etc. The match condition may include a vehicle location match between the request location and the vehicle location of a publisher vehicle104. The match condition may include a stationary location match between the request location and the stationary location of a publisher station110. The match condition may include a time match between the request time period and the published data time. The match condition may include a type match between the publisher data type and the subscriber data type.

As discussed above, the publisher vehicle104may opt-in to sharing its sensor data. The publisher's sensor data may be limited to a particular region and/or to a particular time. For example, the match conditions may include geofenced locations. Thus, the match condition may include a region match between the vehicle location and the allowed region. The match condition may include a time match between the allowed time and the published data time. After determining operation408, control passes to transmitting operation410.

At transmitting operation410, the server106transmits the vehicle sensor data and/or stationary sensor data to the service subscriber116upon occurrence of the match condition. For example, when a publisher vehicle104is at location X and a subscribing vehicle118is planning to go to location X (or is on its way to location X) and enables push notifications for available parking spots114at location X, the server106may transmit notification of an available parking spot114based on the publisher vehicle's104sensor data if the publisher vehicle104has allowed sensor data sharing at location X, within the time frame of the requested data, and of the data type requested. After transmitting operation410, control passes to training operation412.

At training operation412, the server106may train a machine learning model to determine recommended data sources. The machine learning model may be trained based on historical subscriber requests from the service subscriber116. The machine learning model may alternatively or additionally be trained based on historical subscriber requests from a plurality of other service subscribers. For example, the machine learning model may use collaborative filtering to learn data sources usage patterns of other users similar to the service subscriber116.

In one configuration, AI/ML prediction may be incorporated for both subscriber (as described above) and publisher (auto suggest to publish based on geolocation or time based notification to maximize incentives). For example, the server instructions may include instructions to train a machine learning model to determine recommended data sources based on historical sensor data from other publisher vehicles. The machine learning model may suggest, to a publisher vehicle104, sharing video/camera data at a particular location and/or time period based on historical publisher patterns in sharing video/camera feed. After training operation412, control passes to providing operation414(seeFIG.4B).

At providing operation414, the server106provides the recommended data sources to the service subscriber116. The machine learning model may learn user patterns of accessing sensor data. With time the machine learning model checks if sensor data from vehicles similar to the subscriber's vehicle are available and auto-suggests to the subscriber access to other sensor data. The server106may check if data sources from vehicles similar to the subscriber vehicle118are available and may recommend to the service subscriber116such data sources.

Consider, for example, a scenario where the service subscriber116often visits location X every Friday and Saturday nights and parks their subscriber vehicle118at a multilevel parking lot. The subscriber vehicle118is not equipped with connected cameras and the service subscriber116uses the server106to check the subscriber vehicle's surroundings at 11 PM before walking to the subscriber vehicle118. With time the machine learning model can learn the service subscriber's patterns and auto suggests the service subscriber116view a camera feed when the subscriber vehicle118is parked at location Y on Tuesdays and Wednesdays at 8 PM. In another example a publisher vehicle is parked at location X every day from 9 AM to 5 PM and publishes video feed data. With time the system learns this pattern and auto suggests the owner to share his video feed at X between 9 AM to 5 PM.

In another scenario, several service subscribers may receive a video data stream from a particular vehicle sensor108when they are parked at location Y from 11 PM to 6 AM. If the service subscriber116is in location Y for the first time, the server106may recommend the same video data stream to the service subscriber116. After providing operation414, control passes to receiving operation416.

At receiving operation416, the server106receives a financial payment from the service subscriber116. Subscriptions can be, for example, on a need basis (for a few days), weekly, monthly, or as part of a special weekend package. After receiving operation416, control passes to providing operation418.

At providing operation418, the server106provides a financial incentive for transmission of vehicle sensor data by the publisher vehicle104. For example, the publisher vehicle104may receive discounts, coupons, incentives for opting-in to the system102.

Providing operation414may additionally or alternatively include recommending to the publisher vehicle sensor data to publish by providing the recommended sensor data for publication to the publisher vehicle104. The machine learning model may learn user patterns of publishing sensor data and auto-suggest to the publisher sensor data to publish to maximize, for example, geolocation data coverage and/or time of day data coverage.

FIG.5shows an example method for sensor access performed by the service subscriber116. As noted above, the service subscriber116may be a portable device, such as a smartphone or a computer carried by a subscriber vehicle118.

The method may include a sending operation502that sends a subscriber request for remote data by the service subscriber116to the server106. The request may include notification preferences, such as the duration, proximity, and specific times/days, the remote data is requested. It is contemplated that the server106may offer, for example, sensor data of traffic ahead of the subscriber vehicle118and unmanned aerial vehicle (drone) sensor data. After sending operation502, control passes to receiving operation504.

At receiving operation504, the service subscriber116receives the vehicle sensor data from the server106. The sensor data may be, for example, a live camera feed from a publisher vehicle104. In one configuration, the server106issues push notifications for sensor data available at publisher vehicles104and/or publisher stations110. Push notifications can be configured for a location, day, time window, and so on. After receiving operation504, control passes to displaying operation506.

At displaying operation506, the vehicle sensor data is displayed at the service subscriber116. In one configuration the sensor data is displayed a vehicle display120at the subscriber vehicle118. For example, the vehicle display120may show available parking locations114within 200 meters of the subscriber vehicle118.

The descriptions of the various examples and implementations have been presented for purposes of illustration but are not intended to be exhaustive or limited to the implementations disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described implementations. The terminology used herein was chosen to best explain the principles of the implementations, the practical application or technical enhancements over technologies found in the marketplace, or to enable others of ordinary skill in the art to understand the implementations disclosed herein.

As will be appreciated, the methods and systems described may be implemented as a computer program product. The computer program product may include a computer readable storage medium (or media) having computer readable program instructions thereon for causing a processor to carry out operations discussed herein.

The computer readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device. The computer readable storage medium may be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of the computer readable storage medium includes the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device such as punch-cards or raised structures in a groove having instructions recorded thereon, and any suitable combination of the foregoing. A computer readable storage medium, as used herein, is not to be construed as being transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission media (e.g., light pulses passing through a fiber-optic cable), or electrical signals transmitted through a wire.

Computer readable program instructions described herein can be downloaded to respective computing/processing devices from a computer readable storage medium or to an external computer or external storage device via a network, for example, the Internet, a local area network, a wide area network and/or a wireless network. The network may comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and/or edge servers. A network adapter card or network interface in each computing/processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium within the respective computing/processing device.

Computer readable program instructions for carrying out operations may be assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state-setting data, or either source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C++ or the like, and conventional procedural programming languages, such as the “C” programming language or similar programming languages. The computer readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider). In some implementations, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGA), or programmable logic arrays (PLA) may execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry.

Various implementations are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer readable program instructions.

These computer readable program instructions may be provided to a processor of a general-purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. These computer readable program instructions may also be stored in a computer readable storage medium that can direct a computer, a programmable data processing apparatus, and/or other devices to function in a particular manner, such that the computer readable storage medium having instructions stored therein comprises an article of manufacture including instructions which implement aspects of the function/act specified in the flowchart and/or block diagram block or blocks.

The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process, such that the instructions which execute on the computer, other programmable apparatus, or other device implement the functions/acts specified in the flowchart and/or block diagram block or blocks.

The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the block may occur out of the order noted in the Figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or acts or carry out combinations of special purpose hardware and computer instructions.

All terms used in the claims are intended to be given their plain and ordinary meanings as understood by those skilled in the art unless an explicit indication to the contrary in made herein. In particular, use of the singular articles such as “a,” “the,” “said,” etc. should be read to recite one or more of the indicated elements unless a claim recites an explicit limitation to the contrary. Use of “in response to” and “upon determining” indicates a causal relationship, not merely a temporal relationship.

The disclosure has been described in an illustrative manner, and it is to be understood that the terminology which has been used is intended to be in the nature of words of description rather than of limitation. Many modifications and variations of the present disclosure are possible in light of the above teachings, and the disclosure may be practiced otherwise than as specifically described.