Patent ID: 12254116

DETAILED DESCRIPTION

As described above, previous technologies fail to provide efficient and reliable solutions for determining a more optimal memory resource configuration for network nodes to execute tasks, obfuscate confidential information in task logs, and detect anomalies in blockchain transactions. Embodiments of the present disclosure and its advantages may be understood by referring toFIGS.1through7.FIGS.1through7are used to describe a system and method for determining memory resource configuration for network nodes to operate in a distributed computing environment, a system and method for detecting and obfuscating confidential information in task logs, and a system and method for detecting anomalies in blockchain transactions and updating a distributed blockchain ledger based on the detected anomalies.

System Overview

FIG.1illustrates an embodiment of a system100that is generally configured to 1) determine a more optimal memory resource configuration184for network nodes130to operate in a distributed computing network106, 2) detect and obfuscate confidential information180in task logs104, and 3) update a distributed ledger160of a blockchain network140based on detecting anomalies410in blockchain transactions162. In one embodiment, system100comprises a server170communicatively coupled with one or more computing devices120(e.g., computing devices120aand120n) associated with respective third-party vendors122(e.g., vendors122aand122n), one or more network nodes130(e.g., network nodes130aand130n), and a blockchain network140via a network110. Network110enables communication among the components of the system100. Each third-party vendor122may be an organization that provides a service and/or a product to its users. Each network node130may be a computing device or node in a distributed computing network106. Blockchain network140may include blocks142(e.g., blocks142a-d). Each block142may store a record of distributed ledger160that is distributed among the blocks142. The server170comprises a processor172in signal communication with a memory176. Memory176stores software instructions178that when executed by the processor172, cause the processor172to perform one or more operations described herein.

The organization108may provide services and/or products to its users. For example, the organization108may provide services and/or products via a software, mobile, and/or a web application, collectively referred to herein as an application102. Users or clients of the organization108may use the application102to request services and/or products. The server170implements the application102by the processor172executing the software instructions178. The application102may have a graphical user interface (e.g., a webpage) that allows the users to interact with the application102.

When the users interact with the application102and/or when the application102is executed and running, multiple tasks112may be performed. The tasks112may include executing the code used to implement the application102, such as when a user opens a webpage on the application102, a task112to load the webpage may be executed. The tasks112may be processed using the network nodes130in a distributed computing network106. For example, a task112may be allocated to several network nodes130to be processed. The output of executing the task112may result in task logs104. The task logs104include records of events associated with the operations of the application102and operations performed on the application102by users accessing the application102.

One or more network nodes130may send the task logs104to the server170. The server170maintains the task logs104for troubleshooting, archiving, and other purposes. In some cases, the application102may fail, e.g., due to an error or a bug in code that is used to implement the application102. For example, the application102may fail and a webpage, or a user interface page, a menu on the application102may not load. In such cases, a respective task log104may include the event that led to the failure of the application102and operations that happened at the failure time.

In certain embodiments, the organization108(e.g., via the server170) may communicate the task logs104to a third-party vendor122for debugging, troubleshooting, and determining the cause of the failure. In some cases, the task logs104may include confidential information180. The confidential information180may include private user information, such as social security number, name, address, card number, and proprietary information related to the network nodes130, such as hostname, port number, database name, network node name, communication protocol, etc. The third-party vendor122may not have the authority to access the confidential information180. Thus, the server170may be configured to detect and obfuscate the confidential information180in the task logs104before sending the task logs104to the third-party vendor122.

System Components

Network

Network110may be any suitable type of wireless and/or wired network. The network110is not connected to the Internet or public network. The network110may include all or a portion of an Intranet, a peer-to-peer network, a switched telephone network, a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), a personal area network (PAN), a wireless PAN (WPAN), an overlay network, a software-defined network (SDN), a virtual private network (VPN), a mobile telephone network (e.g., cellular networks, such as 4G or 5G), a plain old telephone (POT) network, a wireless data network (e.g., WiFi, WiGig, WiMax, etc.), a long-term evolution (LTE) network, a universal mobile telecommunications system (UMTS) network, a peer-to-peer (P2P) network, a Bluetooth network, a near field communication (NFC) network, and/or any other suitable network that is not connected to the Internet. The network110may be configured to support any suitable type of communication protocol as would be appreciated by one of ordinary skill in the art.

Third-Party Vendor

Each third-party vendor122may be an organization that provides services and/or products to its users. A third-party vendor122may receive task logs104for troubleshooting and debugging a failure that has led to the application102failing as described above. For example, the third-party vendor122may receive the task logs104from the server170via the computing device120associated with the third-party vendor122.

Each third-party vendor122may be associated with one or more computing devices120. In the illustrated embodiment, the third-party vendor122ais associated with the computing device120a, and the third-party vendor122nis associated with the computing device120n.

Each of the computing devices120aand120nis an instance of the computing device120. Computing device120is generally any device that is configured to process data and interact with users. Examples of the computing device120include, but are not limited to, a personal computer, a desktop computer, a workstation, a server, a laptop, a tablet computer, a mobile phone (such as a smartphone), etc. The computing device120may include a user interface, such as a display, a microphone, keypad, or other appropriate terminal equipment usable by users. The computing device120may include a hardware processor, memory, and/or circuitry (not explicitly shown) configured to perform any of the functions or actions of the computing device120described herein. For example, a software application designed using software code may be stored in the memory and executed by the processor to perform the functions of the computing device120. The computing device120is configured to communicate with other devices via the network110.

Distributed Computing Network

Distributed computing network106may be a cloud of network nodes130, and generally configured to execute tasks112in a distributed environment. For example, the server170executes or runs the application102so that users or clients of the organization108can access the application102from their computing devices, e.g., smartphones, desktop computers, laptops, etc. (not shown).

When the server170executes the application102, the server170may employ the distributed computing network106to execute the tasks112, similar to that described above. The task112may be divided into smaller portions and each portion may be forwarded to a different network node130. The network nodes130may execute the task112and produce or output task logs104. The task logs104are transmitted to the server170.

Each of the network nodes130aand130nmay be an instance of a network node130. Network node130is generally any device that is configured to process data, such as the tasks112. Examples of the network node130include, but are not limited to, a personal computer, a desktop computer, a workstation, a server, a virtual machine, and the like. The network node130may include a hardware processor, memory, and/or circuitry (not explicitly shown) configured to perform any of the functions or actions of the network node130described herein. For example, a software application designed using software code may be stored in the memory and executed by the processor to perform the functions of the network node130. The network node130is configured to communicate with other devices via the network110. The network node130may interchangeably be referred to as a container130.

Blockchain Network

Blockchain network140comprises a cloud of computer systems (referred to herein as blocks142) and is generally configured to keep records of blockchain ledger160and any other data, communications and interactions among the blocks142, and the blockchain transactions162. The blockchain network140may comprise any number of blocks142. Each block142may comprise a computing device, a virtual machine, and/or the like. In the present disclosure, a block142may interchangeably be referred to as a network node, a node, or a network device. The blockchain network140generally refers to a distributed database (e.g., distributed ledger160) shared between a plurality of network nodes142in a network. The system100may employ any suitable number of devices (e.g., network nodes142) to form a distributed network that maintains the blocks in form of a blockchain. The blockchain links together the blocks142of data which may include the blockchain ledger160.

Each network node142comprises a blockchain ledger160(e.g., stored in the memory148) that is configured to store a copy of the blockchain140(not explicitly shown), which contains every blockchain transaction162executed in the network and any other data. The blockchain140links together blocks142of data which comprise identifiable units called blockchain transactions162. Blockchain transactions162may comprise information, files, or any other suitable type of data, such as data associated with digital documents, tasks112, task logs104, user information, timestamps of tasks112, timestamps of task logs104, and any other type of information.

Each block142in the blockchain140comprises a hash value154and information derived from a preceding block142. For example, every block142in the blockchain140includes a hash152of the previous block142. By including the hash152, the blockchain140comprises a chain of blocks142from a genesis block142to the current block142. Each block142is guaranteed to come after the previous block142chronologically because the previous block's hash152would otherwise not be known. In one embodiment, blocks142in a blockchain140may be linked together by identifying a preceding block142with a cryptographic checksum (e.g., secure hash algorithm (SHA)-256) of its contents (e.g. blockchain transactions162, and additional metadata stored in the memory148) which serves as each block's unique identifier. Links are formed by storing the cryptographic checksum identifier of one block142in the metadata of another block142, such that the former block142becomes the predecessor of the latter block142. In this way, the blocks142form a chain that can be navigated from block-to-block by retrieving the cryptographic checksum of a particular block's predecessor from the particular block's own metadata. Each block142is computationally impractical to modify once it has been in the blockchain140because every block142after it would also have to be regenerated. These features protect data stored in the blockchain140from being modified by bad actors. Thus, these features improve the information security of the data stored in the blockchain140.

The consensus module158is configured to establish a consensus among the blocks142about the present state of the distributed ledger160. For example, the consensus module158may be executed by the processor144executing the software instructions150to implement a procedure through which all the blocks142of the blockchain network140reach a common agreement about the present state of the distributed ledger160. In this way, consensus module158in each block142achieves reliability in the blockchain network140and establish trust between the blocks142in a distributed computing environment. The consensus module158implements a consensus protocol to perform its operations. Essentially, the consensus protocol makes sure that every new block142that is added to the blockchain140is the one and only version of the truth that is agreed upon by all the nodes142in the blockchain140.

When a network node142publishes an entry (e.g. a blockchain transaction162) in its blockchain ledger160, the blockchain140for all other network nodes142in the distributed network is also updated with the new entry. Thus, data published in a blockchain140is available and accessible to every network node142with a blockchain ledger160. This allows the data stored in the block142to be accessible for inspection and verification at any time by any device with a copy of the blockchain ledger160.

Each of the blocks142a-142dis an instance of a block142. Each block142may comprise a processor144in signal communication with a memory148and a network interface146.

Processor144comprises one or more processors operably coupled to the memory148. The processor144is any electronic circuitry, including, but not limited to, state machines, one or more central processing unit (CPU) chips, logic units, cores (e.g., a multi-core processor), field-programmable gate array (FPGAs), application-specific integrated circuits (ASICs), or digital signal processors (DSPs). The processor144may be a programmable logic device, a microcontroller, a microprocessor, or any suitable combination of the preceding. The one or more processors are configured to process data and may be implemented in hardware or software. For example, the processor144may be 8-bit, 16-bit, 32-bit, 64-bit, or of any other suitable architecture. The processor144may include an arithmetic logic unit (ALU) for performing arithmetic and logic operations. The processor144may register the supply operands to the ALU and stores the results of ALU operations. The processor144may further include a control unit that fetches instructions from memory and executes them by directing the coordinated operations of the ALU, registers and other components. The one or more processors are configured to implement various software instructions. For example, the one or more processors are configured to execute software instructions150to perform one or more functions described herein. In this way, processor144may be a special-purpose computer designed to implement the functions disclosed herein. In an embodiment, the processor144is implemented using logic units, FPGAs, ASICs, DSPs, or any other suitable hardware. The processor144is configured to operate as described inFIGS.1-7.

Network interface146is configured to enable wired and/or wireless communications (e.g., via network110). The network interface146is configured to communicate data between the network node142and other devices (e.g., computing devices120), server170, other network nodes142, databases, systems, or domains. For example, the network interface146may comprise a WIFI interface, a local area network (LAN) interface, a wide area network (WAN) interface, a modem, a switch, or a router. The processor144is configured to send and receive data using the network interface146. The network interface146may be configured to use any suitable type of communication protocol as would be appreciated by one of ordinary skill in the art.

Memory148may be volatile or non-volatile and may comprise a read-only memory (ROM), random-access memory (RAM), ternary content-addressable memory (TCAM), dynamic random-access memory (DRAM), and static random-access memory (SRAM). Memory148may be implemented using one or more disks, tape drives, solid-state drives, and/or the like. The memory148may store any of the information described inFIGS.1-7along with any other data, instructions, logic, rules, or code operable to implement the function(s) described herein when executed by processors144. The memory148is operable to store software instructions150, previous block hash value152, hash value154, data156, consensus module158, blockchain ledger160, and/or any other data and instructions. The data156may include timestamps of reception of tasks112, execution of tasks112, index of the block142, etc. The hash value154may be used to uniquely identify the corresponding network node142. For example, the hash value154may include an alphanumerical string. The hash152of the previous block142may include a hash value152of the previous block142generated before the corresponding block142. The order and place of the block142in the blockchain140may be determined by the hash152of the previous block142. The software instructions150may comprise any suitable set of software instructions, logic, rules, or code operable to execute the processor144to perform the functions of the processor144and the block142described herein.

Each block142may include information derived from a preceding block142. For example, every block142in the blockchain includes a hash152of the previous block142. By including the hash152of the previous block142, the blockchain network140includes a chain of blocks142ato142dfrom a genesis block142a(or a block not shown to the left of the block142ain the example ofFIG.1) to the latest block142d(or a block not shown to the right of the block142din the example ofFIG.1). Each block142is guaranteed to come after the previous block142chronologically because the previous block's hash value152would otherwise not be known.

Server

Server170is generally a device that is configured to process data and communicate with computing devices (e.g., computing devices120), blockchain network140, network nodes130, databases, systems, etc., via the network110. The server170may be associated with the organization108. The server170is generally configured to oversee the operations of the processor172as described further below in conjunction with the operational flow200of system100described inFIG.2, the operational flow300of system100described inFIG.3, the operational flow400of system100described inFIG.4, and methods500-700described inFIG.5-7, respectively.

Processor172comprises one or more processors operably coupled to the memory176. The processor172is any electronic circuitry, including, but not limited to, state machines, one or more CPU chips, logic units, cores (e.g., a multi-core processor), FPGAs, ASICs, or DSPs. For example, one or more processors may be implemented in cloud devices, servers, virtual machines, and the like. The processor172may be a programmable logic device, a microcontroller, a microprocessor, or any suitable combination of the preceding. The one or more processors are configured to process data and may be implemented in hardware or software. For example, the processor172may be 8-bit, 16-bit, 32-bit, 64-bit, or of any other suitable architecture. The processor172may include an ALU for performing arithmetic and logic operations, registers the supply operands to the ALU and store the results of ALU operations, and a control unit that fetches instructions from memory and executes them by directing the coordinated operations of the ALU, registers and other components. The one or more processors are configured to implement various instructions. For example, the one or more processors are configured to execute instructions (e.g., software instructions178) to implement the processor172. In this way, processor172may be a special-purpose computer designed to implement the functions disclosed herein. In an embodiment, the processor172is implemented using logic units, FPGAs, ASICs, DSPs, or any other suitable hardware. The processor172is configured to operate as described inFIGS.1-7. For example, the processor172may be configured to perform one or more operations of methods500,600, and700as described inFIGS.5-7, respectively.

Network interface174is configured to enable wired and/or wireless communications. The network interface174may be configured to communicate data between the server170and other devices, systems, or domains. For example, the network interface174may comprise an NFC interface, a Bluetooth interface, a Zigbee interface, a Z-wave interface, a radio-frequency identification (RFID) interface, a WIFI interface, a LAN interface, a WAN interface, a MAN interface, a PAN interface, a WPAN interface, a modem, a switch, and/or a router. The processor172may be configured to send and receive data using the network interface174. The network interface174may be configured to use any suitable type of communication protocol.

The memory176may be volatile or non-volatile and may comprise read-only memory (ROM), random-access memory (RAM), ternary content-addressable memory (TCAM), dynamic random-access memory (DRAM), and static random-access memory (SRAM). The memory176may include one or more of a local database, cloud database, network-attached storage (NAS), etc. The memory176comprises one or more disks, tape drives, or solid-state drives, and may be used as an over-flow data storage device, to store programs when such programs are selected for execution, and to store instructions and data that are read during program execution. The memory176may store any of the information described inFIGS.1-7along with any other data, instructions, logic, rules, or code operable to implement the function(s) described herein when executed by processor172. For example, the memory176may store applications102, task logs104, tasks112, software instructions178, confidential information180, memory resource configuration engine182, memory resource configuration184, obfuscation engine188, machine learning module190, patterns192, training dataset194, probability score196, anomaly detection engine198, historical memory resources210, memory categories214, total historical memory resources216, execution times218, linear equations224, memory resource configuration184, memory resource utilization212, threshold memory resource utilization222, anomalies410, and/or any other data or instructions. The software instructions178may comprise any suitable set of instructions, logic, rules, or code operable to execute the processor172and perform the functions described herein, such as some or all of those described inFIGS.1-7.

Memory Resource Configuration Engine

Memory resource configuration engine182may be implemented by the processor172executing software instructions178, and is generally configured to determine a more optimal memory resource configuration184for network nodes130.

One approach to configure the network nodes130is to configure their memory resources using a default memory resource configuration220. In the default memory resource configuration220, a default value for amount of memory resources is assigned or allocated to different memory categories214of each network node130.

The memory categories214may be used for various operations at a network node130. The memory categories214associated with a container130may include a first memory resource for data storage, a second memory resource for an input and output data transfer to a hard drive, a third memory resource for a memory buffer used for data transfer between a central processing unit and the hard drive, a fourth memory resource for a cache memory used for executing instructions of the central processing unit, and a fifth memory resource used for virtual memory buffering. For example, the memory categories214may include the first memory resource related to for data size (e.g., 2048 megabytes (Mb), the second memory resource related to Disk Input/Output that is input and output operations involving a physical memory disk (e.g., 197 Mb for data transfer between the hard disk and RAM), the third memory resource related to RAM (e.g., 5120 Mb), the fourth memory resource related to Cache (e.g., 360 Mb), and the fifth memory resource related to Java virtual memory (JVM), e.g., 10×Cache×1024 Mb. However, this approach suffers from several drawbacks. For example, the default memory resource configuration220may not an optimal configuration to run every task112. Some tasks112may need more memory resources, while other tasks112may need fewer memory resources.

The memory resource configuration engine182is configured to determine a more optimal memory resource configuration184for each container130to execute tasks112and future tasks112. This process is described in greater detail inFIG.2.

Obfuscation Engine

Obfuscation engine188may be implemented by the processor172executing software instructions178, and is generally configured to detect and obfuscate confidential information180in task logs104. In certain embodiments, the obfuscation engine188may implement a machine learning module190to perform its operations. In certain embodiments, the machine learning module190may include a density-based clustering algorithm that does not require any predefined number of clusters to determine patterns192in the text of log files104.

In certain embodiments, the machine learning module190may utilize a heterogeneous density-based spatial clustering of application and noise algorithm for various application logs104to segregate and determine a series of patterns192to identify confidential information180, obfuscate the confidential information180, and store the task log104with obfuscated confidential information180in blockchain ledger160of the blockchain network140.

In certain embodiments, the machine learning module190may include a support vector machine, neural network, random forest, k-means clustering, etc. In certain embodiments, the machine learning module190may be implemented by a plurality of neural network layers, convolutional neural network layers, Long-Short-Term-Memory (LSTM) layers, Bi-directional LSTM layers, recurrent neural network layers, and the like. In certain embodiments, the machine learning module190may include text processing, text parsing, text analysis, or any other linguistic analysis algorithm. In certain embodiments, the machine learning module190may perform word tokenization, sentence tokenization, word segmentation, sentence segmentation, word-tagging, sentence tagging, word sequences, sentiment analysis, and/or any other linguistic pattern analysis.

The task logs104may be in various data formats or data structures. Thus, in certain embodiments, the machine learning module190is configured to detect confidential information180in a task log104that is in any data format, such as unstructured data format, structured data format, and semi-structured data format. The task logs104may be compressed by any compression technique. Various compression techniques may cause the text in the task logs104to be different than its original state (i.e., uncompressed state). In certain embodiments, the machine learning module190may be configured to detect confidential information180in a task log104that is compressed by any compression technique.

The obfuscation engine188may implement the machine learning module190to detect the confidential information180in task logs104. In response, the obfuscation engine188obfuscates the detected confidential information180.

The machine learning module190is trained to detect confidential information180using a training dataset194. The training dataset194may include a lexicon of words, numbers, and/or symbols that are associated with the confidential information180and/or are the confidential information180. In the training process, the machine learning module190is given historical task logs104that each may or may not include confidential information180. If a historical task log104includes confidential information180, it is labeled as such and the confidential information180is marked (e.g., indicated as confidential information). If a historical task log104does not include confidential information180, it is labeled as such indicating that it does not include confidential information180.

The obfuscation engine188feeds the historical task logs104to the machine learning module190. The machine learning module190parses the historical task logs104and learns the patterns192of the text in each historical task logs104. In this process, the machine learning module190extracts features of the historical task logs104, where the features are represented by a vector of numerical values. The machine learning module190learns the relationships and associations between different words, numbers, and symbols in the historical task logs104.

The machine learning module190may be implemented by a set of rules to identify the confidential information180. The set of rules may include a rule that indicates if a word is followed by a port number, it is a confidential information180, a rule that indicates if a word is followed by a date or timestamp, it is confidential information180, among others. The set of rules may also indicate a type of confidential information180, e.g., a rule may indicate if a word is followed by a data or timestamp, it is confidential information180and its is type is a server name. In this manner, the machine learning module190may determine patterns192of text in the task logs104.

In the testing process, the machine learning module190is given unlabeled task logs104and is asked to predict which portions or words of the task log104is among the confidential information180.

In refining process, the machine learning module190goes through a backpropagation process where the results of the machine learning module190are evaluated and used as feedback to refine weight and bias values of the machine learning module190. The operation of detecting and obfuscating the confidential information180is described in greater detail inFIG.3.

Anomaly Detection Engine

Anomaly detection engine198may be implemented by the processor172executing software instructions178, and is generally configured to detect anomalies in blockchain transactions162and update the distributed ledger160based on the detection of anomalies410. The operation of detecting anomalies in blockchain transactions162and updating the distributed ledger160based on the detected anomalies410is described in greater detail inFIG.4.

Example Operational Flow for Determining Memory Resource Configurations for Containers

FIG.2illustrates an example operational flow200of system100ofFIG.1for determining memory resource configurations184for containers130. As discussed inFIG.1, tasks112are sent to containers130to be executed in the distributed computing network106. Typically, the containers130are configured with a default memory resource configuration220which is not optimal to execute various tasks112. For example, the default memory resource configuration184may result in a memory resource utilization212that is less than a threshold memory resource utilization222(e.g., less than 50%, 40%, etc.) at the network nodes130executing the tasks112. In another example, the default memory resource configuration184may lead to an increase in application102downtime due to the unresolved failure in the task112and task logs104. The system100may implement the operational flow200to determine a more optimal memory resource configuration184for the containers130to execute tasks112. As a result, the efficiency in executing tasks112is increased and the application102downtime is reduced.

Determining Historical Memory Resource Utilization

The server170may implement the memory resource configuration engine182to perform the operational flow200. The memory resource configuration engine182accesses historical tasks112. The historical tasks112may include the tasks112that the containers130have performed using the default memory resource configuration220.

The memory resource configuration engine182determines historical memory resources210used to execute each of the historical tasks112. The historical memory resources210may be the memory resources used in the default memory resource configuration220. The historical memory resources210may be associated with memory categories214used for various operations of a container130.

The memory resource configuration engine182determines total historical memory resources216allocated for each memory category214. An example table illustrating a default memory resource configuration220for a container130is illustrated in Table 1.

TABLE 1Example default memory resource configuration 220.Memory categoryAmount of memory214resource (Mb)Data size2048Disk I/O197RAM5120Cache360JVM10 * Cache * 1024

In Table 1, a total memory resource is divided or allocated to five memory categories214to perform various operations of a container130. The example in Table 1 is for illustration purposes only and is not meant to limit the score of the present disclosure. In other examples, other memory categories214may be included in the default memory resource configuration220. In other examples, other memory resources may be allocated to each memory category214.

An example table illustrating the historical tasks112for different applications102, historical memory resources210, and total historical memory resources216, and time taken 18 to execute each historical task112is illustrated in Table 2.

TABLE 2Example historical memory resources 210used to execute historical tasks 112.Taskutilizationpool/TotalhistoricalMemorymemorycategoryHistoricalHistoricalHistoricalHistoricalresources214 (Mb)task 112atask 112btask 112ctask 112d216Data Size2403222346120Disk I/O3121031215440RAM3134310306670Cache2435243605270JVM371434146150Execution16060152001709016030time 218hourshourshourshours

In the example of Table 2, the first historical task112ais associated with a first application102, the second historical task112bis associated with a second application102, the third historical task1120cis associated with a third application102, and the fourth historical task112dis associated with a fourth application102. In this example, the first historical task112amay be associated with a first third-party vendor122, the second historical task112bmay be associated with a second third-party vendor122, the third historical task112cmay be associated with a third third-party vendor112, and the fourth historical task112dmay be associated with a fourth third-party vendor112.

In the example of Table 2, the historical task112ahas used 240 Mb data size, 31 Mb disk I/O, 31 Mb RAM, 24 Mb Cache, and 37 Mb JVM. The historical task112atook 16060 hours to complete, i.e., the execution time218of the historical task112a. In the example of Table 2, the historical task112bhas used 32 Mb data size, 210 Mb disk I/O, 34 Mb RAM, 35 Mb Cache, and 14 Mb JVM. The historical task112btook 15200 hours to complete, i.e., the execution time218of the historical task112b. In the example of Table 2, the historical task112chas used 22 Mb data size, 31 Mb disk I/O, 310 Mb RAM, 24 Mb Cache, and 34 Mb JVM. The historical task112ctook 17090 hours to complete, i.e., the execution time218of the historical task112c. In the example of Table 2, the historical task112dhas used 34 Mb data size, 21 Mb disk I/O, 30 Mb RAM, 360 Mb Cache, and 14 Mb JVM. The historical task112dtook 16030 hours to complete, i.e., the execution time218of the historical task112d.

In the example of Table 2, the total historical memory resource216for the data size memory category214across all historical tasks112is 6120 Mb. In the example of Table 2, the total historical memory resource216for the disk I/O memory category214across all historical tasks112is 5440 Mb. In the example of Table 2, the total historical memory resource216for the RAM memory category214across all historical tasks112is 6670 Mb. In the example of Table 2, the total historical memory resource216for the Cache memory category214across all historical tasks112is 5270 Mb. In the example of Table 2, the total historical memory resource216for the JVM memory category214across all historical tasks112is 6150 Mb.

In certain embodiments, the memory resource configuration engine182determines the historical memory resources210, total historical memory resources216, and execution time218based on analyzing the execution of the historical tasks112. For example, the memory resource configuration engine182may receive a report of the execution of the historical tasks112from one or more containers130.

Determining a More Optimal Memory Resource Configuration to Execute Tasks

Now that the memory resource configuration engine182has determined the parameters described above, it may analyze these parameters to determine a more optimal memory resource configuration184. For example, as discussed above, using the default memory resource configuration220, the memory resource utilization212may become less than the threshold memory resource utilization222. To improve the memory resource utilization of the container130executing the tasks112, the memory resource configuration engine182may perform one or more operations below.

The memory resource configuration engine182determines the memory resource utilization212based on analyzing the historical memory resource210allocations to each task112, total historical memory resources216for each memory category214, and execution time218for each task112. The memory resource utilization212may be represented by a percentage number, 50%, 45%, etc.

The memory resource configuration engine182compares the memory resource utilization212with the threshold memory resource utilization222. For example, assume that the memory resource configuration engine182determines that the memory resource utilization212in executing at least on task112from the historical tasks112is less than the threshold memory resource utilization222, e.g., less than 50%, 45%, etc. In response, the memory resource configuration engine182determines a memory resource configuration184based on the historical memory resources210and total historical memory resources216.

The memory resource configuration184is determined to be used for configuring each container130such that the memory resource configuration184yields the memory resource utilization more than the threshold memory resource utilization222. In this process, the memory resource configuration engine182may be tasked to solve an example Equation (1). The Equation (1) may be one of the linear equations224that the memory resource configuration engine182is asked to solve to determine the parameters for the memory resource configuration184.

MAX⁡(X⁢1,X⁢2,X⁢3,X⁢4,X⁢5)=total⁢memory⁢1*X⁢1+total⁢memory⁢2*X⁢2+total⁢memory⁢3*X⁢3+total⁢memory⁢4*X⁢4+total⁢memory⁢5*X⁢5Equation⁢(1)

The parameter MAX is a name of the function used to calculate the five unknown parameters X1, X2, X3, X4, and X5. The parameters X1, X2, X3, X4, and X5 are variables to be determined. The parameters X1, X2, X3, X4, and X5 are values of amount of memory resources that can be added to and/or reduced from a corresponding memory resource210.

The total memory1parameter is the total memory resources216for the first memory category214, e.g., data size (which in the example of Table 2 is 6120). The total memory2parameter is the total memory resources216for the second memory category214, e.g., disk I/O (which in the example of Table 2 is 5440). The total memory3parameter is the total memory resources216for the third memory category214, e.g., RAM (which in the example of Table 2 is 6670). The total memory4parameter is the total memory resources216for the fourth memory category214, e.g., Cache (which in the example of Table 2 is 5270). The total memory5parameter is the total memory resources216for the fifth memory category214, e.g., JVM (which in the example of Table 2 is 6150). Thus, considering the example of Table 2, the equation (1) may be adapted to equation (2) below.

MAX⁡(X⁢1,X⁢2,X⁢3,X⁢4,X⁢5)=6120*X⁢1+5440*X⁢2+6670*X⁢3+5270*X⁢4+6150*X⁢5Equation⁢(2)

The memory resource configuration engine182may be given the following linear equations as constraints. These linear equations may be included in the linear equations224given to the memory resource configuration engine182to solve. Considering the example of Table 2, the linear equations considered as constraints (included in the linear equations224) to solve the Equation (1) may be as equations (3) to (6).

240*X⁢1+31*X⁢2+31*X⁢3+24*X⁢4+37*X⁢5<=16060Equation⁢(3)32*X⁢1+210*X⁢2+34*X⁢3+35*X⁢4+14*X⁢5<=15200Equation⁢(4)22*X⁢1+31*X⁢2+310*X⁢3+24*X⁢4+34*X⁢5<=17090Equation⁢(5)34*X⁢1+21*X⁢2+30*X⁢3+360*X⁢4+14*X⁢5<=16030Equation⁢(6)

Since the parameters X1, X2, and X3 are memory resources, non-negativity constraints may be given to the memory resource configuration engine182, as X1>=0, X2>=0, and X3>=0. In certain embodiments, further non-negativity constraints of X4>=0 and X5>=0 may also be given to the memory resource configuration engine182.

Based on the given example in Table 2, the memory resource configuration engine182determines deviation memory resources226(i.e., X1, X2, X3, X4, and X5) to be added to a corresponding memory resource category214. Based on the deviation memory resources226, the memory resource configuration engine182determines the memory resource configuration184.

Table 3 illustrates an example memory resource configuration184.

TABLE 3Example memory resource configuration 184.Memory categoryAmount of memoryDeviation memory214resource 228 (Mb)resources 226 (Mb)Data size20480 (no change)X1Disk I/O19741.63X2RAM51207.72X3Cache36026.86X4JVMCache + 1024375.27X5

In the example of Table 3, the first deviation memory resource226is 0. This means that for the data size memory category214, it is determined that no changes to the data size are recommended. This is because if the size of the data is increased beyond the recommended and standard size collections, additional load will be added to the cluster of containers130and make the pool of containers130out of memory resources for any task execution. Thus, in the optimal memory resource configuration184, the deviation memory resource226for the data size is 0.

In the example of Table 3, the second deviation memory resource226is 41.63 Mb for the disk I/O memory category214. This means that in the optimal memory resource configuration184, the memory resources to be allocated for the disk I/O is 197 Mb+41.63 Mb=238.63 Mb.

In the example of Table 3, the third deviation memory resource226is 7.72 Mb for the RAM memory category214. This means that in the optimal memory resource configuration184, the memory resources to be allocated for the RAM is 5120 Mb+7.72 Mb=5127.72 Mb.

In the example of Table 3, the fourth deviation memory resource226is 26.86 Mb for the Cache memory category214. This means that in the optimal memory resource configuration184, the memory resources to be allocated for the Cache memory is 360 Mb+26.86 Mb=386.86 Mb.

In the example of Table 3, the fifth deviation memory resource226is 375.27 Mb for the JVM memory category214. This means that in the optimal memory resource configuration184, the memory resources to be allocated for the JVM memory is Cache+1024 Mb+375.27 Mb. This value may vary depending on the Cache memory.

The total memory resources230for the optimal memory resource configuration184may be 2,727,522 Mb. The total memory resources230may be the optimal task utilization pool capacity for a container130memory resource configuration. The cluster of containers130may utilize this memory capacity at maximum throttle in an efficient manner for any automated task112.

The memory resource configuration engine182may configure each container130according to the determined memory resource configuration184. The memory resource configuration engine182may allocate the containers130configured according to the determined memory resource configuration184to execute future tasks112.

In certain embodiments, the memory resource configuration engine182may determine a set of execution times218taken to execute the historical tasks112, where each execution time218is a time period taken to execute each respective historical task112. Thus, determining the memory resource configuration184may be based on the execution time218for each historical task112, the historical memory resources210, and the total historical memory resources216.

In certain embodiments, determining the memory resource configuration184may include determining the total memory resources to be allocated to each container130, e.g., 2,727,522 Mb as described above. Thus, the memory resource configuration engine182may allocate, from the total memory resources, a set of particular memory resources that includes the memory resource228plus the deviation memory resources226to a respective memory category214, as described above with respect to Table 3.

In certain embodiments, determining the memory resource configuration184may include solving at least four linear equations224in which the particular amount of memory resources (i.e., the deviation memory resources226) and the total memory resources230are unknown parameters, and the historical memory resources210, total historical memory resources216, and the execution time218for each memory category214are known parameters.

In certain embodiments, the at least four linear equations224(e.g., equations (3) to (6) indicated as constraints above) are solved such that the amount of the total memory resources230is maximized. In certain embodiments, the at least four linear equations224(e.g., equations (3) to (6) indicated as constraints above) are solved such that the deviation memory resources226are maximized.

Once the containers130are configured according to the operational flow200, the tasks112are fed to the containers130for processing, parsing, generating task logs104, detecting confidential information180, and obfuscating the confidential information180.

Example Operational Flow for Detecting and Obfuscating Confidential Information in Task Logs

FIG.3illustrates an example operational flow300of system100for detecting and obfuscating confidential information in task logs104. Assume that the containers130are configured according to the operational flow200described inFIG.2, the tasks112are fed to the containers130for processing, parsing, generating task logs104, detecting confidential information180, and obfuscating the confidential information180.

In processing the task logs104, the obfuscation engine188is implemented to detect and obfuscate the confidential information180. As described inFIG.1, the task logs104may be in various data formats, data structure, and compressed by various compression techniques. For example, a task log104may be in a structured data format, and another task log104may be in an unstructured data format. The task logs104may be compressed by any compression technique. Various compression technique may cause the text in the task logs104to be different than its original state (i.e., uncompressed state). The obfuscation engine188(e.g., via the machine learning module190) is configured to detect confidential information180in task logs104that are in various data formats, data structure, and compressed by various compression techniques.

In certain embodiments, the machine learning module190may implement a density-based clustering algorithm as below. The server170may receive the tasks112from the third-party vendors122(via the computing devices120). The server170may send the tasks112to the containers130for processing in a distributed computing network106. The containers130process the tasks112and generate task logs104, similar to that described inFIGS.1and2. During this process, the task logs104are fed to the obfuscation engine188to detect and obfuscate the confidential information180. The obfuscation engine188may implement the machine learning module190to perform its operations. The machine learning module190is trained using the training dataset194, similar to that described inFIG.1.

Determining Patterns in the Task Log

The machine learning module190parses the task logs104and detects patterns192of text that are and/or associated with confidential information180. An example pattern192of text that are and/or associated with confidential information180is shown inFIG.3. The machine learning module190may extract features from the task log104, where the features are represented by a vector of numerical values.

In determining the pattern192, the machine learning module190may parse the task log112and tokenize each portion of the task log112. Each portion of the task log112may be a word, a number, a symbol, and/or any combination thereof. In certain embodiments, a portion of the task log112may be a line or a portion of a line.

The machine learning module190may examine or analyze each portion of the task log112. For example, the machine learning module190may determine a port320, a database name322, and a hostname324in a portion (e.g., a line) of a task log112. The machine learning module190may determine that the port320, database name322, and hostname324are related based on the training dataset194that indicates as such. The machine learning module190may also determine that the port320, database name322, and hostname324are confidential information180based on the training dataset194that indicates as such. The machine learning module190may detect a database management system (DBMS) name326in the portion of the task log112. The machine learning module190may determine that the DBMS326is related to or associated with the port320, database name322, and hostname324based on the training dataset194that indicates as such. The machine learning module190may also determine that the DBMS326is confidential information180based on the training dataset194that indicates as such. The machine learning module190may detect a user328in the portion of the task log112. The machine learning module190may detect that the port320, database name322, hostname324, and DBMS326are associated with the user328based on the training dataset194that indicates as such. The machine learning module190may determine that only the user328is authorized to access and view the port320, database name322, hostname324, and DBMS326because they are associated with the user328. Thus, the obfuscation engine188may obfuscate the port320, database name322, hostname324, and DBMS326, and only allow the user328to access this information (e.g., making it available to the user328by decrypting it upon request) and prevent other users328from accessing this information.

The machine learning module190may determine that the port320, database name322, hostname324, DBMS326, and user328form the pattern192. The machine learning module190generates the pattern192by associating or linking these parameters with each other. The pattern192may be a hierarchical pattern and relationship between the port320, database name322, hostname324, DBMS326, and user328. In this manner, the machine learning module190generates a pattern192that is associated with or includes confidential information180. The machine learning module190may use this example pattern192as a template to identify and detect similar patterns192(that include port320, database name322, hostname324, DBMS326, and user328) in the task logs112. Similarly, the machine learning module190may determine and generate other patterns192that include other portions of the task log112.

The machine learning module190determines whether a word332is among keywords334that are known to be confidential information180(as indicated in the training dataset194) as described below. The obfuscation engine188may perform these operations for each word332in each portion of the task log104. The obfuscation engine188compares the word323with each of set of plurality of keywords334that are known to be among the confidential information180. In this process, the obfuscation engine188extracts a first set of features336from the word323. The first set of features336indicates an identity of the word332(e.g., what word is it), among other attributes, including its meaning, where it is used, where it is used in other portions of the task log104, where it is used in other task logs104, etc. The first set of features336is represented by a first vector338of numerical values. The obfuscation engine188extracts a second set of features340from each keyword334. For example, assume that the obfuscation engine188extracts the second set of features340from a first keyword334. The second set of features340indicates an identity of the first keyword334(e.g., what word is it), among other attributes, including its meaning, where it is used, where it is used in other portions of the task log104, where it is used in other task logs104, etc. The second set of features340is represented by a second vector342. The obfuscation engine188compares the first vector338with the second vector342. In this process, the obfuscation engine188may perform a dot product between the first vector338and the second vector342. The obfuscation engine188may determine an Euclidian distance between the vectors338and342. Thus, the obfuscation engine188may determine the similarity between the word332and keyword334. If the obfuscation engine188determines that the Euclidian distance between vectors338and342is less than a threshold percentage (e.g., less than 1%), it determines that the word332matches or corresponds to the keyword334. Otherwise, it determines that they are different. In certain embodiments, the obfuscation engine188may determine a percentage of numerical values in the first vector338that correspond or match with counterpart numerical values in the second vector342. The obfuscation engine188compares the percentage of the numerical values in the first vector338that correspond or match counterpart numerical values in the second vector342with a threshold percentage (e.g., 90%, 95%, etc.). If the percentage of the numerical values in the first vector338that correspond or match counterpart numerical values in the second vector342exceeds the threshold percentage, the obfuscation engine188determines that the word332matches or corresponds to the keyword334(or the word332is among the keywords334). Otherwise, the obfuscation engine188determines that the word332does not match or correspond to the keyword334. If it is determined that the word332matches the keyword334, the obfuscation engine188determines that the word332is among the confidential information180. Otherwise, the obfuscation engine188determines that the word332is not among the confidential information180.

Clustering Confidential Information

The obfuscation engine188may perform the clustering operation330to cluster the confidential information180. The clustering operation330may include one or more operations described below.

An example plot310illustrating example clustering of datapoints312of confidential information180and outliers314is shown inFIG.3. The machine learning module190may represent each portion of a task log104by coordinates (x, y) in the plot310. For example, each work or number in a task log104may be represented by coordinates in the plot310. Each coordinate may be a datapoint312.

The machine learning module190may implement a density-based clustering algorithm that includes several parameters including Epsilon (Eps) and minPts (minimum points) to cluster the confidential information180. The parameter Eps defines the radius of the neighborhood around a point x, where x is a datapoint312with coordinates of confidential information180in the two-dimension plot310. The parameter MinPts is the minimum number of neighbors within the Eps radius. The parameter MinPts may be number of dimensions in the task log104, as MinPts>=dimension+1. The parameter MinPts may be 2*dimension, if MinPts of at least 3 is used to train the machine learning module190and determine the hierarchical clustering of the confidential information180. This configuration may negate most (or all) of outliers314(e.g., noisy data in the clustering process).

The machine learning module190may cluster the datapoints312of the confidential information180as shown in the example plot310. In the example plot310, the cluster316includes the datapoints312of the confidential information180that may be associated with a particular pattern192. Further, in the example plot310, the outliers314are excluded from the cluster316because the outliers314are determined to not to be among the confidential information180. The corresponding description below described an example process of training the machine learning module190to perform the clustering operation330. The machine learning module190may be trained by computing the equation (7) below.

Model=DBSCAN⁡(eps=0.3,min_samples=1000).fit⁡(x)Equation⁢(7)

The parameter “Model” is a name used to save the output of the machine learning module190. The parameter eps is the Epsilon parameter described above. The min_samples parameter indicates the minimum number of samples to process.

The machine learning module190may execute the equation (7) to perform the clustering operation330. The output of the equation (7) may be one or more clusters316of confidential information180. In certain embodiments, each cluster316may be associated with a different pattern192. In certain embodiments, each cluster316may be associated with a different type of confidential information180(e.g., port, database name, server name, etc.).

Determining the Probability Score of Confidential Information

In certain embodiments, the machine learning module190may determine the distance between spatial coordinates (i.e., the datapoints312) representing words and numbers in the task log104to determine whether they belong to the same cluster316. For example, machine learning module190may determine the distance between datapoints A and B as below.

Dist⁡(A,B)=1-Score(Ai,Bi)/Max⁡(len⁡(A),len⁡(B))⁢Where,i=1⁢to⁢Min⁡(len⁡(A)⁢len⁡(B))⁢Score(a,b)=k⁢1⁢if⁢a=b⁢Score(a,b)=0⁢otherwiseEquation⁢(8)

Where the Dist is a distance function to calculate the distance between datapoints A and B (e.g., two datapoints312). The score parameter is a probability score196that represents a probability of a word, text, number, and/or a symbol in the task log104is confidential information180. The Max(len(A), len(B)) function calculates which of len(A) or len(B) is the maximum. The Min(len(A),len(B)) function calculates which of len(A) or len(B) is the minimum.

In certain embodiments, the parameter K1 may be 1 by default while implementing log distance function to add or reduce weight on the pattern matching fields between two portions of a task log104. If a=b, the output of the Score(a,b) is K1. Otherwise, the Score(a,b) is zero. For example, if the probability score196associated with a portion of the task log104is 90%, it is determined that there is 90% probability that the portion of the task log104is among the confidential information180.

In certain embodiments, the machine learning module190may determine a probability score196for each portion of the task log104. For example, the machine learning module190may select a portion of the task log104(e.g., a word, a number, a symbol). The machine learning module190may determine a probability score196for the portion of the task log104. In this process, the machine learning module190may extract features of the portion of the task log104and compare them with features of previously known confidential information180in the training dataset194. The extracted features may be represented by a vector of numerical values.

The machine learning module190determines a percentage (e.g., more than 90%) of the features of the portion of the task log104corresponds or matches with counterpart features of previously known confidential information180in the training dataset194. The determined percentage of the features of the portion of the task file104that match the counterpart features of the previously known confidential information180may be the probability score196. An example portion of a task log is illustrated in Table 4.

TABLE 4Example portion of a task log 104.2021-06-12 19:10 INFO URL jdbc:hive2://dyys-hive-new-wyxxbt1.bank.com:10000/;ssl=true; default_db;2021-06-12 19:15 INFO Server connect tolxxyyyzza2.com

For example, assume that this example portion of the task log104is given to the machine learning module190. The machine learning module190may segregate this portion to produce an output as shown in Table 5.

TABLE 5Example output of the machine learning module 190.Jun. 12, 202119:10INFOjdbc:hive2://dyys-hive-new-10000sslTrueDefault_db;wyxxbt1.bank.comJun. 12, 202119:15INFOServerlxxyyyzza2.comconnect to

The machine learning module190may determine the patterns192in this portion of the task log104by determining the relationships of segregated parts. The machine learning module190may predict the identity of each segregated part as shown in Table 6.

TABLE 6Example output of the machine learning module 190 predicting theidentity of each part of the task log portion in Table 5.datetimeINFOjdbc-stringHostnamePortboolDatabasedatetimeINFOmessageHostname

In the example Table 6, it is determined that the first column is data, second column is time, third column is info (i.e., information about the current line, in other examples it can be warning, error, etc.), fourth column is the name of a server (in the first row), and message (indicating that the server is connecting to another device), fifth column is hostname, sixth column is port number, seventh column is indicating if the secure socket layer (SSL) protocol is enabled or not, and the eighth column is the database name. The machine learning module190may determine whether each part (column) is or associated with confidential information180based on training dataset194. In this manner, the machine learning module190detects the patterns192, confidential information180and cluster the confidential information180.

Once the confidential information180in the task log104is determined, the obfuscation engine188obfuscates the confidential information180. In this process, the obfuscation engine188may use an encryption function, such as SHA, MD5, and/or any random number generator.

Example Operational Flow for Updating a Distributed Ledger Based on Anomalies Detected in Blockchain Transactions

FIG.4illustrates an example operational flow400of system100(seeFIG.1) for updating a distributed ledger160based on anomalies410detected in blockchain transactions162. In certain embodiments, the operational flow400may begin when a private blockchain140is created. For example, the server170may initiate creating the private blockchain140. The server170may create the private blockchain140when a genesis (i.e., the first) block142ais created and a blockchain ledger160is generated to maintain records of blockchain transactions162.

In certain embodiments, various blockchain ledgers160may be assigned to corresponding vendors122. For example, each of the various blockchain ledgers160may be associated with a particular application102that is associated with a particular vendor122.

Conducting Blockchain Transactions

In certain embodiment, each operation of the operational flow200ofFIG.2and operational flow300ofFIG.3may be stored and considered as a blockchain transaction162. For example, determining the optimal memory resource configuration184described in operational flow200inFIG.2may be considered as a blockchain transaction162. In another example, configuring the containers130with the optimal memory resource configuration184described in operational flow200inFIG.2may be considered as another blockchain transaction162. In another example, training the machine learning module190with the training dataset194to detect the confidential information180described in operational flow300inFIG.3may be considered as another blockchain transaction162. In another example, detecting the patterns192described in operational flow300inFIG.3may be considered as another blockchain transaction162. In another example, clustering the confidential information180described in operational flow300inFIG.3may be considered as another blockchain transaction162. In another example, obtaining the results (e.g., the confidential information180) from the machine learning module190described in operational flow300inFIG.3may be considered as another blockchain transaction162.

The anomaly detection engine198may track the progress of each blockchain transaction162. Each blockchain transaction162may be associated with a particular vendor122because each blockchain transaction162may be associated with a particular task log104and respective application102. The anomaly detection engine198may detect groups of blockchain transactions162that are related to various third-party vendors122.

The anomaly detection engine198may generate various blockchain ledgers160for different groups of blockchain transactions162that are associated with a different third-party vendors122. For example, the anomaly detection engine198may determine that the first group of blockchain transactions162aare related to the vendor122a. In response, the anomaly detection engine198may store the blockchain transactions162ain the blockchain ledger160a. The anomaly detection engine198may communicate the blockchain transactions162ato the vendor122a.

Similarly, the anomaly detection engine198may detect that the second group of blockchain transactions162nare associated with the vendor122n. In response, the anomaly detection engine198may store the blockchain transactions162nin the blockchain ledger160n. The anomaly detection engine198may communicate the blockchain transactions162nto the vendor122n. In this manner, each vendor122may receive a corresponding ledger160that it is supposed to receive. Thus, other vendors122may not receive data that they are not supposed to receive (e.g., task logs104that include confidential information180not related to those vendor122). Each vendor122may receive its respective task logs104with obfuscated confidential information180(e.g., included in the respective blockchain ledger160).

Creating Hash Values for Blocks in the Blockchain

The server170and/or the blocks142may generate hash values154for the blocks142. The hash value154may be an alphanumerical value uniquely identifying the respective block142. For example, the server170, the generis block142a, and/or any other block142may generate the hash value154for the generis block142a.

Each blockchain transaction162may be stored in a different block142. For example, determining the optimal memory resource configuration184described in operational flow200inFIG.2(that is considered as blockchain transaction162a) may be stored in a first block142a. In another example, obtaining the results (e.g., the confidential information180) from the machine learning module190(that is considered as blockchain transactions162n) described in operational flow300inFIG.3may be stored in block142n. Block142nmay be one of the blocks142of the blockchain network140described inFIG.1. In certain embodiments, the blockchain ledgers160a-nmay be passed or distributed to all blocks142.

Evaluating Blockchain Transactions for Detecting Anomalies

The anomaly detection engine198may evaluate each blockchain transaction162for determining whether it includes an anomaly410. For example, anomalies410may include 1) the memory resource configuration184does not lead to a more optimized memory utilization that is more than the threshold memory resource utilization222(seeFIG.2), 2) the machine learning module190prediction of confidential information180is not accurate (e.g., when training the machine learning module190with the training dataset194and determining that the machine learning module190has not predicted confidential information180correctly in historical task logs104in which the confidential information180is labeled), 3) the machine learning module190has predicated that a task log104does not include any confidential information180(where the task log104is known to include confidential information180), 4) a task log104has not been evaluated or skipped due to an error, and/or other anomalies410. If the anomaly detection engine198detects an anomaly410in a blockchain transaction162, the blockchain transaction162is rejected.

In certain embodiments, a user may review the blockchain transactions162and indicate if it includes an anomaly410. The anomaly detection engine198may learn from the behavior of the user to predict whether a future blockchain transaction162includes an anomaly410, in supervised machine learning.

If it is determined that the blockchain transaction162does not include anomalies410, the blockchain transaction162is accepted. In certain embodiments, in response, the anomaly detection engine198may update the corresponding blockchain ledger160to include information that indicates the blockchain transaction162does not include an anomaly410. In certain embodiments, the corresponding blockchain ledger160may further be updated to include any addition, modification, deletion, obfuscation that occurred in the respective log files104.

If it is determined that the blockchain transaction162includes an anomaly410, the blockchain transaction162is rejected. In certain embodiments, in response, the anomaly detection engine198may remove the blockchain transaction162from its respective blockchain ledger160. In certain embodiments, in response, the anomaly detection engine198may update the corresponding blockchain ledger160to include information that indicates the blockchain transaction162includes an anomaly410.

In certain embodiments, it is determined that a blockchain transaction162failed in case of privacy violation by another vendor122. For example, if it is determined that a vendor122has received a task log104associated with another vendor122(due to an error), and has accessed confidential information180included in the task log104, the anomaly detection engine198determines that the corresponding blockchain transaction162is failed due to privacy violation by the other vendor122.

In certain embodiments, it is determined that a blockchain transaction162includes an anomaly410, a message may be communicated to authorities and respective users to initiate an investigation of the detected anomaly410.

Example Method for Determining a Memory Resource Configuration for Containers to Execute Tasks

FIG.5illustrates an example flowchart of a method500for determining a memory resource configuration184for containers130to execute tasks112. Modifications, additions, or omissions may be made to method500. Method500may include more, fewer, or other operations. For example, operations may be performed in parallel or in any suitable order. While at times discussed as the system100, blockchain network140, server170, or components of any of thereof performing operations, any suitable system or components of the system may perform one or more operations of the method500. For example, one or more operations of method500may be implemented, at least in part, in the form of software instructions178ofFIG.1, stored on non-transitory, tangible, machine-readable media (e.g., memory176ofFIG.1) that when run by one or more processors (e.g., processor172ofFIG.1) may cause the one or more processors to perform operations502-520.

At502, the memory resource configuration engine182accesses a plurality of historical tasks112. For example, the historical tasks112may be received when the applications102are executed, similar to that described inFIG.2.

At504, the memory resource configuration engine182may determine historical memory resources210used to execute each historical task112, where the historical memory resources210are associated with memory categories214. For example, the memory resource configuration engine182may determine the historical memory resources210for data size, disk I/O, RAM, Cache, and JVM memory categories214, similar to that described inFIG.2.

At506, the memory resource configuration engine182determines total historical memory resources216allocated for each memory category214. For example, the memory resource configuration engine182may determine the historical memory resources216allocated for each memory category214across the tasks112, similar to that described inFIG.2.

At508, the memory resource configuration engine182determines time taken (i.e., execution time218) to execute each historical task112. For example, the memory resource configuration engine182may determine the execution time218based on the start time and end time of each historical task112.

At510, the memory resource configuration engine182determines memory resource utilization212for each historical task112. For example, the memory resource configuration engine182may determine the memory resource utilization212for each historical task112based on analyzing how the memory resources210were used in executing a respective historical task112, similar to that described inFIG.2. The memory resource configuration engine182may determine the memory resource utilization212based on analyzing the memory resource210usage and efficiency.

At512, the memory resource configuration engine182determines whether the memory resource utilization212is less than the threshold memory resource utilization222. If it is determined that the memory resource utilization212is less than the threshold memory resource utilization222, method500proceeds to516. Otherwise, method500proceeds to514.

At514, the memory resource configuration engine182configures each network node130according to a default memory resource configuration220.

At516, the memory resource configuration engine182determines a memory resource configuration184to be used for configuring a set of network nodes130. For example, the memory resource configuration engine182may determine the memory resource configuration184by determining the deviation memory resources226, memory resources228, and total memory resources230, similar to that described inFIG.2.

At518, the memory resource configuration engine182configures each network node130according to the determined memory resource configuration184. In response, the network nodes130may be used to process future tasks112.

Example Method for Obfuscating Confidential Information in Task Logs

FIG.6illustrates an example flowchart of a method600for obfuscating confidential information180in task logs104. Modifications, additions, or omissions may be made to method600. Method600may include more, fewer, or other operations. For example, operations may be performed in parallel or in any suitable order. While at times discussed as the system100, blockchain network140, server170, or components of any of thereof performing operations, any suitable system or components of the system may perform one or more operations of the method600. For example, one or more operations of method600may be implemented, at least in part, in the form of software instructions178ofFIG.1, stored on non-transitory, tangible, machine-readable media (e.g., memory176ofFIG.1) that when run by one or more processors (e.g., processor172ofFIG.1) may cause the one or more processors to perform operations602-618.

At602, the obfuscation engine188accesses a task log104. For example, the obfuscation engine188may access the task log104of the plurality of task logs104processed by the network nodes130, similar to that described inFIGS.1and3.

At604, the obfuscation engine188selects a portion of the task log104. The portion of the task log104may be a line of text, a paragraph, or a number of lines (e.g., one, two, three, etc. lines.) The obfuscation engine188may iteratively select a portion of the task log104until no portion is left for evaluation. The obfuscation engine188may perform the operations below for each portion of the task log104.

At606, the obfuscation engine188selects a word332in the portion of the task log104. The obfuscation engine188may iteratively select a word in the portion of the task log104until no word is left for evaluation. The obfuscation engine188may perform the operations below for each word in the portion. The word332may be any of the port320, database name322, hostname324, DBMS326, and user328, described inFIG.3.

At608, the obfuscation engine188determines whether the word332is among the confidential information180. In this process, the obfuscation engine188tokenizes each word323in the portion of the task log104. The obfuscation engine188may feed the task log104to the machine learning module190to perform the operations described herein, similar to that described inFIG.3. If the obfuscation engine188determines that the word332is among the confidential information180, method600proceeds to610. Otherwise, method600returns to606.

At610, the obfuscation engine188determines a hierarchical relationship between words332in the portion of the task log104. For example, the obfuscation engine188determines the hierarchical relationship between the word332and neighboring words in the portion of the task log104. The hierarchical relationship between the words332indicates whether or not the word332is associated with each of the neighboring words332. Assume that the obfuscation engine188may determine that the hierarchical relationship between the word332and neighboring words332indicates that the word332is associated with at least a third word332in the portion of the task log104.

At612, the obfuscation engine188generates a template pattern192comprising the word332and neighboring words332that are associated with the word332. In this process, the obfuscation engine188may compare the template pattern192with information stored in the training dataset194, similar to that described inFIG.3. The template pattern192may include the word332and the at least third word332in the portion. In certain embodiments, the at least third word332may be included in the template pattern192even if the at least third word332is not among the keywords334.

At614, the obfuscation engine188obfuscates the template pattern192, e.g., by encrypting the word332and the at least third word332using an encryption technique or function.

At616, the obfuscation engine188determines whether to select another word332in the portion. The obfuscation engine188determines to select another word332if at least one word332is left for evaluation in the portion. In this case, method600returns to606. Otherwise, method600proceeds to618.

At618, the obfuscation engine188determines whether to select another portion in the task log104. The obfuscation engine188determines to select another portion if at least one portion is left for evaluation in the task log104. In this case, method600returns to604. Otherwise, method600ends.

In certain embodiments, the obfuscation engine188may use the template pattern192to identify other instances of the template pattern192in the task logs104. The obfuscation engine188may obfuscate the identified instances of the template patterns192. The obfuscation engine188communicates the task logs104that include obfuscated instances of the template patterns192to appropriate third party vendors122.

In certain embodiments, each of the plurality of task logs104has a different data format compared to other task logs104. In certain embodiments, the different data format comprises structured data format, semi-structured data format, and unstructured data format. In certain embodiments, each task log104is compressed with a different compression technique compared to other task logs104.

In certain embodiments, the obfuscation engine188may identify a fourth word332in the portion of the task log104. The fourth word332may be a name of a user328who is authorized to access the word332and the at least third word332. The user328may be a third party vendor122, described inFIG.1. After obfuscating the word332and the at least third word332, the obfuscation engine188may transmit the obfuscated word332and the at least third word332to a computing device120associated with the user328. The obfuscation engine188may receive a request to decrypt the obfuscated word332and the at least third word332from the computing device120. The obfuscation engine188may decrypt the obfuscated word332and the at least third word332and transmit it to the computing device120.

Example Method for Updating a Blockchain Ledger Based on Detected Anomalies in Blockchain Transactions

FIG.7illustrates an example flowchart of a method700for updating a blockchain ledger160based on detected anomalies410in blockchain transactions162. Modifications, additions, or omissions may be made to method700. Method700may include more, fewer, or other operations. For example, operations may be performed in parallel or in any suitable order. While at times discussed as the system100, blockchain network140, server170, or components of any of thereof performing operations, any suitable system or components of the system may perform one or more operations of the method700. For example, one or more operations of method700may be implemented, at least in part, in the form of software instructions178ofFIG.1, stored on non-transitory, tangible, machine-readable media (e.g., memory176ofFIG.1) that when run by one or more processors (e.g., processor172ofFIG.1) may cause the one or more processors to perform operations702-712.

At702, the anomaly detection engine198accesses a blockchain network140. In certain embodiments, the server170may generate the blockchain network140, similar to that described inFIG.4.

At704, a blockchain transaction162is conducted on a task log104in the blockchain network140. For example, the blockchain transaction162may be conducted by the server140, anomaly detection engine198, and/or one or more blocks142. The blockchain transaction162may be one of the blockchain transactions162ato162ndescribed inFIG.4. The blockchain transaction162may be associated with obfuscating the confidential information180. The blockchain transaction162may be configuring one or more network nodes130with a particular memory resource configuration184. The blockchain transaction162may be obfuscating the confidential information180. Other examples of the blockchain transaction162are described inFIG.3.

At706, the blockchain transaction162is stored in a blockchain ledger160. For example, the blockchain transaction162may be stored in the blockchain ledger160by the server140, anomaly detection engine198, and/or one or more blocks142.

At708, the anomaly detection engine198determines whether the blockchain transaction162is associated with an anomaly410. The anomaly410may indicate that the result of the blockchain transaction162is unexpected. Examples of the anomaly410are described inFIG.3. The anomaly410may indicate that the memory resource configuration184does not lead to a more optimized memory resource utilization212that is more than a threshold memory utilization percentage222for the network nodes130when they are used to obfuscate the confidential information180. The anomaly410may indicate that a prediction result of a machine learning module190for detecting the confidential information180is less than a threshold prediction percentage (e.g., less than 90%). If it is determined that the blockchain transaction162is associated with the anomaly410, method700proceeds to710. Otherwise, method700proceeds to712.

At710, the anomaly detection engine198removes the blockchain transaction162from the blockchain ledger160. In other embodiments, one or more blocks142may remove the blockchain transaction162from the blockchain ledger160.

At712, the anomaly detection engine198updates the blockchain ledger160by indicating (e.g., in a message) that the blockchain transaction162is not associated with the anomaly410. If an anomaly410is detected, the system100(e.g., via the anomaly detection engine198) may trigger an investigation of the blockchain transaction by authorities. Thus, the blockchain transaction with an anomaly410is escalated and addressed, and not remained undetected.

In certain embodiments, the server170and/or the blocks142identify a vendor122that is associated with the blockchain transaction162and the blockchain ledger160. The server170and/or the blocks142generates a block142and store the blockchain ledger160in the block142. The server170and/or the blocks142distribute the blockchain ledger160to other blocks142. The server170and/or the blocks142update the status of the blockchain ledger160based on whether the anomaly410is detected in the blockchain transaction162.

While several embodiments have been provided in the present disclosure, it should be understood that the disclosed systems and methods might be embodied in many other specific forms without departing from the spirit or scope of the present disclosure. The present examples are to be considered as illustrative and not restrictive, and the intention is not to be limited to the details given herein. For example, the various elements or components may be combined or integrated with another system or certain features may be omitted, or not implemented.

In addition, techniques, systems, subsystems, and methods described and illustrated in the various embodiments as discrete or separate may be combined or integrated with other systems, modules, techniques, or methods without departing from the scope of the present disclosure. Other items shown or discussed as coupled or directly coupled or communicating with each other may be indirectly coupled or communicating through some interface, device, or intermediate component whether electrically, mechanically, or otherwise. Other examples of changes, substitutions, and alterations are ascertainable by one skilled in the art and could be made without departing from the spirit and scope disclosed herein.

To aid the Patent Office, and any readers of any patent issued on this application in interpreting the claims appended hereto, applicants note that they do not intend any of the appended claims to invoke 35 U.S.C. § 112(f) as it exists on the date of filing hereof unless the words “means for” or “step for” are explicitly used in the particular claim.