Abstract:
An SSH protocol-based session parsing method and system. The method includes: step  101 ) receiving an SSH protocol-based session data packet and respectively preprocessing the received data packet as follows: when one data packet contains several complete messages, extracting each complete message from the data packet; when the data packet contains an incomplete message, firstly caching the incomplete message contained in the current data packet, then receiving the subsequent data packet, and finally splicing the message contained in the subsequent data packet with content of the cached message until a complete message is spliced; and step  102 ) extracting a content associated with log generation from each obtained complete message, packaging the content associated with the log according to a set format, and then obtaining a parsed log and finishing the session parsing.

Description:
CROSS REFERENCE TO RELATED APPLICATIONS 
       [0001]    This application is the national phase entry of International Application No. PCT/CN2015/074087, filed on Mar. 12, 2015, which is based upon and claims priority to Chinese Patent Application No. 201410691155.4 filed on Nov. 25, 2014, the entire contents of which are incorporated herein by reference. 
     
    
     TECHNICAL FIELD 
       [0002]    Technical solutions of the present invention address the issue of task scheduling in a distributed environment taking into account a plurality of performance indexes including the remaining resources, load balancing, etc., are suitable for task scheduling in the distributed environment, and particularly relate to a distributed node intra-group task scheduling method and system. 
       BACKGROUND OF THE INVENTION 
       [0003]    At present, in many distributed computing environments. e.g., cloud computing, grid computing, a P2P network, etc., a request task needs to be scheduled and executed among a plurality of selectable nodes. Existing types of scheduling are based upon multi-task scheduling, workflow scheduling, etc. 
         [0004]    Multi-task scheduling refers to that a number of concurrent tasks arrive at a cloud distributed computing environment together, where they need to be scheduled, and resources need to be allocated for them, thus requirement of load balancing of respective execution nodes can be met. Min-Min algorithm and Max-Min algorithm are commonly applied. In the Min-Min algorithm, firstly the minimum completion time for each task in the current task queue on respective processors is predicated, and then the task with the minimum completion time is allocated to a corresponding processor, and also a ready time of the corresponding processor is updated, and the allocated task is removed from the task queue; and this process is repeated to allocate the remaining tasks until the entire task queue is empty. The Min-Min algorithm tends to suffer from load unbalancing. A difference of the Max-Min algorithm from the Min-Min algorithm lies in that after the earliest completion time for each task on the respective processors is determined, the task with the maximum earliest completion time is allocated to a corresponding processor, a ready time of the corresponding processor is updated in a timely manner, and this process is repeated for the remaining tasks. The Max-Min algorithm is improved over the Min-Min algorithm in terms of load balancing. 
         [0005]    Another type of scheduling relates to an algorithm of scheduling multiple levels of timing-related sub-tasks in some workflow so as to meet the requirements of shortening a total execution time and a total amount of consumed energy, etc. EDTS algorithm relates to optimum scheduling of N sub-tasks in a task, and in this algorithm, firstly the time and the amount of consumed energy for the respective sub-tasks executed on all the machines are predicated, and then a total length of time is set for this series of sub-tasks, and given the total length of time together with an existing timing relationship, a sub-task allocation manner which consumes energy as low as possible is found, where the task is decomposed and scheduled for optimum performance thereof. 
         [0006]    In most of the solutions above, the computing and resources allocation of scheduling scheme are performed by a central server, however, centralized computing may not be practicable in a large-scale distributed network, e.g., a large-scale computing cluster, a P2P network, or another distributed computing environment because it is difficult for the efficiency and the computing overhead thereof to meet the requirements of real-time, quasi-real-time, and other applications. 
       SUMMARY OF THE INVENTION 
       [0007]    The invention proposes a distributed scheduling method suitable for solving the condition of single task under a node group constrained by multiple resources and containing multiple service nodes. 
         [0008]    In order to attain the object above, the invention provides a distributed node intra-group task scheduling method, the method comprising: 
         [0009]    step  101 ) collecting resource state information and history service information about each node in a group; 
         [0010]    step  102 ) receiving a service request command, and parsing the request command to obtain a request task; and 
         [0011]    step  103 ) calculating a resource utilization rate increment after each node in the group loads a task and an occupation condition of each kind of resource in the group according to the resource state information and the history service information about each node; 
         [0012]    selecting service execution nodes in the group according to a principle of making the resource utilization rate increment of each node in the group as low as possible and the occupation of each kind of resource in the group as balanced as possible, and providing the request task by the selected service execution nodes. 
         [0013]    Optionally the step  101 ) above further comprises: 
         [0014]    step  101 - 1 ) maintaining, by each node in the group, the resource state information and the history service information of the present node; 
         [0015]    step  101 - 2 ) transmitting, by each node, the maintained resource state information and history service information to a scheduling execution node, wherein the scheduling execution node is configured to select an execution node in the group for a specific task, and to receive the service request command O of a user. 
         [0016]    Parsing the request command comprises: calculating and estimating according to the history service information of the task demand, and obtaining a task demanded resource vector O={o 1 , o 2 , . . . , o N } of the task for N kinds of resources; wherein the history service information comprises actual demand for resources and running time of each executed task. 
         [0017]    Optionally the step  103 ) above further comprises: 
         [0018]    step  103 - 1 ) calculating a resource occupation rate of each node in the group according to the resource state information of the node, wherein the resource state information of the node comprises an available resource vector of the node and an occupied resource vector of the node, by equations of: 
         [0000]        rc   j   ={c   1,j   ,c   2,j   , . . . ,c   N,j } 
         [0000]        c   i,j =( s   i,j   −so   i,j )/ s   i,j    
         [0000]    wherein rc j  represents the current resource occupation rate vector of any one node p j  in the group, and c i,j  represents an occupation rate of the i-th kind of resource for the node p j : 
         [0019]    step  103 - 2 ) calculating a resource occupation rate of each kind of resource of the node p j  over which the task O allocated to the present node is executed: 
         [0020]    step  103 - 3 ) calculating an occupation rate increment of each kind of resource of the node p j  after the task O is loaded onto the present node as compared with before it is loaded, that is, a difference between the resource occupation rate obtained in the step  103 - 2 ) and the resource occupation rate obtained in the step  103 - 1 ), and further obtaining a resource occupation rate increment vector of the task O for any one node p j  in the group; 
         [0021]    step  103 - 4 ) obtaining a resource occupation influence rate of the node p j  according to each element in the resource occupation rate increment vector; 
         [0022]    obtaining an intra-node resource balance degree σ j  according to a variance among the occupation rates of different types of resources of the node p j  by an equation of: 
         [0000]    
       
         
           
             
               σ 
               j 
             
             = 
             
               
                 
                   ∑ 
                   
                     i 
                     = 
                     1 
                   
                   N 
                 
                  
                 
                   
                     ( 
                     
                       
                         ro 
                         
                           i 
                           , 
                           j 
                         
                       
                       - 
                       
                         
                           rt 
                           j 
                         
                         _ 
                       
                     
                     ) 
                   
                   2 
                 
               
               N 
             
           
         
       
     
         [0000]    wherein 
         [0000]    
       
         
           
             
               
                 
                   rt 
                   j 
                 
                 _ 
               
                
               
                 
                   
                     ∑ 
                     
                       i 
                       = 
                       1 
                     
                     N 
                   
                    
                   
                     ro 
                     
                       i 
                       , 
                       j 
                     
                   
                 
                 N 
               
             
             , 
           
         
       
     
         [0000]    ro i,j  represents the occupation rate of the i-th type of resource over which the node p j  executes the task O, N represents the total number of types of resources supportable by all the nodes in the group of nodes, and i represents any one type of resource; 
         [0023]    step  103 - 5 ) determining nodes with lower resource balance degrees σ j  and lower resource occupation influence rates μ j  in the group of nodes as execution nodes to obtain an execution node set P K  by an equation of: 
         [0000]        P   K   ={p   k }=argmin(μ j ,σ j ) T ,ε[1, M′].  
 
         [0000]    wherein M′ represents the number of nodes in the eligible execution node set, and argmin represents a corresponding node which makes μ j  and σ j  get minimum values; and step  103 - 6 ) selecting a node in the execution node set P K  as a final execution node, and then forwarding the task O to the final execution node for processing. 
         [0024]    In order to perform the method above, the invention provides a distributed node intra-group task scheduling system, the system comprising: 
         [0025]    an information collecting module in each node in a group, used for collecting resource state information and history service information of the present node; 
         [0026]    a service command receiving module in a scheduling execution node in the group, used for receiving a service request command, and parsing the request command to obtain a request task; and 
         [0027]    a scheduling processing module in the scheduling execution node in the group, used for 
         [0028]    calculating a resource utilization rate increment after each node in the group loads a task and an occupation condition of each kind of resource in the group according to the resource state information and the history service information about each node: 
         [0029]    selecting service execution nodes according to a principle of making the resource utilization rate increment as low as possible and the occupation of each kind of resource as balanced as possible, and providing the request task by the service execution nodes. 
         [0030]    Optionally the information collecting module further comprises: 
         [0031]    an information obtaining sub-module, used for obtaining the resource state information and the history service information of the present node, wherein the resource state information comprises an available resource vector of the node and an occupied resource vector of the node; and 
         [0032]    a sending sub-module, used for transmitting the maintained resource state information and history service information to the scheduling execution node, wherein the scheduling execution node is configured to select an execution node in the group for a specific task and to receive the service request command O of a user. 
         [0033]    In summary, a group of nodes in the invention includes one or more service execution nodes which are serving entities capable of providing various types of services, wherein a node receiving a user service request O is a scheduling execution node which is a core in the group of nodes, and primarily responsible for deciding on service scheduling, computing, and related operations. Each node in the group of nodes needs to maintain resource state information and history service information of that node, and exchanges it with the scheduling execution node under some rule to prepare for a scheduling selection process. The scheduling execution node firstly calculates the resource utilization rate increment after each node in the group of nodes loads a task and the source occupation condition balance degree among various kinds of resources, as general evaluation criterions, according to the amount of resources demanded for the received task request. Thereafter scheduling nodes are selected, and the principle for selecting is to select an appropriate node as the task execution node in case that the resource utilization rate increment is as low as possible, and the occupation of different types of resources are as balanced as possible. The invention designs a distributed scheduling algorithm (where each node can take a role of the task execution node, without being globally planned) oriented to a single task, the algorithm assigns tasks reasonably under a condition of multiple resources constrain and multiple service nodes, thereby guarantees the efficiency of execution taking into account load balancing, the utilization rate of resources, and other factors so as to improve the overall service performance of the system. 
         [0034]    The invention is technically advantageous over the prior art in that: 
         [0035]    the algorithm in the technical solutions according to the invention is designed for a single task, assigns tasks reasonably under a condition of multiple resources constrain and multiple service nodes, thereby guarantees the efficiency of execution taking into account load balancing, the utilization rate of resources, and other factors so as to improve the overall service performance of the system. 
     
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         [0036]      FIG. 1  is a schematic diagram of searching for a scheduling node according to the invention; 
           [0037]      FIG. 2  is a flow chart of a method for scheduling a task according to the invention. 
       
    
    
     DETAILED DESCRIPTION OF THE INVENTION 
       [0038]    The technical solutions according to the invention will be described below in details with reference to the drawings. 
       Example 1 
       [0039]    The invention provides a distributed node intra-group task scheduling method as illustrated in  FIG. 2 : 
         [0040]    The invention proposes a method of distributed scheduling of a single task with multiple resource constraints among a group of nodes including a plurality of serving nodes. A group of nodes include one or more service execution nodes which are serving entities capable of providing various types of services, wherein a node receiving a user service request O is a scheduling execution node which is a core in the group of nodes, and primarily responsible for deciding on service scheduling, computing, and related operations. Each node in the group of nodes needs to maintain resource state information and history service information of that node, and exchanges it with the scheduling execution node under some rule to prepare for a scheduling selection process. The scheduling execution node firstly calculates the resource utilization rate increment after each node in the group of nodes loads the task and the source occupation condition balance degree among various kinds of resources, as general evaluation criterions, according to the amount of resources demanded for the received task request. Thereafter scheduling nodes are selected, and the principle for selecting is to select an appropriate node as the task execution node in case that the resource utilization rate increment is as low as possible, and the occupation of the different types of resources are as balanced as possible. 
         [0041]    In the distributed node intra-group task scheduling method, the execution nodes are selected in the following steps: 
         [0042]    step 1) it is assumed that a group of nodes P is P={p 1 , p 2 , . . . , p j , . . . , p M }, the total amount of available resources for any one node p j  in the group of nodes will be represented as p j ={s 1,j , s 2,j , . . . , s n,j , . . . s N,j }, and occupied resources of the node p j  will be represented as po j ={so 1,j , so 2,j , . . . , so N,j }. 
         [0043]    Here N represents the total number of types of resources supportable by all the nodes in a group of nodes, and s n,j  represents an upper limit of the n-th type of resource on the node p j  when the node p j  keeps normal operation, the types of resources include but not limited to CPU, memory, and hard disk; and the upper limit of resource can be a system upper limit or a preset upper limit. 
         [0044]    A resource demand vector of the task corresponding to the service request command received by the scheduling execution node is represented as O={o 1 , o 2 , . . . , o N }, where o i  represents the total amount of the i-th type of resources in the group of nodes demanded for the task request; the resources demanded for the task request can be calculated and estimated through a system preset value or history information; and the types of demanded resources include but not limited to a number of CPUs, a size of memory, and a size of hard disk. 
         [0045]    step 2) the amount of remaining resources of the N different kinds of resources of the node p j  can be known from the total amount of available resources, and the amount of occupied resources of the node p j  on the N kinds of resources, and the current resource occupation rate vector can be calculated as rc j ={c 1,j , c 2,j , . . . , c N,j }, wherein, the occupation rate of the i-th kind of resource is c i,j =(s i,j −so i,j /s i,j , and if the node p j  processes the task O, the occupation rate of the resource will be ro j ={r 1,j , r 2,j , . . . , r N,j }, wherein r i,j =(s i,j −so i,j −o i )/s i,j , δ i,j =ro i,j −rc i,j , is defined to represent an increment of the occupation rate of the i-th kind of resource after the task O is loaded onto the node p j  as compared before it is loaded, so a resource occupation rate increment vector of the task O for the node p j  is Δ={δ 1,j , δ 2,j , . . . , δ i,j }, 
         [0000]    
       
         
           
             
               
                 μ 
                 j 
               
               = 
               
                 
                   
                     ∑ 
                     
                       i 
                       = 
                       1 
                     
                     N 
                   
                    
                   
                     δ 
                     
                       i 
                       , 
                       j 
                     
                   
                 
                 N 
               
             
             ; 
           
         
       
     
         [0000]    and an intra-node source balance degree refers to a variance among the resource occupation rate of the different types of resources, 
         [0000]    
       
         
           
             
               
                 σ 
                 j 
               
               = 
               
                 
                   
                     ∑ 
                     
                       i 
                       = 
                       1 
                     
                     N 
                   
                    
                   
                     
                       ( 
                       
                         
                           ro 
                           
                             i 
                             , 
                             j 
                           
                         
                         - 
                         
                           
                             rt 
                             j 
                           
                           _ 
                         
                       
                       ) 
                     
                     2 
                   
                 
                 N 
               
             
             , 
           
         
       
     
         [0000]    wherein 
         [0000]    
       
         
           
             
               
                 rt 
                 j 
               
               _ 
             
              
             
               
                 
                   
                     ∑ 
                     
                       i 
                       = 
                       1 
                     
                     N 
                   
                    
                   
                     ro 
                     
                       i 
                       , 
                       j 
                     
                   
                 
                 N 
               
               . 
             
           
         
       
     
         [0046]    step 3) if the node p j  processes the task O, and the occupation rate of the i-th kind of resource thereof is r i,j &gt;TR i , the node p j  will be removed from the group of nodes, thus resulting in a group of remaining nodes, P′={p′ 1 , p′ 2 , . . . , p′ j , . . . , P′ M′ }; and a resource occupation rate increment vector of the task O for a node p′ j , Δ and μ j , and the intra-node resource balance degree, σ j , will be calculated. 
         [0047]    step 4) our objective is to select such nodes that have good balance degree, i.e., small σ j , and small resource occupation influence, i.e., small μ j  as the execution nodes, so the optimization objective is to select an appropriate node set P K  that satisfies: 
         [0000]        P   K   ={p   k }=argmin(μ j ,σ j ) T   ,jε[ 1, M′] 
 
         [0048]    The problem above is a multi-objective optimization problem which can be solved as appropriate by a number of methods including a gradient descent method, a genetic algorithm, a particle swarm algorithm, and an ant colony algorithm, so that an eligible node set is obtained, and one of the nodes is selected as a final decision node, possibly in a number of selection strategies including but not limited to a strategy of selecting a node randomly, according to a certain probability distribution, or under a certain criterion. In the embodiment of the invention, an appropriate solution will be calculated (that is, the problem will be solved to get an execution node set P k ) by a method as follows (as illustrated in  FIG. 1 ): 
         [0049]    step a) a balance degree search step is defined as Δσ=(σ max −σ min )/M′, wherein o max  and σ min  are the maximum and minimum ones of the σ values corresponding to all the nodes in the group of nodes P′ respectively; and a resource occupation rate increment step is defined as Δμ=(μ max −μ min )/M′, wherein μ max  and μ min  are the maximum and minimum ones of the μ values corresponding to all the nodes in the group of nodes P′ respectively; 
         [0050]    step b) the search space with xε[μ min , μ max ] and yε[σ min , σ max ] is searched for optimum values, wherein initially x(0)=μ min , y(0)=σ min , and t=0; 
         [0051]    step c) if there is no node satisfying μ&lt;&lt;x(t) and ρ&lt;&lt;y(t), then x(t+1)=x(t)+Δμ, y(t+1)=y(t)+Δσ, and t=t+1, and this step c) is repeated; and if there is a node p k  satisfying the conditions, then the node will be added to an eligible node set p k  as a scheduling node; and the process above is terminated, and information about the node set is output. 
         [0052]    With the process above, the eligible node set p k  is obtained, and one of the nodes p k  is selected as a final decision node, possibly in a number of selection strategies including but not limited to a strategy of selecting a node randomly, according to a certain probability distribution, or under a certain criterion. 
         [0053]    step 5) the node p k  selected in the step 4) is the final execution node for the task O, and the scheduling node forwards the task O to the node p k  for processing. 
       Example 2 
       [0054]    The invention further provides a distributed node intra-group task scheduling system, the system comprises: 
         [0055]    an information collecting module in each node in the group, used for collecting resource state information and history service information of the present node; 
         [0056]    a service command receiving module in the scheduling node in the group, used for receiving a service request command, and parsing the request command to obtain a request task; and 
         [0057]    a scheduling processing module in the scheduling execution node in the group, used for: 
         [0058]    calculating a resource utilization rate increment after each node in the group loads a task and an occupation condition of each kind of resource in the group according to the resource state information and the history service information about each node: 
         [0059]    selecting service execution nodes according to a principle of making the resource utilization rate increment as low as possible and the occupation of each kind of resource as balanced as possible, and providing the request task by the service execution nodes. 
         [0060]    Optionally the information collecting module further comprises: 
         [0061]    an information obtaining sub-module, used for obtaining the resource state information and the history service information of the present node, wherein the resource state information comprises an available resource vector of the node and an occupied resource vector of the node; and 
         [0062]    a sending sub-module, used for transmitting the maintained resource state information and history service information to the scheduling execution node, wherein the scheduling execution node is configured to select an execution node in the group for a specific task, and to receive the service request command O of a user. 
         [0063]    Optionally the information obtaining sub-module is further configured to parse the request command, and to obtain a task demanded resource vector O={o 1 , o 2 , . . . o N } of the request task for N kinds of resources according to the task demand. 
         [0064]    Further optionally the scheduling processing module further comprises: 
         [0065]    a first processing sub-module, used for to calculating resource occupation rate of the nodes according to the resource state information of the nodes by equations of: 
         [0000]        rc   j   ={c   1,j   ,c   2,j   , . . . c   N,j } 
         [0000]        c   i,j =( s   i,j   −so   i,j )/ s   i,j    
         [0066]    wherein rc j  represents the current resource occupation rate vector of any one node p j  in the group, and c i,j  represents an occupation rate of the i-th kind of resource for the node p j ; 
         [0067]    a second processing sub-module, used for calculating the resource occupation rate for allocating the task O to the node p j  for processing: 
         [0068]    a third processing sub-module, used for calculating the occupation rate increment of the i-th kind of resource after the task O is loaded onto the node p j  as compared with before it is loaded, and further obtaining a resource occupation rate increment vector of the task O for any one node p j  in the group; 
         [0069]    a resource occupation influence rate obtaining sub-module, used for obtaining a resource occupation influence rate of the node p j  according to respective elements in the resource occupation rate increment vector; 
         [0070]    an intra-node resource balance degree obtaining sub-module, used for obtaining an intra-node resource balance degree σ j  according to a variance among the occupation rates of the different types of resources by an equation of: 
         [0000]    
       
         
           
             
               σ 
               j 
             
             = 
             
               
                 
                   ∑ 
                   
                     i 
                     = 
                     1 
                   
                   N 
                 
                  
                 
                   
                     ( 
                     
                       
                         ro 
                         
                           i 
                           , 
                           j 
                         
                       
                       - 
                       
                         
                           rt 
                           j 
                         
                         _ 
                       
                     
                     ) 
                   
                   2 
                 
               
               N 
             
           
         
       
     
         [0000]    wherein 
         [0000]    
       
         
           
             
               
                 
                   rt 
                   j 
                 
                 _ 
               
                
               
                 
                   
                     ∑ 
                     
                       i 
                       = 
                       1 
                     
                     N 
                   
                    
                   
                     ro 
                     
                       i 
                       , 
                       j 
                     
                   
                 
                 N 
               
             
             , 
           
         
       
     
         [0000]    ro i,j  represents the occupation rate of the i-th type of resource over which the node p j  executes the task O, N represents the total number of types of resources supportable by all the nodes in the group of nodes, and i represents any one type of resource; 
         [0071]    an execution node set obtaining sub-module, used for determining nodes with lower resource balance degrees σ j  and lower resource occupation influence rates μ j  in the group of nodes as execution nodes, so as to get an execution node set P k , specifically the node set P k  is obtained by the equation of: 
         [0000]        P   K   ={p   k }=argmin(μ j ,σ j ) T   ,jε[ 1, M′].  
 
         [0072]    wherein M′ represents the number of nodes in the execution node set meeting the conditions, and argmin represents a corresponding node which makes μ j  and σ j  get minimum values; and 
         [0073]    a selecting sub-module, used for selecting a node in the execution node set P k  as a final execution node, and further forwarding the task O to the final execution node for processing. 
         [0074]    The above types of resources comprise a number of CPUs, memory capacity, or hard disk capacity. 
         [0075]    In summary, the invention proposes a distributed scheduling method suitable for solving the condition of a single task under a node group constrained by multiple resources and containing multiple service nodes. A group of nodes includes one or more service execution nodes which are serving entities capable of providing various types of services, wherein a node receiving a user service request O is a scheduling execution node which is a core in the group of nodes, and primarily responsible for deciding on service scheduling, computing, and related operations. Each node in the group of nodes needs to maintain resource state information and history service information of that node, and exchanges it with the secluding execution node under a certain rule to prepare for a scheduling selection process. The scheduling execution node firstly calculates a resource utilization rate increment after each node in the group loads a task and a resource occupation balance degree among different kinds of resources as main evaluation criterions according to the amount of resources demanded for the received task request. Thereafter a scheduling node is selected according to a principle of selecting an appropriate node as the task execution node in case that the resource utilization rate increment is as low as possible and the occupation of different types of resources is as balanced as possible. 
         [0076]    Finally it should be explained that the aforementioned embodiments are merely used for illustrating rather than limiting the technical solution of the present invention. Although the present invention has been described in detail with reference to the embodiments, those skilled in the art will understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and thereby should all be encompassed within the scope of the claims of the present invention.