Patent Publication Number: US-2022237430-A1

Title: Harmonic densely connecting method of block of convolutional neural network model and system thereof, and non-transitory tangible computer readable recording medium

Description:
RELATED APPLICATIONS 
     The present application is a Continuation-in-part of U.S. application Ser. No. 16/451,034, filed Jun. 25, 2019, which is herein incorporated by reference. 
    
    
     BACKGROUND 
     Technical Field 
     The present disclosure relates to a harmonic densely connecting method of a block of a convolutional neural network model and a system thereof, and a non-transitory tangible computer readable recording medium. More particularly, the present disclosure relates to a harmonic densely connecting method of a block of a convolutional neural network model and a system thereof, and a non-transitory tangible computer readable recording medium which are according to a harmonic densely connected network. 
     Description of Related Art 
     A DenseNet can perform better efficiency on parameter and computation, achieving the same accuracy under fewer parameters and fewer computation operations. However, a layer-input of each of the layer operation steps of the DenseNet should concatenate all of pre-layer outputs of the DenseNet. Because a channel width of a layer-input tensor is increased, a computation of a system is increased and a channel width of a layer-output of each of the layer operation steps is increased. Therefore, an access efficiency of the memory is decreased and a power consumption of the system is increased. 
     Hence, how to reduce the computation of the system and optimize the memory access to reduce the power consumption is a crucial problem. 
     SUMMARY 
     According to one aspect of the present disclosure, a harmonic densely connecting method of a block of a convolutional neural network model is applied to a semantic segmentation and includes an input step, a plurality of layer operation steps and an output step. The input step is performed by a Central Processing Unit (CPU) to store an original input tensor of the block of an input image of the semantic segmentation into a memory. Each of the layer operation steps includes a layer-input tensor concatenating step and a convolution operation step. The layer-input tensor concatenating step is performed by the CPU to select at least one layer-input element tensor of a layer-input set from at least one result tensor and the original input tensor in the memory according to an input connection rule. When a number of the at least one layer-input element tensor of the layer-input set is greater than 1, concatenating all of the at least one layer-input element tensors along a channel dimension, and producing a layer-input tensor. The convolution operation step is performed by the CPU to calculate a convolution operation on the layer-input tensor to produce the at least one result tensor, and then store the at least one result tensor into the memory. The output step is performed by the CPU to output a block output of an output image of the semantic segmentation. The block output is a set of at least one block output element tensor, which is selected from the at least one result tensor and the original input tensor in the memory according to an output connection rule. The semantic segmentation is configured to classify the block of the input image into the block output of the output image. The at least one result tensor of each of the layer operation steps is T i . i is an integer which is larger than 0, and T 0  is the original input tensor. The input connection rule in the layer-input tensor concatenating step satisfies: 
     
       
         
           
             
               T 
               ⁢ 
               
                 S 
                 j 
               
             
             = 
             
               
                 { 
                 
                   
                     
                       
                         T 
                         
                           j 
                           - 
                           
                             2 
                             x 
                           
                         
                       
                       | 
                       
                         j 
                         ⁢ 
                         
                             
                         
                         ⁢ 
                         mod 
                         ⁢ 
                         
                             
                         
                         ⁢ 
                         
                           2 
                           x 
                         
                       
                     
                     = 
                     0 
                   
                   , 
                   
                     j 
                     ≥ 
                     
                       2 
                       x 
                     
                   
                   , 
                   
                       
                   
                   ⁢ 
                   
                     x 
                     ∈ 
                     ℤ 
                   
                   , 
                   
                       
                   
                   ⁢ 
                   
                     x 
                     ≥ 
                     0 
                   
                 
                 } 
               
               . 
             
           
         
       
     
     Wherein TS j  is the layer-input set in the layer-input tensor concatenating step of a jth layer operation step. x is a non-negative integer, and T j-1     x    is the at least one layer-input element tensor. The at least one result tensor in the memory has a channel width, and the channel width of the at least one result tensor satisfies: 
     
       
         
           
             
               Channel 
               ⁡ 
               
                 ( 
                 
                   T 
                   i 
                 
                 ) 
               
             
             = 
             
               k 
               * 
               
                 
                   m 
                   
                     z 
                     i 
                   
                 
                 . 
               
             
           
         
       
     
     Wherein Channel(T i ) is the channel width of T i , k is a constant, m is a constant, and z i  is an integer and satisfies: 
     
       
         
           
             
               z 
               i 
             
             = 
             
               max 
               ⁢ 
               
                 
                   { 
                   
                     
                       
                         x 
                         | 
                         
                           i 
                           ⁢ 
                           
                               
                           
                           ⁢ 
                           mod 
                           ⁢ 
                           
                               
                           
                           ⁢ 
                           
                             2 
                             x 
                           
                         
                       
                       = 
                       0 
                     
                     , 
                     
                         
                     
                     ⁢ 
                     
                       x 
                       ∈ 
                       ℤ 
                     
                     , 
                     
                         
                     
                     ⁢ 
                     
                       x 
                       ≥ 
                       0 
                     
                   
                   } 
                 
                 . 
               
             
           
         
       
     
     According to another aspect of the present disclosure, a system of the harmonic densely connecting method of the block of the convolutional neural network model is applied to semantic segmentation and includes a Central Processing Unit (CPU) and a memory. The CPU performs the harmonic densely connecting method. The memory is electronically connected to the CPU, and stories at least one result tensor and an original input tensor. The harmonic densely connecting method includes an input step, a plurality of layer operation steps and an output step. The input step is performed by the CPU to store the original input tensor of the block of an input image of the semantic segmentation into the memory. Each of the layer operation steps includes a layer-input tensor concatenating step and a convolution operation step. The layer-input tensor concatenating step is performed by the CPU to select at least one layer-input element tensor of a layer-input set from the at least one result tensor and the original input tensor in the memory according to an input connection rule. When a number of the at least one layer-input element tensor of the layer-input set is greater than 1, concatenating all of the layer-input element tensors along a channel dimension, and producing a layer-input tensor. The convolution operation step is performed by the CPU to calculate a convolution operation on the layer-input tensor to produce the at least one result tensor, and then store the at least one result tensor into the memory. The output step is performed by the CPU to output a block output of an output image of the semantic segmentation. The block output is a set of at least one block output element tensor, which is selected from the at least one result tensor and the original input tensor in the memory according to an output connection rule. The semantic segmentation is configured to classify the block of the input image into the block output of the output image. The at least one result tensor of each of the layer operation steps is T i . i is an integer which is larger than 0, and T 0  is the original input tensor. The input connection rule in the layer-input tensor concatenating step satisfies: 
     
       
         
           
             
               T 
               ⁢ 
               
                 S 
                 j 
               
             
             = 
             
               
                 { 
                 
                   
                     
                       
                         T 
                         
                           j 
                           - 
                           
                             2 
                             x 
                           
                         
                       
                       | 
                       
                         j 
                         ⁢ 
                         
                             
                         
                         ⁢ 
                         mod 
                         ⁢ 
                         
                             
                         
                         ⁢ 
                         
                           2 
                           x 
                         
                       
                     
                     = 
                     0 
                   
                   , 
                   
                     j 
                     ≥ 
                     
                       2 
                       x 
                     
                   
                   , 
                   
                       
                   
                   ⁢ 
                   
                     x 
                     ∈ 
                     ℤ 
                   
                   , 
                   
                       
                   
                   ⁢ 
                   
                     x 
                     ≥ 
                     0 
                   
                 
                 } 
               
               . 
             
           
         
       
     
     Wherein TS j  is the layer-input set in the layer-input tensor concatenating step of a jth layer operation step. x is a non-negative integer, and T j-2     x    is the at least one layer-input element tensor. The at least one result tensor in the memory has a channel width, and the channel width of the at least one result tensor satisfies: 
     
       
         
           
             
               Channel 
               ⁡ 
               
                 ( 
                 
                   T 
                   i 
                 
                 ) 
               
             
             = 
             
               k 
               * 
               
                 
                   m 
                   
                     z 
                     i 
                   
                 
                 . 
               
             
           
         
       
     
     Wherein Channel(T i ) is the channel width of T i , k is a constant, m is a constant, and z i  is an integer and satisfies: 
     
       
         
           
             
               z 
               i 
             
             = 
             
               max 
               ⁢ 
               
                 
                   { 
                   
                     
                       
                         x 
                         | 
                         
                           i 
                           ⁢ 
                           
                               
                           
                           ⁢ 
                           mod 
                           ⁢ 
                           
                               
                           
                           ⁢ 
                           
                             2 
                             x 
                           
                         
                       
                       = 
                       0 
                     
                     , 
                     
                         
                     
                     ⁢ 
                     
                       x 
                       ∈ 
                       ℤ 
                     
                     , 
                     
                         
                     
                     ⁢ 
                     
                       x 
                       ≥ 
                       0 
                     
                   
                   } 
                 
                 . 
               
             
           
         
       
     
     According to further another aspect of the present disclosure, a non-transitory tangible computer readable recording medium storing instructions which when executed by a Central Processing Unit (CPU) configured to perform a harmonic densely connecting method of a block of a convolutional neural network model. The harmonic densely connecting method of the block of the convolutional neural network model includes an input step, a plurality of layer operation steps and an output step. The input step is performed by the CPU to store an original input tensor of the block of an input image of the semantic segmentation into a memory. Each of the layer operation steps includes a layer-input tensor concatenating step and a convolution operation step. The layer-input tensor concatenating step is performed by the CPU to select at least one layer-input element tensor of a layer-input set from at least one result tensor and the original input tensor in the memory according to an input connection rule. When a number of the at least one layer-input element tensor of the layer-input set is greater than 1, concatenating all of the layer-input element tensors along a channel dimension, and producing a layer-input tensor. The convolution operation step is performed by the CPU to calculate a convolution operation on the layer-input tensor to produce the at least one result tensor, and then store the at least one result tensor into the memory. The output step is performed by the CPU to output a block output of an output image of the semantic segmentation. The block output is a set of at least one block output element tensor, which is selected from the at least one result tensor and the original input tensor in the memory according to an output connection rule. The semantic segmentation is configured to classify the block of the input image into the block output of the output image. The at least one result tensor of each of the layer operation steps is T i . i is an integer which is larger than 0, and T 0  is the original input tensor. The input connection rule in the layer-input tensor concatenating step satisfies: 
     
       
         
           
             
               T 
               ⁢ 
               
                 S 
                 j 
               
             
             = 
             
               
                 { 
                 
                   
                     
                       
                         T 
                         
                           j 
                           - 
                           
                             2 
                             x 
                           
                         
                       
                       | 
                       
                         j 
                         ⁢ 
                         
                             
                         
                         ⁢ 
                         mod 
                         ⁢ 
                         
                             
                         
                         ⁢ 
                         
                           2 
                           x 
                         
                       
                     
                     = 
                     0 
                   
                   , 
                   
                     j 
                     ≥ 
                     
                       2 
                       x 
                     
                   
                   , 
                   
                       
                   
                   ⁢ 
                   
                     x 
                     ∈ 
                     ℤ 
                   
                   , 
                   
                     x 
                     ≥ 
                     0 
                   
                 
                 } 
               
               . 
             
           
         
       
     
     Wherein TS j  is the layer-input set in the layer-input tensor concatenating step of a jth layer operation step. x is a non-negative integer, and T j-2     x    is the at least one layer-input element tensor. The at least one result tensor in the memory has a channel width, and the channel width of the at least one result tensor satisfies: 
     
       
         
           
             
               Channel 
               ⁡ 
               
                 ( 
                 
                   T 
                   i 
                 
                 ) 
               
             
             = 
             
               k 
               * 
               
                 
                   m 
                   
                     z 
                     i 
                   
                 
                 . 
               
             
           
         
       
     
     Wherein Channel(T i ) is the channel width of T i , k is a constant, m is a constant, and z i  is an integer and satisfies: 
     
       
         
           
             
               z 
               i 
             
             = 
             
               max 
               ⁢ 
               
                 
                   { 
                   
                     
                       
                         x 
                         | 
                         
                           i 
                           ⁢ 
                           
                               
                           
                           ⁢ 
                           mod 
                           ⁢ 
                           
                               
                           
                           ⁢ 
                           
                             2 
                             x 
                           
                         
                       
                       = 
                       0 
                     
                     , 
                     
                         
                     
                     ⁢ 
                     
                       x 
                       ∈ 
                       ℤ 
                     
                     , 
                     
                       x 
                       ≥ 
                       0 
                     
                   
                   } 
                 
                 . 
               
             
           
         
       
     
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
       The present disclosure can be more fully understood by reading the following detailed description of the embodiment, with reference made to the accompanying drawings as follows: 
         FIG. 1  shows a flow chart of a harmonic densely connecting method of a block of a convolutional neural network model according to one embodiment of the present disclosure. 
         FIG. 2  shows a schematic diagram of one example of the harmonic densely connecting method of  FIG. 1 . 
         FIG. 3  shows a schematic diagram of another example of the harmonic densely connecting method of  FIG. 1 . 
         FIG. 4  shows a schematic diagram of further another example of the harmonic densely connecting method of  FIG. 1 . 
         FIG. 5  shows a block diagram of a system of the harmonic densely connecting method of the block of the convolutional neural network model of  FIG. 1 . 
         FIG. 6  shows a schematic diagram of an input image and an output image of the harmonic densely connecting method of  FIG. 1 , which is applied to a semantic segmentation. 
         FIG. 7  shows a schematic diagram of FCHarDNet70 of the harmonic densely connecting method of  FIG. 1 . 
     
    
    
     DETAILED DESCRIPTION 
     The embodiment will be described with the drawings. For clarity, some practical details will be described below. However, it should be noted that the present disclosure should not be limited by the practical details, that is, in some embodiment, the practical details is unnecessary. In addition, for simplifying the drawings, some conventional structures and elements will be simply illustrated, and repeated elements may be represented by the same labels. 
       FIG. 1  shows a flow chart of a harmonic densely connecting method s 100  of a block of a convolutional neural network model according to one embodiment of the present disclosure.  FIG. 2  shows a schematic diagram of one example of the harmonic densely connecting method s 100  of  FIG. 1 . In  FIG. 1  and  FIG. 2 , the harmonic densely connecting method s 100  of the block of the convolutional neural network model includes an input step s 110 , a plurality of layer operation steps s 120  and an output step s 130 . 
     The input step s 110  is for storing an original input tensor of the block into a memory  220  (shown in  FIG. 5 ). Each of the layer operation steps s 120  includes a layer-input tensor concatenating step and a convolution operation step. The layer-input tensor concatenating step is for selecting at least one layer-input element tensor of a layer-input set from at least one result tensor and the original input tensor in the memory  220  according to an input connection rule. When a number of the at least one layer-input element tensor of the layer-input set is greater than 1, concatenating all of the layer-input element tensors along a channel dimension, and producing a layer-input tensor for a convolution operation in each of the layer operation steps s 120 . The convolution operation step is for calculating the convolution operation on the layer-input tensor to produce at least one result tensor and then storing the at least one result tensor into the memory  220 . A total number of the layer operation steps s 120  is N. The output step s 130  is for outputting a block output. The block output is a set of at least one block output element tensor, which is selected from the at least one result tensor and the original input tensor in the memory  220  according to an output connection rule. The at least one result tensor of each of the layer operation steps s 120  is T i . i is an integer which is larger than 0, and T 0  is the original input tensor. The input connection rule in the layer-input tensor concatenating step is satisfied by a formula (1). 
     
       
         
           
             
               
                 
                   
                     T 
                     ⁢ 
                     
                       S 
                       j 
                     
                   
                   = 
                   
                     
                       { 
                       
                         
                           
                             
                               T 
                               
                                 j 
                                 - 
                                 
                                   2 
                                   x 
                                 
                               
                             
                             | 
                             
                               j 
                               ⁢ 
                               
                                   
                               
                               ⁢ 
                               mod 
                               ⁢ 
                               
                                   
                               
                               ⁢ 
                               
                                 2 
                                 x 
                               
                             
                           
                           = 
                           0 
                         
                         , 
                         
                           j 
                           ≥ 
                           
                             2 
                             x 
                           
                         
                         , 
                         
                             
                         
                         ⁢ 
                         
                           x 
                           ∈ 
                           ℤ 
                         
                         , 
                         
                           x 
                           ≥ 
                           0 
                         
                       
                       } 
                     
                     . 
                   
                 
               
               
                 
                   ( 
                   1 
                   ) 
                 
               
             
           
         
       
     
     Wherein TS j  is the layer-input set in the layer-input tensor concatenating step of a jth layer operation step. x is a non-negative integer. T j-2     x    is the at least one layer-input element tensor. Because of the input connection rule, a number of the at least one layer-input element tensor is very limited. Therefore, a connection complexity of the harmonic densely connecting method s 100  can be reduced comparing to a full-densely connected network. The at least one result tensor in the memory  220  has a channel width, and the channel width of the at least one result tensor is satisfied by a formula (2). 
     
       
         
           
             
               
                 
                   Channel 
                   ⁢ 
                   
                     
                       
                         ( 
                         
                           T 
                           i 
                         
                         ) 
                       
                       = 
                       
                         k 
                         * 
                         
                           m 
                           
                             z 
                             i 
                           
                         
                       
                     
                     . 
                   
                 
               
               
                 
                   ( 
                   2 
                   ) 
                 
               
             
           
         
       
     
     Wherein Channel(T i ) is the channel width of T i . k is a constant. m is a constant, and z i  is an integer and satisfied by a formula (3). 
     
       
         
           
             
               
                 
                   
                     z 
                     i 
                   
                   = 
                   
                     max 
                     ⁢ 
                     
                       
                         { 
                         
                           
                             
                               x 
                               | 
                               
                                 i 
                                 ⁢ 
                                 
                                     
                                 
                                 ⁢ 
                                 mod 
                                 ⁢ 
                                 
                                     
                                 
                                 ⁢ 
                                 
                                   2 
                                   x 
                                 
                               
                             
                             = 
                             0 
                           
                           , 
                           
                               
                           
                           ⁢ 
                           
                             x 
                             ∈ 
                             ℤ 
                           
                           , 
                           
                             x 
                             ≥ 
                             0 
                           
                         
                         } 
                       
                       . 
                     
                   
                 
               
               
                 
                   ( 
                   3 
                   ) 
                 
               
             
           
         
       
     
     In each of the layer operation steps s 120 , the input connection rule is for reducing the connection complexity to be constrained in O(log N), wherein O is a big O notation, while a shortcut depth from any layer to the base layer is also in O(log N). In other words, the shortcut depth from any the layer operation step to the layer operation step  1  is also in O(log N). Thus, the input connection rule achieves a best balance between the shortcut depth and the connection complexity. Because of the connection complexity is reduced, accesses to the at least one layer-input element tensor of the layer-input set which is a part of the at least one result tensor and the original input tensor in the memory  220  is reduced correspondingly, so that the harmonic densely connecting method s 100  can improve a performance and a power-efficiency of the system  200 . 
     In  FIG. 2 , each of the layer operation steps s 120  calculates the convolution operation on the layer-input tensor with the convolutional kernel of each of the layer operation steps s 120  so as to produce the at least one result tensor of each of the layer operation steps s 120 . 
     Please refer to  FIG. 2  and Table 1, Table 1 lists the layer-input set and the at least one result tensor of the layer operation steps s 120 . The input step s 110  is for storing the original input tensor of the block into a memory  220  (e.g., Dynamic Random Access Memory (DRAM) with a local memory for temporary buffering), as shown in  FIG. 5 , so as to perform the layer operation steps s 120 . In  FIG. 2 , the total number of the layer operation steps s 120  is equal to 8, and it denotes N=8. In the layer operation step  1 , the layer-input set of the layer operation step  1  is selected from the original input tensor in the memory  220  according to the input connection rule. It denotes TS j =TS 1 ={T 1-2     x   |1 mod 2 x =0,1≥2 x , x∈ , x≥0}={T 0 }, and x={0}. The at least one layer-input element tensor of the layer-input set of the layer operation step  1  is T 0 . Because a number of the at least one layer-input element tensor of the layer-input set of the layer operation step  1  is equal to 1, the layer-input tensor of the layer operation step  1  is T 0 . The convolution operation step of the layer operation step  1  performs a convolution operation on T 0  and a convolutional kernel of the layer operation step  1  so as to produce T 1 , and then stores T 1  into the memory  220 . Further, the channel width of T 1  is Channel(T 1 )=k*m z     1   =k, wherein z 1 =max{x|1 mod 2 x =0, x∈ , x≥0}=0. m is greater than 1.4 and less than 2. 
     
       
         
           
               
               
               
               
             
               
                 TABLE 1 
               
               
                   
               
               
                   
                   
                 result 
                 channel width of 
               
               
                 layer operation steps 
                 layer-input set 
                 tensor 
                 the result tensor 
               
               
                   
               
             
            
               
                 layer operation step 
                 T 0   
                 T 1   
                 k 
               
               
                 layer operation step 
                 T 0 , T 1   
                 T 2   
                 k × m 
               
               
                 layer operation step 
                 T 2   
                 T 3   
                 k 
               
               
                 layer operation step 
                 T 0 , T 2 , T 3   
                 T 4   
                 k × m 2   
               
               
                 layer operation step 
                 T 4   
                 T 5   
                 k 
               
               
                 layer operation step 
                 T 4 , T 5   
                 T 6   
                 k × m 
               
               
                 layer operation step 
                 T 6   
                 T 7   
                 k 
               
               
                 layer operation step 
                 T 0 , T 4 , T 6 , T 7   
                 T 8   
                 k × m 3   
               
               
                   
               
            
           
         
       
     
     In the layer operation step  2 , a layer-input set of the layer operation step  2  is selected from the at least one result tensor and the original input tensor in the memory  220  according to the input connection rule. It denotes TS j =TS 2 ={T 2-2     x   |2 mod 2 x =0, 2≥2 x , x∈ , x≥0}={T 0 , T 1 }, and x is equal to {0, 1}. The at least one layer-input element tensor of the layer operation step  2  is T 0  and T 1 . Because a number of the at least one layer-input element tensor of the layer operation step  2  is greater than 1 and respective to T 0  and T 1 , the layer operation step  2  concatenates T 0  and T 1  along the channel dimension so as to produce a layer-input tensor of the layer operation step  2 . A convolution operation step of the layer operation step  2  performs a convolution operation on the layer-input tensor and a convolutional kernel of the layer operation step  2  so as to produce T 2 , and then stores T 2  into the memory  220 . Further, z 2 =max{x|2 mod 2 x =0, x∈ ,x≥0}=1 because x is equal to {0, 1}. Therefore, the channel width of T 2  is Channel(T 2 )=k*m z     2   =k×m. 
     In the layer operation step  3 , a layer-input set of the layer operation step  3  is selected from at least one result tensor and the original input tensor in the memory  220 , according to the input connection rule. It denotes TS 1 =TS 3 ={T 3-2     x   |3 mod 2 x =0, 3≥2 x , x∈Z, x≥0}={T 2 }, and x={0}. The at least one layer-input element tensor of the layer operation step  3  is T 2 . Because a number of the at least one layer-input element tensor of the layer operation step  3  is equal to 1, the layer-input tensor of the layer operation step  3  is T 2 . The convolution operation step of the layer operation step  3  performs a convolution operation on T 2  and a convolutional kernel of the layer operation step  3  so as to produce T 3 , and then stores T 3  into the memory  220 . Further, z 3 =max{x|3 mod 2 x =0,x∈ ,x≥0}=0 because x is equal to {0}. Therefore, the channel width of T 3  is Channel(T 3 )=k*m z     3   =k. The layer-input tensor concatenating step and the convolution operation step of each of the layer operation steps  4 - 8  are same as above, and will not be described again herein. 
     The output step s 130  of the harmonic densely connecting method s 100  selects the set of the at least one block output element tensor from the at least one result tensor in the memory  220  according to the output connection rule. The output connection rule of the output step s 130  is satisfied by a formula (4). 
     
       
         
           
             
               
                 
                   
                     O 
                     ⁢ 
                     
                         
                     
                     ⁢ 
                     S 
                   
                   = 
                   
                     
                       { 
                       
                         
                           
                             T 
                             q 
                           
                           | 
                           
                             q 
                             ⁢ 
                             
                                 
                             
                             ⁢ 
                             mod 
                             ⁢ 
                             
                                 
                             
                             ⁢ 
                             2 
                           
                         
                         = 
                         
                           
                             1 
                             ⁢ 
                             
                                 
                             
                             ⁢ 
                             or 
                             ⁢ 
                             
                                 
                             
                             ⁢ 
                             q 
                           
                           = 
                           N 
                         
                       
                       } 
                     
                     . 
                   
                 
               
               
                 
                   ( 
                   4 
                   ) 
                 
               
             
           
         
       
     
     Wherein OS is the block output. T q  is the block output element tensor of the block output. q is an integer from 1 to N. N is a total number of the layer operation steps, and N is a positive integer. In  FIG. 2 , the block output is selected from the at least one result tensor and the original input tensor in the memory  220  by the formula (4), and it denotes OS={T q |q mod 2=1 or q=N}={T 1 , T 3 , T 5 , T 7 , T 8 }. Therefore, the block output of the harmonic densely connecting method s 100  of  FIG. 2  includes {T 1 , T 3 , T 5 , T 7 , T 8 }. 
     Please refer to  FIG. 3 ,  FIG. 3  shows a schematic diagram of another example of the harmonic densely connecting method s 100  of  FIG. 1 . In  FIG. 3 , each of the layer operation steps s 120  calculates the convolution operation on the layer-input tensor with the convolutional kernel so as to produce the at least one result tensor of each of the layer operation steps s 120 . The output step s 130  of the harmonic densely connecting method s 100  selects the set of at least one block output element tensor from the at least one result tensor and the original input tensor in the memory  220  according to an output connection rule. The output connection rule of the output step s 130  is satisfied by a formula (5). 
     
       
         
           
             
               
                 
                   OS 
                   = 
                   
                     
                       { 
                       
                         
                           
                             T 
                             q 
                           
                           | 
                           
                             q 
                             ⁢ 
                             
                                 
                             
                             ⁢ 
                             mod 
                             ⁢ 
                             
                                 
                             
                             ⁢ 
                             2 
                           
                         
                         = 
                         
                           
                             1 
                             ⁢ 
                             
                                 
                             
                             ⁢ 
                             or 
                             ⁢ 
                             
                                 
                             
                             ⁢ 
                             q 
                           
                           = 
                           
                             
                               N 
                               ⁢ 
                               
                                   
                               
                               ⁢ 
                               or 
                               ⁢ 
                               
                                   
                               
                               ⁢ 
                               q 
                             
                             = 
                             0 
                           
                         
                       
                       } 
                     
                     . 
                   
                 
               
               
                 
                   ( 
                   5 
                   ) 
                 
               
             
           
         
       
     
     The block output is selected from the at least one result tensor and the original input tensor in the memory  220  by the formula (5), and it denotes OS={T q |q mod 2=1 or q=N or q=0}={T 0 , T 1 , T 3 , T 5 , T 7 , T 8 }. Therefore, the block output of the harmonic densely connecting method s 100  of  FIG. 3  includes {T 0 , T 1 , T 3 , T 5 , T 7 , T 8 }. 
     In order to optimize a memory access of the harmonic densely connecting method s 100  so as to reduce a power consumption. A number of the at least one result tensor is greater than 1. When T l  is calculated and l is divided by 4, at least one of the result tensors storing in the memory  220  is removed according to a removing rule. The removing rule is satisfied by a formula (6). 
     
       
         
           
             
               
                 
                   
                     R 
                     ⁢ 
                     
                       S 
                       l 
                     
                   
                   = 
                   
                     
                       { 
                       
                         
                           T 
                           r 
                         
                         | 
                         
                           
                             T 
                             r 
                           
                           ∈ 
                           
                             
                               T 
                               ⁢ 
                               
                                 S 
                                 l 
                               
                             
                             - 
                             
                               { 
                               
                                 
                                   
                                     T 
                                     c 
                                   
                                   | 
                                   c 
                                 
                                 = 
                                 
                                   
                                     min 
                                     ⁢ 
                                     
                                         
                                     
                                     ⁢ 
                                     
                                       { 
                                       
                                         c 
                                         | 
                                         
                                           
                                             T 
                                             c 
                                           
                                           ∈ 
                                           
                                             T 
                                             ⁢ 
                                             
                                               S 
                                               l 
                                             
                                           
                                         
                                       
                                       } 
                                     
                                   
                                   - 
                                   
                                     { 
                                     
                                       
                                         
                                           T 
                                           a 
                                         
                                         | 
                                         a 
                                       
                                       = 
                                       
                                         max 
                                         ⁢ 
                                         
                                             
                                         
                                         ⁢ 
                                         
                                           { 
                                           
                                             a 
                                             | 
                                             
                                               
                                                 T 
                                                 a 
                                               
                                               ∈ 
                                               
                                                 T 
                                                 ⁢ 
                                                 
                                                   S 
                                                   l 
                                                 
                                               
                                             
                                           
                                           } 
                                         
                                       
                                     
                                     } 
                                   
                                 
                               
                               } 
                             
                           
                         
                       
                       } 
                     
                     . 
                   
                 
               
               
                 
                   ( 
                   6 
                   ) 
                 
               
             
           
         
       
     
     Wherein RS l  is a set of the at least one of the result tensors storing in the memory  220  which can be removed after a lth layer operation step. T r  is one of the result tensors in the memory  220  which can be removed. TS l  is the layer-input set of the lth layer operation step. T c  is one of the layer-input element tensors of the lth layer operation step. T a  is another one of the layer-input element tensors of the lth layer operation step. In other words, in the lth layer operation step, the harmonic densely connecting method s 100  makes it possible to remove the set of the at least one of the result tensors storing in the memory  220  so as to increase an access efficiency of the memory  220 . Therefore, the memory accesses can be reduced so as to decrease the power consumption. 
     Please refer to  FIG. 2 ,  FIG. 3  and Table 1. In the layer operation step  4 , the layer-input set of the layer operation step  4  is selected from the at least one result tensor and the original input tensor in the memory  220  by the formula (1). It denotes TS j =TS 4 ={T 4-2     x   |4 mod 2 x =0, 4≥2 x , x∈ , x≥0}={T 0 , T 2 , T 3 }, and x is equal to {0, 1, 2}. Because a number of the at least one layer-input element tensor of the layer-input set of the layer operation step  4  is equal to 3, the layer-input tensor concatenating step of the layer operation step  4  concatenates T 0 , T 2 , T 3  to produce a layer-input tensor. The convolution operation step of the layer operation step  4  performs a convolution operation on the layer-input tensor with a convolutional kernel so as to produce T 4 . Because T 4  is calculated, the set of the at least one of the result tensors storing in the memory  220  which is going to be removed according to the removing rule is RS 4 ={T r |T r ∈TS l −{T c |c=min{c|T c ∈TS 4 }−{T a |a=max{a|T a ∈TS 4 }}}}={T 2 }, and it denotes T c =T 0 , T a =T 3  and T 2  is removed from the memory  220 . Thus, after performing the layer operation step  4 , there are only T 0 , T 1 , T 3 , T 4  storing in the memory  220 . Therefore, the access efficiency of the memory  220  and the power consumption is decreased. 
     In order to decrease the power consumption of the harmonic densely connecting method s 100 , m is greater than 1.4 and less than 2, and N is power of 2. However, m can be any positive number, and the present disclosure is not limited thereto. 
     Please refer to the  FIG. 4 ,  FIG. 4  shows a schematic diagram of further another example of the harmonic densely connecting method s 100  of  FIG. 1 . In order to reduce the computation of the harmonic densely connecting method s 100 , a part of the layer operation steps s 120  further includes a bottleneck layer step. The bottleneck layer step is for calculating the convolution operation on the layer-input tensor with a bottleneck kernel so as to produce a bottleneck tensor, and a size of the bottleneck kernel is 1×1. Each of the part of the layer operation steps s 120  calculates the convolution operation on the bottleneck tensor with the convolutional kernel so as to produce the at least one result tensor. In other words, in each of the part of the layer operation steps s 120 , the bottleneck layer step performs the convolution operation on the layer-input tensor with the bottleneck kernel so as to produce the bottleneck tensor. Because the size of the bottleneck kernel is 1×1, a parameter size of the bottleneck tensor can be reduced so as to a parameter efficiency of the harmonic densely connecting method s 100  can be enhanced. Then, the convolution operation step calculates the at least one result tensor of each of the part of the layer operation steps s 120  by a convolutional operation on the bottleneck tensor with the convolutional kernel. Therefore, the computation of the part of the layer operation steps s 120  (e.g., the layer operation steps  4  and  8  in  FIG. 4 ) can be reduced. Further, each of the other part of the layer operation steps s 120  (e.g., the layer operation steps  1 - 3  and  5 - 7  in  FIG. 4 ) calculates the convolution operation on the layer-input tensor with the convolutional kernel so as to produce the at least one result tensor. 
     In order to reduce the computation of the harmonic densely connecting method s 100 , a channel width of the bottleneck tensor is satisfied by a formula (7). 
     
       
         
           
             
               
                 
                   
                     Channel 
                     ⁡ 
                     
                       ( 
                       
                         B 
                         b 
                       
                       ) 
                     
                   
                   = 
                   
                     
                       
                         
                           Channel 
                           ⁡ 
                           
                             ( 
                             
                               T 
                               ⁢ 
                               
                                 S 
                                 b 
                               
                             
                             ) 
                           
                         
                         
                           Channel 
                           ⁡ 
                           
                             ( 
                             
                               T 
                               b 
                             
                             ) 
                           
                         
                       
                     
                     × 
                     
                       
                         Channel 
                         ⁡ 
                         
                           ( 
                           
                             T 
                             b 
                           
                           ) 
                         
                       
                       . 
                     
                   
                 
               
               
                 
                   ( 
                   7 
                   ) 
                 
               
             
           
         
       
     
     Wherein B b  is the bottleneck tensor of a bth layer operation step. Channel(B b ) is the channel width of B b . b is a layer index of the bth layer operation step. TS b  is the layer-input set in the layer-input tensor concatenating step of the bth layer operation step. Channel(TS b ) is the summation of the channel width of all layer-input element tensors of TS b . 
     Because of the input connection rule, the channel width of the layer-input tensor of each of even layer operation steps s 120 , such as the layer operation step  2  and the layer operation step  4 , is greater than the channel width of the layer-input tensor of each of odd layer operation steps s 120 , such as the layer operation step  1  and the layer operation step  3 . Therefore, b can be an even positive integer so as to reduce the computation of the harmonic densely connecting method s 100 . In  FIG. 4 , b is satisfied by a formula (8). 
     
       
         
           
             
               
                 
                   
                     b 
                     ⁢ 
                     
                         
                     
                     ⁢ 
                     mod 
                     ⁢ 
                     
                         
                     
                     ⁢ 
                     4 
                   
                   = 
                   
                     
                       0 
                       ⁢ 
                       
                           
                       
                       ⁢ 
                       and 
                       ⁢ 
                       
                           
                       
                       ⁢ 
                       b 
                     
                     &gt; 
                     0. 
                   
                 
               
               
                 
                   ( 
                   8 
                   ) 
                 
               
             
           
         
       
     
     Please refer to  FIG. 4 , in the layer operation step  7 , the layer-input set of the layer operation step  7  is selected from the memory  220  by the input connection rule. It denotes TS j =TS 7 ={T 7-2     x   |7 mod 2 x =0, 7≥2 x , x∈ , x≥0}={T 6 }, and x is equal to 0. Because a number of the at least one layer-input element tensor of the layer-input set of the layer operation step  7  is equal to 1, the layer-input tensor of the layer operation step  7  is T 6 . Due to 7 mod 4≠0, the layer operation step  7  calculates the convolution operation on T 6  with the convolutional kernel of the layer operation step  7  so as to produce T 7 . 
     Please refer to  FIG. 4 , in the layer operation step  8 , the layer-input set is selected from the at least one result tensor and the original input tensor in the memory  220  by the formula (1). It denotes TS j =TS 8 ={T 8-2     x   |8 mod 2 x =0, 8≥2 x , x∈ , x≥0}={T 0 , T 4 , T 6 , T 7 } and x is equal to {0, 1, 2, 3}. Because a number of the at least one layer-input element tensor of the layer-input set of the layer operation step  8  is equal to 4. The layer-input tensor concatenating step of the layer operation step  8  concatenates T 0 , T 4 , T 6 , T 7  along the channel dimension, and produces the layer-input tensor of the layer operation step  8 . The bottleneck layer step of the layer operation step  8  performs the convolution operation on the layer-input tensor with a bottleneck kernel for calculating the bottleneck tensor of the layer operation step  8 . The channel width of the bottleneck tensor of the layer operation step  8  is 
     
       
         
           
             
               
                 Channel 
                 ⁡ 
                 
                   ( 
                   
                     B 
                     8 
                   
                   ) 
                 
               
               = 
               
                 
                   
                     
                       
                         Channel 
                         ⁡ 
                         
                           ( 
                           
                             T 
                             ⁢ 
                             
                               S 
                               8 
                             
                           
                           ) 
                         
                       
                       
                         Channel 
                         ⁡ 
                         
                           ( 
                           
                             T 
                             8 
                           
                           ) 
                         
                       
                     
                   
                   × 
                   
                     Channel 
                     ⁡ 
                     
                       ( 
                       
                         T 
                         8 
                       
                       ) 
                     
                   
                 
                 = 
                 
                   
                     
                       
                         Channel 
                         ⁡ 
                         
                           ( 
                           
                             T 
                             ⁢ 
                             
                               S 
                               8 
                             
                           
                           ) 
                         
                       
                       
                         k 
                         × 
                         
                           m 
                           3 
                         
                       
                     
                   
                   × 
                   k 
                   × 
                   
                     m 
                     3 
                   
                 
               
             
             , 
           
         
       
     
     and it denotes the channel width of the bottleneck tensor of the layer operation step  8  is less than the channel width of the layer-input tensor of the layer operation step  8 , so that the computation of the layer operation step  8  can be reduced. After the bottleneck layer step of the layer operation step  8 , the convolution operation step of the layer operation step  8  calculates a convolutional operation on B 8  and the convolutional kernel so as to produce T 8 . Therefore, the computation of the harmonic densely connecting method s 100  can be reduced and the parameter efficiency of the harmonic densely connecting method s 100  can be enhanced. 
     Please refer to  FIG. 5 ,  FIG. 5  shows a block diagram of a system  200  of the harmonic densely connecting method s 100  of the block of the convolutional neural network model of  FIG. 1 . The system  200  of the harmonic densely connecting method s 100  of the block of the convolutional neural network model includes a Central Processing Unit (CPU)  210 , the memory  220 . The CPU  210  performs the layer operation steps s 120 . The memory  220  is electronically connected to the CPU  210  and stores the at least one result tensor and the original input tensor. In detail, the CPU  210  performs the layer-input tensor concatenating step and convolution operation step of each of the layer operation steps s 120 . In the layer-input tensor concatenating step, the CPU  210  selects at least one layer-input element tensor of the layer-input set of each of the layer operation steps s 120  from at least one result tensor or the original input tensor in the memory  220  according to the input connection rule. Because of the input connection rule, the channel width of the layer-input tensor of each of the layer operation steps s 120  is reduced. Therefore, the computation of the system  200  can be reduced. 
     In order to reduce the power consumption of the system  200 , the CPU  210  removes at least one of the result tensors storing in the memory  220  according to the formula (6). Therefore, the access efficiency of the memory  220  can be increased, and the power consumption of the system  200  can be reduced. 
     Further, the CPU  210  performs the bottleneck layer step of the parts of the layer operation steps s 120 , so that the computation of the system  200  can be reduced. 
     Please refer to  FIGS. 1, 5, 6 and 7 .  FIG. 6  shows a schematic diagram of an input image  300  and an output image  400  of the harmonic densely connecting method s 100  of  FIG. 1 , which is applied to a semantic segmentation.  FIG. 7  shows a schematic diagram of FCHarDNet70 of the harmonic densely connecting method s 100  of  FIG. 1 . The harmonic densely connecting method s 100  is applied to a semantic segmentation with a harmonic densely connected network (HarDNet) and includes an input step s 110 , a plurality of layer operation steps s 120  and an output step s 130 . The input step s 110  is performed by a CPU  210  to store an original input tensor of the block of the input image  300  of the semantic segmentation into a memory  220 . Each of the layer operation steps s 120  includes a layer-input tensor concatenating step and a convolution operation step. The layer-input tensor concatenating step is performed by the CPU  210  to select at least one layer-input element tensor of a layer-input set from at least one result tensor and the original input tensor in the memory  220  according to an input connection rule. When a number of the at least one layer-input element tensor of the layer-input set is greater than 1, concatenating all of the at least one layer-input element tensors along a channel dimension, and producing a layer-input tensor. The convolution operation step is performed by the CPU  210  to calculate a convolution operation on the layer-input tensor to produce the at least one result tensor, and then store the at least one result tensor into the memory  220 . The output step s 130  is performed by the CPU  210  to output a block output of the output image  400  of the semantic segmentation. The block output is a set of at least one block output element tensor, which is selected from the at least one result tensor and the original input tensor in the memory  220  according to an output connection rule. The semantic segmentation is configured to classify the block of the input image  300  into the block output of the output image  400 , as shown in  FIG. 6 . 
     The at least one result tensor of each of the layer operation steps s 120  is T i . i is an integer which is larger than 0, and T 0  is the original input tensor. The input connection rule in the layer-input tensor concatenating step satisfies the formula (1). The at least one result tensor in the memory  220  has a channel width, and the channel width of the at least one result tensor satisfies the formula (2) and the formula (3). 
     In one embodiment, the harmonic densely connected network may be FCHarDNet70, as shown in  FIG. 7 , but the present disclosure is not limited thereto. In  FIG. 7 , an architecture of FCHarDNet70 includes the input image  300 , a stem block  310 , a plurality of harmonic densely block (HarDBlock) encoders  320   a ,  320   b ,  320   c ,  320   d ,  320   e , a plurality of HarDBlock decoders  330   a ,  330   b ,  330   c ,  330   d , a plurality of skip connections  340 , a plurality of bilinear upsamplings  350  and the output image  400 . The input image  300  is operated by four layer operation steps s 120  to extract features as the stem block  310 . The HarDBlock encoders  320   a ,  320   b ,  320   c ,  320   d ,  320   e  are corresponding to the strides  4 ,  8 ,  16 ,  32 ,  64 , respectively. The HarDBlock decoders  330   a ,  330   b ,  330   c ,  330   d  are corresponding to the strides  32 ,  16 ,  8 ,  4 , respectively. Each of the HarDBlock encoders  320   a ,  320   b ,  320   c ,  320   d ,  320   e  performs an average pooling to extract features. Each of The HarDBlock decoders  330   a ,  330   b ,  330   c ,  330   d  is performed to concatenate the same resolution image in the HarDBlock encoder (corresponding to the skip connections  340 ) and the bilinear upsampled tensor (corresponding to the bilinear upsamplings  350 ), and then calculate the convolution operation to generate the results for the next block. “Concatenating” represents the layer-input tensor concatenating step of each of the layer operation steps s 120 . “Convolution” represents the convolution operation step of each of the layer operation steps s 120 . Accordingly, the input image  300  can be inputted to FCHarDNet70 (Deep learning model) to generate the output image  400  so as to perform the semantic segmentation with improved performance and improved power efficiency. 
     It is understood that the harmonic densely connecting method s 100  is performed by the aforementioned steps. A computer program of the present disclosure stored on a non-transitory tangible computer readable recording medium is used to perform the method described above. The aforementioned embodiments can be provided as a computer program product, which may include a machine-readable medium on which instructions are stored for programming a computer (or other electronic devices) to perform the method based on the embodiments of the present disclosure. The machine-readable medium can be, but is not limited to, a floppy diskette, an optical disk, a compact disk-read-only memory (CD-ROM), a magneto-optical disk, a read-only memory (ROM), a random access memory (RAM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic or optical card, a flash memory, or another type of media/machine-readable medium suitable for storing electronic instructions. Moreover, the embodiments of the present disclosure also can be downloaded as a computer program product, which may be transferred from a remote computer to a requesting computer by using data signals via a communication link (such as a network connection or the like). 
     Although the present disclosure has been described in considerable detail with reference to certain embodiments thereof, other embodiments are possible. Therefore, the spirit and scope of the appended claims should not be limited to the description of the embodiments contained herein. 
     It will be apparent to those skilled in the art that various modifications and variations can be made to the structure of the present disclosure without departing from the scope or spirit of the disclosure. In view of the foregoing, it is intended that the present disclosure cover modifications and variations of this disclosure provided they fall within the scope of the following claims.