Patent ID: 11914376
Assignee: WUHAN UNIVERSITY OF TECHNOLOGY
Field: Control (Instruments)
Classification: CPC G  B | IPC G

Claim 5:
6. The method according to claim 5, wherein a target evaluation network in the decision-making neural network model is determined according to Q(s,a)=ωaTC(s,a), the target evaluation network is updated by optimizing a loss function as, L
    ⁡
    (
    
     ω
     _
    
    )
   
   =
   
    E
    [
    
     
      (
      
       
        r
        ⁡
        (
        
         s
         ,
         a
        
        )
       
       +
       
        γ
        ⁢
        
         Q
         ⁡
         (
         
          
           s
           ′
          
          ,
          
           
            a
            ′
           
           ;
           
            
             ω
             _
            
            a
           
          
         
         )
        
       
       -
       
        Q
        ⁡
        (
        
         s
         ,
         
          a
          ;
          
           
            ω
            _
           
           β
          
         
        
        )
       
      
      )
     
     2
    
    ]
   
  
  ,, and a parameter ωβ is updated by a random gradient descent method, where ωβ is a parameter in an online evaluation network, ωa is a parameter in the target evaluation network, s′ indicates a next state after taking an action a at a state s,a′ is an action taken by the decision-making neural network at s′,r is a reward value, L (ω) represents a loss value between an output value of the trained network and an expected value, ω represents a set of ωa and ωβ, r(s,a) represents a reward value, γ represents a discount factor, Q(s′,a′;ωa) represents a target evaluation value of the target evaluation network, Q(s,a;ωβ) represents a target evaluation value of the online evaluation network, c(s,a) represents a combination of s and a, s represents a state vector, and a represents an action vector.