Patent ID: 11907663
Assignee: NAVER FRANCE
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 11:
12. A system comprising:
a natural language processing (NLP) model trained in a training domain and configured to perform natural language processing on an input dataset;
a means for:
calculating a domain shift metric based on the input dataset; and
calculating a predicted decrease in accuracy of the NLP model attributable to domain shift relative to the training domain based on the domain shift metric; and

a means for selectively triggering a retraining of the NLP model based on the predicted decrease in accuracy of the NLP model,
wherein one of:
(a) the domain shift metric is a proxy A domain (PAD) metric and the means for calculating the domain shift metric is configured to calculate the domain shift metric using the equation

PAD*=1−2ε(G*d(Gf(x))),, where PAD* is the domain shift metric, ε is a predetermined scalar value, and θf and θy are parameters of a first function Gf and a second function G*d learned by minimizing a loss function, and learnable parameters of Gf and G*d are learned parameters; and
(b) the domain shift metric is a reverse classification accuracy (RCA) metric and the means for calculating the domain shift metric is configured to calculate the domain shift metric using the equation:, R
    ⁢
    C
    ⁢
    A
    *
   
   =
   
    
     
      1
      
       U
       s
       ′
      
     
     ⁢
     
      
       ∑
        
      
      
       
        x
        i
       
       ,
       
        
         y
         i
        
        ∈
        
         U
         s
         ′
        
       
      
      
       m
       ′
      
     
     ⁢
     
      𝕀
      [
      
       
        y
        i
       
       =
       
        
         C
         ′
        
        (
        
         x
         i
        
        )
       
      
      ]
     
    
    -
    
     𝕀
     [
     
      
       y
       i
      
      =
      
       
        C
        ¯
       
       (
       
        x
        i
       
       )
      
     
     ]
    
   
  
  ,
 

where RCA* is the aomain snip metric, C′ is a classifier learned on out-of-domain data and C the classifier learned on held-out in-domain data, xi and yi are data-points, Us' represents data in the training domain.