Patent ID: 11907663
Assignee: NAVER FRANCE
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 7:
8. A system comprising:
a training module configured to:
train, using a training dataset in a training domain, a machine learning (ML) model to perform processing on an input dataset;
determine properties of a domain shift metric for the training dataset in the training domain;

an accuracy module configured to:
calculate the domain shift metric based on the input dataset; and
calculate a predicted decrease in accuracy of the ML model attributable to domain shift relative to the training domain based on the domain shift metric using the properties of the domain shift metric for the training dataset; and

a retraining module configured to selectively trigger a retraining of the ML model based on the predicted decrease in accuracy of the ML model, wherein one of:
(a) the domain shift metric is a proxy A domain (PAD) metric and the accuracy module is configured to calculate the domain shift metric using the equation

PAD*=1−2ε(G*d(Gf(x))), where PAD* is the domain shift metric, ε is a predetermined scalar value, and θf and θy are parameters of a first function Gf and a second function G*d learned by minimizing a loss function, and learnable parameters of Gf and G*d are learned parameters; and
(b) the domain shift metric is a reverse classification accuracy (RCA) metric and the accuracy module is configured to calculate the domain shift metric using the equation:, R
    ⁢
    C
    ⁢
    A
    *
   
   =
   
    
     
      1
      
       U
       s
       ′
      
     
     ⁢
     
      
       ∑
        
      
      
       
        x
        i
       
       ,
       
        
         y
         i
        
        ∈
        
         U
         s
         ′
        
       
      
      
       m
       ′
      
     
     ⁢
     
      𝕀
      [
      
       
        y
        i
       
       =
       
        
         C
         ′
        
        (
        
         x
         i
        
        )
       
      
      ]
     
    
    -
    
     𝕀
     [
     
      
       y
       i
      
      =
      
       
        C
        ¯
       
       (
       
        x
        i
       
       )
      
     
     ]
    
   
  
  ,, where RCA* is the domain shift metric, C′ is a classifier learned on out-of-domain data and C the classifier learned on held-out in-domain data, xi and vi are data-points, Us' represents data in the training domain.