Patent Publication Number: US-2022230416-A1

Title: Training of machine learning systems for image processing

Description:
FIELD 
     The present invention relates to an improved method with respect to a generalization for training a machine learning system for computer-based vision, to a training device, to a computer program and to a machine-readable memory medium. 
     BACKGROUND INFORMATION 
     In paper by Luketina, Jelena, et al. “Scalable gradient-based tuning of continuous regularization hyperparameters” International conference on machine learning. 2016 (retrievable online: https://arxiv.org/abs/1511.06727v1), a gradient-based approach is provided for the adaptation of hyperparameters, in which the hyperparameters are adapted in such a way that the model parameter gradients and thus the updates for the validation costs become more advantageous. 
     SUMMARY 
     Differences of the present invention over the aforementioned publication of the authors Luketina et al., include that no further data sets, in particular, validation data sets, become necessary for the training, and that also non-differentiable hyperparameters may be used. 
     The differences have the advantage that a training method is provided, which achieves a particularly high degree of generalization, gradients with respect to the non-differentiable hyperparameters being calculated in an efficient manner. Ultimately, a particularly generalizing and rapid training method is provided. Furthermore, the present invention is particularly data-efficient as a result of the cited differences, since no validation data are required for the training. These are usually costly to create. 
     In a first aspect, the present invention relates to a computer-implemented method for training a machine learning system. The machine learning system may be trained for computer-based vision, such as image processing, preferably image classifications or object detections or image segmentations. In accordance with an example embodiment of the present invention, the method includes the following steps: 
     initializing parameters θ of the machine learning system and of a metaparameter ϕ. The metaparameter may be an arbitrary other parameter of the machine learning system, which is not directly used by the machine learning system, in order to process a data point using the machine learning system. “Not directly” may be understood to mean that in the case of an inference, an output variable of the machine learning system is not ascertained as a function of this parameter of the machine learning system. The metaparameter may additionally or alternatively be a parameter, which characterizes a property or behavior of the training process. Thus, it is possible that the metaparameter characterizes, for example, a manipulation of training data, for example, augmentation, or is a drop-out rate for neurons or layers of the machine learning system or hyperparameters of a gradient descent method, which is applied for optimizing the parameters of the machine learning system, for example, a learning rate α. 
     This is followed by a loop via the subsequent steps, either with a predefined number of iterations t or until a convergence criterion with respect to training progress of the machine learning system is met. 
     The loop starts with a provision of a batch of training data points, in particular, which have been randomly selected from a plurality of training data points. The batch size totals, for example, at least 128 training data points, preferably at least 256 and particularly preferably at least 1024. It has been found namely that a large batch size additionally improves the convergence behavior, in particular, of metaparameters ϕ using the training method described herein. 
     This is followed by a manipulation of the provided training data points or of a training method for optimizing parameters θ of the machine learning system or of a structure of the machine learning system based on the metaparameter ϕ. It is noted that the training data points include in each case one training input data point, as a function of which the machine learning system ascertains its output variables, and in each case encompass training output data points (AKK. Label) assigned to the training data input points. It is further noted that the structure characterizes an architecture of the machine learning system, for example, a sequence of layers or a type of connection between the layers or the neurons that are to be used. 
     This is followed by an ascertainment of a cost function l as a function of instantaneous parameters θ of the machine learning system and of instantaneous metaparameter ϕ, the cost function characterizing a deviation between ascertained output variables of the machine learning system and training output variables. 
     The cost function is preferably ascertained individually for the training data points in the batch. 
     This is followed by an adaptation of instantaneous parameters θ t  as a function of a first gradient ∇ θ     t   l t , which is ascertained with respect to the instantaneous parameters via the ascertained cost function for the training data points, in particular, of the batch, and then if the step of manipulation has been carried out more than once (t&gt;1), the metaparameter is adapted as a function of a second gradient ∇ ϕ     t−1   l t , which has been ascertained with respect to the used metaparameter (immediately) preceding the loop iteration via the ascertained cost function of the instantaneous loop iteration. Thereupon, the loop is started again until the abort criterion of this loop is met. 
     As a result of the additional adaptation of the metaparameter as a function of the metaparameter of the previous loop iteration evaluated in the instantaneous loop iteration, the convergence of the training of the machine learning system is positively influenced, and an improved generalization is achieved. 
     In accordance with an example embodiment of the present invention, it is provided that the second gradient is ascertained with respect to the metaparameter used in the preceding step as a function of a scalar product ∇ θ     t   l t ·∇ ϕ     t−1   l t−1  between the first gradient with respect to the instantaneous parameter via the ascertained cost function and the second gradient with respect to the preceding metaparameter via the ascertained cost function of the immediately preceding loop cycle. This has the advantage that a more efficient estimation of the second gradient is carried out. 
     It is further provided that the second gradient is ascertained as a function of a scalar product r t,i  between first gradient ∇ θ l(θ t , ϕ t ) i  with respect to the instantaneous parameters of the machine learning system for training data points from the previous loop iteration and a gradient ∇ θ L(θt) with respect to the instantaneous parameter of the machine learning system of an averaged sum via the cost functions of the training data points of the selected batch, the scalar product serving as weighting of the second gradient. 
     This has the advantage that a particularly efficient estimation of the second gradient takes place and the method is also applicable for non-differentiable metaparameters, and in particular, for stochastic gradient descent methods. 
     It is further provided that the first gradient is ascertained with respect to the instantaneous parameters of the machine learning system using a gradient descent method, in particular, a stochastic gradient descent method (SGD). 
     The machine learning system may be an image classifier. The image classifier assigns an input image to one or to multiple classes of a predefined classification. The input images used may, for example, be images of nominally identified products manufactured in series. The image classifier may, for example, be trained to assign the input images of one or of multiple of at least two possible classes, which represent a quality assessment of the respective product. 
     The term “image” includes, in principle, each distribution of pieces of information situated in a two-dimensional or multi-dimensional grid. These pieces of information may, for example, be intensity values of image pixels, which may be recorded using an arbitrary imaging modality such as, for example, using an optical camera, using an infrared camera or using ultrasound. Arbitrary other data such as, for example, audio data, radar data or LIDAR data may, however, also be translated into images and then similarly classified. 
     In accordance with an example embodiment of the present invention, it is further provided that the trained machine learning system, which has been trained according to the first aspect, ascertains an output variable as a function of a detected sensor variable of a sensor, as a function of which a control variable may then be ascertained, for example, with the aid of a control unit. 
     The control unit may be used to control an actuator of a technical system. The technical system may, for example, be an at least semi-autonomous machine, an at least semi-autonomous vehicle, a robot, a tool, an operating machine or a flying object such as a drone. The input variable may, for example, be ascertained as a function of detected sensor data and provided to the machine learning system. The sensor data may be detected by a sensor such as, for example, by a camera of the technical system or may alternatively be received from an external source. 
     In further aspects, the present invention relates to a device and to a computer program, each of which is configured to carry out the above method, and to a machine-readable memory medium, on which this computer program is stored. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
       Specific embodiments of the present invention are explained below with reference to the figures. 
         FIG. 1  schematically shows a flowchart of one specific embodiment of the present invention. 
         FIG. 2  schematically shows a representation of temporal dependencies when ascertaining gradients. 
         FIG. 3  schematically shows one exemplary embodiment for controlling an at least semi-autonomous robot. 
         FIG. 4  schematically shows one exemplary embodiment for controlling a manufacturing system. 
         FIG. 5  schematically shows one exemplary embodiment for controlling an access system. 
         FIG. 6  schematically shows one exemplary embodiment for controlling a monitoring system. 
         FIG. 7  schematically shows one exemplary embodiment for controlling a personal assistant. 
         FIG. 8  schematically shows one exemplary embodiment for controlling a medical imaging system. 
     
    
    
     DETAILED DESCRIPTION OF EXAMPLE EMBODIMENTS 
     Machine learning systems, in particular, neural networks, are usually trained with the aid of a so-called gradient descent method. Gradient descent methods are characterized in that parameters, in particular, weights of the machine learning system, are iteratively updated in every training step as a function of a calculated gradient. In this case, the gradients are ascertained via a derivation of a cost function l, the cost function therefore being evaluated on training data and being derived via the parameters of the machine learning system. For the usual gradient descent method, cost function l(θ) is a function of parameters θ of the machine learning system, as well as of ascertained output variables of the machine learning system and provided target output variables, in particular, labels. 
     The present invention begins here in this training method with gradient descent methods and supplements this training method as explained below and as schematically represented in  FIG. 1 . 
     At start S 1  of the training method, metaparameters ϕ, in addition to parameters θ of the machine learning system, are also initialized. It is noted that here two successive metaparameters for the first training steps may be initialized: for example, ϕ 1 , ϕ 2 :=ϕ 1    
     Metaparameter ϕ parameterizes, for example, a data augmentation of the training data, for example, a distribution via distortions of the images or via rotations. 
     Cost function l(θ, ϕ) is also expanded in such a way that the cost function is now also a function of metaparameter ϕ. 
     Actual training step S 2  of the machine learning system, in which parameters θ are updated as a function of the gradient, remains unchanged. This means, a gradient ∇ θ l(θ, ϕ) with respect to metaparameter θ is calculated via cost function l(θ, ϕ), the cost function being evaluated using the instantaneous parameters of instantaneous iteration step t on the respective used training data: l(θ t , ϕ t ). 
     This is followed, in contrast to the usual training method, by an additional optimization step S 3 . In this step, metaparameter ϕ is optimized via an additional gradient descent method. For this purpose, a gradient ∇ ϕ     t−1    with respect to the metaparameter is calculated as a function of the cost function, for this purpose, the cost function being evaluated as a function of metaparameter ϕ t−1  used in immediately preceding training step t−1: l(θ t , ϕ t−1 ). This means, instantaneous metaparameter ϕ t  is updated as a function of the value of preceding metaparameter ϕ t−1 . 
     This adaptation of the metaparameter enabled in such a way between two training iterations t−1,t effectively means that immediately preceding metaparameter ϕ t−1  used on the instantaneously used training data, which have been used for ascertaining the cost function with the instantaneous parameters of the machine learning system, is evaluated. This generates a dependency between successive steps as opposed to the usual training method. Via this further dependency, additional optimization step S 3  for optimizing metaparameter ϕ t  results in the metaparameter being optimized in such a way that when used in the next training step, the metaparameter further minimizes the cost function. As a result, it may be said that a more rapid convergence due to the metaparameter is achieved by this newly introduced dependency, since the metaparameter advantageously influences the optimization of the cost function usually carried out. 
     Once metaparameter ϕ t+1  has then been set for the next training step S 4 : ϕ t−1 ζϕ t −β∇ ϕ     t−1   l, and the parameters of the machine learning system for the next training have also been set: θ t+1 ζθ t −α∇ θ     t   l, training steps S 2  and S 3  just described are carried out again, in particular, carried out multiple times in succession, until a predefined abort criterion is met. It is noted that parameters α,β represent weightings of the gradients. These parameters preferably have a value between 0≤α, β&lt;1. 
     It is noted that in the subsequent training steps before carrying out step S 2 , the training data are augmented in each case as a function of the set metaparameter. It has been found in experiments, however, that the augmentation of the training data has resulted in significant performance improvements only in every n-th training step. Preferably, n=2 is selected here. In one further exemplary embodiment, the gradient descent method for the machine learning system or a structure of the machine learning system may alternatively or in addition be changed after step S 2  as a function of the metaparameter. 
     If the training has been completed by a multiple sequential repetition of step S 2  and S 3 , step S 4  may follow. Herein, the machine learning system just trained is output. 
     In a subsequent step S 5 , the output machine learning system may then be used, for example, to control an actuator. In this case, the machine learning system is able to process data provided to it and the actuator is then activated as a function of the ascertained result of the machine learning system. 
     In one preferred exemplary embodiment, the machine learning system is trained using images in order to classify/segment objects in the images. 
     In order to further improve the training method, gradient ∇ ϕ     t−1   l is determined using the REINFORCE trick. This measure has the advantage that with this trick, non-differentiable metaparameters ϕ are optimizable, for example, because the latter are not constant or because the latter are characterized by a non-constant probability distribution p. 
     For example, distribution p may be a function of metaparameter ϕ and may output a value α i ˜p(⋅; ϕ) for training data point i. For example, α i  may characterize a value of a hyperparameter of the machine learning system (for example, dropout rate) or a training data point selection strategy. Distribution p(⋅; ϕ) may, for example, be a Softmax distribution, which is parameterized by ϕ. 
     For the measure just cited, a scalar product is used, which connects two successive batches of training data. The scalar product is ascertained as follows for the i-th training data point: 
         r   t,i = ∇ θ   l (θ t−1 , ϕ t ) i , ∇ θ   L (θ t )   (Equation 2):
 
     with l( ) i  being the cost function for the i-th training data point, in particular, from the respectively considered batch of training data points containing n-th training data point, and 
     
       
         
           
             
               L 
               ⁡ 
               ( 
               
                 θ 
                 t 
               
               ) 
             
             = 
             
               
                 1 
                 n 
               
               ⁢ 
               
                 
                   ∑ 
                   
                     j 
                     = 
                     1 
                   
                   n 
                 
                   
                 
                   l 
                   ⁡ 
                   ( 
                   
                     
                       θ 
                       t 
                     
                     , 
                     
                       ϕ 
                       t 
                     
                   
                   ) 
                 
               
             
           
         
       
     
     being the cost function of the entire batch of immediately following step t and the scalar product  , . 
     It is provided that scalar product r t,i  is to be interpreted as a reward and the REINFORCE trick is to be applied thereto. Thus, gradient ∇ ϕ     t−1   l t  may now be approximated as follows: 
     (Equation 3): 
     
       
         
           
             
               
                 ∇ 
                 
                   ϕ 
                   
                     t 
                     - 
                     1 
                   
                 
               
               
                 l 
                 t 
               
             
             ≈ 
             
               
                 ∑ 
                 
                   i 
                   = 
                   1 
                 
                 n 
               
               
                 
                   
                     r 
                     
                       t 
                       , 
                       i 
                     
                   
                   · 
                   
                     
                       ∇ 
                       
                         ϕ 
                         t 
                       
                     
                     log 
                   
                 
                 ⁢ 
                 
                   p 
                   ⁡ 
                   ( 
                   
                     
                       a 
                       i 
                     
                     ; 
                     
                       ϕ 
                       t 
                     
                   
                   ) 
                 
               
             
           
         
       
     
       FIG. 2  illustrates the temporal dependencies for ascertaining Equation 2 by way of example for successive steps t=1,2, . . . ,4. 
       FIG. 3  schematically shows an actuator  10  in its surroundings in interaction with a control system  40 . The surroundings are detected at preferably regular temporal intervals in a sensor  30 , in particular, in an imaging sensor such as a video sensor, which may also be provided by a plurality of sensors, for example, a stereo camera. Other imaging sensors are also possible such as, for example, radar, ultrasound or LIDAR. An infrared camera is also possible. Sensor signal S—or in the case of multiple sensors one sensor signal S each—of sensor  30  is transferred to control system  40 . Thus, control system  40  receives a sequence of sensor signals S. Control system  40  ascertains activation signals A therefrom, which are transferred to actuator  10 . 
     Control system  40  receives the sequence of sensor signals S of sensor  30  in an optional receiving unit  50 , which converts the sequence of sensor signals S into a sequence of input images x (alternatively, each sensor signal S may also be directly adopted as an input image x). Input image x may, for example, be a section or a further processing of sensor signal S. Input image x includes individual frames of a video recording. In other words, input image x is ascertained as a function of sensor signal S. The sequence of input images x is fed to a machine learning system, in the exemplary embodiment, the output machine learning system  60  from step S 4 . 
     Machine learning system network  60  ascertains output variables y from input images x. These output variables y may include, in particular, a classification and/or a semantic segmentation of input images x. Output variables y are fed to an optional forming unit, which ascertains therefrom activation signals A, which are fed to actuator  10  in order to activate actuator  10  accordingly. Output variable y includes pieces of information about objects detected by sensor  30 . 
     Actuator  10  receives control signals A, is activated accordingly and carries out a corresponding action. Actuator  10  in this case may include a (not necessarily structurally integrated) control logic, which ascertains from activation signal A a second activation signal, with which actuator  10  is then activated. 
     In one further specific embodiment, control system  40  includes sensor  30 . In still further specific embodiments, control system  40  also includes alternatively or in addition actuator  10 . 
     In further preferred specific embodiments, control system  40  includes a single or a plurality of processors  45  and at least one machine-readable memory medium  46 , on which the instructions are stored which, when they are carried out on processors  45 , then prompt control system  40  to carry out the method according to the present invention. 
     A display unit  10   a  alternatively or in addition to actuator  10  is provided in alternative specific embodiments. 
     In one further exemplary embodiment, control system  40  is used for controlling an at least semi-autonomous robot, here, an at least semi-autonomous motor vehicle  100 . Sensor  30  may, for example, be a video sensor situated preferably in motor vehicle  100 . 
     Machine learning system  60  is preferably configured for the purpose of safely identifying x objects from the input images. Machine learning system  60  may be a neural network. 
     Actuator  10  situated preferably in motor vehicle  100  may, for example, be a brake, a drive or a steering system of motor vehicle  100 . Activation signal A may then be ascertained in such a way that the actuator or actuators  10  is/are activated in such a way that motor vehicle  100  prevents, for example, a collision with objects reliably identified by artificial neural network  60 , in particular, when objects of particular classes, for example, pedestrians, are involved. 
     Alternatively, the at least semi-autonomous robot may also be another mobile robot (not depicted), for example, one which moves by flying, floating, diving or pacing. The mobile robot may, for example, also be an at least semi-autonomous lawn mower or an at least semi-autonomous cleaning robot. In these cases as well, activation signal A may be ascertained in such a way that the drive and/or the steering system of the mobile robot is/are activated in such a way that the at least semi-autonomous robot prevents, for example, a collision with objects identified by artificial neural network  60 . 
     Alternatively or in addition, display unit  10   a  may be activated with activation signal A and, for example, the ascertained safe areas may be displayed. It is also possible, for example, in a motor vehicle  100  including a non-automated steering system that display unit  10   a  is activated with activation signal A in such a way that it outputs a visual or acoustic warning signal when it is ascertained that motor vehicle  100  threatens to collide with one of the reliably identified objects. 
       FIG. 3  shows one exemplary embodiment, in which control system  40  is used for activating a manufacturing machine  11  of a manufacturing system  200  by activating an actuator  10  that controls this manufacturing machine  11 . Manufacturing machine  11  may, for example, be a machine for stamping, sawing, drilling and/or cutting. 
     Sensor  30  may then, for example, be a visual sensor, which detects, for example, properties of manufacturing products  12   a,    12   b.  It is possible that these manufacturing products  12   a,    12   b,  are movable. It is possible that actuator  10  controlling manufacturing machine  11  is activated as a function of an assignment of detected manufacturing products  12   a,    12   b,  so that manufacturing machine  11  correspondingly carries out a subsequent processing step of the correct one of manufacturing products  12   a,    12   b.  It is also possible that by identifying the correct properties of the same one of manufacturing products  12   a,    12   b  (i.e., without a misclassification), manufacturing machine  11  correspondingly adapts the same manufacturing step for a processing of a subsequent manufacturing product. 
       FIG. 5  shows one exemplary embodiment, in which control system  40  is used for controlling an access system  300 . Access system  300  may include a physical access control, for example, a door  401 . Video sensor  30  is configured to detect a person. This detected image may be interpreted with the aid of object identification system  60 . If multiple persons are detected simultaneously, the identity of the persons, for example, may be particularly reliably ascertained by an assignment of the persons (i.e., of the objects) relative to one another, for example, by an analysis of their movements. Actuator  10  may be a lock, which blocks or does not block the access control, as a function of activation signal A, for example, opens or does not open door  401 . For this purpose, activation signal A may be selected as a function of the interpretation of object identification system  60 , for example, as a function of the ascertained identity of the person. Instead of the physical access control, a logical access control may also be provided. 
       FIG. 6  shows one exemplary embodiment, in which control system  40  is used for controlling a monitoring system  400 . This exemplary embodiment differs from the exemplary embodiment shown in  FIG. 5  in that instead of actuator  10 , display unit  10   a  is provided, which is activated by control system  40 . For example, an identity of the objects recorded by video sensor  30  may be reliably ascertained by artificial neural network  60  in order, for example, to deduce therefrom which become suspicious, and activation signal A is then selected in such a way that this object is displayed in a color highlighted manner by display unit  10   a.    
       FIG. 7  shows one exemplary embodiment, in which control system  40  is used for controlling a personal assistant  250 . Sensor  30  is preferably a visual sensor, which receives images of a gesture of a user  249 . 
     Control system  40  ascertains as a function of the signals of sensor  30  an activation signal A of personal assistant  250 , for example, by the neural network carrying out a gesture recognition. This ascertained activation signal A is then conveyed to personal assistant  250  and the latter is thus activated accordingly. This ascertained activation signal A may, in particular, be selected in such a way that it corresponds to an assumed desired activation by user  249 . This assumed desired activation may be ascertained as a function of the gesture recognized by artificial neural network  60 . Control system  40  may then select activation signal A for conveyance to personal assistant  250  as a function of the assumed desired activation and/or may select activation signal A for conveyance to the personal assistant according to assumed desired activation  250 . 
     This corresponding activation may, for example, include that personal assistant  250  retrieves pieces of information from a database and reproduces them in an apprehensible manner for user  249 . 
     Instead of personal assistant  250 , a household appliance (not depicted), in particular, a washing machine, a stove, an oven, a microwave or a dishwasher may also be provided in order to be activated accordingly. 
       FIG. 8  shows one exemplary embodiment, in which control system  40  is used for controlling a medical imaging system  500 , for example, an MRI, an x-ray device or an ultrasound device. Sensor  30  may, for example, be provided in the form an imaging sensor, display unit  10   a  being activated by control system  40 . For example, it may be ascertained by neural network  60  whether an area recorded by the imaging sensor is conspicuous, and activation signal A may then be selected in such a way that this area is displayed in a color highlighted manner by display unit  10   a.