Patent ID: 12217139

DETAILED DESCRIPTION

InFIG.1the method of training a NN according to the first aspect is schematically depicted. The method comprises the steps receiving S1a training set, training S2the NN and optionally evaluating S3a robustness of the trained NN.

In the step of receiving S1a training set, exemplarily a training set T={X; Y} of training input data X=(X1, . . . , X1000) that are hand written images of numbers from 0 to 9 and corresponding ground truth data Y=(Y1, . . . , Y1000) that are the corresponding numbers for the ten classes (C=10) that resemble the possible numbers from 0 to 9 is received.

Here the C=10 classes are one-hot encoded in the following way:0 corresponds to 1 0 0 0 0 0 0 0 0 01 corresponds to 0 1 0 0 0 0 0 0 0 02 corresponds to 0 0 1 0 0 0 0 0 0 03 corresponds to 0 0 0 1 0 0 0 0 0 04 corresponds to 0 0 0 0 1 0 0 0 0 05 corresponds to 0 0 0 0 0 1 0 0 0 06 corresponds to 0 0 0 0 0 0 1 0 0 07 corresponds to 0 0 0 0 0 0 0 1 0 08 corresponds to 0 0 0 0 0 0 0 0 1 09 corresponds to 0 0 0 0 0 0 0 0 0 1

The step of training S2the NN comprises the iterative training steps selecting T1a training sub-set, generating T2current outputs, computing T3a categorical cross-entropy loss, computing T4a predictive entropy loss, computing T5a combined loss, checking T10whether the training converged, updating T11aweights and stopping T13the training.

In the training step of selecting T1a training sub-set, exemplarily a training sub-set B={XB; YB} of training input data XB=(XB,1, . . . , XB,100) that are 100 randomly selected images from the training set T and corresponding ground truth data YB=(YB,1, . . . , YB,100) that are the corresponding numbers is randomly selected from the training set T.

In the training step of generating T2current outputs, current outputs yijof the NN for the sub-set B are generated by forward propagating the exemplarily 100 images of the training input data XBof the training sub-set B in the NN.

In the training step of computing T3a categorical cross-entropy loss, a categorical cross-entropy loss LCCEfor the sub-set B is computed based on the current outputs yijand the corresponding ground truth data YBof the training sub-set B.

In the training step of computing T4a predictive entropy loss, a predictive entropy loss LSis computed. Thereto, non-misleading evidence (probability of one class is set to one and all other probabilities of the remaining classes are set to zero) is removed from the current outputs yij. The remaining current outputs yijare then distributed over the ten classes in order to calculate the predictive entropy loss LS. The predictive entropy loss LSis calculated by the following formula:

LS=∑i=1n∑j=1C-1c⁢log⁢(pi⁢j(1-yi⁢j)+yi⁢j)
where pijis the confidence associated with to the jth class of sample i and yijis its one-hot encoded label.

In the training step of computing T5a combined loss, a combined loss L is computed by adding to the categorical cross-entropy loss LCCEthe predictive entropy loss LSweighted with a predetermined first loss factor λSof exemplarily 0.5.

The combined loss L may be calculated by the following formula:
L=LCCE+λSLS

In the training step of checking T10whether the training converged, it is checked whether the training converged to a predefined lower limit for a convergence rate.

In the step of updating T11aweights, weights wlof the NN are updated based on the combined loss L and a predetermined training rate η of exemplarily 0.2 in case the training did not converge. Thereto, the combined loss L is back-propagated to the weights wlof the NN by changing the weights according to the combined loss L. It is determined how much each weight wlcontributes to the combined loss L and then each weight wlis changed proportionally to its contribution to the combined loss L. Each weight wlof the NN may be adjusted in the respective step of updating weights by the following formula:

wlu⁢p⁢d⁢a⁢t⁢e⁢d=wlcurrent-η⁢∂L∂wlcurrent
where wlcurrentis the current value of the lth weight of the NN, wlupdatedis the updated or adjusted value of the lth weight of the NN and

∂L∂wlc⁢u⁢r⁢r⁢e⁢n⁢t
is the partial derivative of the combined loss L with respect to the lth weight of the NN.

In the step of stopping T13the training, the training of the NN is stopped in case the training converged.

After the training has converged and the iteration of the step of training S2the NN is aborted, the NN is completely trained. Optionally, the trained NN can be evaluated in the optional step of evaluating S3a robustness of the trained NN that comprises the sub-steps providing U1the trained NN with a first set, generating U2perturbed outputs, computing U3the ECEs for the respective generated outputs, providing U4the trained NN with at least one further set of multiple perturbed input datasets, generating U5perturbed outputs of the NN based on the provided at least one further set of multiple perturbed input datasets, computing U6the ECEs for the respective generated outputs and calculating U7micro-averaged ECEs.

In the sub-step of providing U1the trained NN with a first set, the NN is provided with a first set of multiple perturbed input datasets. The multiple perturbed input datasets have been perturbed with a perturbation of a first perturbation-type. Thereby, each input dataset has been perturbed with a different predefined perturbation level εB.

In the sub-step of generating U2perturbed outputs of the NN, perturbed outputs of the NN are generated based on the provided first set multiple perturbed input datasets.

In the sub-step of computing U3the ECEs for the respective generated outputs, the ECEs are computed for the respective generated outputs. The expected calibration errors ECEs are calculated for the respective generated outputs in the same way as described above for the outputs during the step of training S2the NN.

In the sub-step of providing U4the trained NN with at least one further set of multiple perturbed input datasets, the trained NN is provided with at least one further set of multiple perturbed input datasets. The multiple perturbed input datasets have been perturbed with a perturbation of at least one further perturbation-type. Thereby, each input dataset has been perturbed with a different predefined perturbation level εB.

In the sub-step of generating U5perturbed outputs of the NN based on the provided at least one further set of multiple perturbed input datasets, perturbed outputs of the NN are generated based on the provided at least one further set of multiple perturbed input datasets.

In the sub-step of computing U6the ECEs for the respective generated outputs, the ECEs are computed for the respective generated outputs. The expected calibration errors ECEs are calculated for the respective generated outputs in the same way as described above for the outputs during the step of training S2the NN.

In the sub-step of calculating U7micro-averaged ECEs, micro-averaged ECEs are calculated across the generated outputs of each provided set of multiple perturbed input datasets by calculating the average of the respective ECEs. The micro-averaged ECEs enable a detailed assessment of the trustworthiness or rather robustness of the predictions of the trained NN.

InFIG.2the method of training a NN according to the second aspect is schematically depicted. The method according to the second aspect and as depicted inFIG.2is similar to the method according to the first aspect and as depicted inFIG.1. Therefore, only differences between the two will be described in the following. The step of training S2the NN of the method ofFIG.2comprises the training steps sampling T6a perturbation level, generating T7an adversarial set, generating T8perturbed outputs, computing T9an adversarial calibration loss first time updating T11bweights and second time updating T12the weights instead of the steps computing T4a predictive entropy loss, computing T5a combined loss and updating T11aweights of the step of training S2the NN of the method ofFIG.1.

In the training step of sampling T6a perturbation level, a perturbation level εBis randomly sampled from the exemplary set ε that includes the values 0, 0.05, 0.1, 0.15, 0.2, 0.25, 0.3, 0.35, 0.4, 0.45.

In the training step of generating T7an adversarial set, an adversarial set Badvof adversarial input data Xadvis generated. Thereto, a perturbation is randomly selected from a predefined set of perturbations that exemplarily includes the image-based perturbation types left rotation, right rotation, shift in x direction, shift in y direction, xy shift, shear, zoom in x direction, zoom in y direction and xy zoom. Then the selected perturbation is weighted with the sampled perturbation level εBand applied to the training input data XBof the training sub-set B. Thereby, the training input data that is to be perturbed (i.e. respective images of a number) is transformed into the adversarial input data Xadv(perturbed handwritten images of a number) of the adversarial set Badv.

In the training step of generating T8perturbed outputs, perturbed outputs of the NN for the adversarial set Badvare generated by forward propagating the exemplarily 100 perturbed images as the adversarial input data Xadvof the adversarial set Badvin the NN.

In the training step of computing T9an adversarial calibration loss, an adversarial calibration loss Ladvis computed as the Euclidian norm (L2norm) of an expected calibration error ECE. Thereto, the perturbed outputs are grouped into exemplarily M=10 bins, where each bin covers a confidence interval Imof size 0.1:first bin confidence interval Imbetween 0 and 0.1second bin confidence interval Imbetween 0.1 and 0.2third bin confidence interval Imbetween 0.2 and 0.3fourth bin confidence interval Imbetween 0.3 and 0.4fifth bin confidence interval Imbetween 0.4 and 0.5sixth bin confidence interval Imbetween 0.5 and 0.6seventh bin confidence interval Imbetween 0.6 and 0.7eighth bin confidence interval Imbetween 0.7 and 0.8ninth bin confidence interval Imbetween 0.8 and 0.9tenth bin confidence interval Imbetween 0.9 and 1

The expected calibration error ECE is calculated by the following formula:

E⁢C⁢E=∑m=1M❘"\[LeftBracketingBar]"Bm❘"\[RightBracketingBar]"n⁢❘"\[LeftBracketingBar]"acc⁡(Bm)-conf⁡(Bm)❘"\[RightBracketingBar]"
where Bmis a set of indices of samples whose prediction confidence falls into the associated confidence interval Im, conf(Bm) is the average confidence associated to Bmand acc(Bm) is the accuracy associated to Bm.

The adversarial calibration loss Ladvis calculated by the following formula:

La⁢d⁢v=ECE2=∑m=1M❘"\[LeftBracketingBar]"Bm❘"\[RightBracketingBar]"n⁢❘"\[LeftBracketingBar]"acc⁡(Bm)-conf⁡(Bm)❘"\[RightBracketingBar]"2

In the step of first time updating T11bweights, weights wlof the NN are updated first time based on the categorical cross-entropy loss LCCEand the predetermined training rate η in case the training did not converge.

In the step of second time updating T12weights, weights of the NN are updated second time based on the adversarial calibration loss Ladvweighted with a predetermined second loss factor λadvof exemplarily 0.5 and the predetermined training rate η, in case the training did not converge.

InFIG.3the method of training a NN according to the third aspect is schematically depicted. The method according to the third aspect and as depicted inFIG.3is similar to the methods according to the first and second aspects and as depicted inFIGS.1and2. Therefore, only differences between the three will be mentioned in the following. The step of training S2the NN of the method ofFIG.3comprises the training steps T1to T3, T10and T13of the step of training S2the NN of the methods ofFIGS.1and2. Further, the step of training S2the NN of the method ofFIG.3comprises the training steps T4and T5of the step of training S2the NN of the method ofFIG.1. Also the training step of updating T11aweights of the step of training S2the NN of the method ofFIG.1is comprised by the step of training S2the NN of the method ofFIG.3as the training step first time updating T11aweights. Additionally, the step of training S2the NN of the method ofFIG.3comprises the training steps T6to T9and second time updating T12of the step of training S2the NN of the method ofFIG.2. Consequently, the method ofFIG.3combines in its step of training S2the training steps of the steps of training S2of the methods ofFIGS.1and2.

The computer program according to the fourth aspect may comprise instructions which, when the program is executed by a computer, cause the computer to carry out the steps S1to S3including the respective training steps T1to T3and sub-steps U1to U7of the method according to any of the first, second and third aspect and as depicted inFIGS.1to3.

InFIG.4an embodiment of the computer-readable medium20according to the fifth aspect is schematically depicted.

Here, exemplarily a computer-readable storage disc20like a Compact Disc (CD), Digital Video Disc (DVD), High Definition DVD (HD DVD) or Blu-ray Disc (BD) has stored thereon the computer program according to the fourth aspect and as schematically shown inFIGS.1to3. However, the computer-readable medium may also be a data storage like a magnetic storage/memory (e.g. magnetic-core memory, magnetic tape, magnetic card, magnet strip, magnet bubble storage, drum storage, hard disc drive, floppy disc or removable storage), an optical storage/memory (e.g. holographic memory, optical tape, Tesa tape, Laserdisc, Phasewriter (Phasewriter Dual, PD) or Ultra Density Optical (UDO)), a magneto-optical storage/memory (e.g. MiniDisc or Magneto-Optical Disk (MO-Disk)), a volatile semiconductor/solid state memory (e.g. Random Access Memory (RAM), Dynamic RAM (DRAM) or Static RAM (SRAM)), a non-volatile semiconductor/solid state memory (e.g. Read Only Memory (ROM), Programmable ROM (PROM), Erasable PROM (EPROM), Electrically EPROM (EEPROM), Flash-EEPROM (e.g. USB-Stick), Ferroelectric RAM (FRAM), Magnetoresistive RAM (MRAM) or Phase-change RAM).

InFIG.5an embodiment of the data processing system30according to the sixth aspect is schematically depicted.

The data processing system30may be a personal computer (PC), a laptop, a tablet, a server, a distributed system (e.g. cloud system) and the like. The data processing system30comprises a central processing unit (CPU)31, a memory having a random access memory (RAM)32and a non-volatile memory (MEM, e.g. hard disk)33, a human interface device (HID, e.g. keyboard, mouse, touchscreen etc.)34and an output device (MON, e.g. monitor, printer, speaker, etc.)35. The CPU31, RAM32, HID34and MON35are communicatively connected via a data bus. The RAM32and MEM33are communicatively connected via another data bus. The computer program according to the fourth aspect and schematically depicted inFIGS.1to3can be loaded into the RAM32from the MEM33or another computer-readable medium20. According to the computer program the CPU31executes the steps S1to S3including the respective training steps T1to T3and sub-steps U1to U7of the computer-implemented method according to any of the first, second and third aspect and as schematically depicted inFIGS.1to3. The execution can be initiated and controlled by a user via the HID34. The status and/or result of the executed computer program may be indicated to the user by the MON35. The result of the executed computer program may be permanently stored on the non-volatile MEM33or another computer-readable medium.

In particular, the CPU31and RAM32for executing the computer program may comprise several CPUs31and several RAMs32for example in a computation cluster or a cloud system. The HID34and MON35for controlling execution of the computer program may be comprised by a different data processing system like a terminal communicatively connected to the data processing system30(e.g. cloud system).

InFIG.6an embodiment of the data processing system40according to the seventh aspect is schematically depicted.

The data processing system40may be a personal computer (PC), a laptop, a tablet, a server, a distributed system (e.g. cloud system) and the like. The data processing system40comprises a central processing unit (CPU)41, a memory having a random access memory (RAM)42and a non-volatile memory (MEM, e.g. hard disk)43, a human interface device (HID, e.g. keyboard, mouse, touchscreen etc.)44and an output device (MON, e.g. monitor, printer, speaker, etc.)45. The CPU41, RAM42, HID44and MON45are communicatively connected via a data bus. The RAM42and MEM43are communicatively connected via another data bus. A trained NN that was trained with any of the methods according to the first, second and third aspect and as schematically depicted inFIGS.1to3can be loaded into the RAM32from the MEM33or another computer-readable medium20. Accordingly, the trained NN is implemented on the data processing system40and the CPU41can execute predictions based on provided input data and the trained weights of the trained NN. The execution can be initiated and controlled by a user via the HID44. The status and/or result of the executed prediction by the trained NN may be indicated to the user by the MON45. The result may be permanently stored on the non-volatile MEM43or another computer-readable medium.

In particular, the CPU41and RAM42for implementing the trained NN may comprise several CPUs41and several RAMs42for example in a computation cluster or a cloud system. The HID44and MON45for controlling execution of the prediction by the NN may be comprised by a different data processing system like a terminal communicatively connected to the data processing system40(e.g. cloud system).

The first embodiment refers to the web service details of the proposed method. A transformation method for a trained AI model is described as part of a web service, which is accessible via a webpage. Just for example, the trained AI model might be generated based on a trained NN that was trained with any of the methods according to the aspects described above. The method is illustrated withFIG.7, which is described in more detail in the following.

A user100for example accesses a webpage. The front end20of the webpage is for example realized via a web app, e.g. Elastic Beanstalk, which fetches an AWS EC2instantiation. The web app askes the user100to upload a trained neural network, for example with weights exported in a pre-specified format, e.g. hdf5. In addition, it askes to provide a representative validation data set. These user data D10is provided by the user100via the user interface and front end20, which forwards it to a cloud platform200.

According to the first embodiment, there is an additional authentication step realized with an authentication infrastructure21, so that the user100is reliably identified and authenticated. For example, an e-mail address with password is used to register before the user100can upload the data. The user data D10is therefore enhanced with user authentication information.

Within the cloud platform200, a memory201, e.g. a memory provided by AWS S3, a web-based Cloud Storage Service, saves the uploaded user data D10. An event within the AWS S3, which indicates the request for a transformation, is sent as notification D21to a so-called Lambda function as trigger202for the actual transformation method.

The Lambda function is configured so that it calls suitable transformation methods, which are arranged in containers and called depending on the user data D10. For example, the user data D10also comprises an information about which transformation method is desired, e.g. a post-processing or a re-training, or about requirements in terms of delivery time etc.

The trigger202, in this embodiment the Lambda function, starts for example a data processing-Engine for containers, e.g. AWS Fargate, with a trigger call S32and provides user data location. The AI model transformation then is performed in the backend203, for example with a container-orchestration service like AWS ECS, where the Fargate container is executed.

As a result of this transformation, the trustworthy AI model is generated with the method explained in detail in the various embodiments above.

The transformation of the AI model as web service is enabled due to the characteristics of the transformation method, in particular the dependence on the generated generic samples, where only a validation data set is necessary as input, and moreover the usage of a calibration optimization, which does not affect the architecture or structure of the AI model, so that a deployment of the trustworthy AI model on user side is guaranteed.

The trustworthy AI model as well as corresponding process protocols are saved as transformation result D20in the memory201and finally provided to the user100. This happens for example via an AWS S3event that sends the transformation result D20directly to the e-mail address of the authenticated user100, which has been saved in the memory201.

FIG.8shows a schematic flow chart diagram of a method, with the steps of providing S1the trained artificial intelligence model via a user interface of a webservice platform, providing S2a validation data set, which is based on training data of the trained artificial intelligence model,

generating S3generic samples by a computing component of the webservice platform based on the validation data set, transforming S4the trained artificial intelligence model by optimizing a calibration based on the generic samples.

The steps S1and S2of providing the trained artificial intelligence model and the validation data set might be performed decoupled of each other and in a flexible order, as indicated by the exchangeable reference signs inFIG.8. In other embodiments, they can be combined and performed within one method step or simultaneously, as indicated by the dotted arrow.

The step of transforming the trained artificial intelligence model is explained in more detail.

A cross entropy loss term is defined as

ℒj=∑j=1K-yi⁢j⁢log⁡(pi⁢j)
and a uniform loss term as

ℒuniformi=λt⁢∑j=1K-1K⁢log⁡(pi⁢j(1-yi⁢j)+yi⁢j)
with p being the prediction, y a label, λtan annealing coefficient, t an index of a training step, i an index of a sample, j an index of a class and K the number of classes. This term encourages the model towards a uniformly distributed softmax output in case of uncertainty.

A calibration loss term is defined as

ℒECEi=ECCgeni22ECE=∑m=1M❘"\[LeftBracketingBar]"Bm❘"\[RightBracketingBar]"n⁢❘"\[LeftBracketingBar]"acc⁡(Bm)-conf⁡(Bm)❘"\[RightBracketingBar]"
with ECCgenbeing the ECE on the generic samples, with Bmbeing the set of indices of samples whose prediction confidence falls into its associated interval Im. conf (Bm) and acc(Bm) are the average confidence and accuracy associated to Bmrespectively, n the number of samples in the dataset, M a number of bins and m an index of bins. With this calibration loss term, which could be seen as generic calibration loss term, the technical robustness of the AI model for input around an epsilon-neighborhood of the training samples is increased.

These three loss terms are combined for retraining the AI model. As a result of the re-training, the trained AI model has been transformed into a trustworthy AI model, which yields confidence scores matching the accuracy for samples representing a domain shift, in particular gradually shifting away from samples of the validation data set.

FIG.9illustrates a result of an AI model according to the state of the art, showing a schematic diagram of an output of a classifier. On the vertical axis30of the diagram, the confidence scores are shown, on the horizontal axis40, the discrete input data is shown. From left to right, the quality of the input data, here in form of a handwritten figure “6”, decreased due to a distortion in one direction. Those kinds of distortion reflect effects on data input in real life scenarios, which impede accurate prediction of the classifier.

The classifier is trained to assign one of 10 classes to the input data, corresponding toFIGS.0-9.

As one can see, starting at about a range of perturbation of 50, which represents a flexible and arbitrary scale to group distortions and might be a value of epsilon, which has been introduced in the description above, the classifier starts to predict a wrong classification result, “2” in this case, but with a high confidence score over 60%, even increasing with increasing epsilon up to almost 100%. This illustrates an over-confident classifier when data with domain shift is to be classified.

According to the third embodiment, which is illustrated inFIG.10, the same classifier is used, but the underlying AI classification model has been transformed into a trustworthy AI model with the following method.

A set of samples are generated which cover the entire spectrum from in-domain samples to truly out-of-domain samples in a continuous and representative manner. According to this, the fast gradient sign method (FGSM) is used on the basis of the validation data set with sample pairs to generate perturbated samples, with varying perturbation strength. More specifically, for each sample pair in the validation data set, the derivative of the loss is determined with respect to each input dimension and the sign of this gradient is recorded. If the gradient cannot be determined analytically (e.g. for decision trees), it can be resorted to a 0th-order approximation and the gradient can be determined using finite differences. Then, noise $epsilon$ is added to each input dimension in the direction of its gradient. For each sample pair, a noise level can be selected at random, such that the generic data set comprises representative samples from the entire spectrum of domain drift, as shown in the pseudo code of algorithm 1 and explanation.

\begin{algorithm}[H]\caption{PORTAL with trained neural network $f(x)$, a set of perturbation levels$\mathcal{E}=\{ 0.001,0.002,0.004,0.008,0.016,0.032,0.064,0.128,0.256,0.512\}$ ,complexity parameter $\zeta=1$, validation set $(X, Y)$, and empty perturbed validationset $(X_\mathcal{E},Y_\mathcal{E}, Z_\mathcal{E}, Z{circumflex over ( )}r_\mathcal{E})$.}\label{alg1}\begin{algorithmic}[1]\For{(x, y) in (X,Y)}\For{$\epsilon\; \mathrm{in}\; \mathcal{E}$}\State Generate generic sample $x_\epsilon$ using $\epsilon_\zeta=\epsilon/\zeta$\State Use neural network $f(x_\epsilon)$ to compute unnormalized logits$\bm{z_\epsilon}$ and logit range $z_\epsilon{circumflex over ( )}r$\State Add $(x_\epsilon, y, \bm{z_\epsilon}, z_\epsilon{circumflex over ( )}r)$ to$(X_\mathcal{E},Y_\mathcal{E}, Z_\mathcal{E}, Z{circumflex over ( )}r_\mathcal{E})$\EndFor\EndFor\State Initialize $\bm{\theta}$\State Optimize $\bm{\theta}$ using Nelder-Mead optimizer for log-likelihood ofperturbed validation set $\mathcal{L}(\bm{\theta}) = − \sum_{i=1}{circumflex over ( )}{N_mathcal{E}} y_i\log \hat{Q}_i(\bm{ \theta}) = − \sum_{i=1}{circumflex over ( )}{N_\mathcal{E}} y_i \log\sigma_{SM}(\mathbf{z}_i/T(z{circumflex over ( )}r_i;\bm{\theta}))$\end{algorithmic}\end{algorithm}

Algorithm 1 Generation of generic data set Vgbased on validation V, consisting of a collection of labelled samples {(x,y)}, with x being model inputs an y model outputs. N denotes the number of samples in V, ε={0,0.05,0.1,0.15,0.2,0.025,0.3,0.35,0.4,0.45} the set of perturbation levels.

Require: Validation set V and empty generic data set Vg1: for i in 1:N do2: Read sample pair (xi, yi) from V3: Randomly sample ϵifrom ε4: Generate generic sample pair (xg, y) using the FGSM method based on ϵi5: Add (xg, y) to Vg6: end forxgdenotes a generic input generated from x using the FGSM method.

According to an alternative embodiment, the formulation of Algorithm 1 differs in that not only one generic sample is generated per sample pair; but instead FGSM is applied for all available epsilons. Thereby the size of the generic data set can be significantly increased by the size of the set of epsilons. In other words, different perturbation strategies can be used e.g. based on image perturbation. An advantage is that the method according to embodiments of the invention can be applied on black box models where it is not possible to compute the gradient.

Next, a strictly monotonic parameterized function is used to transform the unnormalized logits of the classifier. For example, Platt scaling, temperature scaling, other parameterizations of a monotonic function, or non-parametric alternatives can be used. In an embodiment according to the following equation a novel parameterization is used, which adds additional flexibility to known functions by introducing range-adaptive temperature scaling. While in classical temperature scaling a single temperature is used to transform logits across the entire spectrum of outputs, a range-specific temperature is used for different value ranges.

The following is a formula of an embodiment:

T⁡(zr;θ)=exp_id⁢(θ1(zr+θ2)θ3+θ0)(5)
with θ=[θ0, . . . θ3] parameterizing the temperature T (zr; θ) and zr=max(z)−min(z) being the range of an unnormalized logits tuple z. θ0can be interpreted as an asymptotic dependency on zr. The following function an be used exp_id: x−>{x+1, x>0; exp(x), else} to ensure a positive output. This parameterized temperature is then used to obtain calibrated confidence scores {circumflex over (Q)}ιfor sample i based on unnormalized logits:

Q^ι=max⁢σSM(zi/T⁡(zir;θ))(c)(6)c∫c,T:ℝ→ℝ∫c,T:x→{x❘"\[LeftBracketingBar]"T1❘"\[RightBracketingBar]"if⁢x<C1x-Ch-1❘"\[LeftBracketingBar]"Th❘"\[RightBracketingBar]"+∑l=1h-1⁢Cl-Cl-1❘"\[LeftBracketingBar]"Tl❘"\[RightBracketingBar]"if⁢Ch-1≤x<Ch,h=2,…,Hwith⁢H:=dim⁡(T)c0:=0,cH:=∞

Sigma_SM denotes the softmax function. The parameters of the function (theta) are then determined by optimizing a calibration metric based on the generic data set. Calibration metrics can be the log likelihood, the Brier score or the expected calibration error, see also Algorithm 2.

Algorithm 2 Fit parameterized post-processing model u=∫(z,T), where ∫ is a strictly monotonic function parameterized by parameters T and maps the unnormalized logits z=C(x) of a classifier C to transformed (still unnormalized) logits u. Let g denote a calibration metric that is used to compute a scalar calibration measure w based on a set of logits along with ground truth labels.

Require: Generic set Vg(from algorithm 1), function ∫ with initial parameters T, calibration metric g.1: repeat2: Read sample pairs {(xg,y)} from Vg. Let Y be the set of all labels.3: Compute post-processed logits u=∫(z,T) for all z=C(xadv), comprising set U.4: Perform optimization step and update T to optimize g(U,Y)5: until Optimisation converged6: return Optimized T

In an alternative embodiment of a blackbox classifier where logits are not available, Algorithm 2 can be adapted such that unnormalized logits are generated by computing z=log(C(x)). Optimizers can advantageously be selected according to the form of the metric (e.g. Nelder Mead for piecewise temperature scaling) in a flexible manner.

After the trained AI classification model has been transformed with the method described, the same input data as used in connection with the state of the art model fromFIG.9is now used to be classified by the trustworthy AI classification model. As one can see fromFIG.10, up to a perturbation level of 20, the right class “6” is predicted with the same high confidence as predicted with the prior art method. The confidence level decreases slightly from almost 100% to about 80% for a perturbation level of 30. For 40, there is already a confidence level for the predicted class of only around 50%, so that there is a clear indication, that the prediction is subject to uncertainty. Up from epsilon 50, the trustworthy AI classifier gives a prediction rate of about 10% for essentially all classes na. This translates to a result “no classification possible with sufficient certainty”, so that all of the ten classes might be the correct prediction, leading to a confidence score of 1/10 or 10%.

The transformed trustworthy AI model can in an advantageous manner be used be a non-expert user in an application for AI-based classifying also in safety-critical applications, where a timely recognition of decreasing accuracy for prediction also for input data under domain drift is key and over-confident estimates have to be avoided.

Although specific embodiments have been illustrated and described herein, it will be appreciated by those of ordinary skill in the art that a variety of alternate and/or equivalent implementations exist. It should be appreciated that the exemplary embodiment or exemplary embodiments are only examples, and are not intended to limit the scope, applicability, or configuration in any way. Rather, the foregoing summary and detailed description will provide those skilled in the art with a convenient road map for implementing at least one exemplary embodiment, it being understood that various changes may be made in the function and arrangement of elements described in an exemplary embodiment without departing from the scope. Generally, this application is intended to cover any adaptations or variations of the specific embodiments discussed herein.

In the foregoing detailed description, various features are grouped together in one or more examples for the purpose of streamlining the disclosure. It is understood that the above description is intended to be illustrative, and not restrictive. It is intended to cover all alternatives, modifications and equivalents as may be included within the scope of embodiments of the invention. Many other examples will be apparent to one skilled in the art upon reviewing the above specification.

Specific nomenclature used in the foregoing specification is used to provide a thorough understanding of embodiments of the invention. However, it will be apparent to one skilled in the art in light of the specification provided herein that the specific details are not required in order to practice embodiments of the invention. Thus, the foregoing descriptions of specific embodiments of the present invention are presented for purposes of illustration and description. They are not intended to be exhaustive or to limit embodiments of the invention to the precise forms disclosed; obviously many modifications and variations are possible in view of the above teachings. The embodiments were chosen and described in order to best explain the principles of embodiments of the invention and its practical applications, to thereby enable others skilled in the art to best utilize embodiments of the invention and various embodiments with various modifications as are suited to the particular use contemplated. Throughout the specification, the terms “including” and “in which” are used as the plain-English equivalents of the respective terms “comprising” and “wherein,” respectively. Moreover, the terms “first,” “second,” and “third,” etc., are used merely as labels, and are not intended to impose numerical requirements on or to establish a certain ranking of importance of their objects. In the context of the present description the conjunction “or” is to be understood as including (“and/or”) and not exclusive (“either . . . or”).

Although the present invention has been disclosed in the form of preferred embodiments and variations thereon, it will be understood that numerous additional modifications and variations could be made thereto without departing from the scope of the invention.

For the sake of clarity, it is to be understood that the use of “a” or “an” throughout this application does not exclude a plurality, and “comprising” does not exclude other steps or elements.