Patent ID: 11900236
Assignee: UMNAI LIMITED
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 19:
20. A method for providing an interpretable neural network, comprising:
inputting a set of training data to a black-box predictor model;
recording an output of the black-box predictor model corresponding to the set of training data;
with a feature attribution layer, receiving, from a plurality of relevance estimators, a plurality of coefficients associated with one or more features of the set of training data input to the black-box predictor model, and calculating one or more attribution values of the coefficients;
with an aggregation layer, aggregating the output and forming one or more hierarchical partitions based on the aggregated output, the hierarchical partitions comprising at least one pair of overlapping hierarchical partitions and at least one pair of non-overlapping hierarchical partitions, wherein the aggregation layer is configured to aggregate the output by merging at least one pair of hierarchical partitions and splitting at least one hierarchical partition, and wherein aggregating the output further comprises aggregating the plurality of relevance estimators;
applying at least one linear or non-linear transformation to the partitions to form one or more local models;
constructing rules based on the local models; and
aggregating the rules to form a global interpretable model configured to simultaneously compute, in a single feed-forward step extending from the input to the output without iteration of intervening steps, an answer and an explanation;
wherein the answer comprises a response provided by the interpretable neural network to the input, and wherein the explanation comprises a set or sequence of decisions performed from the input to the output as part of the single feed-forward step.