Patent ID: 11934969
Assignee: INTERNATIONAL BUSINESS MACHINES CORPORATION
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 8:
9. A computer program product comprising a computer readable storage medium having a computer readable program stored therein, wherein the computer readable program, when executed on a computing device, causes the computing device to implement a bias identification engine that identifies bias in the operation of a trained cognitive computing system, the computer readable program causing the computing device to:
configure a bias risk annotator of the bias identification engine to identify a plurality of bias triggers in inputs to the trained cognitive computing system and outputs of the trained cognitive computing system based on a bias risk trigger data structure that specifies terms or phrases that are associated with a bias, wherein the trained cognitive computing system is an artificial intelligence computing system trained by a machine learning computer operation to generate the outputs based on the inputs to the trained cognitive computing system;
execute the trained cognitive computing system on an annotated input to the trained cognitive computing system and generate an annotated output, by the trained cognitive computing system, based on a first corpus of electronic content;
process, by the bias risk annotator, the annotated input and the annotated output to determine whether the annotated input or annotated output comprises a portion of content that contains a bias trigger in the plurality of bias triggers;
in response to at least one of the annotated input or annotated output comprising a portion of content containing a bias trigger in the plurality of bias triggers, transmit, by a bias notification engine of the bias identification engine, a notification to an administrator computing device, wherein the notification specifies the presence of bias in the operation of the trained cognitive computing system; and
in response to at least one of the annotated input or the annotated output comprising a portion of content containing a bias trigger:
testing, by a bias source identification engine of the bias identification engine, the trained cognitive computing system using a set of bias test questions and a bias test corpus having a predetermined balance of biased results and unbiased results, to determine whether the trained cognitive computing system returns biased results from the bias test corpus at a higher amount than the trained cognitive computing system returns unbiased results from the bias test corpus;
determining, by the bias source identification engine, based on results of the testing of the trained cognitive computing system, whether a source of the bias in the operation of the trained cognitive computing system is training of the trained cognitive computing system or bias present in the first corpus used by the trained cognitive computing system to generate the annotated output, based on results of the testing; and
generating, by the bias source identification engine, an output indicating the source of the bias.