PREDICTING FUTURE POSSIBILITY OF BIAS IN AN ARTIFICIAL INTELLIGENCE MODEL

One or more systems, devices, computer program products and/or computer-implemented methods of use provided herein relate to predicting bias in an artificial intelligence (AI) model. A system can comprise a memory configured to store computer executable components; and a processor configured to execute the computer executable components stored in the memory, wherein the computer executable components can comprise a data generation component that can generate a set of structured test data to test likelihood of an AI model generating biased outputs, based on analysis of payload logging data; and an alerting component that can alert a user of likelihood that the AI model will generate the biased outputs, wherein the alerting component can generate an alert in response to at least a first set of records approaching a defined threshold.

BACKGROUND

The subject disclosure relates to machine learning and, more specifically, to predicting future possibility of bias in an artificial intelligence (AI) model.

SUMMARY

The following presents a summary to provide a basic understanding of one or more embodiments described herein. This summary is not intended to identify key or critical elements, delineate scope of particular embodiments or scope of claims. Its sole purpose is to present concepts in a simplified form as a prelude to the more detailed description that is presented later. In one or more embodiments described herein, systems, computer-implemented methods, apparatus and/or computer program products that enable prediction of future possibility of bias in an AI model are discussed.

According to an embodiment, a computer-implemented system is provided. The computer-implemented system can comprise a memory configured to store computer executable components; and a processor configured to execute the computer executable components stored in the memory, wherein the computer executable components can comprise a data generation component that can generate a set of structured test data to test likelihood of an artificial intelligence (AI) model generating biased outputs, based on analysis of payload logging data; and an alerting component that can alert a user of likelihood that the AI model will generate the biased outputs.

According to another embodiment, a computer-implemented method is provided. The method can comprise generating, by a system operatively coupled to a processor, a set of structured test data to test likelihood of an AI model generating biased outputs, based on analysis of payload logging data; and generating, by the system, alerts to a user of likelihood that the AI model will generate the biased outputs.

According to yet another embodiment, a computer program product for predicting bias in an AI model is provided. The computer program product can comprise a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to cause the processor to generate, by the processor, a set of structured test data to test likelihood of the AI model generating biased outputs, based on analysis of payload logging data; and generate, by the processor, alerts to a user of likelihood that the AI model will generate the biased outputs.

DETAILED DESCRIPTION

Bias in AI models is a well-known and well researched problem, and it is a deal-breaker in most scenarios wherein the AI models are deployed into production for actual business use cases. AI models can exhibit bias at any time and detecting and mitigating bias at run-time can defeat the purpose of having an AI model that users can trust. Thus, while techniques may be available for detecting and mitigating bias at runtime after an AI model has exhibited bias, it is desirable to be able to predict beforehand whether an AI model is expected to exhibit in the near future so that the bias can be proactively avoided altogether for fair and trusted AI applications. Bias detected at runtime, can be time-consuming to repair or correct. One or more embodiments discussed herein can predict if an AI model is likely to exhibit bias in the future and give an advanced warning.

Further, AI models are vetted and tested, and several types of tests, including precision tests, accuracy tests, recall tests, bias tests, drift tests, etc., are run on AI models before deploying them in production. However, many AI models exhibit bias at runtime despite of extensive testing, which indicates that there is potential to improve quality of test data sets used to check for bias in the AI models before deployment. Thus, it is also desirable to generate accurate test data sets specific for bias checking so that any possibility of an AI model exhibiting bias is detected at the time of testing. One or more embodiments discussed herein can enable generating a test data set specifically for checking bias in an AI model before deploying the AI model after the AI model is trained or re-trained.

FIG.1illustrates a block diagram of an example, non-limiting system100that can predict future possibility of bias in an AI model in accordance with one or more embodiments described herein. System100can comprise processor102, memory104, system bus106, analysis component108, data generation component110, alerting component112, AI component114, monitoring component116, and computation component118. One or more aspects of the non-limiting system100can be described in conjunction with one or more embodiments ofFIG.2.

Discussion first turns briefly to processor102, memory104and bus106of system100. For example, in one or more embodiments, the system100can comprise processor102(e.g., computer processing unit, microprocessor, classical processor, and/or like processor). In one or more embodiments, a component associated with system100, as described herein with or without reference to the one or more figures of the one or more embodiments, can comprise one or more computer and/or machine readable, writable and/or executable components and/or instructions that can be executed by processor102to enable performance of one or more processes defined by such component(s) and/or instruction(s).

In one or more embodiments, system100can comprise a computer-readable memory (e.g., memory104) that can be operably connected to the processor102. Memory104can store computer-executable instructions that, upon execution by processor102, can cause processor102and/or one or more other components of system100(e.g., analysis component108, data generation component110, alerting component112, AI component114, monitoring component116, and/or computation component118) to perform one or more actions. In one or more embodiments, memory104can store computer-executable components (e.g., analysis component108, data generation component110, alerting component112, AI component114, monitoring component116, and/or computation component118).

System100and/or a component thereof as described herein, can be communicatively, electrically, operatively, optically and/or otherwise coupled to one another via bus106. Bus106can comprise one or more of a memory bus, memory controller, peripheral bus, external bus, local bus, and/or another type of bus that can employ one or more bus architectures. One or more of these examples of bus106can be employed. In one or more embodiments, system100can be coupled (e.g., communicatively, electrically, operatively, optically and/or like function) to one or more external systems (e.g., a non-illustrated electrical output production system, one or more output targets, an output target controller and/or the like), sources and/or devices (e.g., classical computing devices, communication devices and/or like devices), such as via a network. In one or more embodiments, one or more of the components of system100can reside in the cloud, and/or can reside locally in a local computing environment (e.g., at a specified location(s)).

In addition to the processor102and/or memory104described above, system100can comprise one or more computer and/or machine readable, writable and/or executable components and/or instructions that, when executed by processor102, can enable performance of one or more operations defined by such component(s) and/or instruction(s). System100can be associated with, such as accessible via, a computing environment600described below with reference toFIG.6. For example, system100can be associated with a computing environment600such that aspects of processing can be distributed between system100and the computing environment600.

In one or more embodiments, system100can enable prediction of bias in AI model101as well as generation of structured test data119to check for bias in AI model101. AI component114can train two auto-encoders wherein a first auto-encoder (e.g., auto-encoder206ofFIG.2) can determine a first set of records for which AI model101can generate biased outputs and wherein a second auto-encoder (e.g., auto-encoder208ofFIG.2) can determine a second set of records for which AI model101can generate unbiased outputs. Training data105can be structured data used to train AI model101, and the first auto-encoder (e.g., auto-encoder206) and the second auto-encoder (e.g., auto-encoder208) can also be trained based on training data105. This can be achieved by dividing training data105into two groups wherein a first group can comprise biased records and a second group can comprise unbiased records. For example, individual records comprised in training data105can be perturbed and supplied to AI model101, and the outcomes of AI model101for the individual records can be monitored.

For example, an AI model (e.g., AI model101) employed in a banking environment for deciding whether a housing loan can be granted to individuals can be supplied with training data (e.g., training data105) used to train the AI model, to detect bias in the AI model against individuals of a first social group versus individuals of a second social group. Herein, the group comprising the second social group can be considered as a majority group or reference group and the group comprising the first social group can be considered as a minority group or a monitored group. For example, the training data (e.g., training data105) used to train the AI model (e.g., AI model101) can comprise historical records of applicants to whom a housing loan was granted or denied by a bank, such that the training data can be divided into two distribution demographics of biased and unbiased groups, based on a fairness attribute of type of social group, for a first social group and second social group (i.e., two difference social groups or two different values). For example, a record comprised in the training data (e.g., training data105) and belonging to an applicant of the second social group can be supplied as input to the AI model (e.g., AI model101), and thereafter, the same record can be supplied as input to the AI model by changing the social group on the record to that of the monitored group, keeping other attributes of the record the same.

If the AI model (e.g., AI model101) approves a housing loan for the unperturbed record belonging to the applicant of the second social group and denies the housing loan for the perturbed record wherein the social group was changed to that of the monitored group, it can indicate that the AI model (e.g., AI model101) displays individual bias, that is, it can indicate that the AI model is biased towards the specific perturbed record wherein the social group was changed to the first social group. In this scenario, both the original (unperturbed) record and perturbed versions of the original records can be considered biased records. Contrarily, if the AI model (e.g., AI model101) approves the housing loan for the unperturbed record wherein the social group of the applicant was changed to the first social group, both the original (unperturbed) record and the perturbed version of the original record can be considered unbiased records. Individual records of applicants from the monitored group comprised in the training data (e.g., training data105) can also be supplied as input to the AI model (e.g., AI model101), first as an unperturbed record wherein the social group remains unchanged, followed by changing the social group on the record to that of the reference group.

Thus, if the AI model (e.g., AI model101) approves the housing loan for the unperturbed record and denies the housing loan for the perturbed version of the original record, the AI model can be described as displaying individual bias towards the record, since the perturbed and the unperturbed versions of a record can vary only by social group. Both the original record and the perturbed version of the original record can be considered biased records. The process of scoring the AI model (e.g., AI model101) with individual records from the training data (e.g., training data105) can be repeated for individual records in the training data. Based on the results of the AI model (e.g., AI model101), two groups of records can be generated wherein a first group (e.g., a biased group) can comprise records from the training data (e.g., training data105) for which the AI model generated biased outputs and a second group (e.g., an unbiased group) can comprise records from the training data for which the AI model generated unbiased outputs. It is to be appreciated that the social group values discussed herein are for exemplary purposes only, and one or more methods discussed herein are not restricted to specific social groups. As such, in one or more embodiments, other fairness attributes comprising, for example, ages of individuals, can also be considered for bias detection in an AI model since bias in AI models is not restricted to specific groups, and AI models can display bias at any time and for a variety of reasons.

As discussed in one or more embodiments herein, the group of biased records and the group of unbiased records can be respectively used to train the first auto-encoder (e.g., auto-encoder206) and the second auto-encoder (e.g., auto-encoder208) such that the first auto-encoder and the second auto-encoder can respectively detect biased and unbiased records based on an anomaly detection technique. For example, the first auto-encoder (e.g., auto-encoder206) can be a neural network that can be trained using the group of biased records (e.g., biased records generated by AI model101from training data105) such that when data similar to the training data of the first auto-encoder is fed to first auto-encoder, the first auto-encoder does not detect an anomaly, whereas when data dissimilar to the training data (i.e., unbiased records) is fed to the first auto-encoder, the first auto-encoder detects an anomaly. For example, upon detecting an input record as an anomaly, an auto-encoder (e.g., auto-encoder206) can output a value of 1 on a display, indicating that an anomaly has been detected. For example, upon detecting an input record as not an anomaly, the auto-encoder (e.g., auto-encoder206) can output a value of zero, indicating that no anomaly was detected. The second auto-encoder (e.g., auto-encoder208) can be trained similarly wherein it can detect unbiased records as data that is similar to the training data of the second auto-encoder (e.g., unbiased records generated by AI model101from training data105) and biased records as data that is dissimilar to the training data of the second auto-encoder.

Analysis component108can use auto-encoder206and auto-encoder208to analyze payload logging data103. For example, analysis component108can use auto-encoder206to analyze payload logging data103to determine a first set of records in payload logging data103, for which AI model101can generate biased outputs. For example, analysis component108can use auto-encoder208to analyze payload logging data103to determine a second set of records in payload logging data103, for which AI model101can generate biased outputs. Payload logging data103can comprise runtime inputs107that AI model101can analyze upon being deployed for a runtime use case. In the exemplary banking scenario discussed herein, runtime inputs107can comprise runtime records of individuals seeking a housing loan. Runtime inputs107and the resulting outputs of the AI model (e.g., AI model101) can be stored in a payload logging table as payload logging data103.

An analysis component (e.g., analysis component108) can use the first auto-encoder (e.g., auto-encoder206) and the second auto-encoder (e.g., auto-encoder208) to analyze the payload logging data (e.g., payload logging data103) comprising runtime records of individuals seeking a housing loan. For example, the analysis component (e.g., analysis component108) can feed a record from the payload logging data (e.g., payload logging data103) to the first auto-encoder (e.g., auto-encoder206), wherein if the first auto-encoder detects the record as an anomaly (using the anomaly detection technique), it can imply that the record is not biased. Thereafter, the analysis component (e.g., analysis component108) can feed the same record to the second auto-encoder (e.g., auto-encoder208), wherein if the second auto-encoder detects the record as non-anomalous, it can imply that the record is unbiased. Thus, individual records from the payload logging data (e.g., payload logging data103) can be analyzed by the analysis component (e.g., analysis component108) to divide the payload logging data into a biased distribution and an unbiased distribution. The biased distribution can comprise biased runtime records, that is, runtime records for which the AI model (e.g., AI model101) unfairly denies the housing loan to individuals based on a social group attribute, and the unbiased distribution can comprise unbiased runtime records, that is, runtime records for which the AI model demonstrates fairness in approving the housing loan, regardless of the social group of an applicant.

The number of records in the biased distribution, and the number of records in the unbiased distribution can be monitored. If the number of records for which the AI model (e.g., AI model101) denies the housing loan for individuals of the first social group continues to increase, the AI model can be inferred as being biased against individuals of the first social group (i.e., exhibiting group bias against the first social group). Alerting component112can alert a user of likelihood that AI model101will generate biased outputs at runtime in response to at least a quantity of records in the biased distribution approaching a defined threshold. For example, if the number of records for which the AI model (e.g., AI model101) denies the housing loan for individuals of the first social group approaches a defined threshold, it can imply the AI model can be expected exhibit group bias during production, and an alerting component (e.g., alerting component112) can generate an alert to a user. For example, if more than 50 records, resulting from the AI model's bias against individuals of the first social group in approving the housing loan, accumulate in the biased distribution in one week, the alerting component (e.g., alerting component112) can generate an alert that the quantity of biased records under that category can be expected to approach a defined threshold (e.g.,75records in one week). This can further imply that the AI model (e.g., AI model101) can be expected to exhibit group bias against the first social group at runtime.

Herein, the threshold can be user defined based on the type of data being analyzed, type of predictions being made by an AI model, fairness attributes, etc. Monitoring component116can monitor the number of records in the biased distribution and the number of records in the unbiased distribution to enable computation of a disparate impact ratio (DIR)117, wherein DIR117can be a metric for predicting whether AI model101can be expected to exhibit group bias at runtime. DIR117can be computed by computation component118, wherein computation component118can further compute an estimated fairness score for AI model101based on an alert that DIR117is approaching the defined threshold. That is, monitoring component116can continuously monitor the outputs generated by auto-encoder206and auto-encoder208to enable computation of the DIR for AI model101. Computation of DIR117can be based on a sliding window analysis (or moving window analysis). In the exemplary banking scenario discussed herein, for a fairness attribute of type of social group, the group comprising the second social group can be considered as a majority group or reference group and the group comprising the first social group can be considered as a minority group or a monitored group, and the DIR (e.g., DIR117) can be calculated using equation 1.

Determination of whether the AI model (e.g., AI model101) can be expected to exhibit group bias at runtime can be achieved by two methods. For example, in a first method, the monitoring component (e.g., monitoring component116) can monitor the number of individually biased records detected by the first auto-encoder (e.g., auto-encoder206) for the majority and minority groups to detect if the AI model (e.g., AI model101) is biased against the first social group for approving housing loans. In the second method, monitoring component116can directly monitor payload logging data103, to enable computation of the DIR (e.g., DIR117). In the second method, the monitoring component (e.g., monitoring component116) can monitor the numerator and denominator of the DIR (e.g., DIR117) to detect either a steady decline in the value of the numerator or a steady increase in the value of the denominator and to determine if the overall value of the DIR (e.g., DIR117) is approaching a defined threshold. As discussed in one or more embodiments herein, if the overall value of the DIR (e.g., DIR117) approaches the defined threshold, it can be inferred that the AI model (e.g., AI model101) is expected to exhibit bias at runtime against the first social group for approving housing loans. The second method is further elaborated inFIG.3of this specification.

In an ideal world scenario, the DIR (e.g., DIR117) for an AI model (e.g., AI model101) would be 1, assuming that attributes other than the fairness attribute are consistent for individual records being analyzed. However, percentages of biased and unbiased outputs can vary in real-time. For example, the AI model (e.g., AI model101) can be scored with50records belonging to individuals of the second social group and 50 records belonging to individuals of the first social group. If the AI model (e.g., AI model101) approves the housing loan for 20 individuals of the monitored group and 30 individuals of the reference group, the DIR (e.g., DIR117) according to equation 1 can be about 66.67, wherein the 66.67 can represent a quantitative measure of DIR. A user or developer of the AI model (e.g., AI model101) can define a DIR threshold of 60 such that when the DIR value falls below 60 an alert can be generated by an alerting component (e.g., alerting component112) that the AI model can be expected to exhibit group bias. In response to the advance alert, the AI model (e.g., AI model101) can be removed from production and retrained by a data scientist, and the AI model can be subsequently tested for bias before deploying it for real time use cases. Following re-training of AI model101, structured test data119can be generated by data generation component110, specifically for checking bias in AI model101. Structured test data119can be generated to ensure that AI model101is retrained to prevent AI model101from exhibiting bias at runtime.

For example, it can be possible that upon retraining AI model101on a test data set that is not specifically designed for bias checking, that AI model101can exhibit no bias during testing but demonstrate group bias upon being deployed in the market. Structured test data119can comprise a percentage of records from the biased group and the unbiased group of records respectively used to train auto-encoder206and auto-encoder208. Structured test data119can further comprise a percentage of records from the biased distribution and the unbiased distribution respectively generated as a result of scoring auto-encoder206and auto-encoder208with payload logging data103. The percentages of biased and unbiased records used for generating structured test data119can be user defined. If AI model101exhibits group bias during testing with structured test data119, it can imply that AI model101can be expected to exhibit bias at runtime as well, because structured test data119can comprise a percentage of biased and unbiased records used to train AI model101.

FIG.2illustrates an example, non-limiting flow-diagram200of a baseline distribution-based approach for predicting bias in an AI model in accordance with one or more embodiments described herein. While referring here to one or more processes, facilitations and/or uses of the non-limiting flow-diagram200, descriptions provided herein, both above and below, also can be relevant to one or more other non-limiting systems described herein, such as the non-limiting system100, to be described below in detail. Repetitive description of like elements and/or processes employed in respective embodiments is omitted for sake of brevity.

In one or more embodiments herein, system100can be employed to enable prediction of bias in AI model101via a baseline distribution-based approach illustrated inFIG.2. Bias can be predicted for various fairness attributes such as nationality, age, ethnicity, or other social or non-social groups. The baseline distribution-based approach can require identification of baseline data203for predicting bias in AI model101. Baseline data203can comprise training data105used to train AI model101, test data used to test AI model101, or payload logging data (e.g., payload logging data103) received in production for one complete cycle of AI model101, where the length of the cycle can be decided by the usage of the model (e.g., data received by AI model101in one week or one month). If training data105is unavailable or inaccessible, the test data used to test AI model101or payload logging data (e.g., payload logging data103) can be used as baseline data203.

Once baseline data203is identified and collected, identification of baseline data distribution can be performed to identify distributed baseline data205. Identification of baseline data distribution can comprise identification of data distributions that cause AI model101to act in a biased manner. Identification of baseline data distribution identification can further comprise identification of data distributions that cause AI model101to act in an unbiased manner. Distributed baseline data205can comprise records from baseline data203that can cause AI model101to act in a biased manner as well as records from baseline data203that can cause AI model101to act in an unbiased manner. Distributed baseline data205can be identified by perturbing each individual record of baseline data203based on a fairness attribute, scoring AI model101with the individual records, and subsequently recording changes in outputs of AI model101from favorable to unfavorable or from unfavorable to favorable for the individual records. For example, baseline data203can comprise resumes of candidates seeking jobs and AI model101can be tested for prediction of bias towards a particular ethnicity. The ethnicity that AI model101can be unbiased towards can be described as a reference group, whereas other ethnicities can be collectively described as a monitored group.

The individual resumes of the candidates can be perturbed by changing the ethnicities on the original resumes from the reference group ethnicity to the monitored group ethnicity or vice-a-versa, and the perturbed and original records can be supplied to AI model101to record outcomes. If AI model101exhibits an unbiased outcome towards a resume belonging to the reference group by selecting candidates from the reference group for an interview, and if AI model101exhibits a biased outcome when the ethnicity on the resume is changed to one belonging to the monitored group, AI model101can be described as individually biased towards the resume. Further, both the original record and the perturbed record can be described as biased records. If AI model101does not exhibit bias upon changing the ethnicity, both the original record and the perturbed record can be described as unbiased records. Thus, individual records comprised in baseline data203can be perturbed and divided into a biased group and an unbiased group of records, wherein the biased and unbiased groups of records can comprise distributed baseline data205.

The biased group of records from distributed baseline data205can be used to train auto-encoder206such that auto-encoder206can detect an unbiased record as an anomaly based on anomaly detection. Similarly, the unbiased group of records from distributed baseline data205can be used to train auto-encoder208such that auto-encoder208can detect a biased record as an anomaly. Thus, auto-encoder206and auto-encoder208can encode distribution of data where AI model101has or has not exhibited bias. As discussed in one or more embodiments herein, auto-encoder206and auto-encoder208can be used to analyze individual records from payload logging data103to identify the records for which AI model101is likely to exhibit bias at runtime. In the exemplary scenario discussed herein, a resume belonging to the monitored group can be supplied to auto-encoder208and if auto-encoder208detects the resume as an anomaly, it can be imply that the resume belonging to the monitored group is a biased record (i.e., a record towards which AI model101can exhibit bias at runtime). The resume can then be supplied to auto-encoder206, wherein if auto-encoder206does not detect the resume as an anomaly, the resume can be described as a biased record.

Thus, individual records from distributed baseline data205can be divided into a biased distribution comprising one or more biased records and an unbiased distribution comprising one or more unbiased records. In the baseline distribution-based approach, monitoring component116can monitor the number of records in the biased distribution to track the number of individually biased outcomes that AI model101can exhibit towards individuals of both the reference group and the monitored group. Based on the monitoring, computation component118can compute an expected DIR value such that if the expected DIR value appears to cross a threshold, alerting component112can generate an alert to the user. For example, if the threshold on DIR117is set to 0.8, alerting component112can generate an alert to the user if the expected DIR value falls below 0.85. Based on the alert, computation of the actual DIR (e.g., DIR117) based on a sliding window analysis can be triggered and an estimated fairness value for AI model101can be computed by computation component118. Thereafter, corrective action can be taken by re-training AI model101.

Upon re-training, AI model101can be tested for bias using structured test data119. As discussed in one or more embodiments herein, structured test data119can comprise a percentage of records from the biased group and the unbiased group of records comprised in distributed baseline data205. Structured test data119can further comprise a percentage of records from the biased distribution and the unbiased distribution respectively generated as a result of scoring auto-encoder206and auto-encoder208with payload logging data103. The percentages of biased records and unbiased records comprised in structured test data119can be user defined. The baseline distribution-based approach combined with generating structured test data119specific for bias checking can make for a robust approach for bias checking in AI models (e.g., one or more of AI model101).

FIG.3illustrates an example, non-limiting flow-diagram of a payload analysis-based approach for predicting bias in an AI model in accordance with one or more embodiments described herein. While referring here to one or more processes, facilitations and/or uses of the non-limiting flow-diagram300, descriptions provided herein, both above and below, also can be relevant to one or more other non-limiting systems described herein, such as the non-limiting system100, to be described below in detail. Repetitive description of like elements and/or processes employed in respective embodiments is omitted for sake of brevity.

In one or more embodiments herein, system100can be employed to enable prediction of bias in AI model101via a payload analysis-based approach illustrated inFIG.3. Bias can be predicted for various fairness attributes such as nationality, age, ethnicity, or other social or non-social groups. In the payload analysis-based approach, monitoring component116can directly monitor payload logging data103, to enable computation of the DIR (e.g., DIR117). Based on monitoring of payload logging data103, computation component118can compute total number of records for both minority (monitored) and majority (reference) groups, and computation component118can compute total number of records of both minority and majority groups for unbiased outcomes generated by AI model101. The unbiased outcomes can also be described as favorable outcomes in one or more embodiments herein. Computation component118can use the information to further compute DIR117based on a sliding window analysis and a fairness score for AI model101. For example, payload logging data103can comprises a total 80 records belonging to a monitored group and 90 records can belong to a reference group. AI model101can generate favorable outcomes (i.e., unbiased outcomes) for 40 records of the monitored group and for 75 records of the reference group. Thus, the DIR value based on the favorable outcomes and using equation 1 can be computed as:

The above calculation can result in a DIR value of about 0.6. Similar to the baseline distribution-based approach, if DIR117crosses a user-defined threshold, the user can be alerted by alerting component112. For example, in the exemplary calculation presented herein, for a defined threshold of 0.5, an alert can be generated to the user when the DIR value falls below 0.55. A key difference between the baseline distribution-based approach discussed inFIG.2and the payload analysis-based approach discussed herein can be the perturbation of data. The payload analysis-based approach can directly utilize payload logging data103to determine whether AI model101can be expected to exhibit bias towards a particular group at runtime. The payload analysis-based approach can be executed without utilizing auto-encoders to compute a fairness score for AI model101. That is, in the payload analysis-based approach, the estimated fairness score can be computed by computing a first percentage of unbiased outputs (e.g., unbiased outputs of a monitored group) generated by the AI model and a second percentage of unbiased outputs (e.g., unbiased outputs of a reference group) generated by the AI model, based on an unperturbed set of structured training data (e.g., training data105). In the baseline distribution-based approach, the estimated fairness score can be computed by computing a quantity of biased outputs generated by the AI model and a quantity of unbiased outputs generated by the AI model, based on perturbation of individual records of a set of structured training data (e.g., training data105).

FIG.4illustrates a block diagram of an example, non-limiting method400for predicting future possibility of bias in an AI model in accordance with one or more embodiments described herein. Repetitive description of like elements and/or processes employed in respective embodiments is omitted for sake of brevity.

At402, the non-limiting method400can comprise analyzing, by a system operatively coupled to a processor, the payload logging data, using a first auto-encoder, to determine a first set of records in the payload logging data, for which an AI model generates the biased outputs.

At404, the non-limiting method400can comprise analyzing, by the system, the payload logging data, by using a second auto-encoder, to determine a second set of records for which the AI model generates unbiased outputs.

At406, the non-limiting method400can comprise generating, by the system, a set of structured test data to test likelihood of an AI model generating biased outputs, based on analysis of payload logging data.

At408, the non-limiting method400can comprise monitoring, by the system, respective outputs of the first auto-encoder and the second auto-encoder to enable computation of a disparate impact ratio for the AI model based on a sliding window analysis.

At410, the non-limiting method400can comprise generating, by the system, alerts to a user of likelihood that the AI model will generate the biased outputs and generating the alerts in response to at least the first set of records approaching a defined threshold.

At412, the non-limiting method400can comprise computing, by the system, an estimated fairness score for the AI model based on an alert that a disparate impact ratio is approaching the defined threshold.

FIG.5illustrates a block diagram of an example, non-limiting method500for predicting future possibility of bias in an AI model in accordance with one or more embodiments described herein. Repetitive description of like elements and/or processes employed in respective embodiments is omitted for sake of brevity.

At502, the non-limiting method500can comprise monitoring, by the system, payload logging data to compute at least a first quantity of unbiased outputs generated by an artificial intelligence (AI) model and a second quantity of unbiased outputs generated by the AI model to enable computation of a disparate impact ratio for the AI model based on a sliding window analysis.

At504, the non-limiting method500can comprise computing, by the system, an estimated fairness score by computing a first percentage of unbiased outputs generated by the AI model and a second percentage of unbiased outputs generated by the AI model, based on an unperturbed set of structured training data.

At505, the non-limiting method500can determine if an estimated disparate impact ratio approaches a defined threshold. If no, the non-limiting method500returns to504to continue computing the estimated fairness score. If yes, the non-limiting method500generates an alert at506.

At506, the non-limiting method500can comprise generating, by the system, alerts to a user of likelihood that the AI model will generate the biased outputs and generating the alerts in response to at least the first set of records approaching a defined threshold.

At508, the non-limiting method500can comprise computing, by the system, an estimated fairness score for the AI model based on an alert that a disparate impact ratio is approaching the defined threshold.

Terminology

Fairness attribute: The feature column on which the bias can be observed.

Reference group: The majority group for which the model can be expected to be biased.

Monitored group: The minority group for which the model can be expected to be biased against.

Group bias: Bias (by an AI model) against a particular group (e.g., against a particular ethnicity, nationality, or other social or non-social groups).

Disparate impact (or disparate impact ratio): A type of metric to measure group bias (see equation 1). Group bias can be measured via various other metrics.

For simplicity of explanation, the computer-implemented and non-computer-implemented methodologies provided herein are depicted and/or described as a series of acts. It is to be understood that the subject innovation is not limited by the acts illustrated and/or by the order of acts, for example acts can occur in one or more orders and/or concurrently, and with other acts not presented and described herein. Furthermore, not all illustrated acts can be utilized to implement the computer-implemented and non-computer-implemented methodologies in accordance with the described subject matter. Additionally, the computer-implemented methodologies described hereinafter and throughout this specification are capable of being stored on an article of manufacture to enable transporting and transferring the computer-implemented methodologies to computers. The term article of manufacture, as used herein, is intended to encompass a computer program accessible from any computer-readable device or storage media.

The systems and/or devices have been (and/or will be further) described herein with respect to interaction between one or more components. Such systems and/or components can include those components or sub-components specified therein, one or more of the specified components and/or sub-components, and/or additional components. Sub-components can be implemented as components communicatively coupled to other components rather than included within parent components. One or more components and/or sub-components can be combined into a single component providing aggregate functionality. The components can interact with one or more other components not specifically described herein for the sake of brevity, but known by those of skill in the art.

One or more embodiments described herein can employ hardware and/or software to solve problems that are highly technical, that are not abstract, and that cannot be performed as a set of mental acts by a human. For example, a human, or even thousands of humans, cannot efficiently, accurately and/or effectively probe frequency space of a qubit as the one or more embodiments described herein can enable this process. And, neither can the human mind nor a human with pen and paper probe frequency space of a qubit, as conducted by one or more embodiments described herein.

FIG.6illustrates a block diagram of an example, non-limiting, operating environment in which one or more embodiments described herein can be facilitated.FIG.6and the following discussion are intended to provide a general description of a suitable operating environment600in which one or more embodiments described herein atFIGS.1-5can be implemented.

Computing environment600contains an example of an environment for the execution of at least some of the computer code involved in performing the inventive methods, such as AI bias prediction code645. In addition to block645, computing environment600includes, for example, computer601, wide area network (WAN)602, end user device (EUD)603, remote server604, public cloud605, and private cloud606. In this embodiment, computer601includes processor set610(including processing circuitry620and cache621), communication fabric611, volatile memory612, persistent storage613(including operating system622and block645, as identified above), peripheral device set614(including user interface (UI), device set623, storage624, and Internet of Things (IoT) sensor set625), and network module615. Remote server604includes remote database630. Public cloud605includes gateway640, cloud orchestration module641, host physical machine set642, virtual machine set643, and container set644.

PROCESSOR SET610includes one, or more, computer processors of any type now known or to be developed in the future. Processing circuitry620may be distributed over multiple packages, for example, multiple, coordinated integrated circuit chips. Processing circuitry620may implement multiple processor threads and/or multiple processor cores. Cache621is memory that is located in the processor chip package(s) and is typically used for data or code that should be available for rapid access by the threads or cores running on processor set610. Cache memories are typically organized into multiple levels depending upon relative proximity to the processing circuitry. Alternatively, some, or all, of the cache for the processor set may be located “off chip.” In some computing environments, processor set610may be designed for working with qubits and performing quantum computing.

VOLATILE MEMORY612is any type of volatile memory now known or to be developed in the future. Examples include dynamic type random access memory (RAM) or static type RAM. Typically, the volatile memory is characterized by random access, but this is not required unless affirmatively indicated. In computer601, the volatile memory612is located in a single package and is internal to computer601, but, alternatively or additionally, the volatile memory may be distributed over multiple packages and/or located externally with respect to computer601.

END USER DEVICE (EUD)603is any computer system that is used and controlled by an end user (for example, a customer of an enterprise that operates computer601), and may take any of the forms discussed above in connection with computer601. EUD603typically receives helpful and useful data from the operations of computer601. For example, in a hypothetical case where computer601is designed to provide a recommendation to an end user, this recommendation would typically be communicated from network module615of computer601through WAN602to EUD603. In this way, EUD603can display, or otherwise present, the recommendation to an end user. In some embodiments, EUD603may be a client device, such as thin client, heavy client, mainframe computer, desktop computer and so on.

REMOTE SERVER604is any computer system that serves at least some data and/or functionality to computer601. Remote server604may be controlled and used by the same entity that operates computer601. Remote server604represents the machine(s) that collect and store helpful and useful data for use by other computers, such as computer601. For example, in a hypothetical case where computer601is designed and programmed to provide a recommendation based on historical data, then this historical data may be provided to computer601from remote database630of remote server604.

PRIVATE CLOUD606is similar to public cloud605, except that the computing resources are only available for use by a single enterprise. While private cloud606is depicted as being in communication with WAN602, in other embodiments a private cloud may be disconnected from the internet entirely and only accessible through a local/private network. A hybrid cloud is a composition of multiple clouds of different types (for example, private, community or public cloud types), often respectively implemented by different vendors. Each of the multiple clouds remains a separate and discrete entity, but the larger hybrid cloud architecture is bound together by standardized or proprietary technology that enables orchestration, management, and/or data/application portability between the multiple constituent clouds. In this embodiment, public cloud605and private cloud606are both part of a larger hybrid cloud.