## 3 Results
### Model Retraining  
Retraining models is fundamental to make certain that the output is reliable and fair. With the increasing number of ESG commitments and a company's reliance on data and KPIs to drive outcomes and progress, the intersection of AI and ESG, while promising, comes with important continuous monitoring techniques which include model retraining. Kiva's data have evolved over time. Data attributes such as \"Planned Expiration Time\" and \"Tags\" were added to the dataset in late 2012 and 2013, respectively. The number of countries (\"Country Name\") increased over time. Also, the maximum \"Loan Amount\" increased over time, and there are several more examples of such additions and changes. With these changes in mind, techniques we discussed earlier, including encoding, scaling, and others, are a part of the \"re-training\" model. Consider the following min/max scaling example. Let us say that in 2008, the maximum loan value was 125 and data was scaled through dividing all loan amounts by 125. Let us say that in 2009, we had a new maximum loan value of 1000. However, we should not scale the 2009 data with the maximum loan value from 2008, which was 125. The correct approach is to scale the data according to the new maximum loan value of 1000 as depicted in Figure 6 below. This could also be the case for a number of countries where, let us say, in 2008, \"Country Names\" consisted of Zimbabwe, Uganda, and South Africa, and then in 2009, let us say there were two more countries reflected in the data--Vietnam and Congo. Hence, re-encoding is necessary to capture the new values reflected under \"Country Name.\"  
With this example in mind, re-scaling and re-encoding are performed on some attributes due to changes in the data set to demonstrate model performance when a model is trained once versus monthly. In the below example, applying these techniques, including the Gradient Boosting algorithm, and extracting the AUC in both the single-trained model and the monthly retrained model, yields the outcomes in Figures 7 and 8. As part of this analysis, we aim to understand data attributes or feature importance. Understanding those data attributes that are most important to the model can be insightful when it comes to predicting the likelihood of a loan being funded promptly (in this research, within 7 days). Using topic modeling, specifically, BERTopic, the research highlights the most important features of the model, while also incorporating Density-Based Spatial Clustering of Applications with Noise (DBSCAN)--an important technique that detects outliers in the dataset. With model retraining in mind, Figure 9 below shows a ranking of important attributes in 2010, while Figure 10 shows the ranking of important attributes in 2016. Figure 6: Illustrative Example: Rescaling data with updated loan values (2009). Figure 8: Monthly retrained model performance. Figure 7: Single retrained model performance. As you can see from Figures 9 and 10, \"Loan Amount\" and \"Lender Term\" continue to be the most important attributes of the model during 2010 as well as in 2016, while the Philippines was an important feature in the model during 2010, and in 2016, four other countries (not including the Philippines) were one of the top 20 important features to the model--Paraguay, Nicaragua, Peru, and Kenya. Figure 10: Ranking of feature (data attributes) importance (2016). Figure 9: Ranking of feature (data attributes) importance (2010). Figure 11 below depicts an overall process from data cleansing to model retraining that can be incorporated into a governance process that relies on predictability and increased accuracy in model performance. In summary, the process starts with data pre-processing to predict whether a loan will be raised in several days (target variable creation), and then, due to data drift, over time, the accuracy of the model decreases--informing the need for model retraining. This ultimately allows for bias tracking through assessing feature (data attribute) importance.