Original: Explainability in Practice: Estimating Electrification Rates from Mobile Phone Data in Senegal
Revised: Explainability in Practice: Estimating Electrification Rates from Mobile Phone Data in Senegal

Original: Anonymous Author(s)

Original: AffiliationAddress email

Original: Abstract
Revised: Abstract

Original: Explainable artificial intelligence (XAI) provides explanations for not interpretablemachine learning (ML) models.
Revised: Explainable artificial intelligence (XAI) provides explanations for not interpretable machine learning (ML) models.

Original: While many technical approaches exist, there isa lack of validation of these techniques on real-world datasets.
Revised: While many technical approaches exist, there is a lack of validation of these techniques on real-world datasets.

Original: In this work, wepresent a use-case of XAI: an ML model which is trained to estimate electrificationrates based on mobile phone data in Senegal.
Revised: In this work, we present a use-case of XAI: an ML model which is trained to estimate electrification rates based on mobile phone data in Senegal.

Original: The data originate from the Datafor Development challenge by Orange in 2014/15.
Revised: The data originate from the Data for Development challenge by Orange in 2014/15.

Original: We apply two model-agnostic,local explanation techniques and find that while the model can be verified, it isbiased with respect to the population density.
Revised: We apply two model-agnostic, local explanation techniques and find that while the model can be verified, it is biased with respect to the population density.

Original: We conclude our paper by pointingto the two main challenges we encountered during our work: data processing andmodel design that might be restricted by currently available XAI methods, and theimportance of domain knowledge to interpret explanations
Revised: We conclude our paper by pointing to the two main challenges we encountered during our work: data processing and model design that might be restricted by currently available XAI methods, and the importance of domain knowledge to interpret explanations.

Original:  Introduction
Revised:  Introduction

Original: Explainable AI (XAI) provides techniques to better understand machine learning (ML) models.
Revised: Explainable AI (XAI) provides techniques to better understand machine learning (ML) models.

Original: Thisis motivated by their lack of transparency, and an increased use of these models in resource allocationproblems that critically affect individuals, such as hiring, credit rating or in public administration.
Revised: This is motivated by their lack of transparency, and an increased use of these models in resource allocation problems that critically affect individuals, such as hiring, credit rating or in public administration.

Original: While many XAI methods have been proposed, there is a certain lack of work that uses these methodson real-world data and thus confirms their relevance [19].
Revised: While many XAI methods have been proposed, there is a certain lack of work that uses these methods on real-world data and thus confirms their relevance [19].

Original: In this work, we present a use-case of XAI: we train an ML model to estimate electrification rates in
Revised: In this work, we present a use-case of XAI: we train an ML model to estimate electrification rates in

Original: Senegal, and evaluate it using two popular XAI techniques.
Revised: Senegal, and evaluate it using two popular XAI techniques.

Original: The estimation of such socio-economicindicators can support policy planning and is assumed to be a less costly and time-consumingalternative to traditional approaches such as collecting census or survey data.
Revised: The estimation of such socio-economic indicators can support policy planning and is assumed to be a less costly and time-consuming alternative to traditional approaches such as collecting census or survey data.

Original: Policy planninginvolves considerable amounts of resources, thus, requires transparency and accountability.
Revised: Policy planning involves considerable amounts of resources, thus, requires transparency and accountability.

Original: We drawon a dataset of mobile phone data, collected in 2013 and provided during the Data for Developmentchallenge by Orange in 2014/15.
Revised: We draw on a dataset of mobile phone data, collected in 2013 and provided during the Data for Development challenge by Orange in 2014/15.

Original: We combine it with extracts from the 2013 census in Senegal, andestimate the electrification rate around single cell tower locations.
Revised: We combine it with extracts from the 2013 census in Senegal, and estimate the electrification rate around single cell tower locations.

Original: The contribution of our work is twofold: first, we show how XAI methods can be used to verify
Revised: The contribution of our work is twofold: first, we show how XAI methods can be used to verify

Original: an
Revised: an

Original: ML model, and that our model is biased w.r.t.
Revised: ML model, and that our model is biased w.r.t.

Original: population densities.
Revised: population densities.

Revised: To verify means in our case to show that the model indeed relies on features that relate to the predicted outcome, as given by domain knowledge.

Original: Thus, we confirm the relevance of XAI techniques.
Revised: Thus, we confirm the relevance of XAI techniques.

Original: Second, we point towards two challenges of deploying XAI in practice that emergedduring this work: pipeline design, and domain knowledge.
Revised: Second, we point towards two challenges of deploying XAI in practice that emerged during this work: pipeline design, and domain knowledge.

Original: The paper is structured as follows: section 2 provides the relevant background, section 3 informationon the used data, followed by a description of the experiments in section 4.
Revised: The paper is structured as follows: section 2 provides the relevant background, section 3 information on the used data, followed by a description of the experiments in section 4.

Original: Results are discussed insection 5, followed by the limitations in section 6 and we conclude our paper in section 7.[-
-]
Revised: Results are discussed in section 5, followed by the limitations in section 6 and we conclude our paper in section 7.[-
-]

Original:  Background2.1 Explainable AI
Revised:  Background 2.1 Explainable AI

Original: The field of explainable AI can be distinguished along three dimensions: black vs white box approaches, local vs global, and model-agnostic vs model-specific [10].
Revised: The field of explainable AI can be distinguished along three dimensions: black vs white box approaches, local vs global, and model-agnostic vs model-specific [10].

Original: The term white box refersto models that are interpretable, or explainable by desgin, while a black box (BB) model is notinterpretable, or accessible, e.g., due to intellectual property rights.
Revised: The term white box refers to models that are interpretable, or explainable by design, while a black box (BB) model is not interpretable, or accessible, e.g., due to intellectual property rights.

Original: The majority of used ML modelsbelong to the latter, and it is exactly for those models that we have to design explanation techniques.
Revised: The majority of used ML models belong to the latter, and it is exactly for those models that we have to design explanation techniques.

Original: These explanations can be distinguished by scale.
Revised: These explanations can be distinguished by scale.

Original: Local techniques explain the prediction for a singledata instance (often an individual, in our case a single cell phone tower).
Revised: Local techniques explain the prediction for a single data instance (often an individual, in our case a single cell phone tower).

Original: Here, most prominentapproaches are LIME (Local Interpretable Model-agnostic Explanations) [24] and SHAP (SHapley
Revised: Here, most prominent approaches are LIME (Local Interpretable Model-agnostic Explanations) [24] and SHAP (SHapley

Original: Additive exPlanations) [15].
Revised: Additive exPlanations) [15].

Original: Opposed to this, global approaches tackle a full explanation of thesystem, usually by fitting an interpretable surrogate model.
Revised: Opposed to this, global approaches tackle a full explanation of the system, usually by fitting an interpretable surrogate model.

Original: One example is the TREPAN algorithm[7], building a single decision tree over a neural network.
Revised: One example is the TREPAN algorithm [7], building a single decision tree over a neural network.

Original: Last, we differentiate between approachesthat work on any ( model-agnostic ) or only one ( model-specific ) model.
Revised: Last, we differentiate between approaches that work on any ( model-agnostic ) or only one ( model-specific ) model.

Original: LIME [24] and SHAP [15]are considered model-agnostic, TREPAN [7] model-specific.
Revised: LIME [24] and SHAP [15] are considered model-agnostic, TREPAN [7] model-specific.

Original: A full survey of approaches can befound elsewhere, e.g., [10, 17, 2].
Revised: A full survey of approaches can be found elsewhere, e.g., [10, 17, 2].

Original: 2.2 Mobile Phone Data and Electrification
Revised: 2.2 Mobile Phone Data and Electrification

Original: Mobile phones are a rich source of information, providing details about time, length and locationof calls and other data.
Revised: Mobile phones are a rich source of information, providing details about time, length and location of calls and other data.

Original: Combined with the fact that mobile phone penetration is generally high, 3this opens possibilities for research, public policy, infrastructure planning, etc.
Revised: Combined with the fact that mobile phone penetration is generally high,this opens possibilities for research, public policy, infrastructure planning, etc.

Original: Predicting socioeconomic indicators from remotely accessible data is popular, and we observe an interest in usingsuch data in countries of the Global South, where it might be a less costly and time-consumingalternative to traditional approaches such as collecting census or survey data.
Revised: Predicting socioeconomic indicators from remotely accessible data is popular, and we observe an interest in using such data in countries of the Global South, where it might be a less costly and time-consuming alternative to traditional approaches such as collecting census or survey data.

Original: Different indicators canbe predicted, and they vary based on available data and method.
Revised: Different indicators can be predicted, and they vary based on available data and method.

Original: Examples based on mobile phonedata are [5, 27, 26] (estimating socio-economic status and welfare indicators, literacy rate, populationdensities and electric consumption), approaches using additional data sources are [23, 28] (estimatingpoverty measures).
Revised: Examples based on mobile phone data are [5, 27, 26] (estimating socio-economic status and welfare indicators, literacy rate, population densities and electric consumption), approaches using additional data sources are [23, 28] (estimating poverty measures).

Original: In many cases, these studies are framed within the 17 SDGs.
Revised: In many cases, these studies are framed within the 17 SDGs.

Original: In this work, we focus on the estimation of electrification rates using mobile phone data.
Revised: In this work, we focus on the estimation of electrification rates using mobile phone data.

Original: Relevantstudies that investigate the relation of electricity and other indicators such as mobile connectivity orvolume of visitors in Senegal are [12] and [25].
Revised: Relevant studies that investigate the relation of electricity and other indicators such as mobile connectivity or volume of visitors in Senegal are [12] and [25].

Original:  Dataset3.1 Mobile Phone Data
Revised:  Dataset 3.1 Mobile Phone Data

Original: We use mobile phone data provided in the form of pre-aggregated call detail records (CDR), whichwere made available during the second Data for Development challenge launched by Orange in2014/15.
Revised: We use mobile phone data provided in the form of pre-aggregated call detail records (CDR), which were made available during the second Data for Development challenge launched by Orange in 2014/15.

Original: The original data were collected by Sonatel ( Société Nationale des Télécommunications du
Revised: The original data were collected by Sonatel ( Société Nationale des Télécommunications du

Original: Sénégal ), who is the leading telecommunication company in Senegal (market share of 65% in 2013).
Revised: Sénégal ), who is the leading telecommunication company in Senegal (market share of 65% in 2013).

Original: CDRs are generated for billing purposes, and are proprietary.
Revised: CDRs are generated for billing purposes, and are proprietary.

Revised: [Table]

Revised: ratio of outgoing over incoming events

Original: Senegal is a sub-Saharan country, located on the Northern Hemisphere near the Equator, on the West
Revised: Senegal is a sub-Saharan country, located on the Northern Hemisphere near the Equator, on the West

Original: Coast of Africa.
Revised: Coast of Africa.

Original: It covers 196,712 km 2 .
Revised: It covers 196,712 km 2 .

Original: In 2013, the population count approached 14M. 5 , the yearwhen the data were collected.
Revised: In 2013, the population count approached 14M. 5 , the year when the data were collected.

Original: The original dataset contains more than 9 million individual mobilephone numbers, with a hourly resolution.
Revised: The original dataset contains more than 9 million individual mobile phone numbers, with a hourly resolution.

Original: Sonatel anonymized the data, Orange pre-aggregated andprocessed it further.
Revised: Sonatel anonymized the data, Orange pre-aggregated and processed it further.

Original: The resulting dataset holds (cell) tower-to-tower activity, for calls (includingcall length) and text messages separately.
Revised: The resulting dataset holds (cell) tower-to-tower activity, for calls (including call length) and text messages separately.

Original: Spatial coverage between call and text message data isdifferent, text message data less available in the Eastern part of Senegal (see appendix, section A.1).
Revised: Spatial coverage between call and text message data is different, text message data less available in the Eastern part of Senegal (see appendix, section A.1).

Original: These areas fall together with areas that are less electrified which might suggest a connection betweenaccess to electricity, poverty rate, literacy and text message activity (see [12] for a study on the impactof access to electricity).
Revised: These areas fall together with areas that are less electrified which might suggest a connection between access to electricity, poverty rate, literacy and text message activity (see [12] for a study on the impact of access to electricity).

Original: All details on the dataset can be found here [18].[Table]
Revised: All details on the dataset can be found here [18].

Original: ratio of outgoing over incoming events

Original: We process this data as a weighted communication network (CNW, one network per data type) basedon the overall activity of 2013.
Revised: We process this data as a weighted communication network (CNW, one network per data type) based on the overall activity of 2013.

Original: The CNW is directed and uses cell towers as nodes, call and textactivity to determine the weights and direction of edges.
Revised: The CNW is directed and uses cell towers as nodes, call and text activity to determine the weights and direction of edges.

Original: The cell tower locations are used to map theactivity spatially, and we rely on a Voronoi cell tessellation.
Revised: The cell tower locations are used to map the activity spatially, and we rely on a Voronoi cell tessellation.

Original: From each network, we extract 6 featureswhich are listed in table 1, they form the final dataset which has dimensions of 1587 × 18 , i.e. cellstimes features.
Revised: From each network, we extract 6 features which are listed in table 1, they form the final dataset which has dimensions of 1587 × 18 , i.e. cells times features.

Original: Due to space reasons and the focus of this paper, more detailsof the processing are inthe appendix A.2.
Revised: Due to space reasons and the focus of this paper, more details are in the appendix A.2.

Original: 3.2 Electrification Data
Revised: 3.2 Electrification Data

Original: Electrification rate, population count and population density originate from the 2013 census in
Revised: Electrification rate, population count and population density originate from the 2013 census in

Original: Senegal [3].
Revised: Senegal [3].

Original: Pre-processing is identical to Salat et al.
Revised: Pre-processing is identical to Salat et al.

Original: The census contains questions about thesource of lighting for each household and therefore informs us of stable access to electricity.
Revised: The census contains questions about the source of lighting for each household and therefore informs us of stable access to electricity.

Original: In thecontext of this study, stable access means access either to the main power grid or to photo-voltaicsystems that benefit from the year-round high solar irradiation in the country.
Revised: In the context of this study, stable access means access either to the main power grid or to photo-voltaic systems that benefit from the year-round high solar irradiation in the country.

Original: The electrification rateis computed for each Voronoi cell as the ratio between the total number of households with stableaccess and the total population count, assuming a homogeneous distribution of the population insideeach census unit ( Commune ).
Revised: The electrification rate is computed for each Voronoi cell as the ratio between the total number of households with stable access and the total population count, assuming a homogeneous distribution of the population inside each census unit ( Commune ).

Original: The study reports that the resulting electrification rates were in goodagreement with the fine-grained nighttime lights intensity provided by NOAA [20].
Revised: The study reports that the resulting electrification rates were in good agreement with the fine-grained nighttime lights intensity provided by NOAA [20].

Original: The supportingshapefile containing the geographic boundaries and population counts at commune level was providedalongside another previous study[26].
Revised: The supporting shapefile containing the geographic boundaries and population counts at commune level was provided alongside another previous study[26].

Original: Figure 1 shows electrification rates in Senegal.
Revised: Figure 1 shows electrification rates in Senegal.

Original: Higher rates are observed around major cities: Dakar,the Capital, in the Western most part of the country, Saint-Louis in the North West, and the culturally[Figure] significant city of Touba in the center.
Revised: Higher rates are observed around major cities: Dakar, the Capital, in the Western most part of the country, Saint-Louis in the North West, and the culturally significant city of Touba in the center.

Original: The electrification rates are also high along the more densely[Figure] populated Northern/Eastern borders with Mauritania.
Revised: The electrification rates are also high along the more densely populated Northern/Eastern borders with Mauritania.

Original: This follows closely the electric grid of Senegal,[Figure] discussed in [16].
Revised: This follows closely the electric grid of Senegal, discussed in [16].

Original: We therefore observe a correlation between areas with high electrification rate and[Figure] high density of cell towers.[Figure]
Revised: We therefore observe a correlation between areas with high electrification rate and high density of cell towers.

Original:  Experiments 
Revised:  Experiments 4.1 Preparation

Original: [Figure] 4.1 Preparation

Original: Each data row (a Voronoi cell) is labeled by its electrification rate.
Revised: Each data row (a Voronoi cell) is labeled by its electrification rate.

Original: We bin the data such that it holds10 classes, ordered by electrification rate, and with a bin size of 0 .
Revised: We bin the data such that it holds 10 classes, ordered by electrification rate, and with a bin size of 0 .

Original: 1 electrification rate forclass 0, 0 .
Revised: 1 electrification rate for class 0, 0 .

Original: 2 electrification rate for class 1, etc.).
Revised: 2 electrification rate for class 1, etc.).

Original: The resulting distribution is skewed towardshigher values.
Revised: The resulting distribution is skewed towards [Figure] higher values.

Original: 6 We subsample the elements of class 9 (electrification rate 0 .
Revised: 6 We subsample the elements of class 9 (electrification rate 0 .

Original: 9 .. 1 ), for the trainingsets only.
Revised: 9 .. 1 ), for the training sets only.

Original: We sample such that numbers in class 9 are reduced to the average count over all classes.
Revised: We sample such that numbers in class 9 are reduced to the average count over all classes.

Original: We split the dataset to create a training and a test set for the ML classifiers ( 7 : 3 ), and split the testset to create a training and a test set for the explanations ( 7 : 3 ).
Revised: We split the dataset to create a training and a test set for the ML classifiers ( 7 : 3 ), and split the test set to create a training and a test set for the explanations ( 7 : 3 ).

Original: 4.2 Models
Revised: 4.2 Models

Original: Classification We train several ML standard models to estimate electrification rates: a decision tree(DT), a random forest model (RF) containing 100 decision trees, and an extreme gradient boostingmodel (XGB) with default parameters.
Revised: Classification We train several ML standard models to estimate electrification rates: a decision tree (DT), a random forest model (RF) containing 100 decision trees, and an extreme gradient boosting model (XGB) with default parameters.

Original: We compute the accuracy, the mean absolute error (MAE)and the ratio between the MAE and its maximum ( MAE max = 9 ).
Revised: We compute the accuracy, the mean absolute error (MAE) and the ratio between the MAE and its maximum ( MAE max = 9 ).

Original: Explanations We compute local, model-agnostic explanations based on LIME and SHAP.
Revised: Explanations We compute local, model-agnostic explanations based on LIME and SHAP.

Original: We doso for the following reasons: both methods, including their pitfalls, are well known in the community, 120methods are easily accessible, and rely on interpretable features [17].
Revised: We do so for the following reasons: both methods, including their pitfalls, are well known in the community,methods are easily accessible, and rely on interpretable features [17].

Original: LIME [24] uses a randomlygenerated neighborhood, weighted according to a distance and a kernel function, to fit a linearregression around the data instance in focus.
Revised: LIME [24] uses a randomly generated neighborhood, weighted according to a distance and a kernel function, to fit a linear regression around the data instance in focus.

Original: This regression approximates the decision boundary,and its weights are interpreted as the importance of the features.
Revised: This regression approximates the decision boundary, and its weights are interpreted as the importance of the features.

Original: A positive importance pushes theclassifier towards the predicted class, while a negative pulls it away and towards one of the other(here: nine) classes.
Revised: A positive importance pushes the classifier towards the predicted class, while a negative pulls it away and towards one of the other (here: nine) classes.

Original: Therefore, both the sign and magnitude of the importance matter.
Revised: Therefore, both the sign and magnitude of the importance matter.

Original: SHAP [15]explanations are based on a game-theoretic approach, and provide for each feature its contributiontowards the predicted outcome.
Revised: SHAP [15] explanations are based on a game-theoretic approach, and provide for each feature its contribution towards the predicted outcome.

Original: Contributions are not only specific to an instance but also to the class.
Revised: Contributions are not only specific to an instance but also to the class.

Original: If we sum over all contributions w.r.t.
Revised: If we sum over all contributions w.r.t.

Original: to a class C , and add this value on top of the expected value
Revised: to a class C , and add this value on top of the expected value

Original: E C ( f ( x )) , we reach the predicted value of our model f ( x ) , which in turn determines the class (thehighest value wins).
Revised: E C ( f ( x )) , we reach the predicted value of our model f ( x ) , which in turn determines the class (the highest value wins).

Original: Therefore, the sign and magnitude of a feature contribution matter as well.
Revised: Therefore, the sign and magnitude of a feature contribution matter as well.

Original: We compute explanations only for the best performing model, assuming that only this model wouldbe used in deployment.
Revised: We compute explanations only for the best performing model, assuming that only this model would be used in deployment.

Original: As LIME needs to compute some basic data features to generate explanations,it has to be initialized on the explanation training set.
Revised: As LIME needs to compute some basic data features to generate explanations, it has to be initialized on the explanation training set.

Original: We use LIME tabular and retrieve the five mostimportant features ( d = 5 ).
Revised: We use LIME tabular and retrieve the five most important features ( d = 5 ).

Original: Evaluations of explanations are computed over the explanation test set.
Revised: Evaluations of explanations are computed over the explanation test set.

Original: 4.3 Urban and rural areas
Revised: 4.3 Urban and rural areas

Original: We hypothesize that the ML model predicts electrification rates with different accuracy for rural andurban areas, i.e. that the model is biased w.r.t to the population density.
Revised: We hypothesize that the ML model predicts electrification rates with different accuracy for rural and urban areas, i.e. that the model is biased w.r.t to the population density.

Original: Therefore, we identify theseregions in the test datasets, and calculate the disaggregated accuracies.
Revised: Therefore, we identify these regions in the test datasets, and calculate the disaggregated accuracies.

Original: Further, we compute thedisaggregated explanations.
Revised: Further, we compute the disaggregated explanations.

Original: We identify urban regions based on a population density p > 1000 /km 2 ,as previously done in [25], otherwise as rural. 
Revised: We identify urban regions based on a population density p > 1000 /km 2 , as previously done in [25], otherwise as rural. [Table] [Figure]

Original:  Results 
Revised:  Results 5.1 Classification

Original: 5.1 Classification

Original: While all models perform similarly well, best performance is achieved by the random forest model( acc = 0 .
Revised: While all models perform similarly well, best performance is achieved by the random forest model ( acc = 0 .

Original: 516 , MAE = 0 .
Revised: 516 , MAE = 0 .

Original: 972 , see table 2).
Revised: 972 , see table 2).

Original: As the classification is ordered, an MAE around onemeans that on average, the electrification rate is wrongly estimated by 0 . 1 , which is acceptable.
Revised: As the classification is ordered, an MAE around one means that on average, the electrification rate is wrongly estimated by 0 . 1 , which is acceptable.

Original: Notethat while we need a sufficiently high performance of our models to proceed, the focus of this paperis not on model performance.[Table] [Figure]
Revised: Note that while we need a sufficiently high performance of our models to proceed, the focus of this paper is not on model performance.

Original: 5.2 Explanations
Revised: 5.2 Explanations

Original: We display results in figure 2, where we plotted the average importance (left, LIME) and the averagecontribution (right, SHAP) w.r.t.
Revised: We display results in figure 2, where we plotted the average importance (left, LIME) and the average contribution (right, SHAP) w.r.t.

Original: the predicted class.
Revised: the predicted class.

Original: Both for LIME and SHAP, features based ontext message data is highly relevant.
Revised: Both for LIME and SHAP, features based on text message data are highly relevant.

Original: This is especially prominent for outgoing and incoming eventsand the degree centrality, and for class 9.
Revised: This is especially prominent for outgoing and incoming events and the degree centrality, and for class 9.

Original: Only in the case of LIME, we observe negative values (e.g.,total events based on call length, for class 9).
Revised: Only in the case of LIME, we observe negative values (e.g., total events based on call length, for class 9).

Original: The importance, and high contribution of text messagedata for the prediction is in line with the observation that text message activity could be correlatedwith the electrification rate (see also section 3.1).
Revised: The importance, and high contribution of text message data for the prediction is in line with the observation that text message activity could be correlated with the electrification rate (see also section 3.1).

Original: As such, information provided by this type of datais highly relevant for the model, providing cues to better discriminate.
Revised: As such, information provided by this type of data is highly relevant for the model, providing cues to better discriminate.

Original: For SHAP, we also computed the average over feature contributions w.r.t.
Revised: For SHAP, we also computed the average over feature contributions w.r.t.

Original: all possible output classes.
Revised: all possible output classes.

Original: An analysis of this can be found in the appendix (section A.3).
Revised: An analysis of this can be found in the appendix (section A.3).

Original: Both methods confirm that the model relies on features that indeed relate to the predicted outcome.
Revised: Both methods confirm that the model relies on features that indeed relate to the predicted outcome.

Original: While a more in-depth analysis of the model and explanations is needed, this is a first good result.
Revised: While a more in-depth analysis of the model and explanations is needed, this is a first good result.

Original: 5.3 Urban and rural areas
Revised: 5.3 Urban and rural areas

Original: Disaggregated accuracies are displayed in table 2.
Revised: Disaggregated accuracies are displayed in table 2.

Original: While the accuracy for rural areas is always lowerthan the accuracy as computed for the full test set, the opposite is the case for urban areas.
Revised: While the accuracy for rural areas is always lower than the accuracy as computed for the full test set, the opposite is the case for urban areas.

Original: Thisdifference in accuracy means that the model is biased w.r.t.
Revised: This difference in accuracy means that the model is biased w.r.t.

Original: to the population density.
Revised: to the population density.

Original: We also computed disaggregated explanations which show similar patterns as described in theprevious section (see appendix section A.4).
Revised: We also computed disaggregated explanations, depicted in figure 3. They confirm the general relevance of features based on text message data for the prediction, as demonstrated already in the general case (previous section).

Original: Unsurprisingly, they show different distribution ofclasses: while urban areas belong to class 6 or higher, rural areas span all classes.
Revised: Unsurprisingly, they show different distribution of classes: while urban areas belong to class 6 or higher, rural areas span all classes.

Revised: This is also reflected in the relevance of the features based on text message data, we observe highest importance vales (contributions) as provided by LIME (SHAP) for different classes, depending on whether we look at urban or rural sub-populations.

Original: We further observethat the feature importance values, or feature contributions, are consistently higher in magnitude forthe urban compared to rural sub-population.
Revised: We further observe that the feature importance values, or feature contributions, are consistently higher in magnitude for the urban compared to rural sub-population.

Original: This supports our finding that the model is biased w.r.t.the population density.
Revised: This supports our finding that the model is biased w.r.t. the population density.

Original: Limitations
Revised: [Figure] [Table] [Figure] [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: o u t / i n _ c l t e _ s n o u t _ s n i n _ s n d c _ s n cc _ s n o u t / i n _ s n t

Original: {+e _ c n o u t _ c n i n _ c n d c _ c n cc _ c n o u t / i n _ c n t e _ c l o u t _ c l i n _ c l d c _ c l cc _ c l o u t / i n _ c l t e _ s n o u t _ s n i n _ s n d c _ s n cc _ s n o u t / i n _ s n 
+}
Revised: {+e _ c n o u t _ c n i n _ c n d c _ c n cc _ c n o u t / i n _ c n t e _ c l o u t _ c l i n _ c l d c _ c l cc _ c l o u t / i n _ c l t e _ s n o u t _ s n i n _ s n d c _ s n cc _ s n o u t / i n _ s n 
+}

Revised: [Figure]

Revised: p r e d i c t

Revised: [Figure]

Original:  Although the data come from a network provider who is the market leader ( 65% in 2013), they arenot fully representative of the population.
Revised: d c [Figure]  Limitations Although the data come from a network provider who is the market leader ( 65% in 2013), they are not fully representative of the population.

Original: A good starting point to investigate this further is the workby Pestre et al.
Revised: A good starting point to investigate this further is the work by Pestre et al.

Original: Salat et al.
Revised: Salat et al.

Original: [25] point towards other biases, for example due to a shift from themobile phone network to relying increasingly on internet platforms such as Facebook.
Revised: [25] point towards other biases, for example due to a shift from the mobile phone network to relying increasingly on internet platforms such as Facebook.

Original: Accountingfor these biases, including further work on the bias w.r.t.
Revised: Accounting for these biases, including further work on the bias w.r.t.

Original: population density is a next step.
Revised: population density is a next step.

Original: Also, we would like to apply other XAI methods such as LORE [9] to the ML model to understand whetherwe can extract some additional information, and run the explanations across all trained models.
Revised: Also, we would like to apply other XAI methods such as LORE [9] to the ML model to understand whether we can extract some additional information, and run the explanations across all trained models.

Original: Data used in this work are proprietary and private.
Revised: Data used in this work are proprietary and private.

Original: It is connected to individuals (their mobile phone)and to be processed only under their consent.
Revised: It is connected to individuals (their mobile phone) and to be processed only under their consent.

Original: It was provided in an anonymized form and furtheraggregated to safeguard individuals.
Revised: It was provided in an anonymized form and further aggregated to safeguard individuals.

Original: Being proprietary, the data cannot be shared beyond the project.
Revised: Being proprietary, the data cannot be shared beyond the project.

Original: An alternative could be relying on other open source data, such as satellite imagery, which havealready been proven useful for similar projects.
Revised: An alternative could be relying on other open source data, such as satellite imagery, which have already been proven useful for similar projects.

Original: While being fundamentally different to mobile phonedata, extending the XAI use case to these data are a valuable path to follow.
Revised: While being fundamentally different to mobile phone data, extending the XAI use case to these data are a valuable path to follow.

Original: The data we used originate from Senegal, a country of the Global South.
Revised: The data we used originate from Senegal, a country of the Global South.

Original: Being situated in Western
Revised: Being situated in Western

Original: Europe, we should critically reflect on how this might perpetuate power relationships [1].
Revised: Europe, we should critically reflect on how this might perpetuate power relationships [1].

Original: Whileour focus is on providing a use case of XAI, and to primarily verify a specific ML model that canestimate electrification rates in Senegal, we acknowledge the importance of local knowledge anddomain expertise when evaluating data, and specifically when drawing policy implications [1, 14].
Revised: While our focus is on providing a use case of XAI, and to primarily verify a specific ML model that can estimate electrification rates in Senegal, we acknowledge the importance of local knowledge and domain expertise when evaluating data, and specifically when drawing policy implications [1, 14].

Original:  Conclusion
Revised:  Conclusion

Original: In this work, after showing that electrification rates can be estimated from mobile phone data, weapplied two local, model-agnostic explanation tools to verify our model.
Revised: In this work, after showing that electrification rates can be estimated from mobile phone data, we applied two local, model-agnostic explanation tools to verify our model.

Original: Both explanations performwell, and agree with each other on stressing the relevance of text-based data for the predicted outcome.
Revised: Both explanations perform well, and agree with each other on stressing the relevance of text message data for the predicted outcome.

Original: They confirm the general validity of the model.
Revised: They confirm the general validity of the model.

Original: We also showed that our model is biased w.r.t.
Revised: We also showed that our model is biased w.r.t.

Original: topopulation densities, based on disaggregated accuracies and explanations.
Revised: to population densities.

Original: Thus, areas located in arural area receive an unfair prediction, i.e. are more likely to be linked to a wrong electrification rate.
Revised: Thus, areas located in a rural area receive an unfair prediction, i.e. are more likely to be linked to a wrong electrification rate.

Original: While the prediction of socio-economic indicators from remotely accessible data is not novel in itself,it is of high relevance, e.g., to support policy planning.
Revised: While the prediction of socio-economic indicators from remotely accessible data is not novel in itself, it is of high relevance, e.g., to support policy planning.

Original: Verifying such a model using XAI techniquesis certainly important, and novel.
Revised: Verifying such a model using XAI techniques is certainly important, and novel.

Original: This is complementary to the fact that there are generally fewuse-cases applying an XAI method on a real-world problem, most of them centered around the Global
Revised: This is complementary to the fact that there are generally few use-cases applying an XAI method on a real-world problem, most of them centered around the Global

Original: North.
Revised: North.

Original: A recent survey on XAI projects centered around the Global South [21] showed that whilethe body of work in this field is growing rapidly, only 16 of the surveyed papers relate to the Global
Revised: A recent survey on XAI projects centered around the Global South [21] showed that while the body of work in this field is growing rapidly, only 16 of the surveyed papers relate to the Global

Original: South, with only one technical study that has similar focus and design as ours [13].
Revised: South, with only one technical study that has similar focus and design as ours [13].

Original: Our analysis showed that XAI methods can be useful to verify an ML model in practice.
Revised: Our analysis showed that XAI methods can be useful to verify an ML model in practice.

Original: However, wewould like to caution against using these tools blindly, and summarize the challenges that emergedduring our work as follows:
Revised: However, we would like to caution against using these tools blindly, and summarize the challenges that emerged during our work as follows:

Original: Pipeline Design
Revised: Pipeline Design

Original: If the aim is to use an XAI method, data processing and choosing the modeltasks are limited by the fact that most XAI methods focus on tabular, image or text data, and onclassification problems.
Revised: If the aim is to use an XAI method, data processing and choosing the model tasks are limited by the fact that most XAI methods focus on tabular, image or text data, and on classification problems.

Original: Adapting a problem to this could lead to a loss of information, and lowerprediction accuracy.
Revised: Adapting a problem to this could lead to a loss of information, and lower prediction accuracy.

Original: Thus, efforts should be made to provide explanations for other task and datatypes (such as LASTS [11] for time series data, and beyond).
Revised: Thus, efforts should be made to provide explanations for other task and data types (such as LASTS [11] for time series data, and beyond).

Original: Domain Knowledge While an isolated XAI method can be very useful for debugging purposes,domain knowledge is necessary to draw real-world connections, and eventually verify the modelvia the explanation.
Revised: Domain Knowledge While an isolated XAI method can be very useful for debugging purposes, domain knowledge is necessary to draw real-world connections, and eventually verify the model via the explanation.

Original: In our work, an example of such knowledge is the distribution of text messageactivity over Senegal.
Revised: In our work, an example of such knowledge is the distribution of text message activity over Senegal.

Original: While domain knowledge needs to be available in the first place, it is usuallyexternal to the explanation.
Revised: While domain knowledge needs to be available in the first place, it is usually external to the explanation.

Original: A direct integration into XAI methods via symbolic approaches could betherefore highly useful [4, 6].
Revised: A direct integration into XAI methods via symbolic approaches could be therefore highly useful [4, 6].

Original: Work presented in this paper relies on a static network.
Revised: Work presented in this paper relies on a static network.

Original: We initially kept a second version of thedataset in the form of time series.
Revised: We initially kept a second version of the dataset in the form of time series.

Original: Details on these experiments, including the trained ML model andexplanations based on LASTS [11] can be found in the appendix A.5.
Revised: Details on these experiments, including the trained ML model and explanations based on LASTS [11] can be found in the appendix A.5.

Original: The time series data posedseveral challenges, among others, an high need of computing resources, few XAI approaches to useand compare, and a considerably higher amount of domain knowledge needed for the interpretationof the data.
Revised: The time series data posed several challenges, among others, an high need of computing resources, few XAI approaches to use and compare, and a considerably higher amount of domain knowledge needed for the interpretation of the data.

Original: For these reasons, we decided to continue working only on the network data, and leavethe time series data for future work.
Revised: For these reasons, we decided to continue working only on the network data, and leave the time series data for future work.

Original: Appendix
Revised: Acknowledgments and Disclosure of Funding

Original: [Figure]
Revised: This work has received funding from the European Union’s Horizon 2020 research and innovation programme under Marie Sklodowska-Curie Actions (grant agreement number 860630) for the project “NoBIAS - Artificial Intelligence without Bias” ( nobias-project.eu ). This work reflects only the authors’ views and the European Research Executive Agency (REA) is not responsible for any use that may be made of the information it contains. Authors’ contributions Laura State: conceptualization, experiments, paper draft, writing and editing Hadrien Salat: data preparation, paper co-writing and editing Stefania Rubrichi: data curation, paper reviewing, cosupervision Zbigniew Smoreda: data curation, paper reviewing

Original: A 
Revised: A Appendix A.1 Data Distribution

Original: [Figure] [Figure]

Original: Figure 3 shows the distribution of available call data (left panel) and text message data (right panel).
Revised: Figure 4 shows the distribution of available call data (left panel) and text message data (right panel).

Original: While both data are more dense in the Western part of Senegal, text message data are specificallysparse in the Eastern part of the country.
Revised: While both data are more dense in the Western part of Senegal, text message data are specifically sparse in the Eastern part of the country.

Original: A.2 Data Preparation
Revised: A.2 Data Preparation

Original: We build a directed communication network from the data (one per data type).
Revised: We build a directed communication network from the data (one per data type).

Original: Edges are formedbased on the activity in the network.
Revised: Edges are formed based on the activity in the network.

Original: Per cell tower (outgoing site in the original dataset) we aggregateover the receiving tower (incoming site in the original dataset) and sum over respective events, forexample the number of calls between those two towers.
Revised: Per cell tower (outgoing site in the original dataset) we aggregate over the receiving tower (incoming site in the original dataset) and sum over respective events, for example the number of calls between those two towers.

Original: We thereby do not take care of the time stamp.
Revised: We thereby do not take care of the time stamp.

Original: The total number of events per connection, and in the full year 2013, determines the weight of theedge.
Revised: The total number of events per connection, and in the full year 2013, determines the weight of the edge.

Original: This will lead to a full matrix ( M events ).
Revised: This will lead to a full matrix ( M events ).

Original: It has elements on its main diagonal, as calls/messagesappear also within a Voronoi cell.
Revised: It has elements on its main diagonal, as calls/messages appear also within a Voronoi cell.

Original: Also, it is not symmetric as the number of outgoing and incomingcalls/messages between two cell towers is generally not the same.
Revised: Also, it is not symmetric as the number of outgoing and incoming calls/messages between two cell towers is generally not the same.

Original: We repeat this network construction for all three types of data (call numbers, call length and messagedata, ).
Revised: We repeat this network construction for all three types of data (call numbers, call length and message data, ).

Original: We extract six features per cell tower per network, and aggregate them, resulting in a datasetof dimensionality 1587 × 18 .
Revised: We extract six features per cell tower per network, and aggregate them, resulting in a dataset of dimensionality 1587 × 18 .

Original: A.3 Additional Plot SHAP
Revised: A.3 Additional Plot SHAP

Original: t e _ c n o u t _ c n i n _ c n d c _ c n cc _ c n o u t / i n _ c n t e _ c l o u t _ c l i n _ c l d c _ c l cc _ c l o u t / i n _ c l t e _ s n o u t _
Revised: t e _ c n o u t _ c n i n _ c n d c _ c n cc _ c n o u t / i n _ c n t e _ c l o u t _ c l i n _ c l d c _ c l cc _ c l o u t / i n _ c l t e _ s n o u t _

Original: s n i n _
Revised: s n i n _

Original: s n d c _ s n c c
Revised: s n d c _ s n c c

Original: _ s n o u t / i n _ s n 02468p
Revised: _ s n o u t / i n _ s n 02468p

Original: o
Revised: o

Original: ss i b l e o u t p u t c l a ss 0.
Revised: ss i b l e o u t p u t c l a ss 0.

Revised: contribution of a feature w.r.t.

Revised: the possible output class.

Original: Figure 4 show the average feature contribution w.r.t.
Revised: Figure 5 show the average feature contribution w.r.t.

Original: to all possible output classes.
Revised: to all possible output classes.

Original: Why do we observe negative contributions for low classes?
Revised: Why do we observe negative contributions for low classes?

Original: For a single data instance, featuresbased on text message data generally pushes the model towards the predicted class (high positivecontribution) but at the same time away from the other classes (high negative contribution), togetherthey form the set of possible output classes.
Revised: For a single data instance, features based on text message data generally pushes the model towards the predicted class (high positive contribution) but at the same time away from the other classes (high negative contribution), together they form the set of possible output classes.

Original: This effect is more prominent for data instances fromhigher classes, thus positive (negative) contributions are particular high for high (low) classes.
Revised: This effect is more prominent for data instances from higher classes, thus positive (negative) contributions are particular high for high (low) classes.

Original: Opposed, in figure 2, right, for a better comparability with LIME, we only plotted contribution w.r.t.to the predicted class, thus only positive contributions appear.
Revised: Opposed, in figure 2, right, for a better comparability with LIME, we only plotted contribution w.r.t. to the predicted class, thus only positive contributions appear.

Original: A.4 Disaggregated Explanations
Revised: A.4 Disaggregated Explanations

Original: Disaggregated explanations are shown in figure 5. Results show similar patterns as for the aggregated(all test data) case. The different distributions of classes w.r.t the sub-population (rural/urban) isclearly visible. Also, values are consistently higher for urban than for rural areas.t
Revised: Disaggregated explanations, based on SHAP and w.r.t.

Original: e _ c n o u t _ c n i n _ c n d c _ c n cc _ c n o u t / i n _ c n t e _ c l o u t _ c l i n _ c l d c _ c l cc _ c l o u t / i n _ c l t e _ s n o u t _ s n i n _ s n d c _ s n cc _ s n o u t / i n _ s n 02468p r e d i c t e d c l a ss 0. t e _ c n o u t _ c n i n _ c n d c _ c n cc _ c n o u t / i n _ c n t e _ c l o u t _ c l i n _ c l d c _ c l cc _ c l o u t / i n _ c l t e _ s n o u t _ s n i n _ s n d c _ s n cc _ s n o u t / i n _ s n 02468p r e d i c t e d
Revised: possible output classes are shown in figure 6.

Original: [Figure]
Revised: [Figure]

Original: [Table] [Figure] [Table]
Revised: _ c n o u t _ c n i n _ c n d c _ c n cc _ c n o u t / i n _ c n t e _ c l o u t _ c l i n _ c l d c _ c l cc _ c l o u t / i n _ c l t e _ s n o u t _ s n i n _ s n d c _ s n cc _ s n o u t / i n _ s n

Original: [Figure]
Revised: [Figure]

Original: [Table] [Table] [Table] p r e d i c t e  c 
Revised: p o ss i b l e o u t p u t c l a ss

Original: [Figure]
Revised: [Figure]

Original: t e _ c n o u t _ c n i n _ c n d c _ c n cc _ c n o u t / i n _ c n t e _ c l o u t _ c l i n _ c l d c _ c l cc _ c l o u t / i n _ c l t e _ s n o u t _

Original: s n i n _

Original: s n d c _ s n cc _ s n o u t / i n _ s n 02468p

Original: o ss i b l e o u t p u t c l a ss 0.

Original: t e _ c n o u t _ c n i n _ c n d c _ c n cc _ c n o u t / i n _ c n t e _ c l o u t _ c l i n _ c l d c _ c l cc _ c l o u t / i n _ c l t e _ s n o u t _ s n i n _ s n d c _ s n cc _ s n o u t / i n _ s n 02468p o ss i b l e o u t p u t c l a ss 0.

Original: A.5 Time Series Data
Revised: A.5 Time Series Data

Original: In this section, we briefly describe the work on time series data.
Revised: In this section, we briefly describe the work on time series data.

Original: [Table]

Original: A.5.1 Data Processing
Revised: A.5.1 Data Processing

Original: A time series S = s 1 ..
Revised: A time series S = s 1 ..

Original: s T consists of T = 

Revised: s T consists of T =

Original: × 12 ordered data points, each being the monthlyaverage of the aggregated number of events per hour, such that s t , t ∈ 1 .. 24 represent the monthlyaverage of the aggregated number of events per hour in January (“daily activity curve ” for January),etc.
Revised: × 12 ordered data points, each being the monthly average of the aggregated number of events per hour, such that s t , t ∈ 1 .. 24 represent the monthly average of the aggregated number of events per hour in January (“daily activity curve ” for January), etc.

Original: Events are separated by direction (incoming or outgoing).
Revised: Events are separated by direction (incoming or outgoing).

Original: Thus, per cell tower, we create sixtime series.
Revised: Thus, per cell tower, we create six time series.

Original: We refer to the TS dataset based on number of calls as CN, based on length of callsas CL and based on number of text messages as SN, and use out/in for outgoing/incoming activity,respectively.
Revised: We refer to the TS dataset based on number of calls as CN, based on length of calls as CL and based on number of text messages as SN, and use out/in for outgoing/incoming activity, respectively.

Original: We standardize each of the time series separately by applying the min-max scaler asprovided by sklearn.
Revised: We standardize each of the time series separately by applying the min-max scaler as provided by sklearn.

Original: Data labeling and subsampling applies as above.
Revised: Data labeling and subsampling applies as above.

Original: Data partitioning follows [11].
Revised: Data partitioning follows [11].

Original: We find that text message data is heavily imbalanced, and is smaller than the other datasets.
Revised: We find that text message data is heavily imbalanced, and is smaller than the other datasets.

Original: Thus, weexclude this data from the time series analysis.
Revised: Thus, we exclude this data from the time series analysis.

Original: The variational autoencoder that is used in the explanation as displayed below, is trained for k = 50dimensions and over e = 500 epochs.
Revised: The variational autoencoder that is used in the explanation as displayed below, is trained for k =dimensions and over e = 500 epochs.

Original: We used the “out CL” data and model for explanations as itprovides the smallest MAE .
Revised: We used the “out CL” data and model for explanations as it provides the smallest MAE .

Original: A.5.2 Classifications
Revised: A.5.2 Classifications

Original: To classify based on time series data, we use ROCKET (RandOm Convolutional KErnel Transform) [8], a method based on random convolutional kernels for feature extraction and linear classification.
Revised: To classify based on time series data, we use ROCKET (RandOm Convolutional KErnel Transform) [8], a method based on random convolutional kernels for feature extraction and linear classification.

Original: Results are displayed in table 3.
Revised: Results are displayed in table 3.

Original: A.5.3 Explanations
Revised: A.5.3 Explanations

Original: Figure 6 shows a sample explanation by LASTS [11].
Revised: Figure 7 shows a sample explanation by LASTS [11].

Original: Explanations are provided in visual form andas rule, the latter can be read off the plot.
Revised: Explanations are provided in visual form and as rule, the latter can be read off the plot.

Original: The time series belongs to class 4, i.e. has an electrificationrate between 0 . 4 and 0 . 5 , and is correctly classified by the model.
Revised: The time series belongs to class 4, i.e. has an electrification [Table]

Original: In the left panel, the factual rule is plotted against the original time series. The number above the shapelet indicates its index.
Revised: as

Original: The rulereads as follows: “If shapelet no. 12 is contained in the time series, then it is classified as class 4.”
Revised: : “If shapelet no. 12 is contained in the time series, then it is classified as class 4.”

Original: This is mirrored by the rule for instancesbelonging to the opposite class (here: class 0..3, 5..9): ‘Ifshapelet no. 12 isnot contained in the time series, then it is not classified as class 4.”, displayed infigure 6, right, plotted against a synthetically generated time series from a class different to class 4.
Revised: This is mirrored by the rule for instances the ‘If shapelet no. 12 is