# Employing a Radial-Basis Function Artificial Neural Network to Classify Western and Transition European Economies Based on the Emissions of Air Pollutants and on Their Income

Kyriaki Kitikidou, Lazaros Iliadis

► To cite this version:

Kyriaki Kitikidou, Lazaros Iliadis. Employing a Radial-Basis Function Artificial Neural Network to Classify Western and Transition European Economies Based on the Emissions of Air Pollutants and on Their Income. 12th Engineering Applications of Neural Networks (EANN 2011) and 7th Artificial Intelligence Applications and Innovations (AIAI), Sep 2011, Corfu, Greece. pp.141-149, 10.1007/978-3-642-23960-1_18. hal-01571479

HAL Id: hal-01571479

https://inria.hal.science/hal-01571479

Submitted on 2 Aug 2017

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Employing a radial-basis function artificial neural network to classify western and transition European economies based on the emissions of air pollutants and on their income

Kyriaki Kitikidou¹, Lazaros Iliadis²

¹,²Democritus University of Thrace, Department of Forestry and Management of the Environment and Natural Resources, Pandazidou 193, 68200, Orestiada, Greece

¹<kkitikid@fmenr.duth.gr>

**Abstract.** This paper aims in comparing countries with different energy strategies, and demonstrate the close connection between environment and economic growth in the ex-Eastern countries, during their transition to market economies. We have developed a radial-basis function neural network system, which is trained to classify countries based on their emissions of carbon, sulphur and nitrogen oxides, and on their Gross National Income. We used three countries representative of ex-Eastern economies (Russia, Poland and Hungary) and three countries representative of Western economies (United States, France and United Kingdom). Results showed that the linkage between environmental pollution and economic growth has been maintained in ex-Eastern countries.

**Keywords:** Artificial Neural Networks, Atmospheric pollutants, Radial-Basis Function, Transition economies

## 1 Introduction

There is a general assumption that the former socialist economies failed, as regards environmental protection. Nevertheless, failure makes sense only if one has a success to oppose it with. It is interesting to see if the environmental damage associated with the economic growth of socialist systems is comparable with the negative externalities that, since the industrial evolution, have accompanied the economic development of Western countries. Likewise, during the transition to market economies, we should look for a similar development of the forces that have allowed the improvement of environmental performances in Western countries [15].

Transition to a market economy, for all its long-term rewards, has not been an easy process. In addition to the necessary but often difficult social, political and economic adjustments, the countries of Central and Eastern Europe have had to cope with the environmental legacy of inefficient industries, obsolete and polluting technologies, and weak environmental policies ([4]; [5]; [10]).
In 1993, many serious environmental problems have been identified, several that directly threatened human health and required immediate and urgent attention. These included ([12]; [13]):

- high levels of airborne particulates in urban and industrial "hot spots" from coal combustion by domestic users, small-scale enterprises, power and heating plants, and metallurgical plants;

- high levels of sulphur dioxide and other air pollutants in "hot spot" areas from the combustion high-sulphur coal and fuel oil by these and other sources;

- lead concentrations in air and soil linked to airborne emissions from industry and from the use of leaded gasoline; and

- contamination of drinking water, in particular by nitrates in rural well water.

World Bank analyses of the role of economic and environmental reforms in reducing air pollution levels in ex-Eastern countries have used econometric models to adjust for changes in GDP levels, industrial output and energy use. Overall, these analyses suggest that economic reforms, environmental policies, and other developments over the transition have produced real environmental benefits in the advanced reform countries, beyond emissions reductions resulting from declines in output and energy use. In slower reform countries, such declines appear to be the main factors behind the emissions reductions. If these countries were to return to growth without further economic reforms and effective environmental policies, their air pollution emissions could increase accordingly ([21]; [22]).

The aim of this paper is to allow us to understand the nature of environmental paths in socialist countries. It would be interesting to see if there is a connection between transition of the economies of ex-Eastern countries and environmental pollution, in opposition to Western economies. This analysis is based on the comparison of atmospheric pollutant emissions from energy use of major emission intensity components, using artificial neural networks (ANN). Countries are categorized by energy patterns. The innovative part of this research effort lies in the use of a soft computing machine learning approach like the ANN to predict group memberships of countries with different energy strategies.

## 2 Materials and Methods

We have selected three ex-Eastern countries, Russia, Poland and Hungary, and three OECD (Organisation for Economic Co-operation and Development) countries, United States, France and the United Kingdom, representative of different energy strategies [20]. The emissions of atmospheric pollutants in ex-Eastern countries, for the year 1994, were estimated from an independent study, because of the inadequacy of international statistics as regards ex-Eastern countries [20]. All other data are taken from the OECD libraries [11]. The three atmospheric pollutants under investigation here are carbon dioxide (CO2 measured in t/toe), nitrogen oxides (NOx measured in kg/toe) and sulphur dioxide (SO2 measured in t/toe). Emissions were estimated by four intensity components (coal, natural gas, oil, and electricity-heat). The six countries, and the four intensity components, resulted in 24 cases in our data. The
Gross National Income (GNI) per capita was measured in PPP international dollars, i.e. a common currency that eliminates the differences in price levels between countries allowing meaningful volume comparisons of GNI between countries.

Descriptive statistics for all variables are given in **Table 1**.

<table><caption>**Table 1.** Descriptive statistics for all variables used in the analysis.</caption><thead><tr><th></th><th></th><th>CO<sub>2</sub> (t/toe)</th><th>NOx (kg/toe)</th><th>SO<sub>2</sub> (t/toe)</th><th>GNI (PPP int. $)</th></tr></thead><tbody><tr><td rowspan="4">Ex-Eastern countries</td><td>Mean</td><td>3.5</td><td>12.7</td><td>33.0</td><td>6910</td></tr><tr><td>Standard deviation</td><td>1.8</td><td>9.4</td><td>38.9</td><td>982</td></tr><tr><td>Minimum</td><td>2.0</td><td>2.3</td><td>0.0</td><td>6020</td></tr><tr><td>Maximum</td><td>8.4</td><td>38.0</td><td>124.3</td><td>8210</td></tr><tr><td rowspan="4">OECD countries</td><td>Mean</td><td>3.2</td><td>11.3</td><td>19.8</td><td>21510</td></tr><tr><td>Standard deviation</td><td>1.8</td><td>9.7</td><td>22.9</td><td>3503</td></tr><tr><td>Minimum</td><td>0.7</td><td>2.0</td><td>0.0</td><td>18750</td></tr><tr><td>Maximum</td><td>6.7</td><td>34.6</td><td>65.5</td><td>26230</td></tr></tbody></table>

For the performance of the analysis, the radial-basis function (RBF) network model was used ([1]; [6]; [14]; [17]; [19]), from menu of the SPSS v.19 statistical package [8]. We specified that the relative number of cases assigned to the training:testing:holdout samples should be 6:2:1. This assigned 2/3 of the cases to training, 2/9 to testing, and 1/9 to holdout. The architecture of the developed ANN included three neurons in the hidden layer. The transfer functions (hidden layer activation functions and output function) determine the output by depicting the result of the distance function ([2]; [9]). The schematic representation of the neural network with transfer functions is given in **Fig. 1**.
<figure><img src="image_2.png" /></figure>

Hidden layer activation function: Softmax, Output layer activation function: Identity
intcomp = intensity component, countcat = country category

<caption>Fig. 1. Radial-basis function network structure.</caption>
## 3 Results-Discussion

From the analysis, 15 cases (62.5%) were assigned to the training sample, 6 (25.0%) to the testing sample, and 3 (12.5%) to the holdout sample. No cases were excluded from the analysis. The number of units in the input layer is the number of covariates plus the total number of factor levels; a separate unit is created for each category of emission intensity component and GNI, and none of the categories are considered redundant. Likewise, a separate output unit is created for each country category, for a total of 2 units in the output layer. Three units were chosen in the hidden layer.

**Table 2** shows the practical results of using the network. For each case, the predicted response is the category with the highest predicted pseudo-probability. Cells on the diagonal are correct predictions, while cells off the diagonal are incorrect predictions. The RBF network gets 80.0% of the countries. In particular, the RBF model excels at identifying ex-Eastern countries (100.0% for the training, testing and holdout samples). The holdout sample helps to validate the model; here, 66.7% of these cases were correctly classified by the model. This suggests that the RBF model is in fact correct about three out of five times. The lack of correct classification for the OECD countries to the testing sample (0.0%) must be due to the small data set available, which naturally limits the possible degree of complexity of the model [3].

<table><caption>**Table 2.** Confusion matrix.</caption><thead><tr><th rowspan="2">Sample</th><th rowspan="2">Observed</th><th colspan="3">Predicted</th></tr><tr><th>Ex-Eastern country</th><th>OECD country</th><th>Classified correctly</th></tr></thead><tbody><tr><td rowspan="3">Training</td><td>Ex-Eastern country</td><td>9</td><td>0</td><td>100.0%</td></tr><tr><td>OECD country</td><td>3</td><td>3</td><td>50.0%</td></tr><tr><td>Overall Percent</td><td>80.0%</td><td>20.0%</td><td>80.0%</td></tr><tr><td rowspan="3">Testing</td><td>Ex-Eastern country</td><td>2</td><td>0</td><td>100.0%</td></tr><tr><td>OECD country</td><td>4</td><td>0</td><td>0.0%</td></tr><tr><td>Overall Percent</td><td>100.0%</td><td>0.0%</td><td>33.3%</td></tr><tr><td rowspan="3">Holdout</td><td>Ex-Eastern country</td><td>1</td><td>0</td><td>100.0%</td></tr><tr><td>OECD country</td><td>1</td><td>1</td><td>50.0%</td></tr><tr><td>Overall Percent</td><td>66.7%</td><td>33.3%</td><td>66.7%</td></tr></tbody></table>
For categorical dependent variables, the predicted-by-observed chart displays clustered boxplots of predicted pseudo-probabilities for the combined training and testing samples. The x axis corresponds to the observed response categories, and the legend corresponds to predicted categories. The leftmost boxplot shows, for cases that have observed category ex-Eastern country, the predicted pseudo-probability of category ex-Eastern country. The next boxplot to the right shows, for cases that have observed category ex-Eastern country, the predicted pseudo-probability of category OECD country. The third boxplot shows, for cases that have observed category OECD country, the predicted pseudo-probability of category OECD country. Lastly, the fourth boxplot shows, for cases that have observed category OECD country, the predicted pseudo-probability of category OECD country.

From looking at a portion of cases in one boxplot, and the corresponding location of those cases in another boxplot, we determine that ex-Eastern countries are better classified than the OECD countries (the third and fourth boxplots are roughly equivalent). This was the conclusion extracted from the confusion matrix also.

<figure><img src="image_3.png" /><figcaption>Fig. 2. Predicted-by-observed chart.</figcaption></figure>

Receiver operating characteristics (ROC) graphs are useful for organizing classifiers and visualizing their performance [16]. ROC graphs are commonly used in
medical decision making, and in recent years have been used increasingly in machine learning and data mining research. The numbers along the major diagonal represent the correct decisions made, and the numbers of this diagonal represent the errors – the confusion – between the various classes. The true positive rate (sensitivity) of a classifier is estimated as:

$$ tp \text{ rate} \approx \frac{\text{Positives correctly classified}}{\text{Total positives}} $$

while the false positive rate of the classifier is:

$$ fp \text{ rate} \approx \frac{\text{Negatives incorrectly classified}}{\text{Total negatives}} $$

Specificity of the ROC curve is defined as:

$$ specificity=\frac{\text{True negatives}}{\text{False positives}+\text{True negatives}} $$

The ROC curve in Fig. 3 gives us a visual display of the sensitivity by specificity for all possible classification cutoffs. The chart shown here displays two curves, one for each category of the target variable (country category). It seems that, for each category, the probability that the predicted pseudo-probability of a country being in that category is equal for a randomly chosen country in that category to a randomly chosen country not in that category.

<figure><img src="image_4.png" /><figcaption>Fig. 3. ROC curves.</figcaption></figure>
The cumulative gains chart in **Fig. 4a** shows the percentage of the overall number of cases in a given category gained by targeting a percentage of the total number of cases. For example, the first point on the curve for the ex-Eastern country category is approximately at (10%, 30%), meaning that if we score a dataset with the network and sort all of the cases by predicted pseudo-probability of ex-Eastern country, we would expect the top 10% to contain approximately 30% of all of the cases that actually take the category ex-Eastern country. The diagonal line is the baseline curve; the farther above the baseline a curve lies, the greater the gain. The lift chart in **Fig. 4b** is derived from the cumulative gains chart; the values on the y axis correspond to the ratio of the cumulative gain for each curve to the baseline. It provides another way of looking at the information in the cumulative gains chart.

<figure><img src="image_5.png" /><figcaption>Fig. 4a. Gains chart.</figcaption></figure>

<figure><img src="image_6.png" /><figcaption>Fig. 4b. Lift chart.</figcaption></figure>

# 4 Conclusions

The radial-basis function neural network model that was trained to categorize countries, based on atmospheric pollutant emissions from energy use of major emission intensity components and economic indicators, appears to perform pretty well, considering the small number of cases (as more data is given to the model, the prediction becomes more reliable). The development of such models could help us to investigate if there is a disconnection of economic growth and environmental pollution, for countries that are in the process of being economically developed. The four components of emission intensities have generally been unfavourable to the environment in ex-Eastern Europe. It seems that the connection between environment pollution and economic growth is close in the East because of the difficulty these countries have in reducing their energy intensity. While OECD countries have gradually succeeded in disconnecting economic growth and energy consumption, the linkage has been maintained in ex-Eastern countries [20].
Increasing energy prices, macroeconomic stabilization programmes, industrial restructuring and increasing industrial investments will force countries in transition to change their environmental habits [7]. However, environmental measures and foreign aid will not have a lasting, positive impact on the environment without the development of the economic, legal and social institutions of a market economy [18].

## Aknoledgements

We wish to thank Mr James Kitchen, Marketing Manager of Public Affairs & Communications Directorate of OECD, who gave us online access to the OECD publications.

## References

1. Bishop, C.: Neural Networks for Pattern Recognition, 3rd ed. Oxford University Press, Oxford (1995)

2. Bors, A., Pitas, I.: Radial Basis function networks In: Howlett, R., Jain, L (eds.). Recent Developments in Theory and Applications in Robust RBF Networks: 125-153 Heidelberg, NY, Physica-Verlag (2001)

3. Dendek, C., Mańdziuk, J.: Improving Performance of a Binary Classifier by Training Set Selection. Warsaw University of Technology, Faculty of Mathematics and Information Science, Warsaw, Poland (2008)

4. European Bank for Reconstruction and Development (EBRD): Transition Report Update, London (1998)

5. Gaddy, C., Ickes, B.: Russia's Virtual Economy. Foreign Affairs (1998)

6. Haykin, S.: Neural Networks: A Comprehensive Foundation, 2nd ed. Macmillan College Publishing, New York (1998)

7. Hughes, G.: Are the costs of cleaning up Eastern Europe exaggerated? Economic Reform and the Environment. Oxford Review of Economic Policy 7(4): 106-136 (1991)

8. IBM SPSS Neural Networks 19. SPSS Inc (2010)

9. Iliadis, L.: Intelligent Information Systems and Applications in Risk Management. Stamoulis editions, Thessaloniki, Greece (2007)

10. Lovei, M., Levy, B.: Lead Exposure and Health in Central and Eastern Europe. In: Lovei, M. (ed.). Phasing out Lead from Gasoline in Central and Eastern Europe. Health Issues, Feasibility, and Policies. World Bank, Washington DC (1997)

11. OECD/IEA: Energy balances of OECD countries 1993-1994. Paris (1996)

12. OECD: Economic Survey: Russia. Paris (1998)

13. OECD: Environmental Data Compendium. Paris (1997)

14. Ripley, B.: Pattern Recognition and Neural Networks. Cambridge University Press, Cambridge (1996)

15. Shahgedanova, M., Burt, T.: New data on air pollution in the former Soviet Union. Global Environmental Change 4(3): 201-227 (1994)
16. Streiner, D., Cairney, J.: What's Under the ROC? An Introduction to Receiver Operating Characteristics Curves. The Canadian Journal of Psychiatry 52: 121-128 (2007)

17. Tao, K.: A closer look at the radial basis function (RBF) networks. In: Singh, A. (ed.). Conference Record of the Twenty-Seventh Asilomar Conference on Signals, Systems, and Computers. IEEE Computational Society Press, Los Alamitos, California (1993)

18. Toman, M., Simpson, R.: Environmental policies, economic restructuring and institutional development in the former Soviet Union. Resources 116: 20-23 (1994)

19. Uykan, Z., Guzelis, C., Celebi, M., Koivo, H.: Analysis of input-output clustering for determining centers of RBFN. IEEE Transactions on Neural Networks 11: 851-858 (2000)

20. Viguier, L.: Emissions of SO₂, NOx and CO₂ in transition economies: emission inventories and Divisia index analysis. The Energy Journal 20(2): 59-88 (1999)

21. World Bank: National Biodiversity Strategies and Action Plans. Preliminary Summary of Findings from Eastern Europe and Central Asia. World Bank, Washington DC (1998a)

22. World Bank: Transition Toward a Healthier Environment: Environmental Issues and Challenges in the Newly Independent States. World Bank, Washington DC (1998b)