# Enhancing Countries’ Fitness with Recommender Systems on the International Trade Network

Authors: Hao Liao; Xiao-Min Huang; Xing-Tong Wu; Ming-Kai Liu; Alexandre Vidmer; Ming-Yang Zhou; Yi-Cheng Zhang
Journal: Complexity
 (2018)

---

## Abstract

Prediction is one of the major challenges in complex systems. The prediction methods have shown to be effective predictors of the evolution of networks. These methods can help policy makers to solve practical problems successfully and make better strategy for the future. In this work, we focus on exporting countries’ data of the International Trade Network. A recommendation system is then used to identify the products that correspond to the production capacity of each individual country but are somehow overlooked by the country. Then, we simulate the evolution of the country’s fitness if it would have followed the recommendations. The result of this work is the combination of these two methods to provide insights to countries on how to enhance the diversification of their exported products in a scientific way and improve national competitiveness significantly, especially for developing countries.


---

## Body

## 1. Introduction

International trade plays a considerable role in the exchange channels between countries [1, 2]. It is also becoming more important as time goes on. In 1960, international trade accounted for roughly 25% of a country’s total Gross Domestic Product (GDP). Nowadays it accounts for nearly 60% of countries GDP [3]. Historically, various classic economic models were developed to evaluate the dissimilarity of wealth resulting from the exportation of diverse goods [4]. Models based on the gravity equation were also developed and showed to be adequate to explain many of the features of the international trade [5]. Another approach, the Product Space, attempted to illustrate how the nations will develop in the future by projecting the exports data onto a two-dimensional map and observing the diffusion of the export process [6]. Economic models mainly consist of commodity profits, geographical relations, comprehensive productivity, economic structure [7], and a series of macroeconomic elements. In practice, numerous complex external factors, e.g., national policies, religious beliefs, and the country’s current production capacity, are key components in the international trade. Here, we also recognize that plenty of sociologists have constructed grand theories on the empirical study of global economic development [8–10]. These methods and theories were developed by David Snyder and Edward L. Kick [11]; it was the first study of international trade and world economic growth using a network framework in the American Journal of Sociology, 1979. In [12], the authors emphasized the importance of the global trade, and the structure of the modern world system was addressed by multiple-network analysis. The complex network approach was also used in [13], and it was shown that the International Trade Network and the World Wide Web both have collaborative characteristics [14]. Indeed, the International Trade Network is a complex network in terms of structure. With this is in mind, we apply recommendation algorithms that are usually applied on e-commerce systems in order to predict the evolution of the International Trade Network [15–17].Two methods were proposed to assign scores to countries and products using the complex network approach. The basis of these methods is to analyze the relation between countries and their exports and then to rank the countries according to their economic competitiveness and the exported good with the economic advantage they bring to the exporting countries. The first method, the Method of Reflection (MR), proposed the Economic Complexity Index (ECI) to account for the production characteristics of countries. The countries are ranked according to their exporting capacity. The authors pointed out that the method is able to predict the future economic expansion of countries and that the scores obtained with MR indicate which products should be exported in order to maximize the countries’ performance [18]. In [19], a method based on the Markov chain was proposed. The method showed the need to take into account the nonlinear interactions between exported commodities and national diversity. This led to the development of the Fitness and Complexity metric [20]. It is an iterative method based on the nonlinearity of the system. This method was shown to be conceptually more grounded than the previous ones and a better predictor of the economic evolution [21, 22]. This method was applied to forecast the GDP growth in a recent work. The authors reported that their estimates were on average 25 percent more accurate than were those made by the International Monetary Fund [23].In this paper, we start with a detailed definition of the used algorithms, namely, the Probabilistic Spreading algorithm [24], the Heat Spreading algorithm [25], the Degree Increase algorithm [26], and the Time-Aware Probabilistic Spreading algorithms [27]. We then apply these four algorithms to the International Trade Network and compare their performance on two different aspects. The first one is the evaluation of the accuracy of the recommendation, which is a standard evaluation of the recommendation algorithms. The second aspect is the evaluation of the algorithms on their ability to recommend products that would improve the countries’ fitness. This is obtained by the combination of the recommendation results with the Fitness and Complexity scores to simulate the changes in rankings and fitness values of countries after exporting recommended products. The experimental results confirm the validity of the recommendation algorithm on this task and show the validity of our approach to tackle this problem.


## 2. Materials and Methods

### 2.1. International Trade Network

2001 to 2015. This dataset was cleaned usingharmonization techniques in [28] and the similar categories were merged together. Additionally, the countries that had no entries recorded for exportation between 2001 and 2015 were removed. After cleaning procedure, the International Trade Network was comprised of 192 countries and 786 commodities. We use the bipartite network approach to represent the data. In this approach, one set of nodes represents the countries, and the second set of nodes represents the commodities. If a country is considered as an exporter of a commodity, a link connects the country’s node and the commodity node.RCA. The data of the International Trade Network includes country nodes and commodity nodes. These two types of nodes form a bipartite network. One important aspect of our network representation is that it is binary. Either two nodes are connected, or they are not. We then need a criterion to define if a country can be considered as an exporter of a commodity or not. Indeed, even if countries cannot produce a specific commodity, they might export a very small amount of it. Therefore it should not be considered as an exporter of the commodity. In order to quantify the advantage of a country on a commodity, we use the “Revealed Comparative Advantage” (RCA) as a clear constraint to determine whether a country can be considered as an exporter of certain commodities [29].(1)RCAiα=eiα/∑jejα∑βeiβ/∑jβejβ,where eiα is the total amount of export α for country i. If the country i is regarded as an exporter of the commodity α, the export amount of the commodity α should occupy a larger proportion in the total amount of the export goods of the country i. We set the RCA value to 1 and the country-commodity will have links when this pair of nodes satisfies the condition of RCAiα≥1. These nodes and links together constitute the bipartite network of the international trade activities.


### 2.2. Methods

#### 2.2.1. Recommendation System

Recommendation systems aim at recommending the most adequate items for users. In comparison with traditional link prediction, the focus is put on individual nodes rather than individual links. In our case, the main focus of the recommendation process is the countries nodes. For each country node in the system, the algorithms compute a score for every product in the network. If the country is already considered as an exporter of an item, the score corresponding to the product is set to 0 (i.e., a product that is already exported is not recommended). The recommendation list for each country is composed by the topL products with the highest score for the particular country. We now describe the algorithms used in this work. Note that L is set to 20 in this work and it has been shown to have no impact on the significance of the accuracy [17].In order to compare the performance of the five recommender algorithms, we choose a yearT and predict which additional product the countries export at year T+5 (the testing set) based solely on the data up to year T (the training set).Probabilistic Spreading (ProbS). For target user i, the initial resources are first distributed evenly on the item side and then propagated to the user side through a random walk process [24]. In the same way, the resources are then returned to the item side. Both steps are used to allocate resources among neighbors and then spread to the other side. Finally, the score of each item for country i is obtained. The initial resource vector is represented as f(i) and its elements are represented as fα(i)=aiα. The final resource values sα(i) can be written as(2)sαi=∑β=1IWαβfβi.The elements of the redistribution matrix W are derived from Probabilistic Spreading process.(3)Wαβ=1kβ∑j=1Uajαajβkj,where I denotes the number of products and U the number of countries. ajαajβ represents path from item β to item α through country j in two random walks. The ProbS algorithm employs a column-normalized transition matrix. The partition of kβ and kj corresponds to the uniform partition of resources between all links from nodes β and j [30].Heat Spreading (HeatS). The HeatS spreading algorithm evolved from the Probabilistic Spreading algorithm. These two methods are similar in structure and both use random walk processes to the redistribute initial resources. Compared with the ProbS algorithm, resources spread more evenly in the HeatS spreading method and items with only few connections usually benefit from a higher score than with the ProbS algorithm [25].The initial resource vector is set tof(i) according to user’s item collection; the elements fα(i)=aiα can be regarded as the temperature value of the item. Unlike the ProbS algorithm, which uses column normalization transformation matrix, the HeatS algorithm uses row normalization. The spreading process is represented by a matrix as follows, where W′=WT:(4)W′αβ=1kα∑j=1Uajαajβkj.Resource received by the user i is equal to the average resource owned by the user’s collected item. Similarly, item side receives resources transmitted from the user through the averaging process. The item’s score is calculated as(5)hαi=∑β=1IW′αβfβi.Degree Increase (DI). The time information is often overlooked in the evolution of complex networks. In fact, time plays a crucial role in the evolution of information networks [31–33]. The combination of recommendation systems with time dynamics improves the recommendation and allows performing better predictions. It has been proven that the Degree Increase (DI) method can accurately predict the prevalence of items in the future. The increase in popularity of item α within time window τ is(6)Δkαt,τ=kαt-kαt-τ,where item degree kαt=∑iAiαt corresponds to the number of users who have collected item α. The final item score is expressed as(7)Δk′αt,τ=Δkαt,τ+εkαt.In practice, the value of ε must be small enough to ensure that the ranking of commodity popularity growth Δkα(t,τ) does not change.Time-Aware Probabilistic Spreading (TProbS). The ProbS method is characterized by the fact that spread of resources is cumulative; that is, the more popular items are more likely to get high scores and get recommended to users. The Time-Aware Probabilistic Spreading method integrates the ProbS method with the DI method. It is written mathematically as(8)uαi=sαiΔk′αt,τkαtθ,where the parameter θ is an additional parameter to define the length of past time window.


#### 2.2.2. Fitness and Complexity Metrics

The Fitness and Complexity metrics are used to measure the competitiveness of countries and the quality of exported products. The algorithm consists of two nonlinear coupling equations [20, 34], which eventually reach a fixed value through iterative methods and the equations are defined as (9)Fin=∑α=1IaiαQαn-1(10)Qαn=1∑i=1Uaiα1/Fin-1,where Fin indicates the fitness of country i after n iterations. In [35], the convergence of the algorithm and its stopping condition were demonstrated. The higher the fitness value, the more advantageous the variety and complexity of the goods exported by the country. The study [36] shows that the weak economies can analyze how to get out of the poverty trap and increase the diversification of their exports via fitness metrics. Complexity Qαn cannot simply be calculated from the average of countries’ fitness. Successful countries export almost all products and it is unable to infer the complexity of each product from their export data (for example, our data shows that a total of 786 products were included in the International Trade Network from 2001 to 2015, and the United States exported a total of 775 kinds of products). Therefore, the complexity of product should be measured in a nonlinear way; namely, it is essential to reduce contribution of successful countries. Assuming that product α possesses two exporters i and j with fitness values of 0.2 and 15, respectively, the complexity of the product would be 0.197. If the two exporters i and j have fitness values of 10 and 15, the complexity would be 6. These two examples verify the fact that the value of complexity depends mainly on the worst exporter.


#### 2.2.3. Precision and Recall Metrics

In general, recommendation algorithms are compared in their ability to predict the future. In order to evaluate their accuracy, we use two metrics, namely, the Precision and the Recall metrics. We are not interested in the accuracy of the algorithms per se, but the accuracy of the recommendation results is an important evaluation indicator. A higher accuracy indicates that the exporting countries possess the capacity to produce the recommended commodities. Indeed, the countries should be able to produce the recommended commodities in the near future; otherwise the recommendation process would be meaningless.Precision. For each country, the individual Precision is measured as the fraction of recommended products that are eventually exported. If ni is the number of products that are eventually exported by country i and that are actually recommended, then the Precision for country i is Pi=ni/L. The Precision P is the average of Pi over every country.Recall. Recall is similar to Precision, but we use the number of newly exported goods instead of the fixed recommendation list L. If country i exports Ei additional items in the testing set, the individual Recall reads Ri=ni/Ei. The Recall is the average of Ri over every country.


### 2.3. Addition of Products to Countries’ Export Basket

After having obtained the recommendation list from the previously described recommendation methods, we add the goods that are at the top of the recommendation lists to the respective countries’ exports baskets. Then, the fitness value of each country is reevaluated and its change is recorded. However, the calculations of fitness are an iterative nonlinear process, which is coupled with the complexity of the products. Thus, in order to evaluate the changes brought by adding the products to the country’s export basket, only one country is modified at a time. In other words, for a given countryi, only its recommendation list is added to its exports basket and the rest of the network is left untouched. For instance, if we have a high complexity product to the export basket of a low fitness country, the complexity of the product might be greatly reduced, which will have a strong impact on the rest of the Fitness and Complexity values. The simulation process is described as below. For each country, we add the L products in its recommendation list to its export basket. For each product α in the top L part of the recommendation list of country i, we add a link between the country i and the product α in the data. The export volume of product α from country i must satisfy the condition of RCA≥1.


## 2.1. International Trade Network

2001 to 2015. This dataset was cleaned usingharmonization techniques in [28] and the similar categories were merged together. Additionally, the countries that had no entries recorded for exportation between 2001 and 2015 were removed. After cleaning procedure, the International Trade Network was comprised of 192 countries and 786 commodities. We use the bipartite network approach to represent the data. In this approach, one set of nodes represents the countries, and the second set of nodes represents the commodities. If a country is considered as an exporter of a commodity, a link connects the country’s node and the commodity node.RCA. The data of the International Trade Network includes country nodes and commodity nodes. These two types of nodes form a bipartite network. One important aspect of our network representation is that it is binary. Either two nodes are connected, or they are not. We then need a criterion to define if a country can be considered as an exporter of a commodity or not. Indeed, even if countries cannot produce a specific commodity, they might export a very small amount of it. Therefore it should not be considered as an exporter of the commodity. In order to quantify the advantage of a country on a commodity, we use the “Revealed Comparative Advantage” (RCA) as a clear constraint to determine whether a country can be considered as an exporter of certain commodities [29].(1)RCAiα=eiα/∑jejα∑βeiβ/∑jβejβ,where eiα is the total amount of export α for country i. If the country i is regarded as an exporter of the commodity α, the export amount of the commodity α should occupy a larger proportion in the total amount of the export goods of the country i. We set the RCA value to 1 and the country-commodity will have links when this pair of nodes satisfies the condition of RCAiα≥1. These nodes and links together constitute the bipartite network of the international trade activities.


## 2.2. Methods

### 2.2.1. Recommendation System

Recommendation systems aim at recommending the most adequate items for users. In comparison with traditional link prediction, the focus is put on individual nodes rather than individual links. In our case, the main focus of the recommendation process is the countries nodes. For each country node in the system, the algorithms compute a score for every product in the network. If the country is already considered as an exporter of an item, the score corresponding to the product is set to 0 (i.e., a product that is already exported is not recommended). The recommendation list for each country is composed by the topL products with the highest score for the particular country. We now describe the algorithms used in this work. Note that L is set to 20 in this work and it has been shown to have no impact on the significance of the accuracy [17].In order to compare the performance of the five recommender algorithms, we choose a yearT and predict which additional product the countries export at year T+5 (the testing set) based solely on the data up to year T (the training set).Probabilistic Spreading (ProbS). For target user i, the initial resources are first distributed evenly on the item side and then propagated to the user side through a random walk process [24]. In the same way, the resources are then returned to the item side. Both steps are used to allocate resources among neighbors and then spread to the other side. Finally, the score of each item for country i is obtained. The initial resource vector is represented as f(i) and its elements are represented as fα(i)=aiα. The final resource values sα(i) can be written as(2)sαi=∑β=1IWαβfβi.The elements of the redistribution matrix W are derived from Probabilistic Spreading process.(3)Wαβ=1kβ∑j=1Uajαajβkj,where I denotes the number of products and U the number of countries. ajαajβ represents path from item β to item α through country j in two random walks. The ProbS algorithm employs a column-normalized transition matrix. The partition of kβ and kj corresponds to the uniform partition of resources between all links from nodes β and j [30].Heat Spreading (HeatS). The HeatS spreading algorithm evolved from the Probabilistic Spreading algorithm. These two methods are similar in structure and both use random walk processes to the redistribute initial resources. Compared with the ProbS algorithm, resources spread more evenly in the HeatS spreading method and items with only few connections usually benefit from a higher score than with the ProbS algorithm [25].The initial resource vector is set tof(i) according to user’s item collection; the elements fα(i)=aiα can be regarded as the temperature value of the item. Unlike the ProbS algorithm, which uses column normalization transformation matrix, the HeatS algorithm uses row normalization. The spreading process is represented by a matrix as follows, where W′=WT:(4)W′αβ=1kα∑j=1Uajαajβkj.Resource received by the user i is equal to the average resource owned by the user’s collected item. Similarly, item side receives resources transmitted from the user through the averaging process. The item’s score is calculated as(5)hαi=∑β=1IW′αβfβi.Degree Increase (DI). The time information is often overlooked in the evolution of complex networks. In fact, time plays a crucial role in the evolution of information networks [31–33]. The combination of recommendation systems with time dynamics improves the recommendation and allows performing better predictions. It has been proven that the Degree Increase (DI) method can accurately predict the prevalence of items in the future. The increase in popularity of item α within time window τ is(6)Δkαt,τ=kαt-kαt-τ,where item degree kαt=∑iAiαt corresponds to the number of users who have collected item α. The final item score is expressed as(7)Δk′αt,τ=Δkαt,τ+εkαt.In practice, the value of ε must be small enough to ensure that the ranking of commodity popularity growth Δkα(t,τ) does not change.Time-Aware Probabilistic Spreading (TProbS). The ProbS method is characterized by the fact that spread of resources is cumulative; that is, the more popular items are more likely to get high scores and get recommended to users. The Time-Aware Probabilistic Spreading method integrates the ProbS method with the DI method. It is written mathematically as(8)uαi=sαiΔk′αt,τkαtθ,where the parameter θ is an additional parameter to define the length of past time window.


### 2.2.2. Fitness and Complexity Metrics

The Fitness and Complexity metrics are used to measure the competitiveness of countries and the quality of exported products. The algorithm consists of two nonlinear coupling equations [20, 34], which eventually reach a fixed value through iterative methods and the equations are defined as (9)Fin=∑α=1IaiαQαn-1(10)Qαn=1∑i=1Uaiα1/Fin-1,where Fin indicates the fitness of country i after n iterations. In [35], the convergence of the algorithm and its stopping condition were demonstrated. The higher the fitness value, the more advantageous the variety and complexity of the goods exported by the country. The study [36] shows that the weak economies can analyze how to get out of the poverty trap and increase the diversification of their exports via fitness metrics. Complexity Qαn cannot simply be calculated from the average of countries’ fitness. Successful countries export almost all products and it is unable to infer the complexity of each product from their export data (for example, our data shows that a total of 786 products were included in the International Trade Network from 2001 to 2015, and the United States exported a total of 775 kinds of products). Therefore, the complexity of product should be measured in a nonlinear way; namely, it is essential to reduce contribution of successful countries. Assuming that product α possesses two exporters i and j with fitness values of 0.2 and 15, respectively, the complexity of the product would be 0.197. If the two exporters i and j have fitness values of 10 and 15, the complexity would be 6. These two examples verify the fact that the value of complexity depends mainly on the worst exporter.


### 2.2.3. Precision and Recall Metrics

In general, recommendation algorithms are compared in their ability to predict the future. In order to evaluate their accuracy, we use two metrics, namely, the Precision and the Recall metrics. We are not interested in the accuracy of the algorithms per se, but the accuracy of the recommendation results is an important evaluation indicator. A higher accuracy indicates that the exporting countries possess the capacity to produce the recommended commodities. Indeed, the countries should be able to produce the recommended commodities in the near future; otherwise the recommendation process would be meaningless.Precision. For each country, the individual Precision is measured as the fraction of recommended products that are eventually exported. If ni is the number of products that are eventually exported by country i and that are actually recommended, then the Precision for country i is Pi=ni/L. The Precision P is the average of Pi over every country.Recall. Recall is similar to Precision, but we use the number of newly exported goods instead of the fixed recommendation list L. If country i exports Ei additional items in the testing set, the individual Recall reads Ri=ni/Ei. The Recall is the average of Ri over every country.


## 2.2.1. Recommendation System

Recommendation systems aim at recommending the most adequate items for users. In comparison with traditional link prediction, the focus is put on individual nodes rather than individual links. In our case, the main focus of the recommendation process is the countries nodes. For each country node in the system, the algorithms compute a score for every product in the network. If the country is already considered as an exporter of an item, the score corresponding to the product is set to 0 (i.e., a product that is already exported is not recommended). The recommendation list for each country is composed by the topL products with the highest score for the particular country. We now describe the algorithms used in this work. Note that L is set to 20 in this work and it has been shown to have no impact on the significance of the accuracy [17].In order to compare the performance of the five recommender algorithms, we choose a yearT and predict which additional product the countries export at year T+5 (the testing set) based solely on the data up to year T (the training set).Probabilistic Spreading (ProbS). For target user i, the initial resources are first distributed evenly on the item side and then propagated to the user side through a random walk process [24]. In the same way, the resources are then returned to the item side. Both steps are used to allocate resources among neighbors and then spread to the other side. Finally, the score of each item for country i is obtained. The initial resource vector is represented as f(i) and its elements are represented as fα(i)=aiα. The final resource values sα(i) can be written as(2)sαi=∑β=1IWαβfβi.The elements of the redistribution matrix W are derived from Probabilistic Spreading process.(3)Wαβ=1kβ∑j=1Uajαajβkj,where I denotes the number of products and U the number of countries. ajαajβ represents path from item β to item α through country j in two random walks. The ProbS algorithm employs a column-normalized transition matrix. The partition of kβ and kj corresponds to the uniform partition of resources between all links from nodes β and j [30].Heat Spreading (HeatS). The HeatS spreading algorithm evolved from the Probabilistic Spreading algorithm. These two methods are similar in structure and both use random walk processes to the redistribute initial resources. Compared with the ProbS algorithm, resources spread more evenly in the HeatS spreading method and items with only few connections usually benefit from a higher score than with the ProbS algorithm [25].The initial resource vector is set tof(i) according to user’s item collection; the elements fα(i)=aiα can be regarded as the temperature value of the item. Unlike the ProbS algorithm, which uses column normalization transformation matrix, the HeatS algorithm uses row normalization. The spreading process is represented by a matrix as follows, where W′=WT:(4)W′αβ=1kα∑j=1Uajαajβkj.Resource received by the user i is equal to the average resource owned by the user’s collected item. Similarly, item side receives resources transmitted from the user through the averaging process. The item’s score is calculated as(5)hαi=∑β=1IW′αβfβi.Degree Increase (DI). The time information is often overlooked in the evolution of complex networks. In fact, time plays a crucial role in the evolution of information networks [31–33]. The combination of recommendation systems with time dynamics improves the recommendation and allows performing better predictions. It has been proven that the Degree Increase (DI) method can accurately predict the prevalence of items in the future. The increase in popularity of item α within time window τ is(6)Δkαt,τ=kαt-kαt-τ,where item degree kαt=∑iAiαt corresponds to the number of users who have collected item α. The final item score is expressed as(7)Δk′αt,τ=Δkαt,τ+εkαt.In practice, the value of ε must be small enough to ensure that the ranking of commodity popularity growth Δkα(t,τ) does not change.Time-Aware Probabilistic Spreading (TProbS). The ProbS method is characterized by the fact that spread of resources is cumulative; that is, the more popular items are more likely to get high scores and get recommended to users. The Time-Aware Probabilistic Spreading method integrates the ProbS method with the DI method. It is written mathematically as(8)uαi=sαiΔk′αt,τkαtθ,where the parameter θ is an additional parameter to define the length of past time window.


## 2.2.2. Fitness and Complexity Metrics

The Fitness and Complexity metrics are used to measure the competitiveness of countries and the quality of exported products. The algorithm consists of two nonlinear coupling equations [20, 34], which eventually reach a fixed value through iterative methods and the equations are defined as (9)Fin=∑α=1IaiαQαn-1(10)Qαn=1∑i=1Uaiα1/Fin-1,where Fin indicates the fitness of country i after n iterations. In [35], the convergence of the algorithm and its stopping condition were demonstrated. The higher the fitness value, the more advantageous the variety and complexity of the goods exported by the country. The study [36] shows that the weak economies can analyze how to get out of the poverty trap and increase the diversification of their exports via fitness metrics. Complexity Qαn cannot simply be calculated from the average of countries’ fitness. Successful countries export almost all products and it is unable to infer the complexity of each product from their export data (for example, our data shows that a total of 786 products were included in the International Trade Network from 2001 to 2015, and the United States exported a total of 775 kinds of products). Therefore, the complexity of product should be measured in a nonlinear way; namely, it is essential to reduce contribution of successful countries. Assuming that product α possesses two exporters i and j with fitness values of 0.2 and 15, respectively, the complexity of the product would be 0.197. If the two exporters i and j have fitness values of 10 and 15, the complexity would be 6. These two examples verify the fact that the value of complexity depends mainly on the worst exporter.


## 2.2.3. Precision and Recall Metrics

In general, recommendation algorithms are compared in their ability to predict the future. In order to evaluate their accuracy, we use two metrics, namely, the Precision and the Recall metrics. We are not interested in the accuracy of the algorithms per se, but the accuracy of the recommendation results is an important evaluation indicator. A higher accuracy indicates that the exporting countries possess the capacity to produce the recommended commodities. Indeed, the countries should be able to produce the recommended commodities in the near future; otherwise the recommendation process would be meaningless.Precision. For each country, the individual Precision is measured as the fraction of recommended products that are eventually exported. If ni is the number of products that are eventually exported by country i and that are actually recommended, then the Precision for country i is Pi=ni/L. The Precision P is the average of Pi over every country.Recall. Recall is similar to Precision, but we use the number of newly exported goods instead of the fixed recommendation list L. If country i exports Ei additional items in the testing set, the individual Recall reads Ri=ni/Ei. The Recall is the average of Ri over every country.


## 2.3. Addition of Products to Countries’ Export Basket

After having obtained the recommendation list from the previously described recommendation methods, we add the goods that are at the top of the recommendation lists to the respective countries’ exports baskets. Then, the fitness value of each country is reevaluated and its change is recorded. However, the calculations of fitness are an iterative nonlinear process, which is coupled with the complexity of the products. Thus, in order to evaluate the changes brought by adding the products to the country’s export basket, only one country is modified at a time. In other words, for a given countryi, only its recommendation list is added to its exports basket and the rest of the network is left untouched. For instance, if we have a high complexity product to the export basket of a low fitness country, the complexity of the product might be greatly reduced, which will have a strong impact on the rest of the Fitness and Complexity values. The simulation process is described as below. For each country, we add the L products in its recommendation list to its export basket. For each product α in the top L part of the recommendation list of country i, we add a link between the country i and the product α in the data. The export volume of product α from country i must satisfy the condition of RCA≥1.


## 3. Results

### 3.1. Accuracy of the Recommendation Algorithms

The first step in the comparison of the algorithms is to compare their accuracies. We perform the predictions from years 2001 to 2010. The results are shown in Figure1. First, we see that the top performing method is the TProbS method. This is not surprising as this is the one including both the strength of ProbS and time information. The parameter θ=0.2 is constantly the one giving the highest Recall in our simulation and thus is fixed to this value. Obviously, this lessens the impact of degree increase on the network, which is compatible with the expected behavior of countries development. They should not all focus on the same products, but they still follow the ongoing trends. As noted in [17], the HeatS method performs surprisingly well. This method usually performs poorly in online e-commerce networks [25] but has the advantage of recommending products with lower degree. DI and Degree both perform poorly, which is expected as they do not take into account the production capabilities of countries.Figure 1
A comparison of Recall and Precision for the five algorithms. The recommendation is performed at yearT for year T+5, with T ranging from year 2001 to 2010. The results shown are averaged over this time period. For TProbS, we set a value of θ=0.2. The Degree method simply ranks the products according to their degree. All subsequent experiments were based on this recommendation.
(a) (b)


### 3.2. Tier Division

An interesting study is to investigate the impact of the methods on countries with different fitness rank. It is not certain that a country that belongs to the top of the fitness list will behave the same to a country that belongs at the bottom of the list. In order to study this effect, we divide the countries into three different categories according to their fitness rank. The three categories hold the same number of countries and thus are denoted as top tier, middle tier, and low tier. For each category, we compute the average increased ranking and increased fitness after the addition of the recommended exports. The results are shown in Figure2. The countries that benefit the most from these recommendations are middle and low tier countries. In panel (a), the most interesting result is in HeatS. The increased rankings by following its recommendation are much higher than those of TProbS. This is especially true with top tier countries, as following the recommendation of TProbS would even lower a country’s fitness. This comes from the fact that top tier countries need to innovate and produce goods for which there are only a few competitors, while, for middle and low tier countries, it is sufficient to produce additional items. Note that we try adding time to HeatS: the Recall improved to the level of TProbS, but the improvement of fitness was lower than that of HeatS. So we decided to keep only HeatS and TProbS for simplicity.Figure 2
Panel (a) is the average increase of fitness ranking for the three different tiers of countries for the time period 2008–2015. The number of goods added for each country is set toL=20. Panel (b) is the average increased fitness value of those three tiers.
(a) (b)From panel (b), we see that the fitness of top tier countries is the one improving the most, which shows that the recommendation algorithm indeed recommends different complex products in line with the ability of the country. For top tier countries, the difference of fitness between two countries is large, while, for lower tier countries, the fitness difference between two countries is much smaller. This fact explains the result that top tier countries’ fitness improvement is significant but their ranking is only slightly improved, while the low tier countries can increase the ranking a lot in the case of less improvement in fitness. To illustrate better our recommendation results, we select three countries in each tier. Due to the limit of the page size, we only choose the five products with the highest score according to TProbS and HeatS for each country. The products recommended by TProbS and HeatS are shown in Tables1 and 2, respectively. Both tables show the products with different complexity for various tier countries.Table 1
The top five recommended products by TProbS algorithms for three randomly selected countries in different tiers.
Tier Country Recommended products (top 5) Low tier Vanuatu Cigarettes Fruit, fresh or dried Plants and parts of trees used in perfumery; in pharmacy; etc. Sawlogs and veneer logs, of non-coniferous species Non-alcoholic beverages Tonga Spices, except pepper and pimento Sugar confectionery and preparations, non-chocolate Fish, dried, salted or in brine; smoked fish Cement Wood, simply shaped Dominica Meal and flour of wheat and flour of meslin Cask, drums, etc., of iron, steel, aluminium, for packing goods Packing containers, box files, etc., of paper, used in offices Beer made from malt Plastic packing containers, lids, stoppers and other closures  Middle tier Norway Animals, live (including zoo animals, pets, insects, etc.) Fuel wood and wood charcoal Bovine and equine hides, raw, whether or not split Bones, ivory, horns, coral, shells and similar products Soaps, organic products and preparations for use as soap Mauritius Skirts Vegetable products roots and tubers, fresh, dried Leather of other hides or skins Footwear Other outer garments Kyrgyzstan Oil seeds and oleaginous fruits Fish, fresh or chilled, excluding fillet Eggs, birds, and egg yolks, fresh, dried or preserved, in shell Fruit or vegetable juices Jams, jellies, marmalades, etc., as cooked preparations  Top tier Germany Iron, steel, aluminium reservoirs, tanks, etc., capacity 300 lt plus Non-domestic refrigerators and refrigerating equipment, parts Insulated electric wire, cable, bars, etc. Bottles etc. of glass Railway or tramway sleepers Italy Refined sugar etc. Fuel wood and wood charcoal Sugar confectionery and preparations Manufactures of mineral materials (other than ceramic) cotton, not elastic nor rubberized Switzerland Structures and parts of, of iron, steel; plates, rods, and the like Manufactures of mineral materials (other than ceramic) Other furniture and parts thereof Organic surface-active agents Miscellaneous articles of base metalTable 2
The top five recommended products by HeatS algorithms for three randomly selected countries in different tiers.
Tier Country Recommended products (top 5) Low tier Vanuatu Sugars, beet and cane, raw, solid Natural rubber latex; natural rubber and gums Jute, other textile bast fibers, raw, processed but not spun Ores and concentrates of uranium and thorium Nuts edible, fresh or dried Tonga Manila hemp, raw or processed but not spun Coconut (copra) oil Cocoa beans, raw, roasted Waxes of animal or vegetable origin Palm nuts and kernels Dominica Potassium salts, natural, crude Distilled alcoholic beverages Surveying, navigational, compasses, etc., instruments, nonelectrical Figs, fresh or dried Beer made from malt  Middle tier Norway Crustaceans and molluscs, fresh, chilled, frozen, salted Asbestos Radio-active chemical elements, isotopes etc. Ships, boats and other vessels Iron ore agglomerates Mauritius Fabrics, woven Fish, dried, salted or in brine; smoked fish Articles of apparel, clothing accessories of plastic or rubber Household appliances, decorative article, etc., of base metal Headgear and fitting thereof Kyrgyzstan Meat of sheep and goats, fresh, chilled or frozen Iron ore and concentrates, not agglomerated Coffee green, roasted; coffee substitutes containing coffee Cut flowers and foliage Railway or tramway sleepers (ties) of wood  Top tier Germany Photographic and cinematographic apparatus and equipment Organo-sulfur compounds Parts of the pumps and compressor Orthopedic appliances, hearing aids, artificial parts of the body Phenols and phenol-alcohols, and their derivatives Italy Digital central storage units, separately consigned Children’s toys, indoor games, etc. Optical instruments and apparatus Fabrics of glass fiber (including narrow, pile fabrics, lace, etc.) Printing presses Switzerland Electro-medical equipment Chemical products and preparations Parts of steam power units Spectacles and spectacle frames Other chemical derivatives of cellulose; vulcanized fiber


### 3.3. Evolution of Countries’ Fitness

In order to directly study the effect of each algorithm on the country’s fitness value, we randomly select 30 countries in the middle tier and compare the evolution in the average fitness of these countries for the different methods. We compare the normal evolution of countries and the one recommended by these algorithms. The results are shown in Figure3 (panel (a)). For any length of the recommendation list L, HeatS is the top performing by a great margin compared to other methods. For all methods, ranking is consistent for every choice of L, and so the choice of the value is not meaningful, as long as it is reasonably small compared to the total number of items. By looking at the evolution with the length of the recommendation list L, we find that the results are consistent and if the country should follow the recommendation, even if it can add only few products to its export basket. Following the recommendation made by HeatS seems to be adequate in terms of technological requirements (i.e., the countries have the required technology) as well as the best path to increase the country’s fitness. In Figure 3 (panel (b)), the results are shown for different years and for fixed L. Though some methods are better in a specific year than others, it is clear that HeatS is always the top performing method, followed by Degree and DI and then TProbS. This is good news as it shows the robustness of our method, both in the isolated case and in the real evolution of countries’ exports.Figure 3
Panel (a) shows fitness values as a function of the length of the recommendation listL. The results are average over 30 randomly selected countries and over the 15 years spanned by the data. Time evolution of fitness for the thirty selected middle tier countries that are shown in panel (b). The length of the recommendation list is set to L=20.We are also interested in the individual behavior of the fitness evolution. We randomly select four countries and study the effect of the method on the evolution of the fitness values. The comparison of the five methods as a function ofL and the original fitness is shown in Figure 4. Again, HeatS is the top performing method, except for Kazakhstan. For this country, DI and Degree are the top performing methods due to the low fitness of this country. The original fitness values corresponding to the four countries Croatia, Kazakhstan, Poland, and Colombia are 2.0563, 0.7234, 3.7678, and 1.0714, respectively. While we saw in Figure 3 that HeatS is always best on average, it is different when considering individual countries with especially low production. The impact of the recommendation on the increased ranking of the four countries is shown in Figure 5. We see in Figure 5 (panel (b)) that even if HeatS is not the best method, the difference in ranking is quite small, while in other panels it clearly outperforms other methods.Figure 4
Detailed view of the fitness as a function of the length of the recommendation listL for four randomly selected countries. The original fitness values of four countries Croatia, Kazakhstan, Poland, and Colombia are 2.0563, 0.7234, 3.7678, and 1.0714, respectively.
(a)
Croatia (b)
Kazakhstan (c)
Poland (d)
ColombiaFigure 5
Comparison of the increased rankings of the four selected countries. The increased ranking of each method is averaged over different lengths of the recommendation listL.
(a)
Croatia (b)
Kazakhstan (c)
Poland (d)
Colombia


### 3.4. Real Production Ability

In the previous results, we fixed a parameterL and assigned the same number of new products to every country. However, in reality some countries produce many new products in a specific year, while some others struggle more. To reflect this, we use a dynamic length of the recommendation Li, where Li is the length of the recommendation list for country i, equal to its number of new products between times T+5 and T. We build a “virtual network” corresponding exactly to the real one at T+5, except for one country i, for which we replace the links that appear between T and T+5 by those of the recommendation list. This ensures that the network is of constant size. The results for HeatS are shown in Figure 6. We see that HeatS improves greatly the fitness of most countries that follow its recommendation. Only a few countries see their fitness decreasing compared to their original evolution. As a comparison, we see in Figure 7 that TProbS is of no use for top tier countries, but for middle and especially low tier countries they might benefit from it. While lower than for HeatS, the recommendations of TProbS are made of more widespread technologies and so might be easier to follow for the low fitness countries.Figure 6
Difference of fitness resulting from following recommendation of HeatS and real evolution. The countries are sorted according to their fitness value, country 0 being the one with the highest average fitness. Each bar represents a country. The recommendation is performed at yearT for year T+5, with T ranging from year 2001 to 2010. 143 countries of the 181 countries would see their fitness improve by following the recommendation of HeatS.Figure 7
Difference of fitness resulting from following recommendation of TProbS and real evolution. The countries are sorted according to their fitness value, country 0 being the one with the highest average fitness. Each bar represents a country. The recommendation is performed at yearT for year T+5, with T ranging from year 2001 to 2010. 120 countries of the 181 countries would see their fitness improve by following the recommendation of TProbS.


## 3.1. Accuracy of the Recommendation Algorithms

The first step in the comparison of the algorithms is to compare their accuracies. We perform the predictions from years 2001 to 2010. The results are shown in Figure1. First, we see that the top performing method is the TProbS method. This is not surprising as this is the one including both the strength of ProbS and time information. The parameter θ=0.2 is constantly the one giving the highest Recall in our simulation and thus is fixed to this value. Obviously, this lessens the impact of degree increase on the network, which is compatible with the expected behavior of countries development. They should not all focus on the same products, but they still follow the ongoing trends. As noted in [17], the HeatS method performs surprisingly well. This method usually performs poorly in online e-commerce networks [25] but has the advantage of recommending products with lower degree. DI and Degree both perform poorly, which is expected as they do not take into account the production capabilities of countries.Figure 1
A comparison of Recall and Precision for the five algorithms. The recommendation is performed at yearT for year T+5, with T ranging from year 2001 to 2010. The results shown are averaged over this time period. For TProbS, we set a value of θ=0.2. The Degree method simply ranks the products according to their degree. All subsequent experiments were based on this recommendation.
(a) (b)


## 3.2. Tier Division

An interesting study is to investigate the impact of the methods on countries with different fitness rank. It is not certain that a country that belongs to the top of the fitness list will behave the same to a country that belongs at the bottom of the list. In order to study this effect, we divide the countries into three different categories according to their fitness rank. The three categories hold the same number of countries and thus are denoted as top tier, middle tier, and low tier. For each category, we compute the average increased ranking and increased fitness after the addition of the recommended exports. The results are shown in Figure2. The countries that benefit the most from these recommendations are middle and low tier countries. In panel (a), the most interesting result is in HeatS. The increased rankings by following its recommendation are much higher than those of TProbS. This is especially true with top tier countries, as following the recommendation of TProbS would even lower a country’s fitness. This comes from the fact that top tier countries need to innovate and produce goods for which there are only a few competitors, while, for middle and low tier countries, it is sufficient to produce additional items. Note that we try adding time to HeatS: the Recall improved to the level of TProbS, but the improvement of fitness was lower than that of HeatS. So we decided to keep only HeatS and TProbS for simplicity.Figure 2
Panel (a) is the average increase of fitness ranking for the three different tiers of countries for the time period 2008–2015. The number of goods added for each country is set toL=20. Panel (b) is the average increased fitness value of those three tiers.
(a) (b)From panel (b), we see that the fitness of top tier countries is the one improving the most, which shows that the recommendation algorithm indeed recommends different complex products in line with the ability of the country. For top tier countries, the difference of fitness between two countries is large, while, for lower tier countries, the fitness difference between two countries is much smaller. This fact explains the result that top tier countries’ fitness improvement is significant but their ranking is only slightly improved, while the low tier countries can increase the ranking a lot in the case of less improvement in fitness. To illustrate better our recommendation results, we select three countries in each tier. Due to the limit of the page size, we only choose the five products with the highest score according to TProbS and HeatS for each country. The products recommended by TProbS and HeatS are shown in Tables1 and 2, respectively. Both tables show the products with different complexity for various tier countries.Table 1
The top five recommended products by TProbS algorithms for three randomly selected countries in different tiers.
Tier Country Recommended products (top 5) Low tier Vanuatu Cigarettes Fruit, fresh or dried Plants and parts of trees used in perfumery; in pharmacy; etc. Sawlogs and veneer logs, of non-coniferous species Non-alcoholic beverages Tonga Spices, except pepper and pimento Sugar confectionery and preparations, non-chocolate Fish, dried, salted or in brine; smoked fish Cement Wood, simply shaped Dominica Meal and flour of wheat and flour of meslin Cask, drums, etc., of iron, steel, aluminium, for packing goods Packing containers, box files, etc., of paper, used in offices Beer made from malt Plastic packing containers, lids, stoppers and other closures  Middle tier Norway Animals, live (including zoo animals, pets, insects, etc.) Fuel wood and wood charcoal Bovine and equine hides, raw, whether or not split Bones, ivory, horns, coral, shells and similar products Soaps, organic products and preparations for use as soap Mauritius Skirts Vegetable products roots and tubers, fresh, dried Leather of other hides or skins Footwear Other outer garments Kyrgyzstan Oil seeds and oleaginous fruits Fish, fresh or chilled, excluding fillet Eggs, birds, and egg yolks, fresh, dried or preserved, in shell Fruit or vegetable juices Jams, jellies, marmalades, etc., as cooked preparations  Top tier Germany Iron, steel, aluminium reservoirs, tanks, etc., capacity 300 lt plus Non-domestic refrigerators and refrigerating equipment, parts Insulated electric wire, cable, bars, etc. Bottles etc. of glass Railway or tramway sleepers Italy Refined sugar etc. Fuel wood and wood charcoal Sugar confectionery and preparations Manufactures of mineral materials (other than ceramic) cotton, not elastic nor rubberized Switzerland Structures and parts of, of iron, steel; plates, rods, and the like Manufactures of mineral materials (other than ceramic) Other furniture and parts thereof Organic surface-active agents Miscellaneous articles of base metalTable 2
The top five recommended products by HeatS algorithms for three randomly selected countries in different tiers.
Tier Country Recommended products (top 5) Low tier Vanuatu Sugars, beet and cane, raw, solid Natural rubber latex; natural rubber and gums Jute, other textile bast fibers, raw, processed but not spun Ores and concentrates of uranium and thorium Nuts edible, fresh or dried Tonga Manila hemp, raw or processed but not spun Coconut (copra) oil Cocoa beans, raw, roasted Waxes of animal or vegetable origin Palm nuts and kernels Dominica Potassium salts, natural, crude Distilled alcoholic beverages Surveying, navigational, compasses, etc., instruments, nonelectrical Figs, fresh or dried Beer made from malt  Middle tier Norway Crustaceans and molluscs, fresh, chilled, frozen, salted Asbestos Radio-active chemical elements, isotopes etc. Ships, boats and other vessels Iron ore agglomerates Mauritius Fabrics, woven Fish, dried, salted or in brine; smoked fish Articles of apparel, clothing accessories of plastic or rubber Household appliances, decorative article, etc., of base metal Headgear and fitting thereof Kyrgyzstan Meat of sheep and goats, fresh, chilled or frozen Iron ore and concentrates, not agglomerated Coffee green, roasted; coffee substitutes containing coffee Cut flowers and foliage Railway or tramway sleepers (ties) of wood  Top tier Germany Photographic and cinematographic apparatus and equipment Organo-sulfur compounds Parts of the pumps and compressor Orthopedic appliances, hearing aids, artificial parts of the body Phenols and phenol-alcohols, and their derivatives Italy Digital central storage units, separately consigned Children’s toys, indoor games, etc. Optical instruments and apparatus Fabrics of glass fiber (including narrow, pile fabrics, lace, etc.) Printing presses Switzerland Electro-medical equipment Chemical products and preparations Parts of steam power units Spectacles and spectacle frames Other chemical derivatives of cellulose; vulcanized fiber


## 3.3. Evolution of Countries’ Fitness

In order to directly study the effect of each algorithm on the country’s fitness value, we randomly select 30 countries in the middle tier and compare the evolution in the average fitness of these countries for the different methods. We compare the normal evolution of countries and the one recommended by these algorithms. The results are shown in Figure3 (panel (a)). For any length of the recommendation list L, HeatS is the top performing by a great margin compared to other methods. For all methods, ranking is consistent for every choice of L, and so the choice of the value is not meaningful, as long as it is reasonably small compared to the total number of items. By looking at the evolution with the length of the recommendation list L, we find that the results are consistent and if the country should follow the recommendation, even if it can add only few products to its export basket. Following the recommendation made by HeatS seems to be adequate in terms of technological requirements (i.e., the countries have the required technology) as well as the best path to increase the country’s fitness. In Figure 3 (panel (b)), the results are shown for different years and for fixed L. Though some methods are better in a specific year than others, it is clear that HeatS is always the top performing method, followed by Degree and DI and then TProbS. This is good news as it shows the robustness of our method, both in the isolated case and in the real evolution of countries’ exports.Figure 3
Panel (a) shows fitness values as a function of the length of the recommendation listL. The results are average over 30 randomly selected countries and over the 15 years spanned by the data. Time evolution of fitness for the thirty selected middle tier countries that are shown in panel (b). The length of the recommendation list is set to L=20.We are also interested in the individual behavior of the fitness evolution. We randomly select four countries and study the effect of the method on the evolution of the fitness values. The comparison of the five methods as a function ofL and the original fitness is shown in Figure 4. Again, HeatS is the top performing method, except for Kazakhstan. For this country, DI and Degree are the top performing methods due to the low fitness of this country. The original fitness values corresponding to the four countries Croatia, Kazakhstan, Poland, and Colombia are 2.0563, 0.7234, 3.7678, and 1.0714, respectively. While we saw in Figure 3 that HeatS is always best on average, it is different when considering individual countries with especially low production. The impact of the recommendation on the increased ranking of the four countries is shown in Figure 5. We see in Figure 5 (panel (b)) that even if HeatS is not the best method, the difference in ranking is quite small, while in other panels it clearly outperforms other methods.Figure 4
Detailed view of the fitness as a function of the length of the recommendation listL for four randomly selected countries. The original fitness values of four countries Croatia, Kazakhstan, Poland, and Colombia are 2.0563, 0.7234, 3.7678, and 1.0714, respectively.
(a)
Croatia (b)
Kazakhstan (c)
Poland (d)
ColombiaFigure 5
Comparison of the increased rankings of the four selected countries. The increased ranking of each method is averaged over different lengths of the recommendation listL.
(a)
Croatia (b)
Kazakhstan (c)
Poland (d)
Colombia


## 3.4. Real Production Ability

In the previous results, we fixed a parameterL and assigned the same number of new products to every country. However, in reality some countries produce many new products in a specific year, while some others struggle more. To reflect this, we use a dynamic length of the recommendation Li, where Li is the length of the recommendation list for country i, equal to its number of new products between times T+5 and T. We build a “virtual network” corresponding exactly to the real one at T+5, except for one country i, for which we replace the links that appear between T and T+5 by those of the recommendation list. This ensures that the network is of constant size. The results for HeatS are shown in Figure 6. We see that HeatS improves greatly the fitness of most countries that follow its recommendation. Only a few countries see their fitness decreasing compared to their original evolution. As a comparison, we see in Figure 7 that TProbS is of no use for top tier countries, but for middle and especially low tier countries they might benefit from it. While lower than for HeatS, the recommendations of TProbS are made of more widespread technologies and so might be easier to follow for the low fitness countries.Figure 6
Difference of fitness resulting from following recommendation of HeatS and real evolution. The countries are sorted according to their fitness value, country 0 being the one with the highest average fitness. Each bar represents a country. The recommendation is performed at yearT for year T+5, with T ranging from year 2001 to 2010. 143 countries of the 181 countries would see their fitness improve by following the recommendation of HeatS.Figure 7
Difference of fitness resulting from following recommendation of TProbS and real evolution. The countries are sorted according to their fitness value, country 0 being the one with the highest average fitness. Each bar represents a country. The recommendation is performed at yearT for year T+5, with T ranging from year 2001 to 2010. 120 countries of the 181 countries would see their fitness improve by following the recommendation of TProbS.


## 4. Conclusion

In this study, we extended the traditional recommendations, which are usually applied on social networks, to the international trade. Thanks to recent advances in measuring the potential countries’ evolution based solely on the network structure, we designed a method which aims to help countries to find a suitable evolution path among all the possible ones. The study suffers two main limitations. The first one is that the exports categories are roughly defined. Only about 786 categories are present in the dataset. This leads to some categories containing very varied products, such asiron based goods. The second one is that there are important restrictions, or conversely support, to the trade between countries. For instance, USA, Canada, and Mexico recently signed an agreement to open the market of dairy and car parts. On the opposite side, countries might limit their sensitive exports, such as military goods, to specific countries. While the first limitation is difficult to address due to the data limitations, the second one can be added by weighting differently the relationships between countries on specific products.At the same time, the recommendation proved to grasp the countries technological evolution by being able to correctly predict the future, and the method of Fitness and Complexity has been shown in [37–39] to uncover hidden features of the countries’ evolution. It is important to note that the recommendation method should agree with the similar capabilities of a country [40]. Those two ingredients together mixed with our results show remarkable evidence that our methods as supplementary message could help to design objective metrics in order to facilitate the work of policy makers and encourage the development of technology towards the better economic goal.



---

*Source: 5806827-2018-10-18.xml*