# Biodiesel-triple bottom line (TBL): A new hierarchical sustainability assessment framework of principles criteria & indicators (PC&I) for biodiesel production.
## Part II-validation

Sandra Bautista, Manon Enjolras, P. Narvaez, Mauricio Camargo, Laure Morel

<figure><img src="image_1.png" /></figure>

► To cite this version:

Sandra Bautista, Manon Enjolras, P. Narvaez, Mauricio Camargo, Laure Morel. Biodiesel-triple bottom line (TBL): A new hierarchical sustainability assessment framework of principles criteria & indicators (PC&I) for biodiesel production. Part II-validation. Ecological Indicators, 2016, 69, pp.803-817. 10.1016/j.ecolind.2016.04.046. hal-01522500

HAL Id: hal-01522500

https://hal.univ-lorraine.fr/hal-01522500

Submitted on 2 Nov 2021

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>
# Biodiesel–Triple Bottom Line (TBL): A New Hierarchical Sustainability Assessment Framework of Principles Criteria & Indicators (PC&I) for Biodiesel Production. Part II-Validation

S. Bautista¹, ², ³, M. Enjolras², P. Narvaez¹, M. Camargo², L. Morel²

(1) Grupo de Procesos Químicos y Bioquímicos, Department of Chemical and Environmental Engineering, Universidad Nacional de Colombia sede Bogotá, pcnarvaezr@unal.edu.co, and sancbautistar@unal.edu.co

(2) Université de Lorraine, ERPI, F-54000 Nancy, France; mauricio.camargo@univ-lorraine.fr

(3) Grupo de Agua y Desarrollo Sostenible, Department of Environmental Engineering, Universidad Central, Bogotá, Colombia. sbautistar2@ucentral.edu.co

Bautista, S., Enjolras, M., Narvaez, P., Camargo, M., & Morel, L. (2016). Biodiesel-triple bottom line (TBL): A new hierarchical sustainability assessment framework of principles criteria & indicators (PC&I) for biodiesel production. Part II-validation. *Ecological indicators, 69*, 803-817.
[https://doi.org/10.1016/j.ecolind.2016.04.046](https://doi.org/10.1016/j.ecolind.2016.04.046)

## Abstract

Sustainability assessment is a critical issue for the biodiesel production chain. The constant growth of the biodiesel industry has generated important sustainability concerns, such as competition with food production, indirect land use change, and impacts on water, biodiversity and social values. Government policies consider that quantitative, robust and independent sustainability assessment is vital to estimate the extent to which the biodiesel industry impacts sustainability issues. However, weaknesses in the definition of adequate indicators to measure the sustainability of biodiesel production have been identified. In this context, this work proposes a hierarchical assessment framework based on sustainability dimensions, principles, criteria and indicators. To do this, after a systematic literature review, a first version of the framework was proposed. Then, to define the final framework, a validation strategy based on expert survey consultations and a descriptive statistical analysis were developed. In order to define the principles and criteria importance for sustainability assessment of biodiesel, 62 experts answered an online survey assessing three attributes: relevance, ease of measurement and reliability. The first result of the validation analysis was the definition of a framework composed of five dimensions (social, economic, environmental, political and technological), 13 principles, and 30 criteria that would be part of a sustainability assessment of biodiesel production. The second result was the identification of potential groups and relationships between principles and criteria represented through data visualization techniques. The validated framework provides the basis for defining future studies about interdimensional principles and criteria. Also, the proposed sustainability assessment framework could be adapted and applied to biodiesel production in specific contexts.

## Keywords

Biodiesel, sustainability criteria, expert survey, framework

## Introduction
Over the last decade, biofuels, specifically bioethanol and biodiesel, have gained relevance as an important alternative to liquid fossil fuels. The positive impact on sustainability attributed to biofuels contributes to stimulating its production (Kampman et al. 2012). Governments usually justify the application of policies to promote biofuel production based on positive impacts, including a reduction in greenhouse gas emissions, job creation in rural regions of developing countries and diversification of fuel sources, especially in the transport sector (Gnansounou 2011).

In the last decades, impacts caused during the biofuels life cycle, such as the alteration of soil, water, air quality, energy balance, food security, use and land tenure, and investment of public funds to subsidize biofuel production, began to be more closely analyzed (Reinhardt 1996; Kaltschmitt et al. 1997; Finnveden & Ekvall 1998; Dreier et al. 1998; Zemanek & Reinhardt 1999; MacLean et al. 2000; Zah et al. 2009; Hill et al. 2006; Janssen & Rutz 2011; Gnansounou 2011). Furthermore, to make clear the bioenergy consequences for sustainable development, comprehensive analyses about the complex relationship between negative and positive life-cycle impacts of biofuel production, the dynamic characteristics of the impacts, the inclusion of the sustainability dimensions and the long-term studies were promoted (Florin et al. 2013).

Sustainability theory provides strategies for better understanding and assessing the impacts of biofuel production on society. Sustainability assessment encompasses the complex study of social, environmental and economic aspects, as well as their interrelationships and changes over time. Additionally, these studies involve the multiple goals and methodologies to assess sustainability (Markevičius et al. 2010).

In this context, governments, public and private entities, international organizations, non-governmental organizations and researchers have proposed strategies to establish how sustainable biofuel production is. As a consequence, certification systems, regulatory frameworks, international policies and local initiatives have been developed and implemented (Fritsche et al. 2006; Scarlat 2011; FAO & BEFSCI 2011; FAO 2013). A comprehensive summary about certification, initiatives, standards and policies for biofuel references is presented in Appendix 2. However, there is still a need for a comprehensive framework that considers the sustainability of multidimensional assessment, the relationship between dimensions and criteria. In addition, such a framework should also describe the validation processes to define aims and criteria (FAO & BEFSCI 2012).

In the search for strategies to achieve a comprehensive framework that integrate dimensions, principles and criteria of sustainability for biodiesel production, the first part of this work proposed a hierarchical sustainability assessment framework consisting of four levels: sustainability dimensions, principles, criteria and indicators defined through a comprehensive state of the art (Bautista et al. 2016). Five dimensions were included in the framework proposed, which integrates the political and technological dimensions into the three traditional sustainability dimensions (social, economic and environmental) with the aim of strengthening the multidimensional sustainability capabilities.

On that basis, this work develops expert consultation to validate the principles and criteria proposed by Bautista (2016) in the sustainability assessment framework. The expert consultation has been previously used for the validation of conceptual models and complex issues (van der Sluijs et al. 2005; Augusiak et al. 2014). Expert consultation was performed through an online
survey, which allowed us to collect the data to perform a comprehensive analysis applying
methods of descriptive statistics.

This paper is structured as follows: in the first section, the hierarchical assessment framework is described. Then, the second section presents the validation method, which includes expert identification, online survey design and application, and the description of the statistical tools applied to survey answers (measures of central tendency, dispersion of data, principal component analysis and hierarchical ascendant classification). The third section shows the expert profile by geographic region of experts, organization type and expertise area. The fourth section presents the results of the expert consultation, the general analysis of dispersion, the potential correlation between importance attributes and the experts' profile, and the hierarchical ascendant classification of criteria. Finally, section five presents the principles and criteria recommended, the comparison of the framework proposed in this work with other similar approaches, and the potential groups and relationships between principles and criteria of different dimensions.

# 1. Hierarchical assessment framework

The hierarchical assessment framework was defined based on comprehensive analysis of the state of the art. About 400 documents were reviewed, and 103 documents were ultimately selected (Bautista et al. 2016). The document selection was carried out through a deductive process that includes the identification of the following components: 1) Documents that present at least two of the following concepts: goals, objectives or principles, criteria, and indicators. 2) Document about the traditional dimensions of sustainable development (social, economic, environmental). 3) Documents presenting research about the sustainability assessment framework. A document is selected if it is part of two or more groups previously described. The information sources employed were academic and research institutions, governmental and non-governmental institutions involved in sustainability assessment of biodiesel, and scientific databases like Web of Science and Scopus, particularly the following journal databases: American Chemical Society ACS, Science Direct, Springer Journal, and Nature.com.

Associated with sustainability assessment, the environmental, social and economic dimensions are sometimes referred to as the "three pillars" of sustainability or the "triple bottom lines – TBL" (Hacking & Guthrie 2008). The TBL is an approach which incorporates the three dimensions of sustainability, linked to a strong theoretical fit for measuring multidimensionality of sustainability (Lee et al. 2012). Furthermore, in particular for biodiesel production assessment, Bautista (2016) identified two additional dimensions: the political and technological dimension. The political dimension is highly relevant because government policies are essential to promote biodiesel production, creating economic conditions and favorable markets such as subsidies, tax exemptions, and compulsory biodiesel consumption as a diesel mixture. The technological dimension is also relevant, taking into account that emerging technologies are competing with first-generation biodiesel.

The hierarchical sustainability assessment framework proposed by Bautista (2016) consists of four levels. The first level comprises the five dimensions previously mentioned; the second includes 13 principles, defined as premises, bases or universal principles that define the sustainability of a biodiesel supply chain. The third level contains 40 criteria linked to each principle. A criterion is a measurable condition (qualitative or quantitative) that establishes the application of principles. The fourth level corresponds to a set of indicators that describes each
criterion. An indicator is an observable qualitative or quantitative expression, which can describe the characteristics, behaviors or phenomena of reality through the development of one or more variables (see Appendix 1).

Principles and criteria were defined in order to be applied regardless of the economic, social, political or biogeographic context, the technological conditions, or the raw materials used, among other aspects. Furthermore, to assess specific contexts, particular conditions of the biodiesel production chain or scale in a continent, country or region, a set of indicators for each criterion should be defined.

## 2. Validation method of sustainability assessment principles and criteria for biodiesel production

Expert consultation was selected as the validation method for sustainability assessment principles and criteria for biodiesel production, because of its applicability in the validation of conceptual models and complex issues. It has been used by other authors in biofuel research. Some examples can be found in studies developed by Barrett (2011), Brent & Musango (2013), Fiorese et al. (2013) and Tickner (2013) to define the future perspectives of biodiesel through sustainability assessment. Also, expert consultation has been employed to assess the environmental degradation linked to biofuel production (Vermeulen & Cotula 2010; Bisaro et al. 2014).

In order to validate the proposed framework, a validation strategy was implemented following six steps: (1) expert identification, (2) survey design, (3) survey application, (4) descriptive statistical analysis, (5) selection of sustainability assessment principles and criteria for biodiesel production, and (6) data visualization technique. The validation process is shown in Figure 1. The following of each step will be described.
<figure><img src="image_3.png" /><figcaption>Figure 1. Validation method of sustainability assessment principles and criteria for biodiesel production</figcaption></figure>

## 2.1 Expert identification

Experts were identified through a screening of the community involved in biodiesel production and its sustainability assessment through the papers published in peer-reviewed journals, conference participation lists, and members of industry and institutions such as the World Energy Council, the Green Chemistry Development Institute, WIP Renewable Energies, and government institutions of Brazil, Argentina, Colombia, Chile, and India, among others.

In this work, four types of experts were identified according to the following definition: An industrial expert is a person who has worked in industrial biodiesel production processes for a period equal to or longer than five years. A government expert is a person who has worked in governmental institutions that develop policies, laws or politics about biofuels. In the same way, a non-governmental expert is a professional who has worked on biofuel-related topics, especially
in areas linked to sustainable production. Governmental and non-governmental experts are characterized by their participation as key speakers at congresses and other academic events, and as international or national consultants. A university or research expert is a professional who has published at least five papers in a peer-reviewed journal, and these papers have been cited at least 10 times.

This work identified key areas to be taken into account about the expert profile. Initially, nationality and type of organization with which the expert was affiliated (from industry, government, non-governmental institutions, universities or research centers). After this, expertise areas were identified for each professional (knowledge and experience), such as social, economic, political, technological and environmental areas associated with sustainability dimensions of biodiesel production.

## 2.2 Survey design and application

### 2.2.1 Attributes of the importance and classification range

Sustainability assessment importance was defined as the impact of each principle and criterion on the assessment. The importance was the average value of three attributes: relevance, ease of measurement and reliability. The dimensionless ranking scale was null (0), low (1), moderate (2) and high (3); the attributes were defined as follows:

* Relevance: Measures the level of significance and pertinence of the criterion in the sustainability assessment of biodiesel.

* Ease of measurement: Measures the level of accessibility to information and the ability to perform the measurement of the criterion.

* Reliability: Measures how credible or certain the results of the measurement for the criterion are.

The Online survey can be found at

https://docs.google.com/forms/d/1dwBYYCBxfY7ZEp7qyA3bgx5joiYhM8PnHAPMHStVQG Y/viewform

The importance was calculated using equation 1. Subsequently, the principles and criteria were classified, taking into account the computed value of importance and using the ranking scale shown in Table 1.

$$ \text{Equation 1.} \qquad \text{Importance}_i = \frac{\sum \text{Relevance}_i + \text{Ease of measurement}_i + \text{Reliability}_i}{3} $$

Where: “i” is the principle or criteria.

<caption>Table 1. Importance ranking scale of principles and criteria</caption>

### 2.2.2 Survey structure

The survey form was structured by multiple-choice questions about the level of importance (relevance, ease of measurement and reliability) of each principle and criteria. Then, the questions were written and translated into three languages (English, Spanish and French). The survey was sent together with a presentation letter to the experts, in which the goals, ranking scale were explained in detail. Finally, a pilot survey test was applied, enabling survey improvement.

## 2.3 Descriptive statistical analysis
A statistical analysis of the survey results was conducted. Initially, techniques such as central tendency measurements and variability or dispersion measurements were used, and subsequent multivariate data analyses were applied (Principal Components Analysis - PCA and Hierarchical Ascendant Classification - HAC). The algorithm of the descriptive statistical analysis is shown in Figure 2 and the tools are described below.

<figure><img src="image_4.png" /><figcaption>Figure 2. Algorithm of descriptive statistical analysis</figcaption></figure>

### 2.3.1 Dispersion and correlation measurements

The dispersion and correlation techniques applied to the importance attributes ranked (relevance, ease of measurement and reliability) were the correlation matrix, scatter plot and box plot. Initially the correlation matrix was applied; this is a symmetric matrix where the variables are horizontally and vertically equal. In this case, the variables are the importance attributes. The matrix measures
the level of corresponding correlation between each pair of attributes through the coefficient called σ. The σ is expressed by a number ranging from 0 to 1, where a value of 0 indicates that no correlation is observed (Hastie et al. 2009).

Afterwards, a scatter plot was used because it can suggest different correlations between
importance attributes with a certain confidence interval. In addition, it is possible to show linear
and nonlinear relationships between attributes. The correlations may be positive (rising), negative
(falling), or zero (uncorrelated). A line of best fit (alternatively called "tend-line") can be drawn
in order to study the correlation between the variables. Finally, a box plot was applied, which is
a graph based on quartiles, whereby a data set is displayed. It consists of a rectangle, the "box"
and two arms, "whiskers" that provide information about the minimum and maximum values, the
quartiles Q1, Q2 and Q3 or medium, and the existence of outliers and symmetry of the distribution
(Hastie et al. 2009).

2.3.2 Multivariate data analysis

In the study of the answers given by the experts, two types of multivariate analysis data tools were used. The first tool is a Principal Components Analysis (PCA), which delivers a graphical representation of the data according to two principal axes (Syms 2008). PCA defines the principal axes regarding different variables. In this case, we studied on one hand the relevance, the ease of measurement and the reliability of the criteria considered and on the other hand, the experts’ profile (nationality, area of expertise and organization type). The role of the PCA is at first to detect pairs of variables perceived as synonyms (some variables are highly correlated), in order to reduce the dimension of the variables’ space. Secondly, PCA determines which pairs of variables contribute little to the variance of the assessments. Such pairs are considered as irrelevant for a semantic description of the given group of individuals (Petiot & Yannou 2004).

The second tool is Hierarchical Ascendant Classification (HAC). It is based on general linking
data using the Ward criteria aggregation method (Ward 1963). HAC regroups the individuals
according to the similarity of characteristics and behavior (Fort et al. 2011; Ward 1963). The
principle is to create a partition in every stage obtained by aggregating pair-wise the closest
individuals (or group of individuals). The functioning of this technique is based on regrouping
individuals by minimal Euclidean distance on a plan, considering the same variables as for PCA.
In our case, the objective is to identify several groups of criteria, which have the same global
profile according to the scores of relevance, ease of measurement and reliability given by the
experts.

The combination of these two analyses is relevant because it allows the researcher to visualize
the correlated (or similar) variables as well as the identification of the individuals having close
behaviors. This combination also makes it possible to position the groups obtained according to
their evaluation with regard to the three attributes of the criteria (relevance, ease of measurement
and reliability). We can thereby identify the least good and the best-estimated groups in order to
identify similarities and differences between criteria.
## 2.4 Selection of sustainability assessment principles and criteria for biodiesel production

The principles and criteria are selected if they are ranked by survey participants with high and moderate importance, defined as the average value of the three attributes: relevance, ease of measurement and reliability. The criteria ranked with medium relevance, low reliability and low ease of measurement are eliminated. Furthermore, if the criteria obtain high or moderate relevance, but ease of measurement and reliability are low, then the criteria are merged with others. The merging process takes into account the HAC that regroups the individuals according to the similarity of characteristics and behavior and provides potential groups of criteria using the Ward criteria aggregation method (Ward 1963).

## 2.5 Data visualization of principles and criteria selected

Data visualization techniques permit better comprehension of relationships and potential correlations between the three levels of the hierarchical sustainability assessment framework proposed (dimensions, principles and criteria). Also, it is possible to identify central nodes in which it connects principles and criteria of different sustainability dimensions.

# 3. Results

## 3.1 Expert profile description

As a result of the expert consultation described in section 2.1, 310 experts meet the profile defined. Although all the experts were contacted, only 62 of them answered the survey. After that, the analysis of the profile was conducted as explained in section 2.2.

* *Geographic region of the experts:* The 62 experts that answered the survey are from 20 different countries and their nationalities were classified by region. The largest number of experts belongs to Latin America and the Caribbean (68%), the second expert group belongs to the European region (21%), and a small number of experts are from the regions of Asia, North America and Africa.

* *Experts by organization type:* Most of the experts were connected to universities and research institutes (67%), followed by the experts associated with government institutions (17%) and industries (16%).

* *Experts by sustainability dimension:* A wide variety of combinations was observed in the expertise of the professionals consulted; 42% of the experts were linked with one or more sustainability dimensions. 24% of the experts were related to environmental-technological dimensions, and 23% were only in the technological dimension. 11% of the specialists declared expertise in the five dimensions and 61% working in two or more dimensions, while 39% were regarded as authorities in one dimension. The participation of experts from multiple sustainability dimensions was favorable for this research because it brings multiple points of view.

## 3.2 Statistical analysis of principles

To each average value of the three attributes ranked for each principle and criterion, the algorithm shown in Figure 2 was applied. If the average absolute error is acceptable, then the average values of the principle and criteria attribute are statistically reliable.

### 3.2.1 Principles - General analysis dispersion
The use of average values in the subsequent statistical analysis was possible. The attribute ranking of principles is shown in Table 2, while the description of each principle is presented in Appendix 1.

<table><caption>Table 2. Ranking averages of principles' importance</caption><thead><tr><th rowspan="2">Sustainability Dimension</th><th rowspan="2">Principles</th><th colspan="5">Importance</th></tr><tr><th>Rank</th><th>Null</th><th>Low</th><th>Mod.</th><th>High</th></tr></thead><tbody><tr><td rowspan="4">Social</td><td>P1: Respect property and land tenure</td><td>2.7</td><td></td><td></td><td></td><td></td></tr><tr><td>P2: Social acceptability</td><td>2.4</td><td></td><td></td><td></td><td></td></tr><tr><td>P3: Promote responsible work conditions</td><td>2.7</td><td></td><td></td><td></td><td></td></tr><tr><td>P4: Prevent alteration to food supply</td><td>2.6</td><td></td><td></td><td></td><td></td></tr><tr><td rowspan="3">Political</td><td>P5: Relationship between national and international biodiesel promotion policies</td><td>2.1</td><td></td><td></td><td></td><td></td></tr><tr><td>P6: National biodiesel production consistent with international environmental policies</td><td>2.6</td><td></td><td></td><td></td><td></td></tr><tr><td>P7-P: Promote commitment to ethics and transparency</td><td>2.8</td><td></td><td></td><td></td><td></td></tr><tr><td>Economic</td><td>P8: Economic viability</td><td>2.6</td><td></td><td></td><td></td><td></td></tr><tr><td>Technological</td><td>P9: Technology used promotes efficiency and reduction of negative environmental impact</td><td>2.8</td><td></td><td></td><td></td><td></td></tr><tr><td rowspan="4">Environmental</td><td>P10: Maintain or improve the air, soil and water quality</td><td>2.9</td><td></td><td></td><td></td><td></td></tr><tr><td>P11: Positive balance of greenhouse gases</td><td>2.7</td><td></td><td></td><td></td><td></td></tr><tr><td>P12: Promote the conservation and protection of biodiversity</td><td>2.8</td><td></td><td></td><td></td><td></td></tr><tr><td>P13: Energy efficiency and use of renewable energy</td><td>2.8</td><td></td><td></td><td></td><td></td></tr></tbody></table>

As explained in Table 1 (section 2.2), the importance given by the experts to the principles was considered as high if the average value was between 2.4 and 3.0. The principles with high variation coefficients were the same principles that obtained moderate importance ranks (P5-Biodiesel promotion policy, P2-Social acceptability and P8-Economic viability). The principles associated with technological and environmental dimensions had lower variation coefficients and they were ranked with high importance, indicating a great deal of agreement in the ranks given by experts (see Figure 3).
<figure><img src="image_5.png" /><figcaption>Figure 3. Principles importance, average and variation coefficient (Cv)</figcaption></figure>

### 3.2.2 Principles - Principal component analysis

The results of the Principal Component Analysis (PCA) set forth the potential correlation between the principles' importance and the expert profile; these are shown in Figure 4. According to the PCA rules, all these variables are orthogonal, so there was no correlation between them. It was possible to conclude that the principle importance ranking scale does not depend on the type of organization the expert works with. Indeed, according to PCA rules, all these variables are orthogonal, so there was no correlation between them.
<figure><img src="image_6.png" /><figcaption>Figure 4. PCA loading plots concerning the type of organization</figcaption></figure>

## 3.3 Criteria statistical analysis

The average value of the three attributes ranked (relevance, ease of measurement and reliability) for each principle and criterion was statistically significant based on the algorithm shown in Figure 2.

### 3.3.1 Criteria - General analysis dispersion

The average value of each attribute ranked was used in the subsequent statistical analysis because each average ranking obtained an absolute error less than the maximum error absolute with a statistical confidence level of 95% (see statistical algorithm in Figure 2). The attribute ranking of principles is shown in Table 3, and the descriptions of the criteria are presented in Appendix 1.

<table><caption>Table 3. Average ranking criteria for attribute</caption><thead><tr><th rowspan="2">Criteria</th><th colspan="4">Relevance</th><th colspan="4">Ease of measurement</th><th colspan="4">Reliability</th><th colspan="4">Importance</th></tr><tr><th>Grad</th><th>Null</th><th>Low</th><th>Mod.</th><th>High</th><th>Grad</th><th>Null</th><th>Low</th><th>Mod.</th><th>High</th><th>Grad</th><th>Null</th><th>Low</th><th>Mod.</th><th>High</th><th>Grad</th><th>Null</th><th>Low</th><th>Mod.</th></tr></thead><tbody><tr><td>C1-Rights of land access</td><td>2.7</td><td></td><td></td><td></td><td></td><td>1.9</td><td></td><td></td><td></td><td></td><td>1.7</td><td></td><td></td><td></td><td>2.1</td><td></td><td></td><td></td><td></td></tr><tr><td>C2-Land conflicts</td><td>2.6</td><td></td><td></td><td></td><td></td><td>1.5</td><td></td><td></td><td></td><td></td><td>1.6</td><td></td><td></td><td></td><td>1.9</td><td></td><td></td><td></td><td></td></tr><tr><td>C3-National energy security</td><td>2.5</td><td></td><td></td><td></td><td></td><td>2.0</td><td></td><td></td><td></td><td></td><td>2.0</td><td></td><td></td><td></td><td>2.2</td><td></td><td></td><td></td><td></td></tr><tr><td>C4-Environmental noise</td><td>1.9</td><td></td><td></td><td></td><td></td><td>2.0</td><td></td><td></td><td></td><td></td><td>1.8</td><td></td><td></td><td></td><td>1.9</td><td></td><td></td><td></td><td></td></tr><tr><td>C5-Changes in landscape - visual impacts</td><td>1.8</td><td></td><td></td><td></td><td></td><td>1.6</td><td></td><td></td><td></td><td></td><td>1.6</td><td></td><td></td><td></td><td>1.6</td><td></td><td></td><td></td><td></td></tr><tr><td>C6-Local prosperity</td><td>2.6</td><td></td><td></td><td></td><td></td><td>1.9</td><td></td><td></td><td></td><td></td><td>1.9</td><td></td><td></td><td></td><td>2.1</td><td></td><td></td><td></td><td></td></tr><tr><td>C7-Health &amp; safety - employees</td><td>2.8</td><td></td><td></td><td></td><td></td><td>2.3</td><td></td><td></td><td></td><td></td><td>2.2</td><td></td><td></td><td></td><td>2.4</td><td></td><td></td><td></td><td></td></tr><tr><td>C8-Labor laws</td><td>2.9</td><td></td><td></td><td></td><td></td><td>2.2</td><td></td><td></td><td></td><td></td><td>2.2</td><td></td><td></td><td></td><td>2.4</td><td></td><td></td><td></td><td></td></tr><tr><td>C9-Food supply</td><td>2.6</td><td></td><td></td><td></td><td></td><td>1.9</td><td></td><td></td><td></td><td></td><td>1.8</td><td></td><td></td><td></td><td>2.1</td><td></td><td></td><td></td><td></td></tr><tr><td>C10-Biomass local uses</td><td>2.1</td><td></td><td></td><td></td><td></td><td>1.5</td><td></td><td></td><td></td><td></td><td>1.5</td><td></td><td></td><td></td><td>1.7</td><td></td><td></td><td></td><td></td></tr><tr><td>C11-Subsidy scheme</td><td>1.8</td><td></td><td></td><td></td><td></td><td>2.0</td><td></td><td></td><td></td><td></td><td>1.8</td><td></td><td></td><td></td><td>1.9</td><td></td><td></td><td></td><td></td></tr></tbody></table>
<table><thead><tr><th rowspan="2">Criteria</th><th colspan="4">Relevance</th><th colspan="4">Ease of measurement</th><th colspan="4">Reliability</th><th colspan="4">Importance</th></tr><tr><th>Grad</th><th>Null</th><th>Low</th><th>Mod.</th><th>High</th><th>Grad</th><th>Null</th><th>Low</th><th>Mod.</th><th>High</th><th>Grad</th><th>Null</th><th>Low</th><th>Mod.</th><th>High</th><th>Grad</th><th>Null</th><th>Low</th><th>Mod.</th><th>High</th></tr></thead><tbody><tr><td>C12-Advanced biodiesel promotion</td><td>1.7</td><td></td><td></td><td></td><td></td><td>1.9</td><td></td><td></td><td></td><td></td><td>1.9</td><td></td><td></td><td></td><td>1.8</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>C13-Agreement biodiesel blend-diesel</td><td>1.7</td><td></td><td></td><td></td><td></td><td>2.3</td><td></td><td></td><td></td><td></td><td>2.1</td><td></td><td></td><td></td><td>2.0</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>C14-National capability in biodiesel research</td><td>2.2</td><td></td><td></td><td></td><td></td><td>1.9</td><td></td><td></td><td></td><td></td><td>1.9</td><td></td><td></td><td></td><td>2.0</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>C15-Biomass (non-edible) for biodiesel</td><td>2.4</td><td></td><td></td><td></td><td></td><td>1.9</td><td></td><td></td><td></td><td></td><td>1.8</td><td></td><td></td><td></td><td>2.0</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>C16-National amount of land suitable</td><td>2.6</td><td></td><td></td><td></td><td></td><td>1.9</td><td></td><td></td><td></td><td></td><td>1.9</td><td></td><td></td><td></td><td>2.2</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>C17-Emision condition GHE</td><td>2.5</td><td></td><td></td><td></td><td></td><td>1.8</td><td></td><td></td><td></td><td></td><td>1.8</td><td></td><td></td><td></td><td>2.0</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>C18-Ethical commitment</td><td>2.4</td><td></td><td></td><td></td><td></td><td>1.4</td><td></td><td></td><td></td><td></td><td>1.5</td><td></td><td></td><td></td><td>1.7</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>C19-Compliance with local laws</td><td>2.3</td><td></td><td></td><td></td><td></td><td>1.3</td><td></td><td></td><td></td><td></td><td>1.4</td><td></td><td></td><td></td><td>1.7</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>C20-Influence oil market/biodiesel</td><td>2.4</td><td></td><td></td><td></td><td></td><td>2.1</td><td></td><td></td><td></td><td></td><td>2.0</td><td></td><td></td><td></td><td>2.2</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>C21-Influence diesel market/biodiesel</td><td>2.5</td><td></td><td></td><td></td><td></td><td>2.3</td><td></td><td></td><td></td><td></td><td>2.1</td><td></td><td></td><td></td><td>2.3</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>C22-Production biodiesel 1a gen./biodiesel</td><td>2.2</td><td></td><td></td><td></td><td></td><td>2.4</td><td></td><td></td><td></td><td></td><td>2.3</td><td></td><td></td><td></td><td>2.3</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>C23-Influence vegetable oil market/biodiesel</td><td>2.6</td><td></td><td></td><td></td><td></td><td>2.2</td><td></td><td></td><td></td><td></td><td>2.2</td><td></td><td></td><td></td><td>2.3</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>C24- Influence glycerin market/biodiesel</td><td>2.1</td><td></td><td></td><td></td><td></td><td>2.2</td><td></td><td></td><td></td><td></td><td>2.1</td><td></td><td></td><td></td><td>2.1</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>C25-Influence advanced biodiesel market/biodiesel</td><td>2.2</td><td></td><td></td><td></td><td></td><td>2.0</td><td></td><td></td><td></td><td></td><td>2.0</td><td></td><td></td><td></td><td>2.0</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>C26-Influence advanced biodiesel market/veg. oil</td><td>2.6</td><td></td><td></td><td></td><td></td><td>1.8</td><td></td><td></td><td></td><td></td><td>1.8</td><td></td><td></td><td></td><td>2.1</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>C27-Influence advanced biodiesel techno/cost</td><td>2.4</td><td></td><td></td><td></td><td></td><td>1.9</td><td></td><td></td><td></td><td></td><td>1.9</td><td></td><td></td><td></td><td>2.1</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>C28-Influence engine trends</td><td>2.3</td><td></td><td></td><td></td><td></td><td>2.1</td><td></td><td></td><td></td><td></td><td>2.1</td><td></td><td></td><td></td><td>2.2</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>C29-Influence technological learning/cost</td><td>2.3</td><td></td><td></td><td></td><td></td><td>1.8</td><td></td><td></td><td></td><td></td><td>1.8</td><td></td><td></td><td></td><td>2.0</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>C30-Air quality</td><td>2.8</td><td></td><td></td><td></td><td></td><td>2.2</td><td></td><td></td><td></td><td></td><td>2.0</td><td></td><td></td><td></td><td>2.3</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>C31-Efficient use of water</td><td>2.8</td><td></td><td></td><td></td><td></td><td>2.1</td><td></td><td></td><td></td><td></td><td>2.1</td><td></td><td></td><td></td><td>2.4</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>C32-Local water quality</td><td>2.9</td><td></td><td></td><td></td><td></td><td>2.3</td><td></td><td></td><td></td><td></td><td>2.2</td><td></td><td></td><td></td><td>2.4</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>C33-Wastes-managed responsibly</td><td>2.8</td><td></td><td></td><td></td><td></td><td>2.3</td><td></td><td></td><td></td><td></td><td>2.2</td><td></td><td></td><td></td><td>2.4</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>C34-Soil quality</td><td>2.8</td><td></td><td></td><td></td><td></td><td>2.0</td><td></td><td></td><td></td><td></td><td>1.9</td><td></td><td></td><td></td><td>2.2</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>C35-Greenhouse gases</td><td>2.6</td><td></td><td></td><td></td><td></td><td>1.8</td><td></td><td></td><td></td><td></td><td>1.8</td><td></td><td></td><td></td><td>2.1</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>C36-Transformation-natural ecosystems</td><td>2.6</td><td></td><td></td><td></td><td></td><td>1.8</td><td></td><td></td><td></td><td></td><td>1.8</td><td></td><td></td><td></td><td>2.1</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>C37-Influence fragile ecosystems</td><td>2.8</td><td></td><td></td><td></td><td></td><td>2.0</td><td></td><td></td><td></td><td></td><td>2.0</td><td></td><td></td><td></td><td>2.2</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>C38-Influence wildlife</td><td>2.8</td><td></td><td></td><td></td><td></td><td>1.9</td><td></td><td></td><td></td><td></td><td>1.9</td><td></td><td></td><td></td><td>2.2</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>C39-Energy used</td><td>2.4</td><td></td><td></td><td></td><td></td><td>2.0</td><td></td><td></td><td></td><td></td><td>2.0</td><td></td><td></td><td></td><td>2.1</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>C40-Energy savings</td><td>2.2</td><td></td><td></td><td></td><td></td><td>2.1</td><td></td><td></td><td></td><td></td><td>2.0</td><td></td><td></td><td></td><td>2.1</td><td></td><td></td><td></td><td></td><td></td></tr></tbody></table>

Overall, the criteria importance averages were between 1.6 and 2.4. The highest ranked criteria belong to the environmental dimension, and the lowest to the political dimension. Regarding the variation coefficient, the highest values correspond to the criteria of the social and political dimensions, while the lowest to the environmental. This behavior shows that there is major disagreement in what experts ranked about the importance of social and political criteria as shown in Figure 5.
<figure><img src="image_7.png" /><figcaption>Figure 5. Criteria importance, average and variation coefficient (Cv).</figcaption></figure>

Correlation matrix (Table 4) showed that there was high correlation (0.97) between the attributes ease of measurement and reliability, whereas the correlation was very low (0.23) between relevance and reliability, as well as between ease of measurement and relevance (0.15).

<table><caption>Table 4. Correlation matrix between criteria attributes</caption><thead><tr><th></th><th>Reliability</th><th>Ease of measurement</th><th>Relevance</th></tr></thead><tbody><tr><th>Reliability</th><td>1.0</td><td></td><td></td></tr><tr><th>Ease of measurement</th><td><strong>0.97</strong></td><td>1.0</td><td></td></tr><tr><th>Relevance</th><td>0.23</td><td>0.15</td><td>1.0</td></tr></tbody></table>

### 3.3.2 Criteria - Principal Component Analysis (PCA)

In order to analyze the criteria another PCA was conducted. The objective of this analysis was to identify the potential correlations between the collected data. First, the expert profile was analyzed.

Figure 6A shows the PCA loading plots of the geographical areas of the experts compared with the three attributes.

In the first loading plot (Figure 6A), it can be seen that the geographical regions of Europe and of Latin America and the Caribbean, as well as the attributes reliability and ease of measurement are well represented, because the spots indicating these variables were very close to the border of the
circle. The relevance attribute was also well represented but not as clearly as the two other attributes. Therefore, Figure 6A puts forward that the geographical regions of Europe and of Latin America and the Caribbean were not correlated with the three attributes because they were orthogonal to each other. Another result was that the ease of measurement attribute and the reliability attribute were very close on the loading plot, which means the two attributes are correlated. Also in Figure 6A, it can be seen that the Asia geographical region was also orthogonal to the attributes. However, the same cannot be concluded about North America because this variable was not well represented. In conclusion, the loading plots presented highlighting the experts' geographical region did not seem to be correlated with the attributes (R, EM and Re). In this sense, the ranking done by the experts does not depend on their geographical origin.

A similar analysis was conducted in regard to the dimension of expertise of the experts (Figure 6B). The representation was adequate for the social and political proficiency of the experts, as shown in Figure 6B, as well as for the attributes reliability and ease of measurement. Again, ease of measurement and reliability were correlated attributes. The relevance attribute and the economical, technical and environmental knowledge of the experts were moderately represented in the PCA loading plot. In a general way, the fields of expertise were not correlated with the three attributes. Indeed, all the dimensions of expertise are orthogonal to the attributes. This means that the field of expertise of the experts did not influence their criteria ranking.

Figure 6C, presents the results regarding the type of organization. The University organization type was clearly orthogonal to the three attributes. The organization types Government and Industry were moderately well represented, but they do not seem correlated with the three attributes. Therefore, the organization type did not influence the evaluation of the experts regarding the criteria. Finally, this first PCA highlights that the criteria assessment did not depend on the experts' profile.
<figure><img src="image_8.png" /><figcaption>Figure 6 A. PCA loading plot concerning the geographical sector of the experts</figcaption></figure>

<figure><img src="image_9.png" /><figcaption>Figure 6 A. PCA loading plot concerning the geographical sector of the experts</figcaption></figure>

<caption>Figure 6 B. PCA loading plot concerning the dimension of expertise of the experts</caption>

<caption>Figure 6 C. PCA loading plot concerning the type of organization (relevance (R), ease of measurement (EM) and reliability (Re))</caption>

### 3.3.3 Criteria: Hierarchical ascendant classification

HAC was carried out on all the criteria submitted to ranking. The classification objective was to highlight some potential groups of criteria. The HAC dendogram shown in Figure 7 permitted the identification of three criteria groups. The characteristics of each group are shown in Table 5; in general, two of three criteria groups show high and moderate importance to be part of the sustainability assessment framework.
<figure><img src="image_10.png" /><figcaption>Figure 7. Dendogram of the HAC concerning the criteria</figcaption></figure>

<table><caption>Table 5. Characteristics of criteria groups</caption><thead><tr><th colspan="5">Dimensions</th><th rowspan="2">Criteria groups</th><th colspan="4">Relevance</th><th colspan="4">Ease of measurement</th><th colspan="4">Reliability</th></tr><tr><th>Social</th><th>Political</th><th>Economic</th><th>Technological</th><th>Environmental</th><th>Null</th><th>Low</th><th>Mod.</th><th>High</th><th>Null</th><th>Low</th><th>Mod.</th><th>High</th><th>Null</th><th>Low</th><th>Mod.</th><th>High</th></tr></thead><tbody><tr><td></td><td></td><td></td><td></td><td></td><td></td><td rowspan="3">Group 1</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td></td><td></td><td></td><td></td><td></td><td></td><td rowspan="2">Group 2</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td></td><td></td><td></td><td></td><td></td><td></td><td rowspan="2">Group 3</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr></tbody></table>

### 3.3.4 Criteria: potential attribute correlation by sustainability dimension

In order to obtain a higher level of precision, a PCA was conducted according to each sustainability dimension. The objective of this PCA was to propose a criteria representation belonging to one dimension in relation to the three attributes.

The results of this analysis were reflected in a point cloud plot representing the criteria of the considered dimension according to three axes, one for each attribute. For all the dimensions, the attributes ease of measurement and reliability seem to be highly correlated. Thus, they could be thought to represent a single axis.

Concerning the social dimension (Figure 8), the analysis makes it possible to identify the better and worse criteria for each attribute. The most relevant criterion seems to be C1s, rights of land access, and the less relevant is C5s, undesirable visual impacts. The most reliable and easy to
measure criteria were C7s, health and safety for employees, and C8s, respect of labor laws, and the less reliable was C10s, prevent alteration to biomass production for traditional local uses.

<figure><img src="image_11.png" /><figcaption>Figure 8. Point cloud of the social dimension</figcaption></figure>

Regarding the political dimension (Figure 9), the most relevant criterion seems to be C16, national amount of land suitable for biomass, and the less relevant was C12, advanced biodiesel promotion. The most reliable and easy to measure criterion was C13, agreement between national and international biodiesel blend-diesel, and the least were C18, ethical commitment and transparency, and C19, compliance with local laws. Concerning this dimension, it was important to note that the most reliable and easy to measure criterion (C13) was part of the least relevant criteria.

<figure><img src="image_12.png" /><figcaption>Figure 9. Point cloud of the political dimension</figcaption></figure>

Concerning the economical dimension (Figure 10), the most relevant criterion was C23, influence of vegetal oil production on biodiesel production, and the least relevant was C24, influence of glycerin market on biodiesel production. The most reliable and easy to measure criterion was C22, annual production of first-generation biodiesel, and the least reliable and most difficult to measure was C25, influence of advanced biodiesel market on first generation biodiesel
production. In this dimension, the most reliable and easy to measure criterion (C22) was also part of the least relevant criteria.

<figure><img src="image_13.png" /><figcaption>Figure 10. Point cloud of the economical dimension</figcaption></figure>

With respect to the technological dimension (Figure 11), the most relevant criterion was C26, influence of advanced biodiesel production on vegetable oil market, and the least relevant was C29, development and influence of technological learning on biodiesel production. The most reliable and easy to measure criterion was C28, influence of technology trends engines on biodiesel production, and the least reliable was C26. In this dimension, the most relevant criterion (C26) was also the least reliable and the least easy to measure. Criterion C29 was also the least relevant criterion and it was among the less reliable and less easy to measure.

<figure><img src="image_14.png" /><figcaption>Figure 11. Point cloud of the technological dimension</figcaption></figure>

Finally, in the environmental dimension (Figure 12), many criteria were moderately relevant but none was the most relevant. However, the least relevant criterion was C40, energy savings in biodiesel production. The most reliable and easy to measure criteria were C32, local water quality, and C33, waste management responsibility, and the most difficult to measure were C35, amount of greenhouse gases captured, and C36, influence of biodiesel production on transformation of natural ecosystems.
<figure><img src="image_15.png" /><figcaption>Figure 12. Point cloud of the environmental dimension</figcaption></figure>

4. Selection of sustainability assessment principles and criteria

The average ranking of sustainability assessment principles is shown in Table 2. 15% of the principles were ranked as being of moderate importance, and 85% as being of high importance. Thus, all principles were recommended for the sustainability assessment framework. The higher importance rank was obtained by the environmental principle P10 (maintains or improves air, soil and water quality), and the lowest importance rank was for the political principle P5, relationship between national and international biodiesel promotion policies.

Regarding the result in Table 3, 73% of the sustainability criteria obtained high relevance and 27% moderate relevance. Therefore, the criteria ranked with medium relevance, low reliability and low ease of measurement were eliminated: C4s-Environmental noise, C5s-Changes in landscape generating undesirable visual impacts and C10s-Prevent alteration to biomass production for traditional local uses.

Furthermore, seven criteria obtained high or moderate relevance, but the ease of measurement and reliability were low. Therefore, their use in the sustainability assessment framework of biodiesel production was not confirmed. The criteria in doubt were: C11p-Subsidy scheme, C12p-Advanced biodiesel promotion, C18p-Ethical commitment, C2s-Conflicts over land use, C19p-Compliance with local laws, and C14p-National capability in biodiesel research. Therefore, these criteria were merged with other criteria, taking into account the hierarchical ascendant classification (HAC) that provided potential groups of criteria. In short, 13 principles and 30 criteria were recommended to become part of the sustainability assessment framework, as shown in Table 6.

<caption>Table 6. Sustainability assessment criteria recommended</caption>
<table><thead><tr><th>Sustainability Dimension</th><th>Principle</th><th>Criteria</th></tr></thead><tbody><tr><td rowspan="4">Social</td><td>P1: Respect property and land tenure</td><td>C1s-Rights of land access</td></tr><tr><td>P2: Social acceptability</td><td>C2s-<br/>- National energy security<br/>- Local prosperity</td></tr><tr><td>P3: Promote responsible working conditions</td><td>C3s-Health & safety - employees<br/>C4s-Labor laws</td></tr><tr><td>P4: Prevent alteration to food supply</td><td>C5s-Food supply</td></tr><tr><td rowspan="4">Political</td><td>P5: Relationship between national and international biodiesel promotion policies</td><td>C6p<br/>- Agreement national/international biodiesel blend-diesel<br/>- Subsidy scheme<br/>- Advanced biodiesel promotion</td></tr><tr><td>P6: National biodiesel production consistent with international environmental policies</td><td>C7p-Biomass (non-edible) for biodiesel production<br/>C8p-National amount of land suitable for biomass<br/>C9p-Emission condition GHE</td></tr><tr><td>P7: Promote commitment to ethics and transparency</td><td></td></tr><tr><td></td><td></td></tr><tr><td rowspan="7">Economic</td><td rowspan="7">P8: Economic viability</td><td>C10ec-Influence oil market/biodiesel</td></tr><tr><td>C11ec-Influence diesel market/biodiesel</td></tr><tr><td>C12ec<br/>- Production biodiesel 1a gen./biodiesel<br/>- Ethical commitment<br/>- Land conflicts<br/>- Compliance with local laws</td></tr><tr><td>C13ec-Influence vegetable oil market/biodiesel</td></tr><tr><td>C14ec-Influence glycerin market/biodiesel</td></tr><tr><td>C15ec-Influence advanced biodiesel Market/biodiesel</td></tr><tr><td>C16t-Influence advanced biodiesel Market/veg. Oil</td></tr><tr><td></td><td></td></tr><tr><td></td><td></td><td>C17t-Influence advanced biodiesel Techno/cost</td></tr></tbody></table>
<table><thead><tr><th>Sustainability Dimension</th><th>Principle</th><th>Criteria</th></tr></thead><tbody><tr><td rowspan="10">Environmental</td><td rowspan="5">P10: Maintain or improve air, soil and water quality</td><td>C18t-influence of technologie trends engins on biodiesel production</td></tr><tr><td>C19t</td><td>- Influence technological learning/cost<br/>- National capability in biodiesel research</td></tr><tr><td>C20en-Air quality</td></tr><tr><td>C21en-Efficient use of water</td></tr><tr><td>C22en-Local water quality</td></tr><tr><td>C23en-Wastes-managed responsibly</td></tr><tr><td>C24en-Soil quality</td></tr><tr><td>P11: Positive balance of greenhouse gases</td><td>C25en-Greenhouse gases balance</td></tr><tr><td rowspan="3">P12: Promote the conservation and protection of biodiversity</td><td>C26en-Transformation-natural ecosystems</td></tr><tr><td>C27en-Influence fragile ecosystems</td></tr><tr><td>C28en-Influence wildlife</td></tr><tr><td rowspan="2">P13: Energy efficiency and use of renewable energy</td><td>C29en-Energy used</td></tr><tr><td>C30en-Energy savings</td></tr></tbody></table>

## 4.1 Analysis of the principles and criteria recommended

Comparing the new framework for biodiesel sustainability assessment with other similar studies (Bueren & Blom 1996; Mendoza et al. 1999; Fao & Befsci 2007; Cramer et al. 2007; Van Cauwenbergh et al. 2007; Kurka & Blackwood 2013; Florin et al. 2013), several particular characteristics can be pointed out. First, the new framework considered five dimensions for the sustainability assessment in comparison with the three traditional ones. Second, it includes the integration between the sustainability assessment dimensions and the biodiesel supply chain. Third, the relations between the levels of hierarchical structure were identified. That means the dimensions were analyzed through principles, each principle was assessed by a set of criteria and each criterion by a set of indicators. Taking into account the points above, the new framework is a comprehensive sustainable assessment, while also being adaptable to different contexts and biofuel supply chains.

The biofuel sustainability assessment proposed by Buchholz et al. (2009) did not raise a hierarchical assessment framework, nor sustainability principles. Neither political nor
technological criteria were defined. With regard to the validation process recommended by Buchholz et al. (2009), four attributes to evaluate the sustainability criteria were established; one of these was importance. Regarding the last point, in this work importance was defined as the average value of three attributes (relevance, ease of measurement and reliability) to validate principles and criteria that were part of the assessment framework proposed.

Some works about sustainability assessment tools described the performance and applicability of an existing tool for selection, which is potentially useful for sustainability assessment of bioenergy systems (Buytaert et al. 2011; Silva Lora et al. 2011; Cucek & Kravanja 2012; FAO & BEFSCI 2012). The research of Kurka & Blackwood (2013) presents a generic approach for selecting sustainability criteria and indicators by using participatory methodology. Florin et al. (2013) proposed sustainability principles and criteria for small farmers who produce biomass for biodiesel; this was emphasized to indicate definition, and did not take into account the technological dimension. In a different way, this work defined the hierarchical framework, the principles and criteria necessary to perform the assessment. In addition, this work proposed a framework that linked the stages of the biodiesel supply chain with the sustainability assessment.

In the assessment framework definition of this work, the existence of horizontal consistency was considered. This means that principles or criteria appearing at the same level do not have any overlap. Regarding the horizontal relationship between criteria, the Hierarchical Ascendant Classification (HAC), seven potential groups of 40 criteria were identified (Figure 7); in each group there were criteria linked to different sustainability dimensions. After applying the validation strategy, 13 principles and 30 criteria were proposed (Table 6).

## 4.2 Data visualization technique of principles and criteria

With the aim of visualizing the relationships between the five sustainable assessment dimensions, the 13 sustainable principles and the 30 assessment criteria, a display of data in a visualization technique (graph drawing<fnref n="1" />) was applied (Figure 13). The graph drawing is a data visualization technique that better represents heterogeneous information, multi-dimensional data, importance and correlation between data.

In the graph drawing, nodes represent principles and criteria, and edges represent relations between nodes. The node size represented the importance average ranking. A force-directed algorithm was applied that combines a multilevel approach which overcomes local minimums, with techniques such as approximates, short- and long-range force (Hu 2005). In the network, the nodes with more connections were located in central areas, and these central nodes tend to have greater influence on the other nodes. Furthermore, greater proximity between nodes was indicative of similar or equal importance between nodes and, in turn, of node groups on the network.

<footnote n="1">¹ Graph drawing is an area of mathematics and computer science combining methods from geometric graph theory. The basic graph drawing is given a set of nodes with a set of edges (relations); calculate the position of the nodes and the curve to be drawn for each edge. It requires the definition of properties and classification of layout according to the types of graphs to which they can be applied (Herman et al. 2000).</footnote>
<figure><img src="image_16.png" /><figcaption>Figure 13. Principles and criteria framework visualization</figcaption></figure>

Three central nodes were identified, the visualization of which is shown in Figure 13. The first central node (CN1) includes the political, social and technological principles and criteria, and only two environmental ones. In general, in this central node the importance ranking was between moderate and high. In addition, it was observed that the most central nodes were the political criterion C17 about policy conditions pertaining to greenhouse gas emissions in the life cycle of biodiesel production, and the technological criterion C26 regarding the influence of the advanced biodiesel market on the vegetable oil market. Furthermore, greater proximity between social and environmental criteria nodes was identified, because such criteria obtained equal importance ranking.

The second central node (CN2) is related, mainly, to environmental principles and criteria. However, two social criteria and two economic criteria are included. Overall, the highest importance ranking was given to the principles and criteria of this central node. It was identified that the principle P10, about maintaining and improving air, soil and water quality, was the most central node in this group, because it was ranked with the highest importance, like their related criteria.

Finally, the third central node (CN3) shows the highest heterogeneity, probably because it includes economic, environmental, technological and social criteria, whose importance ranking was moderate. In addition, greater proximity between nodes is shown, indicating similar importance.

In short, the three central nodes are comprised by interdimensional principles and criteria. For example, the environmental criteria are present in all groups and environmental criteria in particular are related to social and economic ones. The forces – directed and approximate force
between criteria – shown in Figure 13 call attention to the possibility of evaluating two or more
sustainability dimensions through one criterion.

Conclusions

The new methodological validation permits the integration of expert consultation, descriptive statistical analysis and data visualization to confirm the importance of adding the technological and political dimensions to the three traditional dimensions of the Triple Bottom Line (TBL) for sustainability assessment (social, economic and environmental) in biodiesel production. Likewise, it was possible to establish the importance ranking of principles and criteria that permit assessment of the sustainability of biodiesel production.

As a result, five dimensions, 13 principles and 30 criteria were validated and recommended to become part of the sustainability assessment framework for biodiesel production. Furthermore, three central nodes were identified, comprised of interdimensional principles and criteria that permit the visualization of relationships between dimensions, principles and criteria.

The proposed hierarchical sustainability assessment framework of principles, criteria and indicator (PC&I) for biodiesel production has the following constraints: First, the framework applicability is to renewable energy whose raw material is biomass. Second, for the assessment of a specific context of biodiesel production it is necessary to define a set of indicators for each criterion. Third, most of the experts belong to the geographical area of Latin America, with Universities as their organization and Technological and Environmental as their expertise area; thus it is important to strengthen expert consultation with experts from other geographical areas, organizations and expertise areas. However, it is important to note that, according to the results of the Principal Component Analysis followed in this work, the ranks given by the experts to principles and criteria were not correlated with their geographical origin, type of organization or expertise area.

Regarding the attributes of importance that were proposed (relevance, ease of measurement and reliability) to rank the impact of each principle and criterion on the assessment framework, a high correlation was identified between ease of measurement and reliability. For example, if the measurement of a criterion is easy, it is possible to repeat and verify the measurement, and increase the reliability in regard to the criterion. Therefore, the Ease of Measurement attribute can be eliminated and the importance ranking can be performed through the Relevance and Reliability attributes.

Acknowledgements

The authors would like to thank the program ECOS No. C13P01 (Evaluation-orientation of the Cooperation Scientific and University with Hispanic America), and COLCIENCIAS RC09002012 (Administrative Department of Science, Technology and Innovation of Colombia) for their support in this research work.

References

Augusiak, J., Van den Brink, P.J. & Grimm, V., 2014. Merging validation and evaluation of ecological models to “evaludation”: A review of terminology and a practical approach. *Ecological Modelling*, 280, pp. 117–128.
Barrett, S., 2011. European Expert Group reports on future transport fuels. *Fuel Cells Bulletin*, 2011(2), pp. 12–16.

Bautista, S. et al., 2016. Biodiesel -TBL+: A new hierarchical sustainability assessment framework of PC&I for biodiesel production- Part I. *Ecological Indicators*, 60, pp. 84–107. Available at: http://dx.doi.org/10.1016/j.ecolind.2015.06.020.

Bisaro, A. et al., 2014. Global drivers setting desertification research priorities: Insights from a stakeholder consultation forum. *Land Degradation and Development*, 25(1), pp. 5–16.

Brent, A.C. & Musango, J.K., 2013. Application of a new framework for energy technology sustainability assessment: results from survey. *22nd International conference for the international association of management of technology IAMOT 2013 “Science, Technology and Innovation in the Emerging Market Economies”*, pp. 1–14.

Buchholz, T., Luzadis, V. a. & Volk, T. a., 2009. Sustainability criteria for bioenergy systems: results from an expert survey. *Journal of Cleaner Production*, 17(SUPPL. 1), pp. S86–S98. Available at: http://dx.doi.org/10.1016/j.jclepro.2009.04.015.

Bueren, E.L. Van & Blom, E., 1996. Hierarchical Framework for the Formulation of Sustainable Forest Management Standards, p. 97.

Buytaert, V. et al., 2011. Towards integrated sustainability assessment for energetic use of biomass: A state of the art evaluation of assessment tools. *Renewable and Sustainable Energy Reviews*, 15(8), pp.3918–3933. Available at: http://dx.doi.org/10.1016/j.rser.2011.07.036.

Cramer, J. et al., 2007. Testing Framework for Sustainable Biomass (“Cramer Criteria”) - The Netherlands. , pp. 1–16.

Cucek, L. & Kravanja, Z., 2012. A Review of Footprint analysis tools for monitoring impacts on sustainability, p. 34.

Dreier, T., Geiger, B. & Wagner, U., 1998. Environmental Impacts and System Analysis of Biofuels. In H. Kopetz et al., eds. *Biomass for Energy and Industry: 10th European Conference and Technology Exhibition, proceedings of the International Conference*. Würzburg, pp. 544–548. Available at: http://www.gbv.de/dms/tib-ub-hannover/246435062.pdf.

FAO & BEFSCI, 2011. *A Compilation of Bioenergy Sustainability Initiatives Overview*,

FAO, F. and A.O. of the U.N., 2013. *Biofuels and the sustainability challenge: A global assessment of policies for biofuels and related feedstocks*,

FAO, F. and A.O. of the U.N. & BEFSCI, B. and F.S.C. and I. project, 2012. *A compilation of Tools and Methodologies to Assess the Sustainability of Modern Bioenergy*,

Fao, F. and A.O. of the U.N. & Befsci, B. and F.S.C. and I.P., 2007. Sustainability criteria for biomass and biofuels. , pp. 1–5. Available at: http://www.worldenergy.org/documents/annex8sustcritrgbelg.pdf.
Finnveden, G. & Ekvall, T., 1998. Life-cycle assessment as a decision-support tool—the case of recycling versus incineration of paper. *Resources, Conservation and Recycling*, 24(3-4), pp.235–256. Available at: http://www.scopus.com/inward/record.url?eid=2-s2.0-0031782337&partnerID=tZOtx3y1 [Accessed February 11, 2016].

Fiorese, G. et al., 2013. Advanced biofuels: Future perspectives from an expert elicitation survey. *Energy Policy*, 56, pp.293–311.

Florin, M.J., van de Ven, G.W.J. & van Ittersum, M.K., 2013. What drives sustainable biofuels? A review of indicator assessments of biofuel production systems involving smallholder farmers. *Environmental Science & Policy*, pp. 1–16.

Fort, E., Lobies, P. & Bergeret, A., 2011. Phénomène de rupture professionnelle chez des salariés en consultation en service de maladie professionnelle. Analyse factorielle et classification. *Archives des Maladies Professionnelles et de l'Environnement*, 72(5), pp. 426–438.

Fritsche, U. et al., 2006. *Sustainability Standards for Bioenergy*, Frankfurt.

Gnansounou, E., 2011. Assessing the sustainability of biofuels: A logic-based model. *Energy*, 36(4), pp.2089–2096. Available at: http://dx.doi.org/10.1016/j.energy.2010.04.027.

Hacking, T. & Guthrie, P., 2008. A framework for clarifying the meaning of Triple Bottom-Line, Integrated, and Sustainability Assessment. *Environmental Impact Assessment Review*, 28(2-3), pp. 73–89.

Hastie, T., Tibshirani, R. & Friedman, J., 2009. The Elements of Statistical Learning. *The Mathematical Intelligencer*, 27, p. 721.

Herman, I. et al., 2000. Graph Visualization and Navigation in Information Visualization: A Survey. *IEEE Transactions on visualization and computer graphics*, 6(1), pp. 24–43.

Hill, J. et al., 2006. Environmental, economic, and energetic costs and benefits of biodiesel and ethanol biofuels. *Proceedings of the National Academy of Sciences of the United States of America*, 103(30), pp. 11206–11210.

Hu, Y., 2005. Efficient and high quality force-directed graph drawing. *The Mathematica Journal*, 10(1), pp. 37–71.

Janssen, R. & Rutz, D.D., 2011. Sustainability of biofuels in Latin America: Risks and opportunities. *Energy Policy*, 39(10), pp. 5717–5725. Available at: http://dx.doi.org/10.1016/j.enpol.2011.01.047.

Kaltschmitt, M., Reinhardt, G.A. & Stelzer, T., 1997. Life cycle analysis of biofuels under different environmental aspects. *Biomass and Bioenergy*, 12(2), pp. 121–134. Available at: http://www.scopus.com/inward/record.url?eid=2-s2.0-0030616102&partnerID=tZOtx3y1 [Accessed December 27, 2015].

Kampman, B., Grinsven, A. & Croezen, H., 2012. *Sustainable alternatives for land-based biofuels in the European Union Assessment of options and development of a policy strategy* C. Delft, ed., Delft.
Kurka, T. & Blackwood, D., 2013. Participatory selection of sustainability criteria and indicators for bioenergy developments. *Renewable and Sustainable Energy Reviews*, 24, pp. 92–102. Available at: http://dx.doi.org/10.1016/j.rser.2013.03.062.

Lee, S. et al., 2012. Dynamic and multidimensional measurement of product-service system (PSS) sustainability: a triple bottom line (TBL) -based system dynamics approach. *Journal of Cleaner Production*, 32, pp. 173–182.

MacLean, H.L. et al., 2000. A life-cycle comparison of alternative automobile fuels. *Journal of the Air and Waste Management Association*, 50(10), pp.1769–1779. Available at: http://www.scopus.com/inward/record.url?eid=2-s2.0-0033625794&partnerID=tZOtx3y1.

Markevičius, a. et al., 2010. Trends and sustainability criteria of the production and use of liquid biofuels. *Renewable and Sustainable Energy Reviews*, 14(9), pp. 3226–3231.

Mendoza, G. et al., 1999. Guidelines for applying Multi-Criteria Analysis to the assessment of criteria and indicators. *The Criteria & Indicator Toolbox series*, Available at: http://www.cifor.org/acm/methods/toolbox1.html.

Petiot, J.-F. & Yannou, B., 2004. Measuring consumer perceptions for a better comprehension, specification and assessment of product semantics. *International Journal of Industrial Ergonomics*, 33(6), pp. 507–525.

Reinhardt, G., 1996. Life cycle assessments (LCAs) of liquid biofuels: latest results. In *2nd European Motor Biofuels Forum, Proceedings, Graz, AT.*, Graz, pp. 297–303. Available at: https://www.tib.eu/en/search/id/tema%3ATEMAM97040096579/Life-cycle-assessments-LCAs-of-liquid-biofuels/#morelikethis [Accessed February 11, 2016].

Scarlat, N., 2011. Recent developments of biofuels / bioenergy sustainability certification: A global overview., 39, pp. 1630–1646.

Silva Lora, E.E. et al., 2011. Issues to consider, existing tools and constraints in biofuels sustainability assessments. *Energy*, 36(4), pp.2097–2110. Available at: http://dx.doi.org/10.1016/j.energy.2010.06.012.

Van der Sluijs, J.P. et al., 2005. Combining quantitative and qualitative measures of uncertainty in model-based environmental assessment: the NUSAP system. *Risk analysis: an official publication of the Society for Risk Analysis*, 25(2), pp. 481–92.

Syms, C., 2008. Principal Components Analysis. In S. E. Jørgensen & B. D. Fath, eds. *Encyclopedia of Ecology*. Oxford: Academic Press, pp. 2940–2949.

Tickner, T., 2013. 11th international specialist conference on biofuels in Berlin: Expert forums on “biodiesel”, “bioethanol”, “biomethane” and “biomass demand for biofuel production” give insights into the latest developments. *Automotive Industries AI*, 193(12).

Van Cauwenbergh, N. et al., 2007. SAFE-A hierarchical framework for assessing the sustainability of agricultural systems. *Agriculture, Ecosystems and Environment*, 120(2-4), pp. 229–242.
Vermeulen, S. & Cotula, L., 2010. Over the heads of local people: consultation, consent, and recompense in large-scale land deals for biofuels projects in Africa. *Journal of Peasant Studies*, 37(4), pp. 899–916.

Ward, J.H., 1963. Hierarchical Grouping to Optimize an Objective Function. *Journal of the American Statistical Association*, 58(301), pp. 236–244.

Zah, R. et al., 2009. Standardized and simplified life-cycle assessment (LCA) as a driver for more sustainable biofuels. *Journal of Cleaner Production*, 17(SUPPL. 1), pp. 102–105.

Zemanek, G. & Reinhardt, G., 1999. First total evaluation of the ecological comparison: Biofuels versus conventional fuels. In R. P. Overend & E. Chornet, eds. *Biomass, a Growth Opportunity in Green Energy and Value-added Products: Proceedings of the 4th Biomass Conference of the Americas*. Biomass, a Growth Opportunity in Green Energy and Value-added Products: Proceedings of the 4th Biomass Conference of the Americas. Elsevier Science, pp. 393–399. Available at: https://books.google.com.co/books?id=Db9GAAAAYAAJ.

Appendix 1. Principles and criteria proposed

Appendix 2. Certification, initiatives, standards and policies for biofuels