Energy Efficiency across Programming Languages
How Does Energy, Time, and Memory Relate?
Rui Pereira

Marco Couto

Francisco Ribeiro, Rui Rua

HASLab/INESC TEC
Universidade do Minho, Portugal
ruipereira@di.uminho.pt

HASLab/INESC TEC
Universidade do Minho, Portugal
marco.l.couto@inesctec.pt

HASLab/INESC TEC
Universidade do Minho, Portugal
fribeiro@di.uminho.pt
rrua@di.uminho.pt

Jácome CunhaJoão Paulo FernandesJoão Saraiva

NOVA LINCS, DI, FCT
Univ. Nova de Lisboa, Portugal
jacome@fct.unl.pt

Release/LISP, CISUC
Universidade de Coimbra, Portugal
jpf@dei.uc.pt

HASLab/INESC TEC
Universidade do Minho, Portugal
saraiva@di.uminho.pt

Abstract
This paper presents a study of the runtime, memory usage
and energy consumption of twenty seven well-known software languages. We monitor the performance of such languages using ten different programming problems, expressedin each of the languages. Our results show interesting findings, such as, slower/faster languages consuming less/moreenergy, and how memory usage influences energy consumption. We show how to use our results to provide software
engineers support to decide which language to use whenenergy efficiency is a concern.
CCS Concepts • Software and its engineering → Software performance; General programming languages;
Keywords Energy Efficiency, Programming Languages, Language Benchmarking, Green Software
ACM Reference format:
Rui Pereira, Marco Couto, Francisco Ribeiro, Rui Rua, Jácome Cunha,João Paulo Fernandes, and João Saraiva. 2017. Energy Efficiency
across Programming Languages. In Proceedings of SLE’17, Vancouver,
BC, Canada, October 23–24, 2017, 12 pages.
https://doi.org/10.1145/3136014.3136031

1

Introduction

Software language engineering provides powerful techniques
and tools to design, implement and evolve software languages. Such techniques aim at improving programmers
Permission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies are notmade or distributed for profit or commercial advantage and that copies bear
this notice and the full citation on the first page. Copyrights for componentsof this work owned by others than ACM must be honored. Abstracting withcredit is permitted. To copy otherwise, or republish, to post on servers or toredistribute to lists, requires prior specific permission and/or a fee. Request
permissions from permissions@acm.org.
SLE’17, October 23–24, 2017, Vancouver, BC, Canada© 2017 Association for Computing Machinery.
ACM ISBN 978-1-4503-5525-4/17/10. . . $15.00
https://doi.org/10.1145/3136014.3136031

productivity - by incorporating advanced features in the language design, like for instance powerful modular and typesystems - and at efficiently execute such software - by developing, for example, aggressive compiler optimizations.
Indeed, most techniques were developed with the main goalof helping software developers in producing faster programs.
In fact, in the last century performance in software languages
was in almost all cases synonymous of fast execution time(embedded systems were probably the single exception).
In this century, this reality is quickly changing and software energy consumption is becoming a key concern forcomputer manufacturers, software language engineers, programmers, and even regular computer users. Nowadays, it
is usual to see mobile phone users (which are powerful computers) avoiding using CPU intensive applications just to
save battery/energy. While the concern on the computers’
energy efficiency started by the hardware manufacturers, it
quickly became a concern for software developers too [27].
In fact, this is a recent and intensive area of research where
several techniques to analyze and optimize the energy consumption of software systems are being developed. Suchtechniques already provide knowledge on the energy efficiency of data structures [14, 26] and android language [24],the energy impact of different programming practices both inmobile [17, 21, 30] and desktop applications [25, 31], the energy efficiency of applications within the same scope [2, 16],or even on how to predict energy consumption in several
software systems [4, 13], among several other works.
An interesting question that frequently arises in the software energy efficiency area is whether a faster program isalso an energy efficient program, or not. If the answer is yes,then optimizing a program for speed also means optimizing
it for energy, and this is exactly what the compiler construction community has been hardly doing since the very
beginning of software languages. However, energy consumption does not depends only on execution time, as shownin the equation Ener дy = Time × Power . In fact, there areseveral research works showing different results regarding
this subject [1, 20, 26, 28, 34, 37]. A similar question arises

SLE’17, October 23–24, 2017, Vancouver, BC, Canada
when comparing software languages: is a faster language, agreener one? Comparing software languages, however, is anextremely complex task, since the performance of a language
is influenced by the quality of its compiler, virtual machine,
garbage collector, available libraries, etc. Indeed, a software
program may become faster by improving its source code,
but also by "just" optimizing its libraries and/or its compiler.
In this paper we analyze the performance of twenty sevensoftware languages. We consider ten different programming
problems that are expressed in each of the languages, following exactly the same algorithm, as defined in the Computer
Language Benchmark Game (CLBG) [11]. We compile/execute such programs using the state-of-the-art compilers,
virtual machines, interpreters, and libraries for each of the
27 languages. Afterwards, we analyze the performance ofthe different implementation considering three variables:
execution time, memory consumption and energy consumption. Moreover, we analyze those results according to thelanguages’ execution type (compiled, virtual machine andinterpreted), and programming paradigm (imperative, functional, object oriented, scripting) used. For each of the execution types and programming paradigms, we compiled asoftware language ranking according to each variable considered. Our results show interesting findings, such as, slower/faster software languages consuming less/more energy, andhow memory usage influences energy consumption. Moreover, we discuss how to use such results to provide software
engineers support to decide which language to use whenenergy efficiency is a concern.
This work builds on previous work [6] where it was presented a framework to allow the monitoring of the energy
consumption of executable software programs. In that work,
the C-based framework was used to define a preliminary
ranking of ten languages (where only energy was considered). We reuse the energy monitoring framework (briefly
described in Section 2.2) to analyze the energy efficiencyof 27 languages and (almost) 270 programs. We have also
extended it in order to monitor memory consumption, as
well.
This paper is organized as follows: Section 2 exposes thedetailed steps of our methodology to measure and compareenergy efficiency in software languages, followed by a presentation of the results. Section 3 contains the analysis anddiscussion on the obtained results, where we first analyze
whether execution time performance implies energy efficiency, then we examine the relation between peak memory usage and memory energy consumption, and finally we
present a discussion on how energy, time and memory relate in the 27 software languages. In Section 4 we discuss
the threats to the validity of our study. Section 5 presents
the related work, and finally, in Section 6 we present theconclusions of our work.

R. Pereira et. al.

2

Measuring Energy in Software
Languages

The initial motivation and primary focus of this work is tounderstand the energy efficiency across various programming languages. This might seem like a simple task, but it
is not as trivial as it sounds. To properly compare the energy efficiency between programming languages, we mustobtain various comparable implementations with a good
representation of different problems/solutions.
With this in mind, we begin by trying to answer the following research question:• RQ1: Can we compare the energy efficiency of software
languages? This will allow us to have results in which
we can in fact compare the energy efficiency of popular programming languages. In having these results,
we can also explore the relations between energy consumption, execution time, and memory usage.
The following subsections will detail the methodology
used to answer this question, and the results we obtained.

2.1

The Computer Language Benchmarks Game

In order to obtain a comparable, representative and extensive
set of programs written in many of the most popular andmost widely used programming languages we have explored
The Computer Language Benchmarks Game [11]. (CLBG).
The CLBG initiative includes a framework for running,
testing and comparing implemented coherent solutions fora set of well-known, diverse programming problems. The
overall motivation is to be able to compare solutions, within
and between, different programming languages. While theperspectives for comparing solutions have originally essentially analyzed runtime performance, the fact is that CLBG
has recently also been used in order to study the energy
efficiency of software [6, 20, 24].
In its current development stage, the CLBG has gathered
solutions for 13 benchmark problems, such that solutions
to each such problem must respect a given algorithm andspecific implementation guidelines. Solutions to each problem are expressed in, at most, 28 different programming
languages.
The complete list of benchmark problems in the CLBG
covers different computing problems, as described in Table 1.
Additionally, the complete list of programming languages in
the CLBG is shown in Table 2, sorted by their paradigms.

2.2

Design and Execution

Our case study to analyze the energy efficiency of software
languages is based on the CLBG.

Energy Efficiency across Programming Languages
Table 1. CLBG corpus of programs.
Benchmarkn-bodyfannkuchredux
spectralnorm
mandelbrot
pidigits
regex-redux
fasta
k-nucleotide
reversecomplement
binary-trees
chameneosredux
meteorcontest
thread-ring

Description
Double precision N-bodysimulation
Indexed access to tiny integer
sequence
Eigenvalue using the power
methodGenerate Mandelbrot set
portable bitmap file
Streaming arbitrary precision
arithmetic
Match DNA 8mers andsubstitute magic patterns
Generate and write random
DNA sequences
Hashtable update andk-nucleotide strings
Read DNA sequences, write
their reverse-complement
Allocate, traverse anddeallocate many binary trees
Symmetrical thread rendezvous
requests
Search for solutions to shapepacking puzzle
Switch from thread to thread
passing one token

Input
50M
12
5,500
16,00010,000fasta
output25M
fasta
outputfasta
output
21
6M
2,098
50M

Table 2. Languages sorted by paradigm
Paradigm
Functional
Imperative
ObjectOriented
ScriptingLanguages
Erlang, F#, Haskell, Lisp, Ocaml, Perl,
Racket, Ruby, Rust;
Ada, C, C++, F#, Fortran, Go, Ocaml,
Pascal, Rust;
Ada, C++, C#, Chapel, Dart , F#, Java,JavaScript, Ocaml, Perl, PHP, Python,
Racket, Rust, Smalltalk, Swift,
TypeScript;
Dart, Hack, JavaScript, JRuby, Lua, Perl,
PHP, Python, Ruby, TypeScript;

From the 28 languages considered in the CLBG, we excluded Smalltalk since the compiler for that language is proprietary. Also, for comparability, we have discarded benchmark problems whose language coverage is below the threshold of 80%. By language coverage we mean, for each benchmark problem, the percentage of programming languages(out of 27) in which solutions for it are available. This criteria
excluded chameneos-redux, meteor-contest and threadring from our study.
We then gathered the most efficient (i.e. fastest) versionof the source code in each of the remaining 10 benchmark
problems, for all the 27 considered programming languages.

SLE’17, October 23–24, 2017, Vancouver, BC, Canada
The CLBG documentation also provides information about
the specific compiler/runner version used for each language,
as well as the compilation/execution options considered (forexample, optimization flags at compile/run time). We strictly
followed those instructions and installed the correct compiler versions, and also ensured that each solution was compiled/executed with the same options used in the CLBG.
Once we had the correct compiler and benchmark solutions
for each language, we tested each one individually to makesure that we could execute it with no errors and that theoutput was the expected one.
The next step was to gather the information about energy consumption, execution time and peak memory usage
for each of the compilable and executable solutions in eachlanguage. It is to be noted that the CLBG already contains
measured information on both the execution time and peakmemory usage. We measured both not only to check theconsistency of our results against the CLBG, but also sincedifferent hardware specifications would bring about different results. For measuring the energy consumption, we usedIntel’s Running Average Power Limit (RAPL) tool [9], whichis capable of providing accurate energy estimates at a veryfine-grained level, as it has already been proven [12, 29]. Also,
the current version of RAPL allows it to be invoked fromany program written in C and Java (through jRAPL [22]).
In order to properly compare the languages, we needed
to collect the energy consumed by a single execution of a
specific solution. In order to do this, we used the system
function call in C, which executes the string values which
are given as arguments; in our case, the command necessary
to run a benchmark solution (for example, the binary-trees
solution written in Python is executed by writing the command /usr/bin/python binarytrees.py 21).
The energy consumption of a solution will then be theenergy consumed by the system call, which we measured
using RAPL function calls. The overall process (i.e., the workflow of our energy measuring framework 1 ) is described in
Listing 1.
...
for ( i = 0 ; i < N ; i ++) {time_before = getTime (...) ;// performs initial energy measurement
rapl_before (...) ;// executes the programsystem ( command ) ;// computes the difference between// this measurement and the initial one
rapl_after (...) ;time_elapsed = getTime (...) - time_before ;
...
}
...

Listing 1. Overall process of the energy measuring
framework.
1 The measuring framework and the complete set of results are publicly

available at https://sites.google.com/view/energy-efficiency-languages

SLE’17, October 23–24, 2017, Vancouver, BC, Canada
In order to ensure that the overhead from our measuring framework, using the system function, is negligible ornon-existing when compared to actually measuring with
RAPL inside a program’s source code, we design a simple
experiment. It consisted of measuring the energy consumption inside of both a C and Java language solution, using
RAPL and jRAPL respectively, and comparing the results tothe measurements from our C language energy measuring
framework. We found the resulting differences to be insignificant, and therefore negligible, thus we conclude that we
could use this framework without having to worry about
imprecisions in the energy measurements.
Also, we chose to measure the energy consumption andthe execution time of a solution together, since the overhead
will be the same for every measurement, and so this should
not affect the obtained values.
The memory usage of a solution was gathered using thetime tool, available in Unix-based systems. This tool runs a
program given as an argument, and summarizes the system
resources used by that program, which includes the peak ofmemory usage.
Each benchmark solution was executed and measured 10
times, in order to obtain 10 energy consumption and execution time samples. We did so to reduce the impact of cold
starts and cache effects, and to be able to analyze the measurements’ consistency and avoid outliers. We followed thesame approach when gathering results for memory usage.
For some benchmark problems, we could not obtain any
results for certain programming languages. In some cases,
there was no source code available for the benchmark problem (i.e., no implementation was provided in a concrete language which reflects a language coverage below 100%).2
In other cases, the code was indeed provided but either thecode itself was already buggy or failing to compile or execute,
as documented in CLBG, or, in spite of our best efforts, we
could not execute it, e.g., due to missing libraries.3 From
now on, for each benchmark problem, we will refer as its
execution coverage to the percentage of (best) solutions forit that we were actually able to successfully execute.
All studies were conducted on a desktop with the following specifications: Linux Ubuntu Server 16.10 operating
system, kernel version 4.8.0-22-generic, with 16GB of RAM,a Haswell Intel(R) Core(TM) i5-4460 CPU @ 3.20GHz.
2.3

Results

The results from our study are partially shown in this section,with the remainder shown in the online appendix for thispaper 1 . Table 3, and the left most tables under Results - A.
Data Tables in the appendix, contains the measured data fromdifferent benchmark solutions. We only show the results
2 In these cases, we will include an n.a. indication when presenting theirresults.
3 In these cases, we will include an n.e. indication when presenting theirresults.

R. Pereira et. al.
for binary-trees, fannkuch-redux, and fasta within thepaper, which are the first 3 ordered alphabetically. Each row
in a table represents one of the 27 programming languages
which were measured.
The 4 rightmost columns, from left to right, represent the
average values for the Energy consumed (Joules), Time of
execution (milliseconds), Ratio between Energy and Time,and the amount of peak memory usage in Mb. The Energy
value is the sum of CPU and DRAM energy consumption.
Additionally, the Ratio can also be seen as the average Power,
expressed in Kilowatts (kW). The rows are ordered according
to the programming language’s energy consumption, fromlowest to highest. Finally, the right most tables under Results- A. Data Tables contain the standard deviation and averagevalues for our measured CPU, DRAM, and Time, allowing
us to understand the variance.
The first column states the name of the programming languages, preceded by either a (c), (i), or (v) classifying them as
either a compiled, interpreted, or virtual-machine language,
respectively. In some cases, the programming language namewill be followed with a ↑x /↓y and/or ⇑x /⇓y symbol. The first
set of arrows indicates that the language would go up by
x positions (↑x ) or down by y positions (↓y ) if ordered byexecution time. For example in Table 3, for the fasta benchmark, Fortran is the second most energy efficient language,
but falls off 6 positions down if ordered by execution time.
The second set of arrows states that the language would go
up by x positions (⇑x ) or down by y positions (⇓y ) if orderedaccording to their peak memory usage. Looking at the same
example benchmark, Rust, while the most energy efficient,would drop 9 positions if ordered by peak memory usage.
Table 4 shows the global results (on average) for Energy,
Time, and Mb normalized to the most efficient language inthat category. Since the pidigits benchmark solutions only
contained less than half of the languages covered, we did notconsider this one for the global results. The base values are
as follows: Energy for C is 57.86J, Time for C is 2019.26ms, andMb for Pascal is 65.96Mb. For instance, Lisp, on average,
consumes 2.27x more energy (131.34J) than C, while taking
2.44x more time to execute (4926.99ms), and 1.92x morememory (126.64Mb) needed when compared to Pascal.
To better visualize and interpret the data, we also generated two different sets of graphical data for each of thebenchmarks. The first set, Figures 1-3 and the left most figures under Results - C. Energy and Time Graphs in the appendix, contains the results of each language for a benchmark,consisting of three joint parts: a bar chart, a line chart, and ascatter plot. The bars represent the energy consumed by thelanguages, with the CPU energy consumption on the bottom
half in blue dotted bars and DRAM energy consumption on
the top half in orange solid bars, and the left y-axis representing the average Joules. The execution time is represented
by the line chart, with the right y-axis representing averagetime in milliseconds. The joining of these two charts allow

Energy Efficiency across Programming Languages
us to better understand the relationship between energy andtime. Finally, a scatter plot on top of both represents the ratio between energy consumed and execution time. The ratio
plot allows us to understand if the relationship between energy and time is consistent across languages. A variation inthese values indicates that energy consumed is not directly
proportional to time, but dependent on the language and/or
benchmark solution.
The second set, Figures 4-6 and the right most figures
under Results - C. Energy and Time Graphs in the appendix,
consists of two parts: a bar chart, and a line chart. The blue
bars represent the DRAM’s energy consumption for each of
the languages, with the left y-axis representing the average
Joules. The orange line chart represents the peak memory
usage for each language, with the right y-axis representing
the average Mb. The joining of these two allows us to lookat the relation between DRAM energy consumption and thepeak memory usage for each language in each benchmark.
By turning to the CLBG, we were able to use a large setof software programming languages which solve various
different programming problems with similar solutions. This
allowed us to obtain a comparable, representative, and extensive set of programs, written in several of the most popular
languages, along with the compilation/execution options,and compiler versions. With these joined together with ourenergy measurement framework, which uses the accurate Intel RAPL tool, we were able to measure, analyze, and compare
the energy consumption, and in turn the energy efficiency,of software languages, thus answering RQ1 as shown with
our results. Additionally, we were also able to measure theexecution time and peak memory usage which allowed us
to analyze how these two relate with energy consumption.
The analysis and discussion of our results is shown in thenext section.

3

Analysis and Discussion

In this section we will present an analysis and discussion on
the results of our study. While our main focus is on understanding the energy efficiency in languages, we will also try
to understand how energy, time, and memory relate. Additionally, in this section we will try to answer the following
three research questions, each with their own designatedsubsection.
• RQ2: Is the faster language always the most energy
efficient? Properly understanding this will not only
address if energy efficiency is purely a performance
problem, but also allow developers to have a greater
understanding of how energy and time relates in alanguage, and between languages.
• RQ3: How does memory usage relate to energy consumption? Insight on how memory usage affects energy consumption will allow developers to better understandhow to manage memory if their concern is energy
consumption.

SLE’17, October 23–24, 2017, Vancouver, BC, Canada• RQ4: Can we automatically decide what is the best programming language considering energy, time, and memory usage? Often times developers are concerned withmore than one (possibly limited) resource. For example, both energy and time, time and memory space,energy and memory space or all three. Analyzing these
trade-offs will allow developers to know which programming languages are best in a given scenarios.
3.1

Is faster, greener?

A very common misconception when analyzing energy consumption in software is that it will behave in the sameway execution time does. In other words, reducing the execution time of a program would bring about the sameamount of energy reduction. In fact, the Energy equation,
Energy (J) = Power (W) x Time(s), indicates that reducing time implies a reduction in the energy consumed.
However, the Power variable of the equation, which cannot be assumed as a constant, also has an impact on theenergy. Therefore, conclusions regarding this issue diverge
sometimes, where some works do support that energy andtime are directly related [37], and the opposite was also observed [20, 28, 34].
The data presented in the aforementioned tables and figures lets us draw an interesting set of observations regarding
the efficiency of software languages when considering both
energy consumption and execution time. Much like [1] and[26], we observed different behaviors for energy consumption and execution time in different languages and tests.
By observing the data in Table 4, we can see that the C
language is, overall, the fastest and most energy efficient.
Nevertheless, in some specific benchmarks there are more
efficient solutions (for example, in the fasta benchmark itis the third most energy efficient and second fastest).
Execution time behaves differently when compared to energy efficiency. The results for the 3 benchmarks presented
in Table 3 (and the remainder shown in the appendix) show
several scenarios where a certain language energy consumption rank differs from the execution time rank (as the arrows
in the first column indicate). In the fasta benchmark, forexample, the Fortran language is second most energy efficient, while dropping 6 positions when it comes to execution
time. Moreover, by observing the Ratio values in Figures 1
to 3 (and the remainder in the appendix under Results - C. Energy and Time Graphs), we clearly see a substantial variation
between languages. This means that the average power is
not constant, which further strengthens the previous point.
With this variation, we can have languages with very similarenergy consumptions and completely different execution
times, as is the case of languages Pascal and Chapel in thebinary trees benchmark, which energy consumption differ roughly by 10% in favor of Pascal, while Chapel takes
about 55% less time to execute.

SLE’17, October 23–24, 2017, Vancouver, BC, Canada

R. Pereira et. al.

Table 3. Results for binary-trees, fannkuch-redux, and fasta(c) C(c) C++(c) Rust ⇓2(c) Fortran ⇑1(c) Ada ⇓1(c) Ocaml ↓1 ⇑2(v) Java ↑1 ⇓16(v) Lisp ↓3 ⇓3(v) Racket ↓4 ⇓6(i) Hack ↑2 ⇓9(v) C# ↓1 ⇓1(v) F# ↓3 ⇓1(c) Pascal ↓3 ⇑5(c) Chapel ↑5 ⇑4(v) Erlang ↑5 ⇑1(c) Haskell ↑2 ⇓2(i) Dart ↓1 ⇑1(i) JavaScript ↓2 ⇓4(i) TypeScript ↓2 ⇓2(c) Go ↑3 ⇑13(i) Jruby ↑2 ⇓3(i) Ruby ⇑5(i) PHP ⇑3(i) Python ⇑15(i) Lua ↓1(i) Perl ↑1(c) Swiftbinary-trees
Energy
Time
39.80
1125
41.23
1129
49.07
1263
69.82
2112
95.02
2822
100.743525
111.843306
149.5510570
155.8111261
156.714497
189.7410797
207.1315637
214.6416079
237.297265
266.147327
270.1511582
290.2717197
312.1421349
315.1021686
636.71
16292
720.53
19276
855.1226634
1,397.51
42316
1,793.4645003
2,452.04 209217
3,542.20
96097
n.e.

Ratio
0.035
0.037
0.039
0.033
0.034
0.029
0.034
0.014
0.014
0.035
0.018
0.013
0.013
0.033
0.036
0.023
0.017
0.015
0.015
0.039
0.037
0.032
0.033
0.040
0.012
0.037

Mb
131
132
180
133
197
148
1120
373
467
502
427
432
256
335
433
494
475
916
915
228
1671
482
786
275
1961
2148

(c) C ⇓2(c) C++ ⇑1(c) Rust ⇓11(c) Swift ⇓5(c) Ada ⇓2(c) Ocaml ↓1(c) Chapel ↑1 ⇓18(v) Lisp ↓3 ⇓15(v) Java ↑1 ⇓13(c) Fortran ⇓1(c) Go ↑2 ⇑7(c) Pascal ⇑10(v) F# ↓1 ⇓7(v) C# ↑1 ⇓5(i) JavaScript ↓1 ⇓2(c) Haskell ↑1 ⇑8(i) Dart ⇓7(v) Racket ⇑3(v) Erlang ⇑3(i) Hack ⇓6(i) PHP(i) TypeScript ↓4 ⇑4(i) Jruby ↑1 ⇓4(i) Lua ↓3 ⇑19(i) Perl ↑2 ⇑12(i) Python ↑2 ⇑14(i) Ruby ↑2 ⇑17

fannkuch-redux
EnergyTime
215.926076
219.896123
238.306628
243.816712
264.987351
277.277895
285.397853
309.029154
311.388241
316.508665
318.518487
343.559807
395.0310950
399.3310840
413.9033663
433.6814666
487.2938678
1,941.5343680
4,148.38
1018395,286.77
1154905,731.88
125975
6,898.48
516541
7,819.03
2191488,277.87
635023
11,133.49 249418
12,784.09 279544
14,064.98 315583

Ratio
0.036
0.036
0.036
0.036
0.036
0.035
0.036
0.034
0.038
0.037
0.038
0.035
0.036
0.037
0.012
0.030
0.013
0.044
0.041
0.046
0.046
0.013
0.036
0.013
0.045
0.046
0.045

Mb
2
1
16
7
4
3
53
43
35
12
2
2
34
29
26
7
46
18
18
119
34
26
669
2
12
12
8

(c) Rust ⇓9(c) Fortran ↓6(c) C ↑1 ⇓1(c) C++ ↑1 ⇓2(v) Java ↑1 ⇓12(c) Swift ⇓9(c) Go ↓2(c) Ada ↓2 ⇑3(c) Ocaml ↓2 ⇓15(c) Chapel ↑5 ⇓10(v) C# ↑4 ⇓5(i) Dart ⇓6(i) JavaScript ⇓1(c) Pascal ↓1 ⇑13(i) TypeScript ↓2 ⇓10(v) F# ↑2 ⇑3(v) Racket ↓1 ⇑5(c) Haskell ↑2 ⇓8(v) Lisp ⇓2(i) Hack ⇓3(i) Lua ⇑18(i) PHP ↓1 ⇑13(v) Erlang ↑1 ⇑12(i) Ruby ↓1 ⇑2(i) JRuby ↑1 ⇓2(i) Python ↓1 ⇑18(i) Perl ↑1 ⇑8

Figure 1. Energy and time graphical data for binary-trees

Figure 2. Energy and time graphical data for fannkuch-redux

Figure 3. Energy and time graphical data for fasta

fasta
Energy
26.15
27.62
27.64
34.88
35.86
37.06
40.45
40.45
40.78
40.88
45.35
63.61
64.84
68.63
82.72
93.11
120.90205.52231.49237.70347.37430.73477.81
852.30912.93
1,061.41
2,684.33

Time
931
1661
973
1164
1249
1405
1838
2765
3171
1379
1549
4787
5098
5478
6909
5360
8255
5728
15763
17203
24617
29508
27852
61216
49509
74111
61463

Ratio
0.028
0.017
0.028
0.030
0.029
0.026
0.022
0.015
0.013
0.030
0.029
0.013
0.013
0.013
0.012
0.017
0.015
0.036
0.015
0.014
0.014
0.015
0.017
0.014
0.018
0.014
0.044

Mb
16
1
3
4
41
31
4
3
201
53
35
49
30
0
271
27
21
446
75
120
3
14
18
104
705
9
53

Energy Efficiency across Programming Languages

SLE’17, October 23–24, 2017, Vancouver, BC, Canada

Figure 4. Energy and memory graphical data for binary-trees

Figure 5. Energy and memory graphical data for fannkuch-redux

Figure 6. Energy and memory graphical data for fastaCompiled languages tend to be, as expected, the fastest
and most energy efficient ones. On average, compiled languages consumed 120J to execute the solutions, while forvirtual machine and interpreted languages this value was
576J and 2365J, respectively. This tendency can also be observed for execution time, since compiled languages took
5103ms, virtual machine languages took 20623ms, and interpreted languages took 87614ms (on average). Grouped by
the different paradigms, the imperative languages consumed
and took on average 125J and 5585ms, the object-oriented
consumed 879J and spent 32965ms, the functional consumed
1367J and spent 42740ms and the scripting languages consumed 2320J and spent 88322ms.
Moreover, the top 5 languages that need less energy andtime to execute the solutions are: C (57J, 2019ms), Rust (59J,
2103ms), C++ (77J, 3155ms), Ada (98J, 3740ms), and Java
(114J, 3821ms); of these, only Java is not compiled. As expected, the bottom 5 languages were all interpreted: Perl
(4604J), Python (4390J), Ruby (4045J), JRuby (2693J), and Lua(2660Js) for energy; Lua (167416ms), Python (145178ms),

Perl (132856ms), Ruby (119832ms), and TypeScript (93292ms)
for time.
The CPU-based energy consumption always represents
the majority of the energy consumed. On average, for thecompiled languages, this value represents 88.94% of the energy consumed, being the remaining portion assigned to
DRAM. This value is very similar for virtual machine (88.94%)
and interpreted languages (87.98%). While, as explained in
the last point, the overall average consumption for these 3
language types is very different, the ratio between CPU and
DRAM based energy consumption seems to generally maintain the same proportion. This might indicate that optimizing
a program to reduce the CPU-based energy consumption will
also decrease the DRAM-based energy consumption. However, it is interesting to notice that this value varies more forinterpreted languages (min of 81.57%, max of 92.90%) whencompared to compiled (min of 85.27%, max of 91.75%) or
virtual machine languages (min of 86.10%, max of 92.43%).
With these results, we can try to answer the question
raised in RQ2: Is the faster language always the most energy

SLE’17, October 23–24, 2017, Vancouver, BC, Canada

R. Pereira et. al.

Table 4. Normalized global results for Energy, Time, andMemoryTotal(c) C(c) Rust(c) C++(c) Ada(v) Java(c) Pascal(c) Chapel(v) Lisp(c) Ocaml(c) Fortran(c) Swift(c) Haskell(v) C#(c) Go(i) Dart(v) F#(i) JavaScript(v) Racket(i) TypeScript(i) Hack(i) PHP(v) Erlang(i) Lua(i) Jruby(i) Ruby(i) Python(i) Perl

Energy
1.00
1.03
1.34
1.70
1.98
2.14
2.18
2.27
2.40
2.52
2.79
3.10
3.14
3.23
3.83
4.13
4.45
7.91
21.50
24.02
29.30
42.23
45.98
46.54
69.91
75.88
79.58

(c) C
(c) Rust(c) C++(c) Ada(v) Java(c) Chapel(c) Go(c) Pascal(c) Ocaml(v) C#(v) Lisp(c) Haskell(c) Swift(c) Fortran(v) F#(i) JavaScript(i) Dart(v) Racket(i) Hack(i) PHP(v) Erlang(i) Jruby(i) TypeScript(i) Ruby(i) Perl(i) Python(i) Lua

Time
1.00
1.04
1.56
1.85
1.89
2.14
2.83
3.02
3.09
3.14
3.40
3.55
4.20
4.20
6.30
6.52
6.67
11.27
26.99
27.64
36.71
43.44
46.20
59.34
65.79
71.90
82.91

(c) Pascal(c) Go(c) C(c) Fortran(c) C++(c) Ada(c) Rust(v) Lisp(c) Haskell(i) PHP(c) Swift(i) Python(c) Ocaml(v) C#(i) Hack(v) Racket(i) Ruby(c) Chapel(v) F#(i) JavaScript(i) TypeScript(v) Java(i) Perl(i) Lua(v) Erlang(i) Dart(i) Jruby

Mb
1.00
1.05
1.17
1.24
1.34
1.47
1.54
1.92
2.45
2.57
2.71
2.80
2.82
2.85
3.34
3.52
3.97
4.00
4.25
4.59
4.69
6.01
6.62
6.72
7.20
8.64
19.84

efficient? By looking solely at the overall results, shownin Table 4, we can see that the top 5 most energy efficientlanguages keep their rank when they are sorted by execution
time and with very small differences in both energy and timevalues. This does not come as a surprise, since in 9 out of 10benchmark problems, the fastest and most energy efficient
programming language was one of the top 3. Additionally, it
is common knowledge that these top 3 language (C,C++, andRust) are known to be heavily optimized and efficient for
execution performance, as our data also shows. Thus, as time
influences energy, we had hypothesized that these languages
would also produce efficient energy consumptions as they
have a large advantage in one of the variables influencing
energy, even if they consumed more power on average.
Nevertheless, if we look at the remaining languages inTable 4, we can see that only 4 languages maintain the sameenergy and time rank (OCaml, Haskel, Racket, and Python),while the remainder are completely shuffled. Additionally,
looking at individual benchmarks we see many cases where
there is a different order for energy and time.
Moreover, the tables in Results - A. Data Tables in the
appendix also allows us to understand that this question
does not have a concrete and ultimate answer. Although
the most energy efficient language in each benchmark isalmost always the fastest one, the fact is that there is no
language which is consistently better than the others. This
allows us to conclude that the situation on which a language
is going to be used is a core aspect to determine if that
language is the most energy efficient option. For example,
in the regex-redux benchmark, which manipulates strings

using regular expressions, interpreted languages seem tobe an energy efficient choice (TypeScript, JavaScript and
PHP, all interpreted, are in the top 5), although they tend to
be not very energy efficient in other scenarios. Thus, theanswer for RQ2 is: No, a faster language is not always themost energy efficient.
3.2

Memory impact on energy

How does memory usage affect the memory’s energy consumption? There are two main possible scenarios which
may influence this energy consumption: continuous memory usage and peak memory usage. With the data we havecollected, we will try to answer the latter scenario.
The top 5 languages, which can also be seen in Table 4,
which needed the least amount of memory space (on average)
to execute the solutions were: Pascal (66Mb), Go (69Mb),
C (77Mb), Fortran (82Mb), and C++ (88Mb); these are allcompiled languages. The bottom 5 languages were: JRuby(1309Mb), Dart (570Mb), Erlang (475Mb), Lua (444Mb), andPerl (437Mb); of these, only Erlang is not an interpreted
language.
On average, the compiled languages needed 125Mb, thevirtual machine languages needed 285Mb, and the interpreted needed 426Mb. If sorted by their programming paradigm, the imperative languages needed 116Mb, the objectoriented 249Mb, the functional 251Mb, and finally the scripting needed 421Mb.
Additionally, the top 5 languages which consumed the lastamount of DRAM energy consumption (average) were: C (5J),
Rust (6J), C++ (8J), Ada (10J), and Java (11J); of these, onlyJava is not a compiled language. The bottom 5 languages
were: Lua (430J), JRuby (383J), Python (356J), Perl (327J),
and Ruby (295J); all are interpreted languages. On average,
the compiled languages consumed 14J, the virtual machine
languages consumed 52J, and the interpreted languages consumed 236J.
Looking at the visual data from Figures 4-6, and the rightmost figures under Results - C. Energy and Time Graphs in
the appendix, one can quickly see that there does not seemto be a consistent correlation between the DRAM energy
consumption and the peak memory usage. To verify this, wefirst tested both the DRAM energy consumption and peakmemory usage for normality using the Shapiro-Wilk [32]test. As the data is not normally distributed, we calculated
the Spearman [38] rank-order correlation coefficient. The
result was a Spearman ρ value equal to 0.2091, meaning it
is between no linear relationship (ρ = 0) and a weak uphillpositive relationship (ρ = 0.3).
While we did expect the possibility of little correlation
between the DRAM’s energy consumption and peak memory
usage, we were surprised that the relationship is almost
non-existent. Thus, answering the first part of RQ3, thisindicates that the DRAM’s energy consumption has very

Energy Efficiency across Programming Languages
little to do with how much memory is saved at a given point,
but possibly more of how it is used.
As future work, we wish to measure the continuous memory usage, or in other words the total amount of memory
used over time, to understand if this is what leads to higher
DRAM energy consumption. We expect there to be a stronger
relationship between these two as factors such as garbage
collection, cache usage, register location, and the data management efficiency of each language (read/write) to have astrong impact on the energy consumption.
3.3

Energy vs. Time vs. Memory

There many are situations where a software engineer has
to choose a particular software language to implement hisalgorithm according to some functional or non functional
requirements. For instance, if he is developing software forwearables, it is important to choose a language and apply
energy-aware techniques that help to save battery. Anotherexample is the implementation of tasks that run in background. In this case, execution time may not be a main concern, and they may take longer than the ones related to theuser interaction.
With the fourth research question RQ4, we try to understand if it is possible to automatically decide what isthe best programming language when considering the energy consumption, execution time, and peak memory usageneeded by their programs, globally and individually. In otherwords, if there is a “best” programming languages for all
three characteristics, or if not, which are the best in eachgiven scenario.
To this end, we present in Table 5 a comparison of threecharacteristics of the languages: energy consumption, execution time, and peak memory usage. To be able to compare
the languages using more than one characteristic at a time
we use a multi-objective optimization algorithm to sort theselanguages. This is also called Pareto optimization [7, 8]. It
is necessary to use such an algorithm because in some caseit may happen that no solution simultaneously optimizes
all objectives. For our example, energy, time, and memory
are the optimization objectives. In these cases, a dominantsolution does not exist, but each solution is a set, in our case,of software languages. Here, the solution is called the Pareto
optimal.
We used this technique, and in particular the software
available at [36], to calculate different rankings for the analyzed software languages. In Table 5 we present four multiobjective rankings: time & memory, energy & time, energy& memory, and energy & time, & memory. For each ranking, each line represents a Pareto optimal set, that is, a set
containing the languages that are equivalent to each otherfor the underlying objectives. In other words, each line isa single rank or position. A single software language in aposition signifies that the language was clearly the best for
the analyzed characteristics. Multiple languages in a line

SLE’17, October 23–24, 2017, Vancouver, BC, Canada
imply that a tie occured, as they are essentially similar; yetultimately, the languages lean slightly towards one of theobjectives over the other as a slight trade-off.
The most common performance characteristics of software languages used to evaluate and choose them are execution time and memory usage. If we consider these two
characteristics in our evaluation, C, Pascal, and Go are equivalent. However, if we consider energy and time, C is the bestsolution since it is dominant in both single objectives. If
we prefer energy and memory, C and Pascal constitute thePareto optimal set. Finally, analyzing all three characteristics,
this scenario is very similar as for time and memory.
It is interesting to see that, when considering energy andtime, the sets are usually reduced to one element. This means,that it is possible to actually decide which is the best language. This happens possibly because there is a mathematical
relation between energy and time and thus they are usuallytight together, thus being common that a language is dominant in both objectives at the same time. However, there
are cases where this is not true. For instance, for Pascal andChapel it is not possible to decide which one is the best as
Pascal is better in energy and memory use, but worse in
execution time. In these situations the developer needs tointervene and decide which is the most important aspect tobe able to decide for one language.
It is also interesting to note that, when considering memory use, languages such as Pascal tend to go up in the
ranking. Although this is natural, it is a difficult analysis
to perform without information such as the one we present
in this paper.
Given the information presented in Table 5 we can try
to answer RQ4: Can we automatically decide what isthe best software language considering energy, time,and memory usage? If the developer is only concernedwith execution time and energy consumption, then yes, it isalmost always possible to choose the best language. Unfortunately, if memory is also a concern, it is no longer possible
to automatically decide for a single language. In all the otherrankings most positions are composed by a set of Pareto optimal languages, that is, languages which are equivalent given
the underlying characteristics. In these cases, the developer
will need to make a decision and take into consideration
which are the most important characteristics in each particular scenario, while also considering any fuctional/nonfunctional requirements necessary for the development of
the application. Still, the information we provide in this paper
is quite important to help group languages by equivalence
when considering the different objectives. For the best of ourknowledge, this is the first time such work is presented. Notethat we provide the information of each individual characteristic in Table 4 so the developer can actually understandeach particular set (we do not show such information in Table 5 to avoid cluttering the paper with to many tables withnumbers).

SLE’17, October 23–24, 2017, Vancouver, BC, Canada

R. Pereira et. al.

Table 5. Pareto optimal sets for different combination of objectives.
Time & Memory
Energy & Time
Energy & Memory
Energy & Time & Memory
C • Pascal • GoC
C • Pascal
C • Pascal • Go
Rust • C++ • Fortran
Rust
Rust • C++ • Fortran • Go
Rust • C++ • Fortran
Ada
C++
Ada
Ada
Java • Chapel • Lisp • Ocaml
Ada
Java • Chapel • Lisp
Java • Chapel • Lisp • Ocaml
Haskell • C#
Java
OCaml • Swift • Haskell
Swift • Haskell • C#
Swift • PHP
Pascal • Chapel
C# • PHP
Dart • F# • Racket • Hack • PHP
F# • Racket • Hack • Python
Lisp • Ocaml • Go
Dart • F# • Racket • Hack • Python
JavaScript • Ruby • Python
JavaScript • Ruby
Fortran • Haskell • C#
JavaScript • Ruby
TypeScript • Erlang
Dart • TypeScript • Erlang
Swift
TypeScript
Lua • JRuby • Perl
JRuby • Perl
Dart • F#
Erlang • Lua • Perl
Lua
JavaScript
JRuby
Racket
TypeScript • Hack
PHP
ErlangLua • JRuby
Ruby

4

Threats to Validity

The goal of our study was to both measure and understandthe energetic behavior of several programming languages,
allowing us to bring about a greater insight on how certain
languages compare to each other mainly in terms of energy
consumption, but also performance and memory. We present
in this subsection some threats to the validity of our study,
divided into four categories [3], namely: conclusion validity,
internal validity, construct validity, and external validity.
Conclusion Validity From our experiment it is clear thatdifferent programming paradigms and even languages within
the same paradigm have a completely different impact onenergy consumption, time, and memory. We also see interesting cases where the most energy efficient is not the fastest,
and believe these results are useful for programmers. For abetter comparison, we not only measured CPU energy consumption but also DRAM energy consumption. This allowed
us to further understand the relationship between DRAM
energy consumption and peak memory usage, while also
understanding the behavior languages have in relation theenergy usage derived from the CPU and DRAM. Additionally, the way we grouped the languages is how we felt is themost natural to compare languages (by programming paradigm, and how the language is executed). Thus, this was thechosen way to present the data in the paper. Nevertheless,
all the data is available and any future comparison groups
such as “.NET languages” or “JVM languages” can be very
easily analyzed.
Internal Validity This category concerns itself with what
factors may interfere with the results of our study. Whenmeasuring the energy consumption of the various different

programming languages, other factors alongside the different
implementations and actual languages themselves may contribute to variations, i.e. specific versions of an interpreter or
virtual machine. To avoid this, we executed every language
and benchmark solution equally. In each, we measured theenergy consumption (CPU and DRAM), execution time, andpeak memory 10 times, removed the furthest outliers, andcalculated the median, mean, standard deviation, min, andmax values. This allowed us to minimize the particular statesof the tested machine, including uncontrollable system processes and software. However, the measured results are quiteconsistent, and thus reliable. In addition, the used energy
measurement tool has also been proven to be very accurate.
Construct Validity We analyzed 27 different programming languages, each with roughly 10 solutions to the proposed problems, totaling out to almost 270 different cases.
These solutions were developed by experts in each of the
programming languages, with the main goal of "winning" byproducing the best solution for performance time. While thedifferent languages contain different implementations, they
were written under the same rules, all produced the same
exact output, and were implemented to be the fastest andmost efficient as possible. Having these different yet efficientsolutions for the same scenarios allows us to compare thedifferent programming languages in a quite just manner as
they were all placed against the same problem. Albeit certain paradigms or languages could have an advantage forcertain problems, and others may be implemented in a not so
traditional sense. Nevertheless, there is no basis to suspectthat these projects are best or worst than any other kind we
could have used.

Energy Efficiency across Programming Languages
External Validity We concern ourselves with the generalization of the results. The obtained solutions were the best
performing ones at the time we set up the study. As the CLBG
is an ongoing "competition", we expect that more advanced
and more efficient solutions will substitute the ones we obtained as time goes on, and even the languages’ compilers
might evolve. Thus this, along with measurements in different systems, might produce slightly different resulting values
if replicated. Nevertheless, unless there is a huge leap within
the language, the comparisons might not greatly differ. The
actual approach and methodology we used also favors easy
replications. This can be attributed to the CLBG containing
most of the important information needed to run the experiments, these being: the source code, compiler version,and compilation/execution options. Thus we believe theseresults can be further generalized, and other researchers canreplicate our methodology for future work.

5

Related Work

The work presented in this paper extends the work presented by [6], where the energy consumption monitoring
approach for different programming languages was introduced. The main focus of [6] was the methodology and thecomparison of the CPU-based energy efficiency in 10 of the
28 languages. We made a wider and more in-depth analysis,
since in addition to including all languages, we also included
the DRAM-based energy consumption and peak memory usage values, and presented a discussion on how energy, timeand energy relate in software, and on different languages
divided by type and paradigm.
The CLBG benchmark solutions have already been usedfor validation purpose by several research works. Amongother examples, CLGB was used to study dynamic behaviorof non-Java JVM languages [19], to analyze dynamic scripting languages [35] and compiler optimizations [33], or evento benchmark a JIT compiler for PHP [15]. At the best of ourknowledge, CLGB was only used once for energy consumption analysis. In [20], the authors used the provided Haskell
implementations, among other benchmarks, to analyze theenergy efficiency of Hakell programs from strictness andconcurrency perspectives, while also analyzing the energy influence of small implementation changes. The authors of [24]also used CLBG to compare JavaScript, Java, and C++ inan Android setting.
While several works have shown indications that a moretime efficient approach does not always lead to the mostenergy efficient solution [1, 20, 24, 26, 28, 34], these results
were not the intended focus nor main contribution, but moreof a side observation per se. We focused on trying to understand and directly answer this question of how energy
efficiency and time relate.
Nevertheless, the energy efficiency in software problem
has been growing in interest in the past few years. In fact,
studies have emerged with different goals and in different

SLE’17, October 23–24, 2017, Vancouver, BC, Canada
areas, with the common vision of understanding how development aspects affect the energy consumption in diversified software systems. For instance, for mobile applications,
there are works focused on analyzing the energy efficiency
of code blocks [5, 18], or just monitoring how energy consumption evolves over time [10]. Other studies aimed at amore extensive energy consumption analysis, by comparing
the energy efficiency of similar programs in specific usage
scenarios [4, 16], or by providing conclusions on the energy
impact of different implementation decisions. Several otherworks have shown that several factors, such as different design patterns [21, 30], coding practices [20, 25, 28, 31], anddata structures [14, 22, 23, 26], actually have a significant
influence in the software’s energy efficiency.

6

Conclusions

In this paper, we first present an analysis and comparison of
the energy efficiency of 27 well-known software languages
from the popular software repository The Computer Language Benchmarks Game. We are able to show which were themost energy efficient software languages, execution types,
and paradigms across 10 different benchmark problems.
Through also measuring the execution time and peakmemory usage, we were able to relate both to energy tounderstand not only how memory usage affects energy consumption, but also how time and energy relate. This allowed
us to understand if a faster language is always the mostenergy efficient. As we saw, this is not always the case.
Finally, as often times developers have limited resources
and may be concerned with more than one efficiency characteristic we calculated which were the best/worst languages
according to a combination of the previous three characteristics: Energy & Time, Energy & Peak Memory, Time & Peak
Memory, and Energy & Time & Peak Memory.
Our work helps contribute another stepping stone in bringing more information to developers to allow them to become
more energy-aware when programming.
Acknowledgments
We would like to thank Luís Cruz (University of Porto) for the helpthat he provided. This work is financed by the ERDF – European
Regional Development Fund through the Operational Programme
for Competitiveness and Internationalisation - COMPETE 2020
Programme and by National Funds through the Portuguese funding
agency, FCT - Fundação para a Ciência e a Tecnologia within project
POCI-01-0145-FEDER-016718. The first author is also sponsored by
FCT grant SFRH/BD/112733/2015.

References[1] Sarah Abdulsalam, Ziliang Zong, Qijun Gu, and Meikang Qiu. 2015.
Using the Greenup, Powerup, and Speedup metrics to evaluate software energy efficiency. In Proc. of the 6th Int. Green and Sustainable
Computing Conf. IEEE, 1–8.
[2] Shaiful Alam Chowdhury and Abram Hindle. 2016. GreenOracle:
estimating software energy consumption with energy measurement
corpora. In Proceedings of the 13th International Conference on Mining

SLE’17, October 23–24, 2017, Vancouver, BC, Canada
Software Repositories, MSR 2016, Austin, TX, USA, May 14-22, 2016.
49–60.
[3] Thomas D Cook and Donald T Campbell. 1979. Quasi-experimentation:
design & analysis issues for field settings. Houghton Mifflin.
[4] Marco Couto, Paulo Borba, Jácome Cunha, João P. Fernandes, Rui
Pereira, and João Saraiva. 2017. Products go Green: Worst-Case Energy
Consumption in Software Product Lines. (2017).
[5] Marco Couto, Tiago Carção, Jácome Cunha, João Paulo Fernandes,
and João Saraiva. 2014. Detecting Anomalous Energy Consumption
in Android Applications. In Programming Languages: 18th Brazilian
Symposium, SBLP 2014, Maceio, Brazil, October 2-3, 2014. Proceedings,
Fernando Magno Quintão Pereira (Ed.). 77–91.
[6] Marco Couto, Rui Pereira, Francisco Ribeiro, Rui Rua, and João Saraiva.
2017. Towards a Green Ranking for Programming Languages. In Programming Languages: 21st Brazilian Symposium, SBLP 2017, Fortaleza,
Brazil, September, 2017.
[7] K. Deb, M. Mohan, and S Mishra. 2005. Evaluating the ε -domination
based multiobjective evolutionary algorithm for a quick computationof Pareto-optimal solutions. Evolutionary Computation Journal 13, 4(2005), 501–525.
[8] K. Deb, A. Pratap, S. Agarwal, and T. Meyarivan. 2002. A Fast andElitist Multiobjective Genetic Algorithm: NSGA-II. Trans. Evol. Comp6, 2 (2002), 182–197.
[9] Martin Dimitrov, Carl Strickland, Seung-Woo Kim, Karthik Kumar, andKshitij Doshi. 2015. Intel® Power Governor. https://software.intel.com/
en-us/articles/intel-power-governor. (2015). Accessed: 2015-10-12.
[10] F. Ding, F. Xia, W. Zhang, X. Zhao, and C. Ma. 2011. Monitoring Energy
Consumption of Smartphones. In Internet of Things (iThings/CPSCom),2011 Int. Conf. on and 4th Int. Conf. on Cyber, Physical and Social Computing. 610–613.
[11] Isaac Gouy. The Computer Language Benchmarks Game. http://benchmarksgame.alioth.debian.org/[12] Marcus Hähnel, Björn Döbel, Marcus Völp, and Hermann Härtig. 2012.
Measuring energy consumption for short code paths using RAPL.
SIGMETRICS Performance Evaluation Review 40, 3 (2012), 13–17.
[13] Shuai Hao, Ding Li, William G. J. Halfond, and Ramesh Govindan. 2013.
Estimating Mobile Application Energy Consumption Using Program
Analysis. In Proc. of the 2013 Int. Conf. on Software Engineering (ICSE’13). IEEE Press, 92–101.
[14] Samir Hasan, Zachary King, Munawar Hafiz, Mohammed Sayagh,Bram Adams, and Abram Hindle. 2016. Energy profiles of java collections classes. In Proc. of the 38th Int. Conf. on Software Engineering.
ACM, 225–236.
[15] Andrei Homescu and Alex Şuhan. 2011. HappyJIT: A Tracing JIT
Compiler for PHP. SIGPLAN Not. 47, 2 (Oct. 2011), 25–36.
[16] R. Jabbarvand, A. Sadeghi, J. Garcia, S. Malek, and P. Ammann. 2015.
EcoDroid: An Approach for Energy-based Ranking of Android Apps. In
Proc. of 4th Int. Workshop on Green and Sustainable Software (GREENS’15). IEEE Press, 8–14.
[17] Ding Li and William G. J. Halfond. 2014. An Investigation Into EnergySaving Programming Practices for Android Smartphone App Development. In Proceedings of the 3rd International Workshop on Green andSustainable Software (GREENS).
[18] Ding Li, Shuai Hao, William GJ Halfond, and Ramesh Govindan. 2013.
Calculating source line level energy information for android applications. In Proc. of the 2013 Int. Symposium on Software Testing andAnalysis. ACM, 78–89.
[19] Wing Hang Li, David R. White, and Jeremy Singer. 2013. JVM-hosted
Languages: They Talk the Talk, but Do They Walk the Walk?. In Proc.
of the 2013 Int. Conf. on Principles and Practices of Programming on theJava Platform: Virtual Machines, Languages, and Tools (PPPJ ’13). ACM,
101–112.
[20] Luís Gabriel Lima, Gilberto Melfe, Francisco Soares-Neto, Paulo
Lieuthier, João Paulo Fernandes, and Fernando Castor. 2016. Haskell
in Green Land: Analyzing the Energy Behavior of a Purely Functional

R. Pereira et. al.
Language. In Proc. of the 23rd IEEE Int. Conf. on Software Analysis,
Evolution, and Reengineering (SANER’2016). IEEE, 517–528.
[21] Mario Linares-Vásquez, Gabriele Bavota, Carlos Bernal-Cárdenas,
Rocco Oliveto, Massimiliano Di Penta, and Denys Poshyvanyk. 2014.
Mining energy-greedy API usage patterns in Android apps: an empirical study. In Proc. of the 11th Working Conf. on Mining Software
Repositories. ACM, 2–11.
[22] Kenan Liu, Gustavo Pinto, and Yu David Liu. 2015. Data-oriented characterization of application-level energy optimization. In Fundamental
Approaches to Software Engineering. Springer, 316–331.
[23] Irene Manotas, Lori Pollock, and James Clause. 2014. SEEDS: A Software Engineer’s Energy-Optimization Decision Support Framework.
In Proc. of the 36th Int. Conf. on Software Engineering. ACM, 503–514.
[24] Wellington Oliveira, Renato Oliveira, and Fernando Castor. 2017. A
study on the energy consumption of Android app development approaches. In Proceedings of the 14th International Conference on Mining
Software Repositories. IEEE Press, 42–52.
[25] R. Pereira, T. Carção, M. Couto, J. Cunha, J. P. Fernandes, and J. Saraiva.
2017. Helping Programmers Improve the Energy Efficiency of Source
Code. In Proc. of the 39th Int. Conf. on Soft. Eng. Companion. ACM.
[26] Rui Pereira, Marco Couto, João Saraiva, Jácome Cunha, and João Paulo
Fernandes. 2016. The Influence of the Java Collection Framework on
Overall Energy Consumption. In Proc. of the 5th Int. Workshop on Greenand Sustainable Software (GREENS ’16). ACM, 15–21.
[27] Gustavo Pinto, Fernando Castor, and Yu David Liu. 2014. Mining
questions about software energy consumption. In Proc. of the 11th
Working Conf. on Mining Software Repositories. ACM, 22–31.
[28] Gustavo Pinto, Fernando Castor, and Yu David Liu. 2014. Understanding energy behaviors of thread management constructs. In Proc. ofthe 2014 ACM Int. Conf. on Object Oriented Programming Systems Languages & Applications. ACM, 345–360.
[29] Efraim Rotem, Alon Naveh, Avinash Ananthakrishnan, Eliezer Weissmann, and Doron Rajwan. 2012. Power-Management Architecture of
the Intel Microarchitecture Code-Named Sandy Bridge. IEEE Micro 32,
2 (2012), 20–27.
[30] Cagri Sahin, Furkan Cayci, Irene Lizeth Manotas Gutierrez, James
Clause, Fouad Kiamilev, Lori Pollock, and Kristina Winbladh. 2012.
Initial explorations on design pattern energy usage. In Green andSustainable Software (GREENS), 2012 1st Int. Workshop on. IEEE, 55–61.
[31] Cagri Sahin, Lori Pollock, and James Clause. 2014. How do code
refactorings affect energy usage?. In Proc. of the 8th ACM/IEEE Int.
Symposium on Empirical Software Engineering and Measurement. ACM,36.
[32] SS Shaphiro and MB Wilk. 1965. An analysis of variance test fornormality. Biometrika 52, 3 (1965), 591–611.
[33] Vincent St-Amour, Sam Tobin-Hochstadt, and Matthias Felleisen. 2012.
Optimization Coaching: Optimizers Learn to Communicate with Programmers. In Proc. of the ACM Int. Conf. on Object Oriented Programming Systems Languages and Applications (OOPSLA ’12). ACM, 163–
178.
[34] Anne E. Trefethen and Jeyarajan Thiyagalingam. 2013. Energy-aware
software: Challenges, opportunities and strategies. Journal of Computational Science 4, 6 (2013), 444 – 449.
[35] Kevin Williams, Jason McCandless, and David Gregg. 2010. Dynamic
Interpretation for Dynamic Scripting Languages. In Proc. of the 8th
Annual IEEE/ACM Int. Symposium on Code Generation and Optimization(CGO ’10). ACM, 278–287.
[36] Matthew
WoodruffandJon
Herman.
2013.
pareto.py:aε − nondominat ion
sortingroutine.
https://github.com/matthewjwoodruff/pareto.py. (2013).
[37] Tomofumi Yuki and Sanjay Rajopadhye. 2014. Folklore confirmed:
Compiling for speed= compiling for energy. In Languages and Compilers for Parallel Computing. Springer, 169–184.
[38] Daniel Zwillinger and Stephen Kokoska. 1999. CRC standard probability and statistics tables and formulae. Crc Press.