Comparing the Energy Efficiency of WebAssembly and
JavaScript in Web Applications on Android Mobile Devices
Max van Hasselt, Kevin Huijzendveld, Nienke Noort, Sasja de Ruijter
Tanjina Islam, Ivano Malavolta
Vrije Universiteit Amsterdam, The Netherlands{ m.p.p.van.hasselt | k.j.huijzendveld | n.noort | s.n.de.ruijter }@student.vu.nl, { t.islam | i.malavolta }@vu.nl

ABSTRACT

1Context. WebAssembly was created as an alternative to JavaScriptfor developing heavy loading web applications. Since JavaScript isknown to have long execution times. A lot of research is alreadyperformed to compare the run-time performance of WebAssembly against that of JavaScript. However, little research is available
that compares the energy consumption of WebAssembly versus
JavaScript.
Goal. With this study we aim to identify the correlation between theenergy consumption and the use of WebAssembly versus JavaScript.
This will aid developers in deciding which method matches theneeds of their project best in terms of energy efficiency.
Method. The subjects of the experiment are WebAssembly and
JavaScript. During the experiment two research questions are defined. For the first research question the programming language
is the independent variable. For the second research question the
web browser is the independent variable. For both research questions is the energy consumption of the Android device in Joules
the dependent variable.
Results. We can confirm that the energy consumption of WebAssembly is less than that of JavaScript. The browser also plays a rolesince the energy consumption of Firefox is significantly smaller
than that of Chrome for both WebAssembly and JavaScript.
Conclusions. This study provides evidence that using WebAssembly
for the development of web applications can reduce the energy
consumption and thus improve the battery life of a user’s Android
device. Developers can use this information when choosing a programming language to develop a web application. Moreover, using
Firefox over Chrome does also reduce the energy consumption of
web applications developed both with WebAssembly and JavaScript.

JavaScript is one of the most used programming language supported by the web. However, interpreted JavaScript is known to
run applications much slower than compiled Android machinecode, from hereon referred to as native code [1] [2]. Problems arise
when adopting high performance computing, like machine learningand 3D visualizations, on the web [3]. JavaScript is not equippedto perform such high performance computing functions, whichleads to long execution times. In an effort to tackle these problems,
a number of major web browser providers like Apple, Microsoft
and Mozilla, decided to collaborate and join forces. This led to thecreation of a new type of low-level code, namely WebAssembly1 .
WebAssembly is a standard that defines low-level binary code. The
standard is designed to be as small as possible and to compile efficiently. It is language independent, meaning that it supports several
programming language by using a compiler like Emscripten [4].
Various experiments have been conducted to analyse the runtime performance of WebAssembly by either comparing WebAssembly to the application’s native code, or in comparison with JavaScript [2,5–8]. Most of these papers use the execution time of benchmark
functions to measure the run-time performance. However, little is
known about the impact on energy consumption of WebAssembly.
Oliveira et al. conducted an experiment on the run-time performance of WebAssembly where the battery consumption was measured in addition to the execution time [6]. Based on the obtained
results, the authors concluded that using WebAssembly so as toimprove the run-time performance of JavaScript reduces the battery
consumption by 39%.
According to the book of Sasu Tarkoma et al., a crucial aspectof consumer satisfaction of smartphones is the battery life [9].
Software is often equipped with suboptimal functions which leads
to inefficient use of the hardware. Since there is little knowledge
about the impact of the use of WebAssembly on energy consumption, this research aims to examine the energy consumption using
WebAssembly in comparison with JavaScript.
The goal of this paper is to identify the correlation between theenergy consumption and the use of WebAssembly versus JavaScript.
This will aid developers in deciding which of these two languages
matches the needs of their project best in terms of energy efficiency.

ACM Reference Format:
Max van Hasselt, Kevin Huijzendveld, Nienke Noort, Sasja de Ruijter and Tanjina Islam, Ivano Malavolta. 2022. Comparing the Energy Efficiency of WebAssembly and JavaScript in Web Applications on Android Mobile Devices. In The International Conference on Evaluation and Assessment in Software
Engineering 2022 (EASE 2022), June 13–15, 2022, Gothenburg, Sweden. ACM,
New York, NY, USA, 10 pages. https://doi.org/10.1145/3530019.3530034

Permission to make digital or hard copies of all or part of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor profit or commercial advantage and that copies bear this notice and the full citationon the first page. Copyrights for components of this work owned by others than ACM
must be honored. Abstracting with credit is permitted. To copy otherwise, or republish,
to post on servers or to redistribute to lists, requires prior specific permission and/or afee. Request permissions from permissions@acm.org.
EASE 2022, June 13–15, 2022, Gothenburg, Sweden© 2022 Association for Computing Machinery.
ACM ISBN 978-1-4503-9613-4/22/06. . . $15.00
https://doi.org/10.1145/3530019.3530034

2

INTRODUCTION

RELATED WORK

Studies on the run-time performance of WebAssembly vsnative/Java- Script code. The motivation behind the study ofHerrera et al. [5] was the increase in power of hardware devices.
They carried out the experiments on the Ostrich benchmark set,
1 https://webassembly.github.io/spec/core/

EASE 2022, June 13–15, 2022, Gothenburg, Sweden

Max van Hasselt, Kevin Huijzendveld, Nienke Noort, Sasja de Ruijter and Tanjina Islam, Ivano Malavolta

which is a collection of numerical programs representing the numerical dwarf categories. The goal was to evaluate the current stateof portable and web-based numerical computing. In their experiment, they compared the run-time performance of WebAssembly tothe run-time performance of JavaScript. The results of their study
reveal that the run-time performance of using WebAssembly isbetter than the run-time performance of using only the numericbenchmarks. In this paper, the Ostrich benchmark set is used aswell, however, the focus lies solely on Android devices, whereas
the focus of the experiment by Herrera et al. is on various types ofdevices. Thus, mobile phones are being compared to desktops, andbrowsers are being compared as well. Another difference is thatthe paper of Herrera et al. focuses on the run-time performance differences between WebAssembly and JavaScript, whereas this study
focuses on the energy efficiency of WebAssembly and JavaScript.
The goal of Jangda et al. [2] is to analyze the run-time performance of WebAssembly compared to the native code, for which
a SPEC CPU benchmark suite is used. This benchmark suite includes several applications that fall under the intended use casesof WebAssembly. Thus, where the study of Herrera et al. [5] andthis study focuses on JavaScript, the study of Jangda et al. focuseson native code in general. Besides this, the results of the study of
Jangda et al. are solely focused on the run-time performance of WebAssembly and native code in web browsers, specifically Chrome
and Firefox. From this study follows that applications compiled to
WebAssembly run slower than applications compiled to native codein both Firefox and Chrome.
Studies on the runtime efficiency of WebAssembly. Similar to
the experiment of Herrera et al. [5], two other related works use theOstrich benchmark set. Firstly, the experiment of Oliveira et al. [6],assesses the use of WebAssembly as a strategy to improve the runtime performance of JavaScript applications in an IoT environment.
From this experiment follows that JavaScript run-time performance
could be improved in terms of execution time, memory usage, andbattery consumption is reduced when using WebAssembly. Theirmain objective is to assess if there is a performance gain and how
much it impacts battery-powered devices. Specifically, they focus
on the IoT environment. Whereas, in this paper, we analyze theimpact on the energy consumption of web applications on Android
mobile devices. Additionally, compared to their study, we analyze
how the choice of web browsers impacts the energy efficiency of
JavaScript and WebAssembly.
Secondly, the experiment of Alamari and Chow [10] follows fromthe new run-time performance capabilities in the browser. These
being the possibility to build universal applications that run on
every machine that has a web browser installed on it. They propose
a design to build web applications that take advantage of these new
run-time performance capabilities. Both of these experiments differ
from this paper, along the lines of its goal. The goal of these experiments is to improve the run-time performance of web applications,whereas the goal of the experiment in this paper is to analyze theenergy efficiency of WebAssembly and JavaScript.
An experiment done by Reiser and Bläser [7] does not makeuse of the Ostrich benchmark set, but this experiment is focusedon runtimes. Its goal is faster and more predictable runtimes for
performance-critical web code. In order to achieve this goal, Reiser

and Bläser create a cross-compiler that translates JavaScript/TypeScript to WebAssembly. From this follows a compute-intense webcode that is faster and has a reduced runtime.
Another experiment that focuses on runtimes is executed by
Lehmann and Pradel [11]. They present Wasabi, a framework whichis created to dynamically analyze WebAssembly. Wasabi is based on
binary instrumentation. This binary instrumentation inserts calls toanalysis functions that are written in JavaScript into a WebAssembly binary. This framework demands a runtime overhead that isreasonable for dynamic analysis, and it makes implementation ofvarious dynamic analyses straightforward.
The run-time performance of JavaScript. So far, most related
work that is discussed focuses on the combination of WebAssembly
and JavaScript. However, there are also studies that focus only on
the run-time performance of JavaScript.
Sandhu et al. [8] compare JavaScript to native languages by focusing on run-time performance and choice of optimal sparse matrix
storage format for sequential parse matrix-vector multiplication(SpMV). SpMv is a kernel that is considered critical regarding the
run-time performance of data-intensive applications. From this experiment follows that the best performing browser demonstrated aslowdown of only 1.2x to 3.9x, that double-precision SpMV is more
efficient than single-precision, and that the optimal storage format
choices are very different for C as compared to JavaScript, andeven quite different between the two browsers. The main difference
between this experiment and the experiment of this paper, is theuse of SpMv to compare the run-time performance of JavaScriptwith the run-time performance of native languages.
An experiment done by Miettinen and Nurminen [12] analyzes
the energy consumption of JavaScript based web applications onmobile phones. From this experiment follows that choosing a proper
library can save up energy when implementing the same functionality. Thus, there are relevant differences when implementing a
JavaScript based application when focusing on energy efficiency.
Where the experiment of Miettinen and Nurminen focuses solely
on the energy efficiency of JavaScript, the experiment of this paper
analyzes the differences in the energy efficiency of both JavaScript
and WebAssembly.
Energy consumption of Google Chrome vs Mozilla Firefox.
Macedo et al. [13] present a study on the energy consumption of
two popular browsers: Google Chrome and Mozilla Firefox. Theirgoal was to measure the energy consumption of the browsers tounderstand which browser is the most appropriate to be used if
energy consumption is of concern to the user. In particular, theymeasure and analyze the energy consumed by the DRAM and CPU
of Chrome and Firefox on different web applications. In general,they conclude that Chrome is a more energy-efficient browser thanFirefox, but Firefox is more consistent in terms of energy efficiency.

3

STUDY DESIGN

The experiment is carried out according to known empirical software engineering guidelines [14–16] . Below we present the designof the experiment. For further details, a complete replication package is available2 . The replication package also allows independent
researchers to verify and replicate the study.
2 https://github.com/S2-group/EASE-2022-energy-web-assembly-rep-pkg

Comparing the Energy Efficiency of WebAssembly and JavaScript in Web Applications on Android Mobile Devices

3.1

Goal and Research Questions

The goal of this study is to analyze the impact of using WebAssemblyover JavaScript on the energy efficiency of Android browser processes.
Our goal is refined into the following two research questions:[RQ1]: How does the use of WebAssembly over JavaScript impact
the energy efficiency of web applications on Android mobile devices?
By answering this research question, software developers will
be given an empirically substantiated perspective on the impact ofcompiling to WebAssembly, as opposed to JavaScript, on the energy
efficiency of their web applications.
[RQ2]: To what extent does the choice of browser influence theenergy efficiency of JavaScript and WebAssembly applications? Different browsers use different JavaScript engines [17][18] to compile
both JavaScript and WebAssembly. These different engines could
produce different results. As such, this study will factor in the typeof browser used during the experiment. The answer to this question
may help end-users who want to reduce their energy consumption
in choosing a mobile web browser.

EASE 2022, June 13–15, 2022, Gothenburg, Sweden

The dependent variable for both research questions is the energy consumption (𝐸) of the Android device in Joules. Energy consumption is computed as follows:
𝑃
𝑇
𝑊 ×
𝑠,
(1)
1000
106
where 𝑃 is the average power consumed by the Android device inmicroWatts, and 𝑇 is the time in milliseconds that each run takes.
𝐸=

3.4

Experimental Hypotheses

In order to answer the two research questions of our study, we
have formulated the following hypotheses. Given that 𝜇 𝐽 𝑎𝑣𝑎𝑆𝑐𝑟𝑖𝑝𝑡
is the average energy consumption of web applications when using
JavaScript and 𝜇𝑊 𝑒𝑏𝐴𝑠𝑠𝑒𝑚𝑏𝑙 𝑦 is the average energy consumptionof web applications when using WebAssembly, then the null andalternate hypotheses for the programming language (𝑝𝑙) in RQ1
are defined as:𝑝𝑙

𝐻 0 : 𝜇 𝐽 𝑎𝑣𝑎𝑆𝑐𝑟𝑖𝑝𝑡 = 𝜇𝑊 𝑒𝑏𝐴𝑠𝑠𝑒𝑚𝑏𝑙 𝑦𝑝𝑙

3.2

Subjects Selection

In our experiment, we consider benchmarking algorithms as the
subject. The benchmark chosen for this experiment is the Ostrich
benchmark [19]. This benchmark is chosen mainly because it isspecifically designed to study the run-time performance of programming languages for numerical code. Each of the benchmark functions contain an equivalent implementation of both JavaScript andC. The Ostrich benchmark is also used in previous studies [5, 6, 10].
Using the compiler Emscripten, the Ostrich benchmark can be compiled from C to the equivalent implementation of WebAssembly.
This will allow us to compare JavaScript and WebAssembly in termsof their energy consumption.
The Ostrich benchmark consists of 12 numerical benchmark
functions. Since four of the functions in this benchmark were crashing when run multiple times in succession, this study uses only 8of these algorithms. Table 1 contains the 8 used benchmark functions together with a description as stated by the developers of theOstrich benchmark[19]. It also shows the number of times eachfunction will be executed during a single run. For each function,
the algorithm has a run time of approximately one minute. Thus,
the amount of calls depends on the speed of the function. The
benchmark functions used in this experiment are written both in
JavaScript and C. The C implementation is compiled to WebAssembly.

3.3

Experimental Variables

This study has two research questions that both introduce one
independent variable. For RQ1, the independent variable is the
programming language. This independent variable has two treatments, namely the raw JavaScript code and the C code compiled
to WebAssembly. Compiled means WebAssembly binary generated
after the compilation step. For RQ2, the independent variable
is the web browser in which the trials will run. This independent
variable has the two treatments Chrome and Firefox.

𝐻𝑎 : 𝜇 𝐽 𝑎𝑣𝑎𝑆𝑐𝑟𝑖𝑝𝑡 ≠ 𝜇𝑊 𝑒𝑏𝐴𝑠𝑠𝑒𝑚𝑏𝑙 𝑦
The null hypothesis states that the average energy consumptionof web applications when using JavaScript or WebAssembly is thesame. Intuitively, the alternative hypothesis states that there is asignificant difference in average energy consumption when using
JavaScript versus WebAssembly.
Similarly, given that 𝛽𝑐ℎ𝑟𝑜𝑚𝑒 is the average energy consumptionof web applications launched in Google Chrome and 𝛽 𝑓 𝑖𝑟𝑒 𝑓 𝑜𝑥 isthe average energy consumption of web applications launched in
Mozilla Firefox, then the null and alternate hypotheses for thebrowser (𝑏) in RQ2 are defined as:
𝐻 0𝑏 : 𝛽𝑐ℎ𝑟𝑜𝑚𝑒 = 𝛽 𝑓 𝑖𝑟𝑒 𝑓 𝑜𝑥
𝐻𝑎𝑏 : 𝛽𝑐ℎ𝑟𝑜𝑚𝑒 ≠ 𝛽 𝑓 𝑖𝑟𝑒 𝑓 𝑜𝑥
Intuitively, the null hypothesis states that there is no significant
difference between the average energy consumption of web applications when using either Google Chrome or Mozilla Firefox
browser. Subsequently, the alternative hypothesis states that there
is a significant difference in average energy consumption whenusing a different web browser.
The following two hypotheses are defined for the interactionof browser and programming language, where 𝜇𝑖 is the effect oftreatment i (JavaScript/WebAssembly) of the programming language factor from RQ1 and 𝛽 𝑗 is the effect of treatment j (Google
Chrome/Mozilla Firefox) of Web browser factor from RQ2. Thenthe null and alternate hypotheses are defined as:
𝑝𝑙,𝑏

𝐻0

: (𝜇𝛽)𝑖 𝑗 = 0 ∀𝑖, 𝑗𝑝𝑙,𝑏: ∃(𝑖, 𝑗) | (𝜇𝛽)𝑖 𝑗 ≠ 0

𝐻𝑎The null hypothesis states that for each pair of treatments (i.e.,
programming language and web browser), there is no significant
difference between the average energy consumption of web applications. However, the alternative hypothesis states that the averageenergy consumption of at least one pair of treatments is significantly less than others.

EASE 2022, June 13–15, 2022, Gothenburg, Sweden

Max van Hasselt, Kevin Huijzendveld, Nienke Noort, Sasja de Ruijter and Tanjina Islam, Ivano Malavolta

Table 1: Ostrich benchmark functions
Number1

Benchmarkfft

Category
Spectral methods2hmm

Graphical models

3
4

lavamdlud

N-body methods
Dense linear algebra

5nqueens

Branch and bound6

nw

Dynamic programming

7

page-rank

Map reduce

8

spmv

Sparse linear algebra

3.5

Description
Application of the Fast Fourier Transform function
A forward-backward algorithm which looks for the unknown
parameters of a hidden Markov modelCalculation of particle potential and relocation within a big 3D space
Application of a lower-uper decomposition on a 1024 x 1024 matrix
An algorithm that computes the number of different
possibilities where there are n queens put on a n x n chess board
and the queens are not attacking eachother
Computation of the optimal alignment of two protein sequences
An algorithm that ranks websites according to their importance, also
used by Google SearchMultiplication of a sparse matrix with a vector

Experiment Design

Based on the subject selection, the variables, and the hypotheses
constructed for this experiment, the design type will be two factors
and two treatments (2F-2T). These two factors are the programming
language from RQ1 and the web browser from RQ2. Each factor
has two treatments. Firstly, the two treatments of the programming
language factor are JavaScript and WebAssembly. Secondly, thetwo treatments of the web browser factor are Chrome and Firefox.
Since there are two factors in this design type, it has to be taken
into account that these two factors might interact with each other.
Thus, not only the effect of the treatments is being modeled, theeffect of the interaction between these is being modeled as well. In
the experimental hypotheses section, this effect is included by thelast hypothesis.
The factorial design considers all possible combinations of treatments, in which we use the same number of benchmark functions
for all cases.
For answering RQ1, the 8 benchmark functions will be executed
once in JavaScript, and once in C++, which will be compiled to
WebAssembly. We used the Ostrich benchmark in our experiment,
where the benchmark functions contain an equivalent implementation of both JavaScript and C. That is why we considered C/C++
over other programming languages. It is important to note that, toanswer RQ1, the benchmark functions will be executed on Google
Chrome for Android, since it is currently the most popular browser
on the Android operating system [20].
For answering RQ2, all subjects are executed for each possiblecombination of browser and programming language. About thebrowsers, Google Chrome and Firefox browsers are chosen since
they follow different approaches to compile towards WebAssembly. Specifically, Chrome’s V8 compiles to machine code [17], andFirefox’s SpiderMonkey compiles to intermediate bytecode [21].
Finally, each trial of our experiment is repeated 30 times, so foreach pair of programming language and web browser. During eachrun, the total energy consumption (Joules) of the Android device
is measured. These results are then used to calculate the energy
efficiency of each algorithm in each web browser. These repetitions
per trial are done to take into account the intrinsic variability ofthe energy measurement process [16].

3.6

Runs
10
3
7
6
3
10
4
15

Data Analysis

To answer RQ1 and RQ2, the collected measures are analyzed infour phases: data exploration, check for normality and transformations, hypothesis testing, and effect size estimation. In the data
exploration phase, we get a first indication of the obtained energy
consumption values via a combination of descriptive statistics, histograms, and box plots. Next, we analyze the distribution of thedata. Specifically, we check if the energy consumption measures arenormally distributed via (i) visually inspecting their Q-Q plots and(ii) applying the Shapiro-Wilk normality test. Since this experimenthas more than one factor, it is greatly desired that these values arenormally distributed. This is because an ANOVA test is possible
when there is a normal distribution, but this is not the case when thedata is not normally distributed. Since there is no non-parametriccounterpart for a two-factor design, an ANOVA test cannot be used
when there is no normal distribution of data [14]. Here, ANOVA
is an example of a parametric test. In non-parametric tests, only
very general assumptions are made and the data does not haveto be normally distributed. If data is normally distributed, we will
apply a single statistical test to test the statistical hypothesis of bothresearch questions. As mentioned before, this test will be two-way
ANOVA. The threshold for 𝛼 will be set to 0.05. If the data is notnormally distributed, the measurement data will be transformed.
As was mentioned by Stevens in his book "Applied Multivariate
Statistics for the Social Sciences", the impact on the validity of nonnormal data is small enough for an ANOVA analysis so that thestatistical test still produces realistic results [22]. The significancelevel will again be 𝛼 = 0.05. If this transformation does not lead tonormality, logistic regression is used. Finally, to statistically assess
the magnitude of the differences between the energy consumptionof the treatments of both factors, the Cliff’s Delta measure will be
used [23]. This is a non-parametric effect size measure that will be
used complementary to the hypothesis testing. Where the ANOVA
test will determine whether there is a statistical significance, theCliff’s Delta measure will determine whether there is a practicalsignificance in the differences between the treatment pairs [24]. The
Cliff’s Delta statistics will be interpreted as proposed by Grissom
et al. [25].

Comparing the Energy Efficiency of WebAssembly and JavaScript in Web Applications on Android Mobile Devices

4

EXPERIMENT EXECUTION

In this section, the technical setup used for executing the experiment will be described. This includes the different devices used, theinfrastructure that connects these devices, and the software tools
used to run the experiment.
Figure 1 shows a schematic overview of the infrastructure of
our experiment. The infrastructure of the experiment consists of
two devices: (i) an Android device on which the subjects of theexperiments will be executing, and (ii) a Raspberry Pi, which will
be used to orchestrate the experiment and is responsible for collecting experimental data from the Android device. The Raspberry
Pi is convenient for this experiment because the USB ports can be
programmatically disabled, ensuring that the phone is not charging during the experiment. The trials are run on a Huawei Nexus
6P phone. The energy consumption of the Android device is measured using the Trepn plugin3 , a software-based power profiler
for Android devices. Trepn is widely used in empirical researchon energy-efficient software [26–28], and it has been tested to be
sufficiently accurate like the hardware-based power monitors (e.g.,
the Monsoon Power Monitor4 ), with an error margin of 99% [29].
Due to version limitations of the Trepn framework, the phone will
run Android 6. The system specs for the Nexus 6P can be seen in
Table 2.
Raspberry Pi

Android device
HTML pages

Pythonscripts
Pythonscripts
Python scripts

JavaScript
4. starts custom
python scripts
7. stops profiler
and close browser
Android Runner

3a. starts one
browser application
with url

6. saves colllected
data

5. opens subjects
Applications
Browser applications
Firefox

2. starts profiler

ChromeTrepn
Profiler3b. starts
collecting data

Figure 1: Overview of the experiment execution
The browsers that are used to execute the subjects are Firefox Nightly version 95.0a1 and Google Chrome 94.0. Before every run in Firefox, due to the duration of the runs, the setting"dom.max_script_run_time" is set to 0, to allow scripts to run indefinitely. Firefox sets this setting to 20 seconds by default. Sucha configuration is not necessary for Chrome, which allows longrunning scripts by default.
We take special care in keeping the execution environment asclean as possible. Specifically, the application data of the browser
will be cleared, so that no caching will influence the power measurements and the run time of the run. Furthermore, to avoid charging
the Android device during execution, before every run, the Raspberry Pi USB port to which the Android device is connected is
3 https://github.com/S2-group/android-runner/tree/master/AndroidRunner/Plugins/

trepn
4 https://www.msoon.com/

OS
ChipsetCPU
GPU
Storage
RAM
ScreenWLAN
USB
Battery

Android 6 (Marshmallow)
Qualcomm MSM8994 Snapdragon 810
Octa-core(4x1.55 GHz Cortex-A53 & 4x2.0 GHz Cortex-A57)
Adreno 430
32 GB eMMC 5.0
3GB
5.7 inch OLED QHD, (2560 x 1440)
Wi-Fi 802.11 a/b/g/n/ac, dual-band, Wi-Fi Direct
Type-C 2.0
LiPo 3450 mAh, Fast charging 15W
Table 3: Raspberry Pi 4B specs

1. stops all
applications
local
storagedisabled. After each run, it is re-enabled to allow for charging between runs. Android-runner provides functionality to achieve this.
For the orchestration of the experiment, a Raspberry Pi 4B will
be used, which is a convenient, low-power micro-computer withdecent specifications and which runs Linux. This device will control the phone and initiate the different trials. Furthermore, theRaspberry Pi will record the Trepn logs of the power consumptionof the phone during the trials. The technical specification of the
Raspberry Pi can be found in Table 3.
To control the experiment, the Raspberry Pi will use Android
Runner [30], a Python framework used to execute experiments on
Android devices. The experiment will be defined using a JSON filedescribing the steps of the experiment. Then, the entire execution of
the experiment is managed by Android Runner with a combinationof Python scripts and Android Debug Bridge (ADB) commands. The
Raspberry Pi will instruct the Nexus 6P to execute the browsers
and open the HTML files containing the subjects. To eliminate
the influence that a fluctuating internet connection may have, theHTML files will be served by the Raspberry Pi over WiFi to the
Nexus 6P.
Table 2: Huawei Nexus 6P specs

HTML pages
WebAssembly

EASE 2022, June 13–15, 2022, Gothenburg, Sweden

OS
CPU
RAM
USB

Raspian GNU/Linux 10 (Buster)
Broadcom BCM2711, Quad core Cortex-A72 (ARM v8)
8GB
USB 3

The timeline of the experiment will be as follows. Firstly, allrunning applications on the Nexus 6P will be stopped, represented
by 1 in Figure 1. This way the influences of other processes on theenergy consumption of the device will be minimized. Secondly, theprofiler will be started, 2 in Figure 1. Third, the browser will be
opened and then instructed to, one by one, open the HTML documents which directs the browser to load the subject with specifictreatment (JavaScript or WebAssembly). Each treatment will be
executed 30 times and each run will last for about 1 minute. Aftereach execution, the execution time and power consumption duringexecution will be measured and stored. The order of the executionof the trials will be randomized to counteract the possible variability of the energy measurements. After each trial execution, thebrowser cache will be cleared. Then step three will be repeated forthe other browser. Finally, the data of all runs will be aggregated,
and the power measurements over time will be converted to Joules.

Max van Hasselt, Kevin Huijzendveld, Nienke Noort, Sasja de Ruijter and Tanjina Islam, Ivano Malavolta

5.1.1 Data Exploration. An overview of the amount of energy thatis consumed (in Joules) when running the experiment is given
in Table 4, and Figure 2. From Table 4 follows a clear averageenergy consumption throughout the experiment of 56,847 Joules
for both programming languages. When looking at JavaScript and
WebAssembly separately, a large difference was found between
the average energy consumption of both languages. Namely, theaverage energy consumption of JavaScript is 81,785 Joules. Thisaverage is larger than 31.91 Joules, which is the average energy
consumption of WebAssembly.

JavaScript and WebAssembly is given in Figure 3. Based on these
histograms, we can infer that the data for both treatments are notnormally distributed.
Distribution JavaScript

Distribution WebAssembly

30

30

20

Frequency

5 RESULTS
5.1 Impact on energy efficiency (RQ1)

Frequency

EASE 2022, June 13–15, 2022, Gothenburg, Sweden

10

20

10

Table 4: Descriptive statistics for RQ1
0
50

100

150

0

Energy consumption (J)

20

40

60

Energy consumption (J)Figure 3: Frequency of energy consumption of RQ1

5.1.2 Check for normality. In order to check the normality ofthe data, a histogram for the distribution of the consumption of150

To confirm this presumption, the Shapiro-Wilks test is executedon the data. From this follows W = 0.91868 and p-value = 2.39e-09 for
JavaScript, and W = 0.95126, p-value = 1.476e-06 for WebAssembly.
Thus, since the p-value of both JavaScript and WebAssembly is
below the significance threshold of 0.05, the data is not normally
distributed. In order to be able to execute an ANOVA test, the datais transformed to normalized data. This is done by using orderNorm
for both JavaScript and WebAssembly, which results in the Q-Qplots shown in Figure 4.
Normality WebAssembly
4

2

2

0

−2

Language0−2−4−3 −2 −1 0

100

Normality JavaScript

4

SampleWhat is noticed as well when looking at Table 4, is the maximum energy consumption of JavaScript and WebAssembly. Thismaximum is almost 100 Joules bigger for JavaScript compared to
WebAssembly.
To get a better understanding of the data, two boxplots are created. Figure 2 illustrates a boxplot of the energy consumption per
language. From the diagram, we get an overview of the energy
consumption for each benchmark function per language type. Fromthe boxplot, we can infer that there is a large difference in energy
consumption values for JavaScript and WebAssembly.

Energy consumption (J)00

SampleMinimum
1st quartile
MedianMean
3rd quartile
Maximum

Energy Consumption (Joules)
Both
JavaScript WebAssembly
3.251
3.251
4.039
24.99635.91222.22642.07372.30227.91356.84781.78531.91072.280130.752
46.031
164.219164.21968.531

1

2

Normality Quantiles

3−4−3 −2 −1 0

1

2

3

Normality Quantiles

JavaScript
WebAssembly

Figure 4: Q-Q-plot of JavaScript and WebAssembly
By looking at these Q-Q-plots, the data seems normally distributed. To confirm this, the Shapiro-Wilks test is executed again.
This leads to a value of W = 0.99978 and p-value = 1 for both
JavaScript and WebAssembly. Thus, this data is normally distributed.

50

0
fft

hmm

lavamdludnqueensnw

page−rank

Benchmark functions

Figure 2: Energy consumption of JavaScript (red) comparedto WebAssembly (green)

5.1.3 Hypothesis Testing. Since the transformed data is normally
distributed and we will only focus on the treatment language, a oneway ANOVA test is executed. Due to the influence of the differencein the output of the benchmarks on the data, a within-subjects factor
is added as well. By adding the benchmarks as a within-subjects

Comparing the Energy Efficiency of WebAssembly and JavaScript in Web Applications on Android Mobile Devices

factor, it is possible to compare these benchmarks under different
conditions. Thus, a one-way repeated measure is executed. Herethe different conditions are the languages used. As can be seen in
Table 5, the p-value is below 0.05 which means that we can reject
𝑅𝑄1
the null hypothesis 𝐻 0 . Therefore it can be said that the use of
JavaScript or WebAssembly has a different impact on the energy
consumption of web applications on Android mobile devices.

EASE 2022, June 13–15, 2022, Gothenburg, Sweden

than the mean of Chrome and JavaScript. Likewise, the minimum
energy consumption of Firefox and JavaScript is larger than theminimum energy consumption of Chrome and JavaScript. This
probably means that the data points that are measured for Firefox
and JavaScript have more outliers than the data points measured
for Chrome and JavaScript.

Table 5: One-way ANOVA for RQ1
Sum Sq261197580563

Mean Sq
2611971389

F value
188.06p-value< 2.2e-16

5.1.4 Effect Size Estimation. By calculating the effect size estimation using Cliff’s Delta and a density plot, the strength of the differences between the treatment pairs will be estimated. Looking at thedensity plot in Figure 5, it is expected that effect size is big considering the height and spread of both curves. The value of Cliff’s delta
estimation is large (0.5907937). This means that there is a strong
difference between the use of JavaScript versus WebAssembly in
Chrome.

150

Energy consumption (J)Language
Residuals

Df
1
418

100

Language
JavaScript
WebAssembly

500ChromeChromeFirefoxFirefoxFigure 6: Boxplot of the energy consumption of JavaScript,
WebAssembly, Chrome and Firefox

Density

0.02

Language
JavaScript
WebAssembly
0.01

0.00
0

50

100

150

Energy consumption (J)Figure 5: Energy consumption density curve of JavaScript
compared to WebAssembly

5.2

Influence of the choice of browser (RQ2)

5.2.1 Data Exploration. In Table 6, and Figure 6 we give an overview
of the energy consumption of each run of the experiment for thebrowsers Chrome and Firefox. Similar to Table 4, Table 6 shows thatthe average energy consumption of JavaScript is a lot larger than
the average energy consumption of WebAssembly. It also showsthat the difference in the average energy consumption between
Chrome and Firefox is not very large. However, Table 6 shows that
Chrome has a larger average energy consumption than Firefox.
Another data point that stands out, is the maximum value ofFirefox combined with JavaScript. This maximum value is larger
than the maximum value of Chrome combined with JavaScript,
even though the mean of Firefox and JavaScript is a lot smaller

The boxplot in Figure 6 shows the energy consumption of JavaScript
and WebAssembly combined with Chrome and Firefox. From thediagram, we get an overview of the energy consumption per language to the type of browser used. Again, the big difference in theenergy consumption between JavaScript and WebAssembly immediately stands out. However, when looking at the different typesof browsers, the difference in energy consumption is smaller. But,
as discussed earlier in this paragraph, this boxplot clearly showsthat this difference does exist between the energy consumption of
Chrome and Firefox. Namely, Chrome consumes more energy thanFirefox. Since the mean of each treatment is larger than the median,it follows that the data is positively skewed. Thus, the data is most
likely not normally distributed.
5.2.2 Check for normality. For the normality check of the data, ahistogram for the treatments is created to give a first impression.
These histograms are shown in Figure 7, and it shows the distribution of the consumption of JavaScript and WebAssembly combinedwith Chrome and Firefox. When solely looking at these histograms,
it looks like none of these data sets are normally distributed.
However, it cannot be assumed that data is not normally distributed by just looking at a histogram of the data. Thus, the ShapiroWilks test is executed as well. The value of W and the p-value of
this test is given in Table 7 for each of the treatments. From thisfollows that the data is indeed not normally distributed, since thep-value is remarkably smaller than 1 for each language-browser
combination.
Since the data has to be normally distributed in order to executea two-way ANOVA test, the data is transformed. This is done by
applying orderNorm to all of the data. The transformed data isshown in the Q-Q-plots of Figure 8.

EASE 2022, June 13–15, 2022, Gothenburg, Sweden

Max van Hasselt, Kevin Huijzendveld, Nienke Noort, Sasja de Ruijter and Tanjina Islam, Ivano Malavolta

Table 6: Descriptive statistics for RQ2

Chrome and JavaScript
3.251
35.91272.30281.785130.752
164.219

Minimum
1st quartile
MedianMean
3rd quartile
Maximum

Chrome and JavaScript

Chrome and WebAssembly

30
20
10

20
10

0

5.2.3 Hypothesis Testing. Assuming the data is normally distributed
after transformation, a two-way ANOVA test is executed to calculate the contribution of both the different languages and browsers.
In Table 8 the results are presented. It can be seen that both thelanguage and the browser affect on the energy consumption of the
web applications since they have a p-value of respectively < 2.2e-16
and 9.355e-07. However, the two treatments are not significantlyinteracting with each other considering the p-value of 0.8782.

0
0

50

100

150

0

Energy consumption (J)

20

40

60

Energy consumption (J)Firefox and JavaScript

Firefox and WebAssembly
Frequency

Frequency

30
20
10

15
10
5

0

0
0

50

100

150

0

Energy consumption (J)

10

20

30

40

Table 8: Two-way ANOVA for RQ2

Energy consumption (J)Figure 7: Distribution of energy consumption of RQ2
4
2
0
−2
−4
−3 −2 −1 0

1

2

3

4
2
0
−2
−4
−3 −2 −1 0

Normality Quantiles

2

3

Firefox & WebAssembly
Sample11Normality Quantiles

Firefox & JavaScript

4
2
0
−2
−4
−3 −2 −1 0

Language
Browser
Language:
Browser
Residuals

Chrome & WebAssembly
SampleSampleChrome & JavaScript

SampleFirefox and WebAssembly
2.755
10.57017.66919.01726.68941.812

From this follows the value of W = 0.99978 and a p-value of 1
for all of the treatments. Thus, the transformed data is normally
distributed.

30

Frequency

Frequency

Energy Consumption (Joules)
Chrome and WebAssembly Firefox and JavaScript
4.039
4.226
22.22623.94027.91352.46631.91069.66946.031115.576
68.531
170.553

2

3

4
2
0
−2
−4
−3 −2 −1 0

Normality Quantiles

1

2

3

Normality Quantiles

Figure 8: Q-Q-plot of JavaScript and WebAssembly combined with Chrome or Firefox
Table 7: Shapiro-Wilks test for RQ2

Chrome and JavaScript
Chrome and WebAssembly
Firefox and JavaScript
Firefox and WebAssemblyW
0.91868
0.95126
0.91861
0.95285

p-value
2.39e-09
1.476e-06
2.36e-09
2.138e-06

These Q-Q-plots look like they show normalized data, but a
Shapiro-Wilks test is executed again to confirm this presumption.

Df
1
1

Sum Sq53055432834

Mean Sq
53055432834

F value
394.6031
24.4207p-value<2.2e-16
9.355e-07

1

32

32

0.02350.8782836

11240251345

5.2.4 Effect Size Estimation. In Table 9 the effect size measures
are shown of all the possible pairs of the two treatments. For threeout of the six treatment pairs, the effect size is large. This meansthat according to Cliff’s Delta estimation, the difference between
the treatments of these pairs is noticeable strong. In one of thetreatment pairs, Firefox JavaScript versus Chrome JavaScript, theeffect size is negligibly weak. Looking at the density plot presented
in Figure 9, it is also shown that the differences between ’Firefox and
JavaScript’ versus ’Chrome and JavaScript’ are smaller comparedto the other treatment pairs. Whereas the other treatment pairs aredifferent in height, spread, or both.
Table 9: Effect Size Estimation for RQ2
Treatment pair
Chrome JS - Chrome WASM
Firefox JS - Firefox WASM
Firefox JS - Chrome JS
Firefox WASM - Chrome WASM
Firefox WASM - Chrome JS
Firefox JS - Chrome WASM

Cliff’s Delta estimate
0.5907937 (large)
0.6470295 (large)
0.1351927 (negligible)
0.4539229 (medium)
0.7060317 (large)-0.4078912 (medium)Comparing the Energy Efficiency of WebAssembly and JavaScript in Web Applications on Android Mobile Devices

Density

0.03

Group

0.02

Chrome and Javascript
Chrome and WebAssembly
Firefox and Javascript
Firefox and WebAssembly
0.01

0.00
0

50

100

150

Energy consumption (J)Figure 9: Energy consumption density curves

6

DISCUSSION

The results of the application of the one-way ANOVA test for RQ1
𝑅𝑄1
allow us to reject the 𝐻 0 null hypothesis. This means that the useof WebAssembly or WebAssembly for web applications has a significantly impact on the energy consumption with respect to JavaScript
on Android mobile devices. Besides a one-way ANOVA test, theCliff’s Delta effect size measure was applied to the data as well. Thismeasure shows a large effect size between the use of JavaScript
versus WebAssembly. Thus, generally, it can be concluded thatthe difference in energy consumption between WebAssembly and
JavaScript is statistically confirmed and it is large.
The results of this paper give developers an indication of theenergy consumption of JavaScript and WebAssembly. This can beused by developers when they have to decide between these two
types of languages for creating a web application.
For what concerns RQ2, the two-way ANOVA test reveals thatboth the language and the browser significantly affect the energy
consumption of web applications. From this test follows as well thatthe language type and the browser type are not significantly interacting with each other. After an ANOVA test, the Cliff’s Delta effectsize measure was applied to the data. This effect size is large forthe pairs Chrome JavaScript versus Chrome WebAssembly, Firefox
JavaScript versus Firefox WebAssembly, and Firefox WebAssembly versus Chrome JavaScript. There is a medium effect size forthe pairs Firefox WebAssembly versus Chrome WebAssembly, andFirefox JavaScript versus Chrome WebAssembly. The effect size isnegligible for Firefox JavaScript versus Chrome JavaScript. Basedon these statistical tests, it can generally be stated that both the language and the browser have an impact on the energy consumptionof web applications.
When looking at the results from the perspective of web developers, the focus is mostly on the fact that WebAssembly consumes
consistently less energy compared to JavaScript. For a web developer, this means that if they have JavaScript functions implemented
which are constantly running in the background (e.g., a poller for
fetching remote data, a thread for real-time image manipulation,etc.), it might be convenient to implement these functions in C and

EASE 2022, June 13–15, 2022, Gothenburg, Sweden

then to compile them to WebAssembly. This will make their web
applications more energy efficient with respect to using a plain
JavaScript implementation.
Based on our empirical results, Firefox tends to consume less energy. This result can be useful for users in some specific conditions.
For example, in the case that their device only has 1% of energy
left, and they want to keep working on their device for the largest
amount of time possible. The mean energy consumption of WebAssembly on the Chrome browser is 31.91 Joules, and on Firefox
is 19.02 Joules, which is 12.89 joules less than Chrome. This leads
to a 40.39% energy saving on average when switching to Firefoxfrom Chrome. To put this into perspective, if 5 million users switch
to Firefox from Chrome, then a total of 2 million Joules of energy
will be saved. This amount of energy is equivalent to the energy
consumed by a 100W bulb for 6 hours.
For browser vendors, it is important to notice that Mozilla
Firefox is an open-source project. Thus, browser vendors can lookat its engine for executing JavaScript and WebAssembly. From this,they can take inspiration in order to make their execution engines
more energy efficient.

7

THREATS TO VALIDITY

This section will provide an analysis of possible threats to validityof the study. The threats are categorized into four different types, as
defined by Cook and Campbell: internal validity, external validity,
construct validity, and conclusion validity [31].
Internal Validity. System processes may impact the energy consumption of the Android device. To mitigate this, the order of
execution of the subjects was randomized, and before every run,the application data of the to be used browser is cleared. As such,the risk of system processes impacting the results of a subject isminimized. It could also be possible that the consumption of theAndroid device is influenced by the condition of the network overwhich the subjects are served, for example when the download
time is increased due to fluctuations in the WiFi connection. To
mitigate this, a dedicated router is used which only connects the
Raspberry Pi and the Android device. Furthermore, the Android
device is placed at a constant distance from the router and the
Raspberry Pi is connected with an ethernet cable. A replication
package is available for independent verification of the setup andthe generated data.
External Validity. The most relevant threat to the external validityof this study is the fact that the subjects are not real-life applications,but benchmarking algorithms designed to be compute-intensive.
This means that the results of this study can be misleading whencompared to the situation in the industry, where mobile web applications typically do not do a minute of intense computation on theclient-side. However, the benchmark that was used in this study,
was used in multiple other studies [5] [6] [10]. A future study
might compare close to real-life web applications implemented
in JavaScript and WebAssembly. We used a Huawei Nexus 6P to
run the trials on. Therefore, the version of Android (Android 6,
Marshmallow) used may not represent the current state of Android(Android 12, Snow Cone). We choose to run the phone on Android 6
due to the compatibility issues with Trepn. Nevertheless, newer devices running newer Android releases may lead to different energy

EASE 2022, June 13–15, 2022, Gothenburg, Sweden

Max van Hasselt, Kevin Huijzendveld, Nienke Noort, Sasja de Ruijter and Tanjina Islam, Ivano Malavolta

measurements; further replications of the performed experiments
can help in mitigating this potential threat to validity.
Construct Validity. To avoid inadequate preoperational explication of constructs, we defined our constructs a priori, before theexperiment execution. All the details related to the design of theexperiment (e.g., the goal, research questions, variables, data analysis procedures) was defined before executing the experiment. We
used the GQM approach to define our goal, which then guided
the definition of the research questions of this study. The hypotheses, dependent and independent variables, and treatments were all
defined during the planning phase of the experiment.
Conclusion Validity. To mitigate this type of threat, we used a
fixed number of treatments for both the research questions. Depending on the research question those treatments changed. We
made use of 8 subjects so that means in total 32 web applications.
We then executed 30 runs per web application and as a result, wehave a relatively large total sample size of 960. The procedures
and tests in our statistical analysis were defined a priori. During
the statistical analysis, it was checked whether the assumptions of
the applied statistical tests were met. This check validated that theright statistical tests was used on the data. The replication package
contains all the raw data and the analysis scripts, this can be used
for independent inspection.

8

CONCLUSIONSIn this paper, the impact on the energy consumption of web applications is measured between web browsers and programming
languages. In our experiment, we used eight different benchmark
functions to measure the energy consumption across the two treatments. The results of this experiment show that there is a significant
difference in energy consumption between the use of WebAssembly
and JavaScript. In addition, the choice of browser also contributed
to the differences in energy consumption. Therefore to improvethe energy efficiency of web applications, we recommend softwaredevelopers to compile web applications to WebAssembly whenpossible. In the same sense, we suggest user to use the browser
Firefox over Chrome, when energy is an issue. Combining these
two factors will lead to the optimization of energy consumption
when using web applications.
As future work, this experiment can be extended by using reallife web applications instead of benchmark functions. Such extension can make the results more representative wrt real-life situations. Additionally, we are interested in running the same experiment on a different device, for example, a smartphone with an iOS
operating system. Lastly, the experiment can be repeated on different browsers. Since the use of web browsers Firefox and Chrome
have an impact on energy consumption, it will be interesting tocompare other web browsers as well. In the future, it will be interesting to perform a trade-off analysis between energy consumption
and the run-time performance of WebAssembly.

REFERENCES[1] A. Rossberg, “Webassembly: high speed at low cost for everyone,” in ML16:
Proceedings of the 2016 ACM SIGPLAN Workshop on ML, 2016.
[2] A. Jangda, B. Powers, E. D. Berger, and A. Guha, “Not so fast: Analyzing theperformance of webassembly vs native code,” Proceedings of the 2019 USENIX
Annual Technical Conference, pp. 107–120, 2019.
[3] D. Frankl, “Machine learning in javascript,” 2014.

[4] “Building to webassembly.” [Online]. Available: https://emscripten.org/docs/
compiling/WebAssembly.html#webassembly[5] D. Herrera, H. Chen, E. Lavoie, and L. Hendren, “Webassembly and javascript
challenge: Numerical program performance using modern browser technologies
and devices,” University of McGill, Montreal: QC, Technical report SABLE-TR-2018-2,
2018.
[6] F. Oliveira and J. Mattos, “Analysis of WebAssembly as a Strategy to Improve
JavaScript Performance on IoT Environments,” Anais Estendidos do X Simpósio
Brasileiro de Engenharia de Sistemas Computacionais, pp. 133–138, 11 2020.
[7] M. Reiser and L. Bläser, “Accelerate javascript applications by cross-compiling towebassembly,” 10 2017, pp. 10–17.
[8] P. Sandhu, D. Herrera, and L. Hendren, “Sparse matrices on the web: Characterizing the performance and optimal format selection of sparse matrix-vector
multiplication in javascript and webassembly,” 09 2018, pp. 1–13.
[9] S. Tarkoma, M. Siekkinen, E. Lagerspetz, and Y. Xiao, Smartphone energy consumption: modeling and optimization. Cambridge University Press, 2014.
[10] J. Alamari and C. E. Chow, “Computation at the edge with webassembly,” in ITNG
2021 18th International Conference on Information Technology-New Generations,S. Latifi, Ed. Cham: Springer International Publishing, 2021, pp. 229–238.
[11] D. Lehmann and M. Pradel, “Wasabi: A framework for dynamically analyzing
webassembly,” Proceedings of the Twenty-Fourth International Conference on Architectural Support for Programming Languages and Operating Systems, 2019.
[12] A. Miettinen and J. Nurminen, “Analysis of the energy consumption of javascript
based mobile web applications,” vol. 45, 05 2010, pp. 124–135.
[13] J. de Macedo, J. Aloísio, N. Gonçalves, R. Pereira, and J. Saraiva, “Energy wars
- chrome vs. firefox: Which browser is more energy efficient?” in 2020 35th
IEEE/ACM International Conference on Automated Software Engineering Workshops(ASEW), 2020, pp. 159–165.
[14] C. Wohlin, P. Runeson, M. Höst, M. C. Ohlsson, B. Regnell, and A. Wesslén,
Experimentation in software engineering. Springer Science & Business Media,
2012.
[15] F. Shull, J. Singer, and D. I. Sjøberg, Guide to advanced empirical software engineering. Springer, 2007.
[16] L. Ardito, R. Coppola, M. Morisio, and M. Torchiano, “Methodological guidelines for measuring energy consumption of software applications,” Scientific
Programming, vol. 2019, p. 1–16, 2019.
[17] G. inc. V8. [Online; Consulted 20 September 2021]. [Online]. Available:
https://v8.dev/[18] M. inc. Spidermonkey. [Online; Consulted 20 September 2021]. [Online].
Available: https://spidermonkey.dev/[19] Khan, Faiz and Foley-Bourgon, Vincent and Kathrotia, Sujay and Lavoie, Erick,“Ostrich benchmark suite.” [Online]. Available: https://github.com/Sable/Ostrich[20] S. G. Stats. Browser market share worldwide. [Online; Consulted 9 January 2022].
[Online]. Available: https://gs.statcounter.com/browser-market-share#monthly202110-202110-bar[21] M. inc. Firefox source documentation. [Online; Consulted 20 September 2021].
[Online]. Available: https://firefox-source-docs.mozilla.org/js/index.html[22] K. A. Pituch, J. Stevens, and J. Stevens, Applied multivariate statistics for the social
sciences. Routledge, 2016.
[23] N. Cliff, “Dominance statistics: Ordinal analyses to answer ordinal questions.”
Psychological Bulletin, vol. 114, no. 3, p. 494–509, 1993.
[24] V. B. Kampenes, T. Dybå, J. E. Hannay, and D. I. Sjøberg, “A systematic review
of effect size in software engineering experiments,” Information and Software
Technology, vol. 49, no. 11-12, p. 1073–1086, 2007.
[25] J. J. K. Robert J. Grissom, Effect Sizes for Research: A Broad Practical Approach,
1st ed. Routledge Academic, 2005. [Online]. Available: libgen.li/file.php?md5=
c8d59f4f79837368fea6d52a6a3da9d2[26] I. Malavolta, G. Procaccianti, P. Noorland, and P. Vukmirovic, “Assessing theimpact of service workers on the energy efficiency of progressive web apps,” in2017 IEEE/ACM 4th International Conference on Mobile Software Engineering and
Systems (MOBILESoft), 2017, pp. 35–45.
[27] Y. Hu, J. Yan, D. Yan, Q. Lu, and J. Yan, “Lightweight energy consumption analysis
and prediction for android applications,” Science of Computer Programming,
vol. 162, pp. 132–147, 2018, special Issue on TASE 2016. [Online]. Available:
https://www.sciencedirect.com/science/article/pii/S0167642317300953[28] M. Couto, J. Saraiva, and J. P. Fernandes, “Energy refactorings for android in
the large and in the wild,” in 2020 IEEE 27th International Conference on Software
Analysis, Evolution and Reengineering (SANER), 2020, pp. 217–228.
[29] M. A. Hoque, M. Siekkinen, K. N. Khan, Y. Xiao, and S. Tarkoma, “Modeling,
profiling, and debugging the energy consumption of mobile devices,” vol. 48,
no. 3, dec 2015. [Online]. Available: https://doi.org/10.1145/2840723[30] I. Malavolta, E. M. Grua, C.-Y. Lam, R. de Vries, F. Tan, E. Zielinski, M. Peters, and
L. Kaandorp, “A Framework for the Automatic Execution of Measurement-based
Experiments on Android Devices,” in 35th IEEE/ACM International Conference on
Automated Software Engineering Workshops (ASEW ’20). ACM, 2020.
[31] C. Cook, Quasi-experimentation - Design and Analysis Issues for Field Settings.,
1979.