Benchmarking R vs RRO
Autor: Carlos Roberto | Publicado: 2015-07-21T13:39:00-07:00


Actually R has been
considered the most powerful programming language for statistical computing,
machine learning and graphics. The popularity of R has increased substantially
in recent years.



I have been playing with R language in machine learning
projects since the beginning of 2013 and so, I have seen lots of interesting
discussions in forums and articles about R, RStudio, Python, Hadoop, etc. 



Some weeks ago I was reading an article about RRO
acquisition (the Revolution R Open) by Microsoft. The comments about RRO performance
took my attention: one of the features that differentiate RRO from R is its Multithreaded Performance
capability. That is a very nice news because we know that in statistical
analysis and machine language modeling, most of the time we have tasks that are
computationally intensive that lead us to consider options, like optimized
libraries, that can decrease the R processing time. Checking the RRO documentation
I could see that RRO has libraries that “make it possible for so many common R
operations, such as matrix multiply/inverse, matrix decomposition, and some
higher-level matrix operations, to compute in parallel and use all of the
processing power available to reduce computation times”. In the era of
multi-core machines, definitely that is a good news. 



So, I decided to make a benchmark with RRO to see how it
could be useful in the projects I have worked with when compared with R. You
will find more benchmarks in RRO website as well.



For this job, I selected a project I worked last year. Basically
the project is a data analysis that uses dataset from the National Oceanic and
Atmospheric Administration’s (NOAA). The project is simple but it takes about 5
hours of execution time. I will not discuss the data analysis project here because
this is beyond the scope of this post but if you are interested in the details
about the project analysis, you will find it here in the rpubs website.



Hardware used:
1. Processor: &nbsp;&nbsp;&nbsp;&nbsp; Intel Core i7-3615QM
cpu 2.30GHz
2. Memory: &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 8.0 GB
3. Number of processors: 4 cores or 8 logical cpus



Environment used:
1. System OS: Windows 7 Pro 64-bit
2. R: 3.2.0
3. RRO: 3.2.0
4. RStudio: 0.99.448 



The project code was not changed to run in RRO once the
documentation says that RRO is 100% compatible with R.
To have a better idea about how long time the processing takes to execute the R
tasks, I divided the entire processing time in 12 phases, where there are different
R tasks like dataset download, unpack, read, order, summation, aggregate and so
on. You can better visualize these phases in the final graphic.



Lessons learned.


The
table below shows the performance, in minutes, for each API in each phase of
the processing


 

  
  
Phase

  
  
R

  
  
RRO

  
  
RRO.MKL

  
  
Desc

 

  
  
1

  
  
1.04

  
  
0.74

  
  
0.76

  
  
Download 47 Mb

 

  
  
2

  
  
6.09

  
  
6.08

  
  
6.20

  
  
Unpack and read file

 

  
  
3

  
  
0.00

  
  
0.00

  
  
0.00

  
  
Select data of interest

 

  
  
4

  
  
0.13

  
  
0.14

  
  
0.11

  
  
Sum columns and create new one

 

  
  
5

  
  
0.03

  
  
0.03

  
  
0.03

  
  
Aggregate

 

  
  
6

  
  
0.00

  
  
0.00

  
  
0.00

  
  
Filter out zero data

 

  
  
7

  
  
0.00

  
  
0.00

  
  
0.00

  
  
Order and select top 10

 

  
  
8

  
  
0.00

  
  
0.00

  
  
0.00

  
  
Get new subset

 

  
  
9

  
  
265.96

  
  
266.35

  
  
265.35

  
  
Loop and apply multiplier

 

  
  
10

  
  
0.01

  
  
0.01

  
  
0.01

  
  
Aggregate

 

  
  
11

  
  
0.00

  
  
0.00

  
  
0.00

  
  
Order and show top 10

 

  
  
12

  
  
0.02

  
  
0.00

  
  
0.00

  
  
Order and plot top 5

 



The graphic below represents the processing table.






Looking
to the graphic we can see that there is no significant difference between the
use of 3 APIs. Although the processing time takes about 5 hours, it looks like this
project does not fill the requirements to make use the advantage of&nbsp;multi-threaded&nbsp;performance because it does not use the optimized functions described in RRO&nbsp;documentation. So, the advantage of RRO performance is a good news for data
scientists and R developers but the R user must have in mind that not all R
code will use the RRO performance (at least considering the version used in
this test). Enjoy.