R vs. Python
for Data
Science?
Norm Matloff
Dept. ofComputer
Science
University ofCalifornia,
Davis

R vs. Python for Data Science?
Norm Matloff
Dept. of Computer Science
University of California, Davis

Invited Talk
SDSS 2020

URL for these slides:
http://heather.cs.ucdavis.edu/RvsPythonForDS.pdf

R vs. Python
for Data
Science?
Norm Matloff
Dept. ofComputer
Science
University ofCalifornia,
Davis

Where I’m coming fromR vs. Python
for Data
Science?

Where I’m coming from

Norm Matloff
Dept. ofComputer
Science
University ofCalifornia,
Davis• User of both languages since near the beginning.

R vs. Python
for Data
Science?

Where I’m coming from

Norm Matloff
Dept. ofComputer
Science
University ofCalifornia,
Davis• User of both languages since near the beginning.
• Former S user, transitioned early to R (“free S”).

R vs. Python
for Data
Science?

Where I’m coming from

Norm Matloff
Dept. ofComputer
Science
University ofCalifornia,
Davis• User of both languages since near the beginning.
• Former S user, transitioned early to R (“free S”).
• Hated PERL, thus welcomed Python early in its
development.

R vs. Python
for Data
Science?

Where I’m coming from

Norm Matloff
Dept. ofComputer
Science
University ofCalifornia,
Davis• User of both languages since near the beginning.
• Former S user, transitioned early to R (“free S”).
• Hated PERL, thus welcomed Python early in its
development.
• Switched to Python for my admin tasks.

R vs. Python
for Data
Science?

Where I’m coming from

Norm Matloff
Dept. ofComputer
Science
University ofCalifornia,
Davis• User of both languages since near the beginning.
• Former S user, transitioned early to R (“free S”).
• Hated PERL, thus welcomed Python early in its
development.
• Switched to Python for my admin tasks.
• Later, switched to R for my admin tasks, not just data
science.

R vs. Python
for Data
Science?

Where I’m coming from

Norm Matloff
Dept. ofComputer
Science
University ofCalifornia,
Davis• User of both languages since near the beginning.
• Former S user, transitioned early to R (“free S”).
• Hated PERL, thus welcomed Python early in its
development.
• Switched to Python for my admin tasks.
• Later, switched to R for my admin tasks, not just data
science.
• Teach both languages.

R vs. Python
for Data
Science?

Where I’m coming from

Norm Matloff
Dept. ofComputer
Science
University ofCalifornia,
Davis• User of both languages since near the beginning.
• Former S user, transitioned early to R (“free S”).
• Hated PERL, thus welcomed Python early in its
development.
• Switched to Python for my admin tasks.
• Later, switched to R for my admin tasks, not just data
science.
• Teach both languages.
• Author of several books on, or using, R.

R vs. Python
for Data
Science?

Where I’m coming from

Norm Matloff
Dept. ofComputer
Science
University ofCalifornia,
Davis• User of both languages since near the beginning.
• Former S user, transitioned early to R (“free S”).
• Hated PERL, thus welcomed Python early in its
development.
• Switched to Python for my admin tasks.
• Later, switched to R for my admin tasks, not just data
science.
• Teach both languages.
• Author of several books on, or using, R.
• Former Editor-in-Chief, The R Journal.

R vs. Python
for Data
Science?

Where I’m coming from

Norm Matloff
Dept. ofComputer
Science
University ofCalifornia,
Davis• User of both languages since near the beginning.
• Former S user, transitioned early to R (“free S”).
• Hated PERL, thus welcomed Python early in its
development.
• Switched to Python for my admin tasks.
• Later, switched to R for my admin tasks, not just data
science.
• Teach both languages.
• Author of several books on, or using, R.
• Former Editor-in-Chief, The R Journal.
Thus a definite bias toward R, but am also an enthusiastic
Pythonista.

R vs. Python
for Data
Science?
Norm Matloff
Dept. ofComputer
Science
University ofCalifornia,
Davis

Overview

R vs. Python
for Data
Science?
Norm Matloff
Dept. ofComputer
Science
University ofCalifornia,
Davis

Overview
Here I will argue in favor of R or Python on each of the below
criteria.

R vs. Python
for Data
Science?
Norm Matloff
Dept. ofComputer
Science
University ofCalifornia,
Davis

Overview
Here I will argue in favor of R or Python on each of the below
criteria. (If your favorite criterion is missing, please bring it up
in Q&A.)R vs. Python
for Data
Science?
Norm Matloff
Dept. ofComputer
Science
University ofCalifornia,
Davis

Overview
Here I will argue in favor of R or Python on each of the below
criteria. (If your favorite criterion is missing, please bring it up
in Q&A.)• Elegance.
• Learning curve• Available libraries for Data Science• Machine learning• Statistical sophistication• Parallel computation• C/C++ interface and performance enhancement• Object orientation, metaprogramming• Language unity• Linked data structures• Online help

R vs. Python
for Data
Science?
Norm Matloff
Dept. ofComputer
Science
University ofCalifornia,
Davis

EleganceR vs. Python
for Data
Science?

Elegance

Norm Matloff
Dept. ofComputer
Science
University ofCalifornia,
Davis

Clear win for Python.
Personally, I really appreciate Python’s clean lines:
if x > y:
z = 5
w = 8

versusif (x > y)
{
z = 5
w = 8
}

Python class structure cleaner than the various R structures.

R vs. Python
for Data
Science?
Norm Matloff
Dept. ofComputer
Science
University ofCalifornia,
Davis

Learning curveR vs. Python
for Data
Science?

Learning curve

Norm Matloff
Dept. ofComputer
Science
University ofCalifornia,
Davis

Huge win for R.

R vs. Python
for Data
Science?

Learning curve

Norm Matloff
Dept. ofComputer
Science
University ofCalifornia,
Davis

Huge win for R.
• I like to say,
R was developed by statisticians for statisticians.
(Replace statisticians by data scientists if you wish.)R vs. Python
for Data
Science?

Learning curve

Norm Matloff
Dept. ofComputer
Science
University ofCalifornia,
Davis

Huge win for R.
• I like to say,
R was developed by statisticians for statisticians.
(Replace statisticians by data scientists if you wish.)• But I’d also say,Python was developed by computer scientists for computer scientists.

R vs. Python
for Data
Science?

Learning curve

Norm Matloff
Dept. ofComputer
Science
University ofCalifornia,
Davis

Huge win for R.
• I like to say,
R was developed by statisticians for statisticians.
(Replace statisticians by data scientists if you wish.)• But I’d also say,Python was developed by computer scientists for computer scientists.
• In Data Science, many people from backgrounds otherthan Computer Science or the like.

R vs. Python
for Data
Science?

Learning curve

Norm Matloff
Dept. ofComputer
Science
University ofCalifornia,
Davis

Huge win for R.
• I like to say,
R was developed by statisticians for statisticians.
(Replace statisticians by data scientists if you wish.)• But I’d also say,Python was developed by computer scientists for computer scientists.
• In Data Science, many people from backgrounds otherthan Computer Science or the like.
• Python, especially in usage of libraries, really requires
some computer systems sophistication.

R vs. Python
for Data
Science?
Norm Matloff
Dept. ofComputer
Science
University ofCalifornia,
Davis

Learning curve cont’d.

R vs. Python
for Data
Science?

Learning curve cont’d.

Norm Matloff
Dept. ofComputer
Science
University ofCalifornia,
Davis• Python libraries can be tricky to configure, even for thesystems-savvy, while most R packages run right out of thebox.

R vs. Python
for Data
Science?

Learning curve cont’d.

Norm Matloff
Dept. ofComputer
Science
University ofCalifornia,
Davis• Python libraries can be tricky to configure, even for thesystems-savvy, while most R packages run right out of thebox.
• To even get started in Data Science with Python, onemust learn a lot of material not in base Python, e.g.,
NumPy, Pandas and matplotlib.

R vs. Python
for Data
Science?

Learning curve cont’d.

Norm Matloff
Dept. ofComputer
Science
University ofCalifornia,
Davis• Python libraries can be tricky to configure, even for thesystems-savvy, while most R packages run right out of thebox.
• To even get started in Data Science with Python, onemust learn a lot of material not in base Python, e.g.,
NumPy, Pandas and matplotlib.
• By contrast, matrix types and basic graphics are built-in tobase R.

R vs. Python
for Data
Science?

Learning curve cont’d.

Norm Matloff
Dept. ofComputer
Science
University ofCalifornia,
Davis• Python libraries can be tricky to configure, even for thesystems-savvy, while most R packages run right out of thebox.
• To even get started in Data Science with Python, onemust learn a lot of material not in base Python, e.g.,
NumPy, Pandas and matplotlib.
• By contrast, matrix types and basic graphics are built-in tobase R. The novice can be doing simple data analyses
within minutes.

R vs. Python
for Data
Science?

Learning curve cont’d.

Norm Matloff
Dept. ofComputer
Science
University ofCalifornia,
Davis• Python libraries can be tricky to configure, even for thesystems-savvy, while most R packages run right out of thebox.
• To even get started in Data Science with Python, onemust learn a lot of material not in base Python, e.g.,
NumPy, Pandas and matplotlib.
• By contrast, matrix types and basic graphics are built-in tobase R. The novice can be doing simple data analyses
within minutes.
• This alone should make Python a non-starter for Data
Science.

R vs. Python
for Data
Science?

Learning curve cont’d.

Norm Matloff
Dept. ofComputer
Science
University ofCalifornia,
Davis• Python libraries can be tricky to configure, even for thesystems-savvy, while most R packages run right out of thebox.
• To even get started in Data Science with Python, onemust learn a lot of material not in base Python, e.g.,
NumPy, Pandas and matplotlib.
• By contrast, matrix types and basic graphics are built-in tobase R. The novice can be doing simple data analyses
within minutes.
• This alone should make Python a non-starter for Data
Science.
• Of central importance, so I will elaborate here...

R vs. Python
for Data
Science?
Norm Matloff
Dept. ofComputer
Science
University ofCalifornia,
Davis

Learning curve cont’d.

R vs. Python
for Data
Science?

Learning curve cont’d.

Norm Matloff
Dept. ofComputer
Science
University ofCalifornia,
Davis

Example, trying to install Keras on one of my machines:
Found e x i s t i n g i n s t a l l a t i o n : p i p 8 . 1 . 1
U n i n s t a l l i n g pip −8.1.1:
S u c c e s s f u l l y u n i n s t a l l e d pip −8.1.1
S u c c e s s f u l l y i n s t a l l e d pip −7.1.2

It took a working version of the package installer pip andinexplicably uninstalled it, replacing it with an older version!
Even a systems-savvy person like me might have trouble
tracking down the problem.

R vs. Python
for Data
Science?
Norm Matloff
Dept. ofComputer
Science
University ofCalifornia,
Davis

Learning curve cont’d.

R vs. Python
for Data
Science?

Learning curve cont’d.

Norm Matloff
Dept. ofComputer
Science
University ofCalifornia,
Davis

As an example, I asked a Python sophisticate to install a libraryfor PHATE, a visualization tool, thinking what a novice wouldsee:
I tried it...using PyCharm...as IDE. I started
off with a fresh install on a new computer,and I did run into some problems...Numpy.distutils.system info.NotFoundError:
No lapack/blas resources found...[the problem] after
doing some google searching... is coming from somemissing dependencies. According to stack overflow,
one way around this is...
He did say things went much better with Anaconda, but to me
his experience epitomizes the problem:
Python is unnecessarily requiring too much expertise in theuser.

R vs. Python
for Data
Science?
Norm Matloff
Dept. ofComputer
Science
University ofCalifornia,
Davis

Learning curve,xkcd

R vs. Python
for Data
Science?
Norm Matloff
Dept. ofComputer
Science
University ofCalifornia,
Davis

Learning curve,xkcd
Data Science version of The Scream:R vs. Python
for Data
Science?
Norm Matloff
Dept. ofComputer
Science
University ofCalifornia,
Davis

Available libraries for Data ScienceR vs. Python
for Data
Science?

Available libraries for Data Science

Norm Matloff
Dept. ofComputer
Science
University ofCalifornia,
Davis

Slight edge to R.

R vs. Python
for Data
Science?

Available libraries for Data Science

Norm Matloff
Dept. ofComputer
Science
University ofCalifornia,
Davis

Slight edge to R.
• PyPI large but limited for data science.

R vs. Python
for Data
Science?

Available libraries for Data Science

Norm Matloff
Dept. ofComputer
Science
University ofCalifornia,
Davis

Slight edge to R.
• PyPI large but limited for data science.
• My (admittedly cursory) search for fast determination ofnearest-neighbors on PyPI produced nothing. CRAN has
at least 2 pkgs for R.

R vs. Python
for Data
Science?

Available libraries for Data Science

Norm Matloff
Dept. ofComputer
Science
University ofCalifornia,
Davis

Slight edge to R.
• PyPI large but limited for data science.
• My (admittedly cursory) search for fast determination ofnearest-neighbors on PyPI produced nothing. CRAN has
at least 2 pkgs for R.
• The following (again, cursory) searches in PyPI turned up
nothing: EM algorithm; log-linear model; Poisson
regression; instrumental variables; spatial data; familywise
error rateR vs. Python
for Data
Science?

Available libraries for Data Science

Norm Matloff
Dept. ofComputer
Science
University ofCalifornia,
Davis

Slight edge to R.
• PyPI large but limited for data science.
• My (admittedly cursory) search for fast determination ofnearest-neighbors on PyPI produced nothing. CRAN has
at least 2 pkgs for R.
• The following (again, cursory) searches in PyPI turned up
nothing: EM algorithm; log-linear model; Poisson
regression; instrumental variables; spatial data; familywise
error rateR vs. Python
for Data
Science?
Norm Matloff
Dept. ofComputer
Science
University ofCalifornia,
Davis

Machine learning

R vs. Python
for Data
Science?

Machine learning

Norm Matloff
Dept. ofComputer
Science
University ofCalifornia,
Davis

Slight edge to Python.

R vs. Python
for Data
Science?

Machine learning

Norm Matloff
Dept. ofComputer
Science
University ofCalifornia,
Davis

Slight edge to Python.
• For many in ML, machine learning = neural networks.

R vs. Python
for Data
Science?

Machine learning

Norm Matloff
Dept. ofComputer
Science
University ofCalifornia,
Davis

Slight edge to Python.
• For many in ML, machine learning = neural networks.
• Since NNs have been developed mainly by CS people, themore sophisticated libraries, esp. for image classification,
tend to be in Python.

R vs. Python
for Data
Science?

Machine learning

Norm Matloff
Dept. ofComputer
Science
University ofCalifornia,
Davis

Slight edge to Python.
• For many in ML, machine learning = neural networks.
• Since NNs have been developed mainly by CS people, themore sophisticated libraries, esp. for image classification,
tend to be in Python.
• But random forests, gradient boosting etc. have beendeveloped mainly by stat people, and R has excellent
packages for these.

R vs. Python
for Data
Science?

Machine learning

Norm Matloff
Dept. ofComputer
Science
University ofCalifornia,
Davis

Slight edge to Python.
• For many in ML, machine learning = neural networks.
• Since NNs have been developed mainly by CS people, themore sophisticated libraries, esp. for image classification,
tend to be in Python.
• But random forests, gradient boosting etc. have beendeveloped mainly by stat people, and R has excellent
packages for these.
• Want to do NNs in R? RStudio put in a huge effort todevelop the R keras package, and it’s excellent. H2O too.

R vs. Python
for Data
Science?
Norm Matloff
Dept. ofComputer
Science
University ofCalifornia,
Davis

Statistical sophistication

R vs. Python
for Data
Science?

Statistical sophistication

Norm Matloff
Dept. ofComputer
Science
University ofCalifornia,
Davis

Big win for R.

R vs. Python
for Data
Science?

Statistical sophistication

Norm Matloff
Dept. ofComputer
Science
University ofCalifornia,
Davis

Big win for R.
• Again: R was developed by statisticians for statisticians.

R vs. Python
for Data
Science?

Statistical sophistication

Norm Matloff
Dept. ofComputer
Science
University ofCalifornia,
Davis

Big win for R.
• Again: R was developed by statisticians for statisticians.
• I find that Python ML people are more interested in the
CS side of a method, e.g. fast sorting, than theprobabilistic meaning of the model.

R vs. Python
for Data
Science?

Statistical sophistication

Norm Matloff
Dept. ofComputer
Science
University ofCalifornia,
Davis

Big win for R.
• Again: R was developed by statisticians for statisticians.
• I find that Python ML people are more interested in the
CS side of a method, e.g. fast sorting, than theprobabilistic meaning of the model.
• They tend to downplay the stat, and often don’t
understand it.

R vs. Python
for Data
Science?

Statistical sophistication

Norm Matloff
Dept. ofComputer
Science
University ofCalifornia,
Davis

Big win for R.
• Again: R was developed by statisticians for statisticians.
• I find that Python ML people are more interested in the
CS side of a method, e.g. fast sorting, than theprobabilistic meaning of the model.
• They tend to downplay the stat, and often don’t
understand it.
• I was appalled recently to see one of the most prominent
ML people state in his book that standardizing the data tomean-0, variance-1 means one is assuming the data are
Gaussian — absolutely false and misleading.

R vs. Python
for Data
Science?
Norm Matloff
Dept. ofComputer
Science
University ofCalifornia,
DavisParallel computation

R vs. Python
for Data
Science?

Parallel computation

Norm Matloff
Dept. ofComputer
Science
University ofCalifornia,
DavisLet’s call it a tie.

R vs. Python
for Data
Science?

Parallel computation

Norm Matloff
Dept. ofComputer
Science
University ofCalifornia,
DavisLet’s call it a tie.
• Python multiprocessing package much improved frombefore.

R vs. Python
for Data
Science?

Parallel computation

Norm Matloff
Dept. ofComputer
Science
University ofCalifornia,
DavisLet’s call it a tie.
• Python multiprocessing package much improved frombefore.
• Python currently has better GPU access.

R vs. Python
for Data
Science?

Parallel computation

Norm Matloff
Dept. ofComputer
Science
University ofCalifornia,
DavisLet’s call it a tie.
• Python multiprocessing package much improved frombefore.
• Python currently has better GPU access.
• But still, R parallel package is much easier to use.

R vs. Python
for Data
Science?
Norm Matloff
Dept. ofComputer
Science
University ofCalifornia,
Davis

C/C++ interface and performanceenhancement

R vs. Python
for Data
Science?
Norm Matloff
Dept. ofComputer
Science
University ofCalifornia,
Davis

C/C++ interface and performanceenhancement

Slight win for R.
• Python has SWIG, PyPy, Cython, variants.

R vs. Python
for Data
Science?
Norm Matloff
Dept. ofComputer
Science
University ofCalifornia,
Davis

C/C++ interface and performanceenhancement

Slight win for R.
• Python has SWIG, PyPy, Cython, variants.
• Lots of excitement about Pybind11.

R vs. Python
for Data
Science?
Norm Matloff
Dept. ofComputer
Science
University ofCalifornia,
Davis

C/C++ interface and performanceenhancement

Slight win for R.
• Python has SWIG, PyPy, Cython, variants.
• Lots of excitement about Pybind11.
• But the versatility of R’s Rccp is really much more
powerful.

R vs. Python
for Data
Science?
Norm Matloff
Dept. ofComputer
Science
University ofCalifornia,
Davis

C/C++ interface and performanceenhancement

Slight win for R.
• Python has SWIG, PyPy, Cython, variants.
• Lots of excitement about Pybind11.
• But the versatility of R’s Rccp is really much more
powerful.
• And R’s new ALTREP has tremendous promise.

R vs. Python
for Data
Science?
Norm Matloff
Dept. ofComputer
Science
University ofCalifornia,
Davis

Language unity

R vs. Python
for Data
Science?

Language unity

Norm Matloff
Dept. ofComputer
Science
University ofCalifornia,
Davis• Python has now successfully accomplished transition from
2.7 to 3.x.

R vs. Python
for Data
Science?

Language unity

Norm Matloff
Dept. ofComputer
Science
University ofCalifornia,
Davis• Python has now successfully accomplished transition from
2.7 to 3.x.
• By contrast, R is rapidly devolving into two mutually
unintelligible dialects/communities, ordinary R and the
Tidyverse.

R vs. Python
for Data
Science?

Language unity

Norm Matloff
Dept. ofComputer
Science
University ofCalifornia,
Davis• Python has now successfully accomplished transition from
2.7 to 3.x.
• By contrast, R is rapidly devolving into two mutually
unintelligible dialects/communities, ordinary R and the
Tidyverse.
• To some degree, that split also falls along the lines ofpeople who do statistics and those who view Data Scienceas graphics and data wrangling.

R vs. Python
for Data
Science?

Language unity

Norm Matloff
Dept. ofComputer
Science
University ofCalifornia,
Davis• Python has now successfully accomplished transition from
2.7 to 3.x.
• By contrast, R is rapidly devolving into two mutually
unintelligible dialects/communities, ordinary R and the
Tidyverse.
• To some degree, that split also falls along the lines ofpeople who do statistics and those who view Data Scienceas graphics and data wrangling.
• I’m a skeptic re Tidy
(http://github.com/matloff/TidyverseSkeptic), but nomatter what one’s view is, this split is not good for R.

R vs. Python
for Data
Science?
Norm Matloff
Dept. ofComputer
Science
University ofCalifornia,
Davis

Linked data structures

R vs. Python
for Data
Science?

Linked data structures

Norm Matloff
Dept. ofComputer
Science
University ofCalifornia,
Davis

Win for Python.

R vs. Python
for Data
Science?

Linked data structures

Norm Matloff
Dept. ofComputer
Science
University ofCalifornia,
Davis

Win for Python.
• E.g. binary trees.
• Easy in Python, hard in R.
• Not common in Data Science.
• There is the R package datastructures.

R vs. Python
for Data
Science?
Norm Matloff
Dept. ofComputer
Science
University ofCalifornia,
Davis

Online help

R vs. Python
for Data
Science?

Online help

Norm Matloff
Dept. ofComputer
Science
University ofCalifornia,
Davis

Big win for R.

R vs. Python
for Data
Science?

Online help

Norm Matloff
Dept. ofComputer
Science
University ofCalifornia,
Davis

Big win for R.
• R’s help() generally more helpful than Python’s.
• Also, example(), vignettes.
• Same for R’s generic functions. When I’m using a new
package, I know that I can probably use print(), plot(),
summary(), and so on, while I am exploring.

R vs. Python
for Data
Science?
Norm Matloff
Dept. ofComputer
Science
University ofCalifornia,
Davis

A small example

R vs. Python
for Data
Science?

A small example

Norm Matloff
Dept. ofComputer
Science
University ofCalifornia,
Davis• OMSI online exam tool (github.com/matloff/omsi).
• Rather complex client/server app.
• Written by a highly talented team of students under mydirection.
• I had them write the exam tool itself in Python, as I
thought it would be easier to get top students who knew
Python well.
• But I wrote the companion grading code, also rather
complex, myself. And I wrote in R, my preference.
• Not a stat/Data Science app at all.
• Yet R was just as usable as Python in this app.
• Unlike some claims to the contrary, Yes, R in fact IS a”real” language!