Julia For Data Science. Python and R are undoubtedly the... | by Bernard Brenyah | Towards Data Science

Currently, Python and R are undoubtedly the most widely used programming languages in the machine learning world. Unfortunately, the high-level and elegant abstractions from these programming usually come at a cost, especially when working with large systems. Before we even begin, this post will not be some endless bashing of either Python or R so any reader with such expectations should look elsewhere. Rather, this post seeks to enlighten and inform readers about the potential addition of Julia to their toolbox. At the end of the day, all these programming languages are mere tools with different use cases.
Julia is described as a dynamic programming language which focuses on being highly performant even with high-level abstractions. As a result, this language is well suited for numerical and scientific computing. Julia was co-created by 4 computer scientists (3 of which are pictured below ) in 2009.
This line has been one of the most-used references, to sum up, the main advantage of this programming language. Some readers may have read about the infamous “two language problem”. For the uninitiated ones, the two language problem simply describes the situation where;
developers have to re-write the critical and performant parts of a codebase in a low-level language like C/C++ as high-level languages like Python/R can be slow with certain tasks due to their high-level abstractions
It is comparably easier to quickly prototype new ideas in high-level languages like Python and R. A major reason why these languages have such a dominant position in the data science ecosystem which heavily relies on trying new ideas as quickly as possible. The tradeoff (in terms of speed) here is acceptable for small and medium scale applications but very noticeable on large scale systems and applications.
Julia is being dubbed as the language that can solve this two-language problem by offering the friendliness of a high-level language like Python/R while matching the speed of a low-level language like C/C++.
I had heard about Julia but I always wondered about its usage. Is it as easy as it is with Python/R?
This post simply tries to test this assertion by building from scratch multivariate linear regression using gradient descent with Julia. I have been using both Python and R for some years now so I decided to see how fast I can use Julia to quickly solve such a task. This post won’t be a benchmark test between Julia and other languages as such tests are well covered but rather a focus on the user-friendliness of the language to someone who is used to Python/R. I will end the post with some final thoughts after this process.
The dataset that will be used for this test is the Combined Cycle Power Plant Data Set (Sheet5) from the UCI data repository. The aim here is to generate a model that can predict Net hourly electrical energy output (EP) from 4 numerical (continuous) features:
Temperature (AT) in the range 1.81°C and 37.11°C.
Exhaust Vacuum (V) in the range 25.36–81.56 cm Hg.
Ambient Pressure (AP) in the range 992.89–1033.30 millibar.
Relative Humidity (RH) in the range of 25.56% to 100.16%.
The plan of attack for this experiment is just like a typical modelling workflow where the data will be:
loaded into memory: The data is loaded into memory as a DataFrame object that most readers will already be familiar with this code snippet:
pre-processed for algorithms: With the data loaded successfully loaded into memory, let’s now split it into a training and validation set using a ratio of 80% for the training of the model with 20% withheld for the evaluation of the generated model. This code snippet below does the trick (but a more robust way is to shuffle the indices). The goal is to quickly get up and running so this will suffice for now.
As you may have noticed, the features are on different scales (from percentages to millibars) which might be problematic for most algorithms so it’s common practice to standardise features so that they can be on the same ‘scale’. The two functions in the code snippet below are used to standardise both the training and testing features.
Our data is now pre-processed and ready for modelling.
model fitting & evaluation: The objective here is to find a linear model that best minimises the good old mean square error function. Our mean squared cost function is essentially a half mean squared function and it’s defined as follows:
With our cost function defined, let’s deploy gradient descent to find us the best weights for each of the features (including a constant as well) that minimises this cost function. The next function takes care of that:
NB: This implementation implements gradient descent without regularisation (which I encourage you to add to your own implementations).
Let’s now check if gradient descent works and minimises our cost function by plotting our cost history vector J:
As one can see from the plot, the cost drops as the number of iterations increases until it flatlines at some point. It appears that gradient descent is doing a good job of finding the weights that minimise our cost function.
It’s now time to make some predictions and evaluate the performance of our new shiny regression model. Two new functions are used to generate predictions and score both the training and testing datasets.
From the scoring function, the Root Mean Square Error (RMSE) for the training and testing set are:
RMSE for Training Set: 4.578258826855693RMSE for Testing Set: 4.473104537374325
Let’s now look at the predictive powers (in terms of R-squared value) of the model in both datasets as well with this function:
Training R2 score for test sets: 0.9274046053983883Testing R2 score for test sets: 0.9335248094191678
The scores appear to show that some decent predictive powers (0.927 & 0.934)!
However, the testing score is better than the training score which appears to suggest that the model is underfitting data. Tinkering with the alpha term (α) and/or the number of iterations (n_iter) should squeeze more performance from this model but I will leave that up to readers to tinker with.
From this experiment, I naturally have some likes and dislikes with Julia based on my previous interactions with Python and R.
Likes
Native Unicode/notation support: My favourite Julia is undoubtedly its native support for Unicode and scientific notations. There’s something cool and neat about using ‘θ’ instead of ‘theta’, ‘α’ instead of ‘alpha’ and in a script. It’s a language built from ground up for scientific/numeric computing! I found myself using smiling every time I had to use scientific notation.
Readable syntax: The syntax of a language plays a key role in its ability to quickly prototype ideas in. Julia’s syntax is not that different from Python. Personally, Julia feels like a love child between Python and Matlab in terms of its syntax. A quick lookup of the documentation, in most cases, was I needed to get past some bad syntax calls. Additionally, it’s very easy to share and read scientific code with real scientific notations!
Support for existing Python/R ecosystem: As a young language, it is expected that some much-needed packages are missing from the eco-system. It is, however, refreshing to know that existing Python/R packages can be used in Julia. This functionality allows Julia to enjoy the benefits of matured eco-systems and avoids the needless need to re-invent certain wheels.
Dislikes
Slow pre-compiling time: The first thing one notices is the time it takes to load packages in Julia compared to Python/R. Loading of packages every time requires pre-compiling which does take some noticeable seconds (more than 10 seconds). This can be frustrating for Python/R users who are so used to instantaneous package imports.
Young language/eco-system: There is almost always a package for some common data wrangling tasks in either Python/R. Julia, as a newcomer, naturally doesn’t have such a native rich eco-system that established languages enjoy. However, this issue can be addressed for now with the support for imports from both Python and R using PyCall & JuliaCall packages.
Lack of traditional OOP support: Part of the charm and wide-spread adoption of Python for data science application is the neat and convenient way functions are constructed with object-oriented programming. For example, it would have been neat and tidy if all the pre-processing functions as well all the learned parameters were available under one class. Julia adopts the “multiple dispatch” paradigm which helps it to be so fast compared to other high-level languages.
Julia is a promising language that is built from the ground up for numerical and scientific computing. In this post, I have shown that it’s relatively easy to prototype in Julia and successfully implemented a multivariate linear regression using gradient descent. It’s one language that every data should at least keep and a digital eye on. There are some amazing data science packages out there for Julia. Grab some dataset, play with Julia and let me know what you think? The best way to get up and running, in my opinion, is to install JuliaPro which comes with an IDE using Atom.
As always I look forward to feedback (good or bad!). The script with all the functions for this post can be found on the GitHub repository for this blog. Finally, you can check out other posts on my personal blog. Until the next post, happy coding in Julia!