The Serious Downsides To The Julia Language In 1.0.3 | by Emmett Boudreau | Towards Data Science

I have talked a lot about how much I love Julia. If I was asked what my favorite programming language right now, “ Julia” would certainly be my reply. On the other hand, something I don’t talk about as much are the things I don’t like about Julia. Although today I would like to address some of these weaknesses and personal gripes I have with the language, I firmly believe that the benefits outweigh any compromises that might have to be made to use Julia. If you have never used Julia, I would highly encourage you to check it out! And with my recommendation, I’m also going to prelude my gripes with the reasons I love it so much.
Julia is very much a functional language, and loves being functional. With that, a great example is what I demonstrated in my looping speed test, Julia works really well with functional recursion.
Functional recursive loops are my personal favorite appearance-wise to utilize when doing statistical work. Typically in other languages however, like Python, we run into roadblocks in speed whenever recursion is used, and this is a big disadvantage in my opinion for statistical computing.
Okay, so Julia is a functional language for statistics...
Must be pretty low-level.
Julia really does break the rules, and every year it seems to inch closer and closer to being as versatile as Python.
Genie is a great example, as it is actually a framework for developing web-back-ends and pipelines inside of Julia, which is pretty high-level if I do say so myself.
Julia is fast, and I’m not just saying that. Julia’s typical run speeds are similar to that of C. This makes deployment quicker than a traditional Python-based web-app that needs to do calculations, especially if you’re deploying to an inactive server, like on Heroku for example. Julia is capable of processing and manipulating data that you can’t even read into Python. Julia’s speed and efficiency is really trifling when working with data-sets of twenty — thirty million, and still coming out with a calculation time below twenty seconds.
While I wouldn’t say Julia is necessarily a replacement for Scala, what I will say is that it certainly could have the potential to replace Scala. I find myself hardly ever needing to utilize Spark when I can use Julia, which is great because of my next advantage:
Now quick isn’t in reference to the speed we discussed, of course, but in reference to effort time. Typically in a powerful language like Julia, a significant downside is some pretty meaty syntax, and if the language is unique, as Julia is, this can be daunting and off-putting. Julia is easy to setup, and even has ways to be integrated into Spark as well as Python. If you’d like to learn more about Julia and Spark, check this out!
Though fairly critical and minor, the biggest one that I wanted to mention out of the gate is DataFrames.jl. Now don’t get me wrong, I like DataFrames to some degree... But particularly annoying is the way that DataFrames are displayed to the user, and I don’t really understand the reasons for doing such a thing this way. Of course, this could be a case of a performance issue I’m unaware of, but I’m definitely uncertain.
Firstly, to read in a CSV file, we don’t use DataFrames.jl like we would use Pandas in Python, or even use R’s internal tables. First, we have to
Pkg.add("CSV")
which will add an entire module for the single purpose of reading a CSV, more on this later.
Whenever we go to read in our CSV after adding CSV though, we are greated with a message at the top of our DataFrame:
“ omitted printing of 68 columns”
Of course, that’s no issue, but how do we show all of the columns? Well it’s nothing too complicated, just:
show(df,allcols=true)
But the issue that I have is what returns after that:
Oh my goodness! What year is this?
Maybe you could argue this isn’t too serious, but try sorting through hundreds and hundreds of features inside of this little iframe, rather than just running
df.head()
and then try to not complain about it. Compared to Pandas, and R DataFrames, Julia certainly takes last place for this! With that in mind, with some thought applied, I consider it quite possible that they fully intend for you to use Pandas with PyCall, but this is purely speculation.
Remember what I was talking about with using CSV.jl as apposed to being able to read a CSV with DataFrames.jl? (which you can, DataFrames.read_table, but it does some really funky stuff and throws a deprecation warning.) Well that whole scattering of functions and packages carries over into nearly everything in Julia.
If I wanted to use a decision tree, for example, I’d have to google up DecisionTree.jl and Pkg.add() it.
Pkg.add("DecisionTree")
Well that’s simple enough, so what’s the problem?
Every single model you want is contained inside of a different module, all with their own individual methods, parameters, constructors, syntax, and documentation. On the documentation spectrum, it can go from two example lines of code that don’t really explain anything all the way up to completely overwhelming one-page documentation (which is my favorite kind!)
My third and final gripe is with Julia’s package manager, Pkg. Pkg is great, I actually love it, and the fact that Pkg has its own REPL that you can enter from the Julia REPL with ] at anytime makes it that much sweeter.
However, these packages do have a fair bit of bugs, dependency issues, and random little quirks. Typically this is not too bad, but occasionally you’ll install a new module and end up rebuilding twenty different packages over and over again, and the process can get a bit annoying if you have to do it with a Genie web-app for example, not only committing the execution on your personal computer, but also doing the same on your server.
Although I will admit: these are fairly small gripes, and often don’t disturb the Julia ecosystem too much, I do feel like this is some information someone looking to try Julia out would like to know. Julia is great now, but with eyes into the future rather than the present, I could certainly see Julia taking off and becoming a core language among Data Scientists. A lot of people might not realize that Julia is still a baby, having just entered 1.0 last year. And with a language being so young, of course the packages are going to be scattered!
Since I have been working in Julia, Julia has already seen substantial package growth. I often find I think of something that I want to do, and there’s a package up on GitHub to do it exactly, which is awesome. The packages are expanding, and that truly is what makes a the language DS-viable for the future. For ML we have XGBoost.jl, Lathe.jl, MLBase, and Flux.jl:
Pkg.add("XGBoost")Pkg.add("Lathe")Pkg.add("Flux")Pkg.add("MLBase")
For stats we have StatsBase.jl, GLM.jl, Lathe.jl, and Distributions.jl:
Pkg.add("StatsBase")Pkg.add("GLM")Pkg.add("Lathe")Pkg.add("Distributions")
For Preprocessing we have MLDataUtils.jl, and Lathe.jl:
Pkg.add("MLDataUtils")Pkg.add("Lathe")
And with Julia’s superior package manager, development of said packages is surprisingly easy!
So I definitely encourage anybody who has an interest sparked(pun intended) in Julia to jump into it now, and help develop the language and the package base. I strongly believe that with its speed and use, we’ll see a rise in the use of Julia in corporate ecosystems around the globe. I’m very excited for the future of Julia, and It’ll probably always hold its place as my favorite language.