Could Pluto Be A Real Jupyter Replacement? | by Emmett Boudreau | Towards Data Science

Are you a Julia developer who is sick and tired of Jupyter Notebook always getting in the way of your programming?
Me niether.
Despite my overall lack of need for a new notebook system, as I don’t even use the one I have that often, I decided to take a look at Pluto.jl because of recommendation from a friend. Pluto.jl is a fully-featured web/markdown/code notebook that comes in the form of a simple Julia package. It features all of the great things that you might expect from your typical Jupyter notebook. For a while, I actually used a notebook written in Scala to write Scala, it made me miss Jupyter, and frankly, its replacement,
the Spylon kernel
put a new tombstone on Scala’s data analysis grave. All of this is to say, while I am a huge proponent of Julia, I am not sure that this is going to change my mind at all — and the theme of this could turn out to be more of a question,
Why not just use Jupyter?
Although I would say that I consider myself a skeptic to these kind of things, I certainly would like to give Pluto.jl a chance. Firstly, I wanted to see what some of the key differences are between Pluto and Jupyter. If there are improvements to be had by using Pluto, then it might end up being a better choice for me as a whole — because I write a lot of Julia.
The first place I decided to check for information was the Pluto.jl Github README markdown file, of course. It was at this moment I realized my criticism of Pluto.jl was completely unfounded. The notebook is actually incredibly smart, and is constantly analyzing the code you write. For example, if I were to have an out-of-state code that has not been ran with a dependency I am trying to use in it, Pluto will automatically run that code for me.
Furthermore, whenever a variable or function is changed, Pluto automatically runs every other code cell associated with it. This is convenient, and completely cuts out the go to the top of the notebook and spam shift+enter activity. It was a great change of pace not having to worry as much about the reproducability of my notebook, as well. Julia has great methods to managing packages, so as a result most of the time dependency issues will never be felt by another programmer. Adding to that, though is the effort that has been made to have code run the state of the kernel.
Taking a quote directly from the Pluto.jl README,
At any instant, the program state is completely described by the code you see.
This is a big problem that I have with. This is made even worse with JIT, which works a lot better when you don’t need to restart and run all cells all of the time. If you’d like to learn more about reproducible research and the fundamental problems I have ran into with Jupyter, here are two articles I have written on that exact topic!:
towardsdatascience.com
towardsdatascience.com
A great example of what makes this so cool for analytics is that I could be actively altering data in one cell while simultaneously visualizing it in another cell. Just a change to the way Jupyter handles kernels and states is actually incredibly refreshing. Another great thing is that the Pluto.jl interpreter is made to read Julia code — as in real Julia code. There are no .ipynbs, only .jl files. This means anything developed in Pluto can also be used across the entire spectrum of Julia, and worked on in any session by any programmer with the same code very easily.
I figured if I was going to give Pluto.jl a decent try, I might as well dive into a classic Data Science project with it. For my tech-stack I will be using DataFrames.jl data frames, Gadfly.jl for visualization, and Lathe.jl for statistical analysis and machine-learning — exciting! Of course, to try out Pluto, we are going to need to add it first:
julia>]pkg> add Pluto
After adding Pluto, it can be ran using the Pluto.run() function.
using Pluto;Pluto.run()
Whenever you startup Pluto for the first time, you will be greeted with this page. I think it is a little tedious that you have to manually enter a path into the text-box to open a notebook. The first thing I had to do getting into these notebooks was run pwd() to see where I am on my file-system.
Interestingly, I am in my .Julia directory located at ~/. This is unfortunate because I was hoping to load some .CSV data in. Let’s see if that changes when we save the notebook.
Good news!
So here are a few major concerns I have with getting a project done in this notebook. Firstly, whenever I ctrl+enter, it does not create a new cell. Secondly, stdout does not work in these notebooks. I cannot for the life of me understand why that is, but you cannot print() or println(). I digress, let’s actually get some data read in. Another cool package I recently picked up (Emmett went shopping) is PrettyTables.jl, so hopefully my data frames will at least look good.
This is going to take some getting used to. I do like the way the notebook looks, and there are certainly some aspects I am enjoying, such as the beautiful output.
I am hungry for a categorical problem, so among this small list of features the “ Precip Type” classification really caught my eye. For all I know, however, this data frame could only be a classification of rainy days. Fortunately, we can check the number of unique values in a given array using the Set type.
length(Set(df[Symbol("Precip Type")]))
Which returned 3. Of course, this is plenty of categories for a categorical problem, however, one of the categories happened to be null — meaning that we should probably be cleaning this data instead of dipping our hands deep into the machine-learning cookie jar. Fortunately, the “ Summary” feature had around 27 different classifications on this data-set with around 96,000 observations.
Set(df[Symbol(“Summary”)])
A great way to check for missing observations in the Julia language is to collect the missing observations and compare it to the original length of the data frame.
testmissings = collect(skipmissing(df[:Summary]))length(testmissings)
Good news — it’s the same number! I am no astronomer or weather man, but if I had to guess I would say that there is a pretty good chance that these descriptions could correlate with the humidity. Just to make sure, though, let’s filter the data and perform a test.
It was here where I really got annoyed with the notebook if I am being honest. A lot of things about it were just things that I certainly could not get used to. Easily the most aggravating thing was that I kept deleting three cells at a time trying to remove some text. After all of this time of using it, I still haven’t been able to find out how to add markdown.
I also ended up getting formally lost on how to fix a copy mistake I made where I accidentally altered my data frame. The notebook wouldn’t let me redefine a variable, and made it extremely tedious to work with this relatively basic code. Despite this, I did get a model fit — and it got 40 percent accuracy. After that I thought maybe these features weren’t as correlated as I previously thought... Their statistical tests weren’t showing very much significance, anyway.
Another big complaint I have with Pluto is the documentation browser, it might as well not even be there and at the very least allow one to run the regular help function. I think it is a great idea, however, that maybe just needs some ironing out — and possibly some NLP suggestive text.
I digress, overall I would say I am a fan of the project, but certainly not going to be using it anytime soon. I am not a big fan of notebooks in general, but they do have their uses! If I had to chose between Pluto.jl and Jupyter, I would probably be choosing Jupyter with IJulia. I would love to hope that this would change in the future, however. With a few more things here and there I am certain it could be a promising development environment. However, at this point in time I certainly cannot suggest this... It was oddly frustrating, and to be blunt — it felt like the IDE got in my way more than it helped me.
Hopefully Pluto.jl will be better in the future. There is a chance too that I am not the target audience, or that others might enjoy them a lot more than me. There is certainly some great ideas and genuinely really cool stuff going on under the hood though, and I am somewhat excited to see what will be like in the future. For now though, I will definitely be sticking to Atom and IJulia.