The Struggles I Had When Switching to Python | Towards Data Science

When I started learning data science, I started learning with R. I was a statistics major, and in the world of academia, there is a heavy preference of R over Python. There isn’t anything inherently wrong with learning one or the other, but I had invested several years mastering R before I joined the workforce and Python became a requirement.
I went into my first job with expectations of being able to build models and automate everything using the tools I had in R. My dreams of predicting and eliminating all the work were quickly dashed when I found out that nobody at my job knew much of anything about R. This meant that anything I created would have no support from the IT department or any of my coworkers if I ever moved on. I could automate my own work but if I started moving my way up the corporate ladder, nobody would be able to do my job the way I was.
As time went on, I started working more with the data science team and their IT support team. They introduced me to this wonderful language called Python. As with any new language or tool, there was a bit of a learning curve that came with it. I struggled a lot. Here are some of the things about Python that are different that really tripped me up.
Yup, first on the list is my inability to save data frames after modifying them. In R, that might look something like this (when using dplyr):
dataframe <- dataframe %>%    select(-col1)
In case you don’t know dplyr well, this code simply drops the column named col1. Since the data frame, dataframe, and function, select, are physically separate from each other, I wouldn’t really expect the output to be saved, so without thinking I assign it to dataframe to overwrite it without the col1 column.
To do the same thing in Python, I used to do something like this:
# The wrong way!>>> dataframe.drop('col1')
The first couple times I did this I would spend the following hour or two muttering curse words under my breath, wondering why col1 was still part of my data frame. You might ask, why wouldn’t I just assign the output to the original variable? Well in python we get these nifty things called methods, like .drop(). They are physically attached to the object they are acting upon. To the Python newbie, we want to believe that they will just change the object and we don’t have to save anything. Alas, the wonderful world of Python doesn’t work like that.
# This actually SAVES the modification>>> dataframe = dataframe.drop('col1')
So there you have it, my first struggle switching from R to Python:
I never remember to save my Python data frames after modifying them
This in and of itself isn’t the most confusing thing in the world, just different from the way R handles things. But those familiar with Python data frame methods might know a little trick that will not require you to save your data frame after dropping a column...
Before all you python experts get all up in arms about how you can in fact modify data frames in place, yes, I know about that now. If I really didn’t want to reassign the variable I could just add the inplace=True argument to the .drop() method. Nice and flexible, problem solved, right?
# We don't need to save changes here>>> dataframe.drop('col1', inplace=True)
Sure that works, but what happens when a method doesn’t have an inplace argument or I’m using a function instead of a method? Being new to the language, having options doesn’t exactly make things easier to remember or use.
In terms of having the ability to complete operations in place, I am all for it. Heck, it is even well implemented in the sense that it isn’t the default behavior to modify data frames in place. But that is for data frames.
What about lists? If I create a list of strings, many string methods, like .append, .insert, .pop, and .remove modify the list in place, with no need to save the result. Take this example where we remove 9 from the list (because 7 ate 9, obviously 😏).
>>> x = [7, 8, 9]>>> x.remove(9)>>> print(x)[1] [7, 8]
What makes it worse is that in a typical way Python is taught, you will learn about lists and how they work before you learn anything about a data frame. The confusion comes because in Python, lists are not analogous to data frame columns the same way R vectors are analogous to data frame columns.
So that was my second big struggle when learning Python:
Some objects (lists) are modified in place by their methods, while others (DataFrames) you need to save in order to modify them through their object methods
It is one of those things that needs to be learned, to some extent is just memorizing how each different object works. I guess some things just look different when coming from R to Python. Which brings me to my next struggle...
In R, we often, somewhat carelessly, load all of our packages into our workspace. R will try to find a function in a loaded library by name, and execute if it exists.
Sometimes you load a package and it will “mask” a function from another package. If you aren’t familiar, this just means that it will use the version of the function from the latest package loaded that has the function. If you want to specify a different library’s version of the function, you can do that. Just put the package name before the function, separated by a double colon (::). Here are some examples:
# Uses the "base" library version of filterfilter(x)# Uses the "dplyr" library version of filterlibrary(dplyr)filter(x)# Uses the "base" version, even if it's maskedbase::filer(x)
That’s really all there is to functions in R. It uses the latest loaded version unless you tell it otherwise.
In Python, things are a little different. There are 3 main ways functions can be called. I use the term function a bit loosely here because to an R user, methods look a lot like functions.
A function name, like with len()
A function from a module, like with pd.DataFrame()
A method from a object class, like df.drop()
You might see them in code as follows.
# A functionlen(df)# A function from a modulepd.DataFrame(myDict)# An object class methoddf.drop('col1')
How is this confusing? Well, in Python you need to know how to call what would just be a function in R. In R, if you want to know how many columns a data frame has, you would just use the ncol() function. In Python, there isn’t a built in function to do this. Instead, you can use the len() function on the .columns attribute of a DataFrame.
# Rncol(df)# Pythonlen(df.columns)
Another example how Python functions can be confusing to R users is finding the number of rows in a DataFrame. In R, we once again use a single function, nrow(). In Python, we call len() on the entire data frame instead of its columns attribute.
# Rnrow(df)# Pythonlen(df)
So here we used the exact same len() function to get different pieces of information about our data frame in Python. The drawback is having to remember what to pass as the argument to len(). R keeps it simple. If we want the number of columns, use ncol(). Number of rows? That’s right, nrow().
Let’s do one more example to show the difficulties of knowing where a function lives in Python. Say we want to know both number of rows and number of columns. In R, the dim() function will serve us well. It will return a vector with the number of rows and number of columns. In order to do this in one call in Python, we need to use the .shape attribute. .shape will return a tuple with the number of rows and columns.
# Rdim(df)# Pythondf.shape
So if you’re still with me at this point, we’re not even using a method or function in Python, we’re just looking up an attribute of the data frame. So there you have it:
R has functions to complete tasks, where in Python you may have to use a function, an object method, an object attribute, or some combination to do the same thing
It isn’t wrong, but makes it so much harder to remember how to get information for someone who is used to just using a function.
Let’s try something else. Say we want to create a dataframe, a very common task in both languages. In R, we just use the dataframe function. In python, we have to specify where python should look for the dataframe function, specifically in the pandas library.
# Rx <- data.frame(X = c(1, 2, 3, 4, 5))# Pythonx = pd.DataFrame({X: [1, 2, 3, 4, 5]})
This is different because we HAVE to specify which package the function is from in Python. R would just use the latest loaded package that contains the function. The benefit of python’s implementation is that you always know the package the function is coming from. This is different, and causes some confusion if you aren’t used to needing to specify which package your function is coming from.
Sometimes in Python you need to specify what module your function is coming from, where R will just guess which package to use
Now the final thing that always got me about Python functions was object methods. They aren’t inherently bad, but they can be confusing for us R users. As we mentioned earlier, they sometimes act on the object in place, and other times they need to be saved over the original object to modify it. This section is about how functions are used, and that is difficult to remember in python.
In R, we call functions by their name, adding objects and options as arguments. Using the pipe operator %>%, you can define an object as the first argument of a function. Each action is telling R to “do this” and each pipe operator is saying “then.” When we string them together, we can complete multiple actions at the same time in a highly readable format.
df %>% select(-col1) %>% summarize(avg_col1 = mean(col1))
Using pipes is the closest analogy I can come up with in R to how object methods work in Python. They can both be strung together, and each step acts as a “do this then” set of instructions. The way they get written is just a little different.
df.drop('col2', 1).mean()
This statement above takes df then drops col2 then gets the average of the remaining values. Similar behavior, but different implementation.
Python methods can be strung together like how we would use the pipe operator in R, just a little less easy to read
No really, it may sound weird, but going to Python I never wanted to look at what errors were saying or put effort into figuring them out. I already mastered R and when things don’t run, I usually already knew what to look for without reading the errors.
After gaining some experience with this, it became much easier to help other R to Python converts. The errors would tell me which of the struggles they were dealing with.
Read your errors carefully and understand what code it is referring to and what it is saying is wrong
In Python, I got a lot more errors than I would in R and it was frustrating. But errors are extremely helpful in learning a language and mastering it. Let’s take this one for example:
>>> '2' + 2Traceback (most recent call last):  File "<stdin>", line 1, in <module>TypeError: Can't convert 'int' object to str implicitly
Errors in python are helpfully labelled for you. The very first part of the error will say Traceback, usually with some text with a file and line number. Not always the most helpful for the beginner for more complex errors, but remember it is there.
The next part of the error text is TypeError. This is a category of error. There are many types of errors in Python (see a whole bunch of them in the official Python documentation). This one is saying something about the types of objects used in the code is causing the error.
The next part of the error gives you more details about the error that was encountered. Here we can see that python doesn’t know how to add an int object to a str object. From here we can easily fix our code. If the goal was to concatenate two strings together, we can add single quotes to the second 2.
>>> '2' + '2'[1] '22'
If the goal was to add two numbers together, we need to eliminate the single quotes on the first one so that it has the correct type.
>>> 2 + 2[1] 4
Either way, reading the error helps us know that the objects need to be of the same type for our code to work.
Not sure what an error means? Search it up online. Some errors are more complex than others and some may be due to some other part of your code. Break it down as simple as you can and work through it piece by piece.
You’ll master python just as quickly as you mastered R in no time. There are many who have come before you who have experience going from one language to another.
Let’s recap the lessons I learned while learning Python as an R user:
In Python, some methods modify objects in place (like lists methods) while others need to save the output over the original variable to modify it (like data frame methods)
R uses functions for almost everything while Python uses a mix of functions, object methods, and object attributes to do the same things
R generally guesses which library your function is from. Python generally needs to be told where to look
R can use the pipe operator %>% to string operations together. Python object methods can sometimes be strung together to achieve the same result
R and Python both provide errors when your code fails. These contain valuable information and learning to read and correct them is the only way to truly master the language
If all else fails, ask someone more experienced on your team or in your class what you are doing wrong. If that doesn’t work, ask the internet. There are tons of free and paid resources out there. Check out some of the ones below:
w3schools.com — a comprehensive knowledge base of Python, R, SQL, and other programming and scripting languages. It is free to everyone and has interactive examples for more languages and concepts.
Datacamp.com (PAID link) — Interactive programming tutorials, mainly focusing on R and Python for data analysis. They offer the first chapter of every course for free, so go ahead and try out their Python for R Users course!
Python for R Users (PAID link) — Written specifically get you up to speed on Python, this book focuses on actionable code and performing the analysis you already know in Python!