An Easy Way to Replace Values in a Pandas DataFrame | by Byron Dolon | Towards Data Science

The Pandas library is widely used as a data processing and analysis tool for cleaning and extracting insights from data.
When working with data, you’ll often want to edit or remove certain pieces of data as a part of your pre-processing steps. This might involve creating new columns from existing ones, or modifying existing columns so they fit within a framework that’s easier to work with. To do this, Pandas provides a wide range of methods that you can use to work with columns of all data types in your DataFrames.
In this piece, let’s take a look specifically at replacing values and sub-strings within columns in a DataFrame. This can come in handy both when you want to replace each value in a column or when you only want to edit part of a value.
If you want to follow along, download the dataset here and load the code below.
import pandas as pddf = pd.read_csv('WordsByCharacter.csv')
Let’s get to it!
The replace method in Pandas allows you to search the values in a specified Series in your DataFrame for a value or sub-string that you can then change. First, let’s take a quick look at how we can make a simple change to the “Film” column in the table by changing “Of The” to “of the”.
# change "Of The" to "of the" - simple regexdf["Film"].replace("Of The", "of the")  # won't work if you do thisdf["Film"].replace("Of The", "of the", regex=True)
Above, all we did was pass in the sub-string we’re looking for as the first argument to the replace method, and then pass in the value we wanted to replace it with as the second argument. You’ll see that we also had to pass regex=True for the operation to work. Otherwise, the replace method would only change column values that were “Of The”, as it would only match entire values.
You could accomplish the same thing we did above by matching the exact string and providing the whole value that you want to change like below.
df["Film"].replace("The Fellowship Of The Ring", "The Fellowship of the Ring")
The problem with this method is that you’ll have to match values on a case-by-case basis. That is, you’ll need to pass each value you want to change, as well as what value you want it to be changed to. For some occasions, it may be easier to use just find and replace everything that matches a regex you define.
However, in those cases that you want to change different values to different replacement values, you won’t have to make multiple calls to the replace method. Instead, you can simply pass a dictionary where the keys are column values you want to search for and values are what you want to replace the original values with. A simple example can be seen below.
# you can do multiple replacements in within one call of the replace method by creating a mapping dictionary# very scalable wooreplacement_mapping_dict = {    "The Fellowship Of The Ring": "The Fellowship of the Ring",    "The Return Of The King": "The Return of the King"}df["Film"].replace(replacement_mapping_dict)
Before, we simply passed regex=True so that we could match on sub-strings. Let’s take this a step further and actually write some regex to match on. You’ll see in the “Chapter” column, the chapter names are preceded by the chapter number and a colon. Let’s do a little bit of data cleaning and remove these using regex in the replace method.
# remove number number colon from "Chapter" - bit more complex regex example# regex is just <= 3 repetitions of any numeric character (0-9) immediately followed by a colon and a spacedf["Chapter"].replace(r"\d{,3}: ", "", regex=True)
All we did is pass r"\d{,3}: " to match three repetitions or less of a numeric character followed by a colon and a space (this would catch “01: ”, “02: ”, ..., “100: ”, and so on). Whenever this is found inside a column value, it would be removed from the string, as the second argument we passed was a blank string.
Alternatively, if you didn’t want to pass regex=True every time you called the replace method, you could instead accomplish the same result from above as follows:
# alternate way of doing the same thing (instead of having to pass regex=True at the end)# if you had multiple things you want to replace you can put them all in a listregex_list = [r"\d{,3}: ", r"The "]df["Chapter"].replace(regex=regex_list, value="")
The result is the same as before, with a few differences that make this method a bit more clear. First, if you had multiple regex that you wanted to match on, you can define them in a list and pass it as a keyword argument in the replace method. Then, you need to explicitly pass another keyword argument value to define what you want the replacement value to be. Someone (including your future self) looking at your code will likely have a very easy time understanding what it does and expanding on it.
And that’s all!
This is a fairly common process when it comes to cleaning data, so I hope you found this quick introduction to the Pandas replace method useful in your own work. There’s a lot more that Pandas has to offer, so feel free to check out some of my other pieces on it for some more hands-on examples on what you can easily do with Pandas.
Good luck on your data journeys!
More by me:- Check for a Substring in a Pandas DataFrame- Conditional Selection and Assignment With .loc in Pandas- 2 Easy Ways to Get Tables From a Website With Pandas- 5 (and a half) Lines of Code for Understanding Your Data with Pandas- Top 4 Repositories on GitHub to Learn Pandas