Using the Pandas Append Function for Dataframes | by Matt Przybyla | Towards Data Science

DatasetPandasAppendTutorial CodeSummaryReferences
Dataset
Pandas
Append
Tutorial Code
Summary
References
The dataset used in this analysis and tutorial for the pandas append function is a dummy dataset created to mimic a dataframe with both text and numeric features. Feel free to use your own csv file with either or both text and numeric columns to follow the tutorial below.
Pandas [2] is one of the most common libraries used by data scientists and machine learning engineers. It is mainly used in the exploratory data analysis step of building a model, as well as the ad-hoc analysis of model results. It also contains several functions, including the append function.
While I previously wrote an article [3] about the pandas query function that referenced adding multiple rows, I thought it would be useful to highlight the benefits of the append function itself. The append function works by adding new rows to an existing dataframe. The rows are created by utilizing the Series function, which stores values that will then be appended to the order of columns that are listed in the index argument.
Some benefits of the append function include:
works with pandas dataframe
works in conjunction with pandas Series function
quick method of adding new data to existing data
In the code snippet below, the dummy data was missing some information. In order to solve this issue, I looked to the append method. Using the append function, I could easily add new rows without losing the data and integrity of the original dataframe. First, I imported the pandas library and created the alias ‘pd’ — this alias helped to use the pandas library in a short-hand format. Next, I read in my dataframe from an existing csv file, referencing the location and path that it is on my local drive. I printed out the dataframe to see what the first five rows looked like. The next step is where I setup up the code for the append function. Using the Series function to, in order, place values — either text or numeric in this case as a list, and then indexing it on the respective columns itself, the new rows were written out and saved as ‘rows’.
The actual append function itself is fairly simple; passing the rows as the first argument, and then setting the ‘ignore_index’ to ‘True’, the index labels were not used. Lastly, I checked to see if my rows were made correctly by using the tail function that returns the dataframe with the last three rows from the append code.
Here is a screenshot of all the code I used below:
Here is the code [5] for easy referencing. Feel free to import your own data and reference your own file pathway, as well as change the data values in the Series to see how appending works in your python environment or jupyter notebook.
# All the python code below for use:# import libraryimport pandas as pd# read in your dataframedf = pd.read_csv('/Users/data.csv')# print out your first five rowsdf.head()# write out new rowsrows = [pd.Series([200, 200, 30,'Yellow','Label_1'], index=df.columns),pd.Series([200, 120, 40,'Red','Label_2'], index=df.columns),pd.Series([120, 40, 200,'Red','Label_3'], index=df.columns)]# append the multiple rowsnew_df = df.append(rows, ignore_index=True)# check the newest 3 rows you madenew_df.tail(3)
To see my code in python-format rather than a screenshot or written out, here is the embedded gist [5]:
The append function is a simple and fast way to add new data to your dataframe. The Series function was also referenced in order to save the rows that would later be appended. I hope this tutorial is helpful to you when creating your data and performing exploratory data analysis on your dataframe, before ultimately creating a data science and machine learning model. Thank you for reading! As mentioned above, please feel free to read my similar article that goes over that pandas query function.
[1] T.Lundqvist, Unsplash (2017)
[2] pandas, pandas (2020)
[3] M.Przybyla, article (2020)
[4] M.Przybyla, screenshot (2020)
[5] M.Przybyla, gist (2020)