Change Column Data Type in Python Pandas  | Towards Data Science

Working with data is rarely straightforward. Mostly one needs to perform various transformations on the imported dataset, to make it easy to analyze.
In all of my projects, pandas never detect the correct data type for all the columns of the imported dataset. But at the same time, Pandas offer a range of methods to easily convert the column data types.
Here, you will get all the methods for changing the data type of one or more columns in Pandas and certainly the comparison amongst them.
Throughout the read, the resources are indicated with 📚, the shortcuts are indicated ⚡️ with and the takeaways are denoted by 📌. Don’t forget to check out an interesting 💡 project idea at the end of this read.
You can quickly follow along with this Notebook 📌.
To make it easier to understand for you, Let’s create a simple DataFrame.
Using this example, it will be much easier to understand — how to change the data type of columns in Pandas.
This method is used to assign a specific data type to a DataFrame column.Let’s assign int64 as the data type of the column Year. With the commands .head() and .info(), the resulting DataFrame can be quickly reviewed.
df1 = df.copy()df1["Year"] = df1["Year"].astype("int64")df1.head()df1.info()
Similarly, the column can be changed to any of the available data types in Python. However, if the data type is not suitable for the values of the column, by default this method will throw a ValueError.
Hate ValueErrors???
Pandas have the solution. The 2nd optional argument in this method ì.e. errors gives you the freedom to deal with the errors. This option defaults to raise, meaning, raise the errors and do not return any output. Simply, assign ‘ignore’ to this argument to ignore the errors and return the original value.
df1["Car"] = df1["Car"].astype("int64", errors='ignore')
❓ Want to change the data type of all the columns in one go ❓
⚡️ Just pass the dictionary of column name & data type pairs to this method and the problem is solved.
df1 = df1.astype({"Year": "complex", "Rating": "float64",\                  "Car": 'int32'}, errors='ignore')
Much simpler, assign a single data type to all the columns by directly passing the data type in astype() , just like the below example.
df1 = df1.astype("int64", errors='ignore')df1.head()df1.info()
As shown in the above picture, the Dtype of columns Year and Rating is changed to int64, whereas the original data types of other non-numeric columns are returned without throwing the errors.
📚 pandas.DataFrame.astype()
Well well, there is no such method called pandas.to_DataType(), however, if the word DataType is replaced by the desired data type, you can get the below 2 methods.
This method is used to convert the data type of the column to the numerical one. As a result, the float64 or int64 will be returned as the new data type of the column based on the values in the column.
df2 = df.copy()df2["Rating"]=pd.to_numeric(df2["Rating"])df2.info()
Here the column gets converted to the DateTime data type. This method accepts 10 optional arguments to help you to decide how to parse the dates.
df2 = df.copy()df2["RealDate"] = pd.to_datetime(df2["Service"])df2.info()
❓ Need to change the data types of multiple columns at a time ❓
⚡ ️Use the method .apply()
df2[["Rating", "Year"]] = df2[["Rating",\                               "Year"]].apply(pd.to_numeric)
Similar to pandas.DataFrame.astype() the method pandas.to_numeric() also gives you the flexibility to deal with the errors.
📚 pandas.to_numeric()📚 pandas.to_datetime()
This method will automatically detect the best suitable data type for the given column. By default, all the columns with Dtypes as object will be converted to strings.
df3 = df.copy()dfn = df3.convert_dtypes()dfn.info()
As per my observation, this method offers poor control over the data type conversion
📚 pandas.DataFrame.convert_dtypes()
Summing up,
In this quick read, I demonstrated how the data type of single or multiple columns can be changed quickly. I frequently use the method pandas.DataFrame.astype() as it provides better control over the different data types and has minimum optional arguments. Certainly, based on analysis requirements, different methods can be used, such as converting the data type to datetime64(ns) the methodpandas.to_datetime() is much straightforward.
Become a Medium member today & get ⚡ unlimited ⚡ access to all the Medium stories.
Sign up here and Join my email subscriptions
When you sign-up here and choose to become a paid Medium member, I will get a portion of your membership fee as a reward.
It can be a good idea to start with a new dataset, assess and clean it by practicing Data Wrangling techniques and store it in a SQL Database to finally visualize it in Power BI.
Additionally, this project idea can be implemented with the resources given in it. As I always say, I am open to constructive feedback and knowledge sharing through LinkedIn.