How To Change DataTypes In Pandas in 4 Minutes | by Benedikt Droste | Towards Data Science

When I worked with pandas for the first time, I didn’t have an overview of the different data types at first and didn’t think about them any further. At the latest when you want to do the first arithmetic operations, you will receive warnings and error messages, so you have to deal with the data types. We will have a look at the following commands:
1. to_numeric() — converts non numeric types to numeric types (see also to_datetime())
2. astype() — converts almost any datatype to any other datatype
We will first look at to_numeric()which is used to convert non-numeric data. astype() is the Swiss army knife which can convert almost anything to anything. Let ́s start!
We start with creating a dataframe:
>>> amount_list = [1,2,'3',2,'4',5]>>> cost_list = [1,2,'3',2,'4',5]>>> date_list = ['2020-01-03', '2020-02-03','2019-12-12','2019-11-14','2020-01-02','2020-02-03']>>> category_list = ['XZ',1,3,'PT','LM',4]>>> df = pd.DataFrame({'Date':date_list,'Items': 'Car Saxophone Curler Car Slingshot Duff'.split(), 'Customer': 'Homer Lisa Marge Lisa Bart Homer'.split(), 'Amount': amount_list, 'Costs': cost_list, 'Category':category_list})>>> df         Date      Items Customer Amount Costs Category0  2020-01-03        Car    Homer      1     1       XZ1  2020-02-03  Saxophone     Lisa      2     2        12  2019-12-12     Curler    Marge      3     3        33  2019-11-14        Car     Lisa      2     2       PT4  2020-01-02  Slingshot     Bart      4     4       LM5  2020-02-03       Duff    Homer      5     5        4
We have six columns in our dataframe. The first column contains dates, the second and third columns contain textual information, the 4th and 5th columns contain numerical information and the 6th column strings and numbers.
Let’s check the data type of the fourth and fifth column:
>>> df.dtypesDate        objectItems       objectCustomer    objectAmount      objectCosts       objectCategory    objectdtype: object
As we can see, each column of our data set has the data type Object. This datatype is used when you have text or mixed columns of text and non-numeric values. We change now the datatype of the amount-column with pd.to_numeric():
>>> pd.to_numeric(df['Amount'])0 11 22 33 24 45 5Name: Amount, dtype: int64
The desired column can simply be included as an argument for the function and the output is a new generated column with datatype int64. If we had decimal places accordingly, Pandas would output the datatype float. It is important that the transformed column must be replaced with the old one or a new one must be created:
>>> df['Amount'] = pd.to_numeric(df['Amount'])>>> df.dtypesDate        objectItems       objectCustomer    objectAmount       int64Costs       objectCategory    objectdtype: object
With the .apply method it ́s also possible to convert multiple columns at once:
>>> df[['Amount','Costs']] = df[['Amount','Costs']].apply(pd.to_numeric)>>> df.dtypesDate        objectItems       objectCustomer    objectAmount       int64Costs        int64Category    objectdtype: object
That was easy, right? You probably noticed we left out the last column, though. There are obviously non-numeric values there, which are also not so easy to convert. If we just try it like before, we get an error message:
>>> pd.to_numeric(df['Category'])Traceback (most recent call last):  File "pandas\_libs\lib.pyx", line 1897, in pandas._libs.lib.maybe_convert_numericValueError: Unable to parse string "XZ"During handling of the above exception, another exception occurred: [...]ValueError: Unable to parse string "XZ" at position 0
to_numeric()accepts an error argument. We can use corce and ignore. With coerce all non-convertible values are stored as NaNs and with ignore the original values are kept, which means that our column will still have mixed datatypes:
>>> pd.to_numeric(df['Category'], errors='coerce')    0    NaN1    1.02    3.03    NaN4    NaN5    4.0Name: Category, dtype: float64>>> pd.to_numeric(df['Category'], errors='ignore')    0    XZ1     12     33    PT4    LM5     4Name: Category, dtype: object
As you may have noticed, Pandas automatically choose a numeric data type. In most cases, this is certainly sufficient and the decision between integer and float is enough. However, sometimes we have very large datasets where we should optimize memory usage. This can be achieved with downcasting:
>>> pd.to_numeric(df['Amount'],downcast='integer')         0    11    22    33    24    45    5Name: Amount, dtype: int8
In this example, Pandas choose the smallest integer which can hold all values.
Using the astype() method. you can specify in detail to which datatype the column should be converted. The argument can simply be appended to the column and Pandas will attempt to transform the data. We can take the example from before again:
>>> df['Amount'].astype(int)         0    11    22    33    24    45    5Name: Amount, dtype: int32>>> df['Amount'].astype(float)         0    1.01    2.02    3.03    2.04    4.05    5.0Name: Amount, dtype: float64df['Amount'].astype(str) 0    11    22    33    24    45    5Name: Amount, dtype: object
You can define the data type specifically:
>>> df['Amount'].astype(np.int64)         0    11    22    33    24    45    5Name: Amount, dtype: int64
Also with astype() we can change several columns at once as before:
>>> df[['Amount','Costs']].astype(int)    Amount  Costs0       1      11       2      22       3      33       2      24       4      45       5      5
A difference to to_numeric is that we can only use raise and ignore as arguments for error handling. Raise is the default option: errors are displayed and no transformation is performed. With ignore errors will be ignored and values that cannot be converted keep their original format:
>>> df['Category'].astype(int, errors='ignore') 0    XZ1     12     33    PT4    LM5     4Name: Category, dtype: object
We have seen how we can convert columns to pandas with to_numeric() and astype(). To_numeric() has more powerful functions for error handling, while astype() offers even more possibilities in the way of conversion. It is important to be aware of what happens to non-numeric values and use the error arguments wisely.
If you enjoy Medium and Towards Data Science and didn’t sign up yet, feel free to use my referral link to join the community.