Tips for Data Mapping and Replacing with Pandas and NumPy | by Chi Nguyen | Towards Data Science

In order to summarize main characteristics, spot anomalies, and visualize information, you should know how to rearrange and transform datasets. In other words, transforming data helps you play with your dataset, make sense of it, and gather as many insights as you can. In this article, I will show you some of my commonly used methods to play with data, and hope this would be helpful.
I will create a simple score dataset, which includes information about different classes’ grades.
Input: info = {'Class':['A1', 'A2', 'A3', 'A4','A5'],        'AverageScore':[3.2, 3.3, 2.1, 2.9, 'three']}data = pd.DataFrame(info)
Output:
As the Average Score of Class A5 in our data is a string object, I want to replace it with a corresponding number for easier data manipulation. The procedure to replace “three” with “3” is as following:
Input:data['AverageScore'] = data['AverageScore'].replace(['three'],'3')data['AverageScore'] = data['AverageScore'].astype('float')
Above is a simple case of replacing a value with another value in a specific column. For more reference, below are some more examples:
If you want to replace multiple values with multiple new values for a specific column, use this:
data['column name'] = data['column name'].replace(['1st old value','2nd old value',...],['1st new value','2nd new value',...])
In case you want to replace multiple values with a new value for a specific column:
data['column name'] = data['column name'].replace(['1st old value','2nd old value',...],'new value')
Suppose I want to add a column evaluating if the grade is good or bad, I will write down a mapping of each grade to the kind of corresponding assessment comment. This can be done in 4 different ways by using bins, pandas cut()function, np.digitize()function, np.select()or your own function with if . Let’s see how I am dealing with each method.
You only need to define your limits and category names, then apply them to a numeric column using pandas.cut(). In this case, I will define three intervals, which correspond to four bin limits, to evaluate each class’s score. In details:
0 < AverageScore ≤2: Below average
2 < AverageScore ≤ 3.1: Average
3.1 < AverageScore ≤5: Good
Input:bins = [0, 2, 3.1, 5]comment = ['Below Average', 'Average', 'Good']data['Evaluation'] = pd.cut(data['AverageScore'], bins, labels=comment)
Output:
The idea of Numpy.digitize() is to get the indices of the bins to which each value belongs. Similarly in this case, you can also define your bin boundaries and category names like the case with pd.cut(). What difference is to create an additional dictionary and use that dictionary to map the category names.
Input:bins = [0, 2, 3.1, 5]comment = ['Below Average', 'Average', 'Good']dictionary = dict(enumerate(comment, 1))data['Evaluation2'] = np.vectorize(dictionary.get)(np.digitize(data['AverageScore'], bins))
Output:
Numpy.select(condition list, choice list)method returns an array drawn from elements in choice list , depending on condition in the list. Therefore, we can easily obtain the desired values by stating the evaluation conditions, which are three intervals with limitation, and the corresponding choice list values. See the code below for reference.
Input:evaluate = [data['AverageScore'].between(0,2), data['AverageScore'].between(2,3.1), data['AverageScore'].between(3.1,5)]values = ['Below Average', 'Average', 'Good']data['Evaluate3'] = np.select(evaluate, values, 0)
Output:
You can even write your own function to apply. The logic is actually quite simple, as you can see:
Input: def evaluate(values):    if values >=0 and values <=2:        return 'Below Average'    elif values >2 and values <= 3.1:        return 'Average'    else:         return 'Good'#Applying functiondata['Evaluation4'] = data['AverageScore'].apply(lambda x: evaluate(x))
Output:
As you can see, all of the four solutions produce the same results. I just want to provide you with different approaches to this problem so that in the future you can apply these data manipulation flexibly in different scenarios.
Above are some of my tips for mapping and replacing data. What I want to convey here is, in some cases when you do exploratory data analysis, you may have to deal with multiple unexpected situations, therefore it’s useful to solve problem with more than one approach. Hope that my article can help you out someday. Good luck!