Pyrentena: a dataset per week
By: Letícia Gerola
Tags: Data Science, Python

I used the ‘train_csv’ file for this analysis, which contained the following columns:

# verifiyng all columns

df.col'umns Index(['Unnamed: 0', 'NU_INSCRICAO', 'NU_ANO', 'CO_MUNICIPIO_RESIDENCIA',

'NO_MUNICIPIO_RESIDENCIA', 'CO_UF_RESIDENCIA', 'SG_UF_RESIDENCIA',

'NU_IDADE', 'TP_SEXO', 'TP_ESTADO_CIVIL',

...

'Q041', 'Q042', 'Q043', 'Q044', 'Q045', 'Q046', 'Q047', 'Q048', 'Q049',

'Q050'],

Lots of internal codes naming the columns, not really intuitive. I started by DataFraming the columns that could possibly answer my questions — and making them more appealing to me:

# creating a dataset with columns that I'm interested in # Q025 stands for internet at home and Q002 stands for mother schooling dataset1 = df[['TP_SEXO', 'NU_IDADE', 'NU_NOTA_MT', 'Q025', 'Q002', 'TP_COR_RACA']] # changin the name of the columns dataset1.columns = ['GENDER', 'AGE', 'MT_GRADE', 'INTERNET', 'MOM_SCHOOL', 'RACE']

df.head() with names of columns altered

That’s better. Since my goal here was to do an exploratory analysis and not really machine learning from it, I decided to change the variables on the columns INTERNET, MOM_SCHOOL and RACE to string, so the conclusions would come out easier. Remember: if you are building a machine learning algorithm, do not make this change — we need all the data so stay numerical!

# race dataset1['RACE'] = dataset1['RACE'].map({0:'Na', 1:'Caucasian', 2:'Black', 3:'Brown', 4:'Yellow', 5:'indigenous'})

To make that change, you have to use the .map and than assign the modifications. I applied the same method on INTERNET and MOM_SCHOOLING.

df.head() with all of the wors on str format

Great! Now we can take a better look at what is happenning here. I’ve started looking at our number for GENDER and RACE and plotting a graph crossing this two variables:

# gender percent gender_data =(dataset1['GENDER'].value_counts()/dataset1.shape[0])*100

print(gender_data) F 61.84268 M 38.15732 Name: GENDER, dtype: float64

Graph 1: gender x race

As we can see, there is a majority of women taking the test (61.8%). Nearly half of the students that applied consider themselves as ‘brown’. Caucasian comes in second place with 33%.

Now It was time to visualize how internet and schooling level of the mother looked like:

Graph 2: internet

Graph 3: Mom aschooling

I’m really unhappy about this third graph. I wanted the bars to be sorted from the least educated to the most educated, also with some sort of gradient pallet for better visualization. Didn’t know how to that on Seaborn, googled some possible solutions on StackOverflow but none of them worked! If anyone knows how to fix this, please, do tell me.

Mooving on, the graphs give us a good idea of how our data is distributed when it comes to these two variables. I’ve decided to look numericly at each one of them by executing the following code:

schooling = (dataset1[‘MOM_SCHOOL’].value_counts()/dataset1.shape[0])*100 print(schooling) No graduation 28.281136

F1 not completed 19.133285

F2 not completed 15.746540

HS not completed 15.171158

Graduation 7.021122

Not known 5.243991

post graduated 4.719592

Never studied 4.683176

Name: MOM_SCHOOL, dtype: float64

Analysing graphs and outputs, we can see that:

-34% dropped out of school before High School.

-15% reached High School, but did not finish it.

-28% finished High School.

With a dataset.describe() I could see the statistics for the numerical variables of this dataset:

dataset1.describe()

We can see that the average grade was around 482 from a total of 1000. This means that the majority of students did not score half of the math test. Also, the standart deviation for math test is incredibly high, probably because of the way ENEM grades are calculated: less students do well in math so their score goes up, furthering them from the mean.

Crossing data

I decided to boxplot the math grade and our mom’s school level and also math grade and internet to take a closer look of how they possibly correlate:

# Box plotting math grade x mother schooling plt.figure(figsize=(13,5)) sns.boxplot(x=dataset1['MOM_SCHOOL'],y=dataset1['MT_GRADE'],

order=['Not known','Never studied','F1 not completed', 'F2 not completed', 'HS not completed','No graduation', 'Graduation'])

plt.xlabel('Level of mother schooling')

plt.ylabel('Math grade')

plt.show()

Graph 4: box plot math x mom school level

You see, on boxplot the ‘order by’ function worked(?!).

Graph 5: math x internet at home

Considering the questions that were asked on the beginning of this article, we can make a few observations:

Math grade increases along with the level of mother schooling. Students whose mother did not go through with high school/graduation have worse grades. Since correlation does not mean causation, we cannont say that having a mom with a degree improves your chance of doing better at math on Enem. Statiscal tests have to be conducted to verify the intensity of this correlation. Having internet at home also seems to have a positive impact on the math grade of the students. Again, statiscal tests have to be conducted to verify the intensity of this correlation.