2024-01-30
1 / 12
readme.md
Table of contents
Programming For Datascience
Programming For Datascience
Table of Contents
1.
Introduction 2. Business
Understanding 3. Data understanding & preparation
Name: Marjolein Gerdes
Datum: 30-01-2024
Machine Translated by Google
readme.md 2024-01-30
2 / 12
4.3 Object Oriented Programming
3.1 First exploration based on the use of the Dataframe Tool 3.2 Second
exploration based on the use of the Dataframe Tool 4. Python code 4.1
Project structure
4.2 Dataframe Tool
4. Occupational status
The project involved conducting a survey to collect data, organizing the data, and using machine learning
techniques to create a predictive model that can determine whether a person should seek professional help based
on their answers to the survey questions (Social media and mental health, 2023).
This dataset was originally collected for a data science and machine learning project that aimed to investigate the
possible correlation between the time an individual spends on social media and its impact on their mental health.
3. Relationship status
1. Age
This document describes the final assignment that was carried out in the context of the module 'Programming For
Data Science' of the Master's program in Applied Data Science. Using a chosen dataset, the following steps
of the CRISP-DM project management method were carried out: Business Understanding, Data Understanding and
Data Preparation. These steps were carried out using the Social Media and Mental Health dataset available on Kaggle.
2. Gender
Citations
The 8 general questions:
The survey used consists of 8 more general questions and 12 Likert scale-based questions that give us points
measuring the frequency or intensity of different aspects of mental health. A low score of 1 generally indicates low
frequency or intensity, and a high score of 5 generally indicates high frequency or intensity.
5 Conclusion and recommendations
This is done using the programming techniques learned during this module.
4.4 Libraries used
The purpose of this assignment is to see whether, using the data from the survey, it is possible to discover factors
(correlations) that influence mental health. So it is not just about the number of hours spent on social media, as was
done in the original study. This is to take a broader look at the data and to assess whether other factors may also
play a major(er) role.
2. Business Understanding
1. Introduction
Machine Translated by Google
2024-01-30
3 / 12
readme.md
3.1 Initial exploration based on the use of the Dataframe Tool
3. Data understanding & preparation
7. Social media platforms used 8.
Time spent - social media use, in hours
The purpose of this phase is to better understand the data, assess their quality and determine whether they are
suitable for achieving the above objectives. This provides a preliminary understanding of the structure, patterns, and
general characteristics. The quality of the data can be assessed, including possible missing values, duplicates,
inconsistencies, and other problems. Based on this, the data can be adjusted (data prepeartion) to improve quality.
To conduct this initial exploration, an application has been developed to read a generic CSV data file and display
the data in various ways (see 4.2). This tool can therefore be used for any CSV data file to do this initial exploration.
At
5. Affiliated organizations
6. Use social media?
15. Comparing oneself to peers [Self-esteem]
The Data Understanding phase is aimed at gaining a deeper insight into the available data.
Easier to read
14. Difficulty Concentrating [ADHD]
Long names Shorter names
13. Worry [Anxiety]
The hypothesis is that the number of hours someone spends on social media has a major influence on the 4
mental health categories. The more time someone spends on social media, the higher the scores will be.
Explanation
20. Sleep Problems [Depression]
12. Easily distracted by social media [ADHD]
It can be seen that the questions are related to 4 health categories: ADHD, Anxiety, Self-esteem and Depression.
Column
10. Distracted by social media [ADHD]
11. Anxiety when social media is not used [Anxiety]
19. Fluctuations in interest [Depression]
18. Feelings of depression [Depression]
9. Aimless use of social media [ADHD]
There are 20 columns and 481 observations. The following points stand out when we look at the data.
Perception Adjustment
17. Seeking Validation Through Social Media [Self-Worth]
Measurements of the frequency or intensity of mental health symptoms are made using the following questions:
16. Feelings about the above comparison [Self-esteem]
Machine Translated by Google
readme.md 2024-01-30
4 / 12
Of
analysis &
prediction
Column
Age
can be included in,
among other things, a
written,
written
Non-binary is
used in
several ways
Use social media
can be taken along
Values are
separated. There
are also rows
that have
empty values
must grant.
Not relevant to
person
Average social
media time
contain.
hit to also
Someone always has
an integer as their age
ways
see if this affects the
experience
can be included in,
among other things, a
So that this column
add to the number of
different
Delete column
So that this column
Come
of the values we would adhere to here
survey
So that this column
Affiliated with
platforms in on too
Gender
seperated
correlation matrix
correlation matrix
Split into columns using dummy
variables. The empty values are converted
to the value 'N/A' (Not applicable)
1 new column
The first step is to split this using dummy
variables. The column itself can then be
deleted.
Timestamp
correlation matrix
1 row has a
number behind it
It then turns out that some categories contain
too few numbers. For this purpose, 1 Other
category is created. There is 1 category
'Are there Others?' This will be removed
because we don't know which one
column
they come
Explanation
predictions
to fill in
Convert True False to 0 and 1
be in one
platforms
are in 1
Converted to numerical values
Merge: Too little
data in the
categories to
come
Categorical
values
Convert to same spelling.
Platforms
Yes in No
Perception Adjustment
to take in
complete
Date of
Machine Translated by Google
3.2 Second exploration based on the use of the Dataframe Tool
5 / 12
readme.md 2024-01-30
Perception Adjustment
can be compared
with each other
Column
Add an extra column indicating
whether someone works or is a student
All mental health
questions
columns with
To see if this affects
mental health
Explanation
So that the new
Create new columns per category
(ADHD, Depression, etc.) that take the sum of
the observations divided by the number
of questions in that category. Delete all
individual question columns.
lead or a
questions
regarding
mental health
do not say
After the adjustments (data preparation) as described in 3.1 have been carried out, a number of visualizations have
been created.
this column
is also possible
The loose one
works.
values
person is
a student or
so much.
From the
Occupation
Machine Translated by Google
readme.md 2024-01-30
6 / 12
The figure above shows that almost 60% of people spend 3 or more hours on social media per day and
almost 80% of people spend 2 hours or more per day.
Machine Translated by Google
7 / 12
readme.md 2024-01-30
The correlation matrix above shows which columns have the most correlation with the 4
health categories. It can be seen that AverageSocialMediaTime has the highest correlation with CountADHD
(0.45) and with CountANXIETY (0.44). Then comes MARRIED with CountDEPRESSION (0.32).
Age, IS_STUDENT and SALARIED_WORKER also show a slight correlation. (For reasons of clarity, the full
correlation matrix is not included in the report).
Machine Translated by Google
readme.md 2024-01-30
8 / 12
4.1 Project structure
4. Python code
about depression. It is noticeable that the Self-esteem questions score lower, with the median being 2.7.
The box plot above shows the average scores of the 4 health categories. A score of 1
This folder contains the Python code
see that the ADHD questions score the highest with a median of 3.5, followed by 3.4 for the questions
Explanation
highest score) indicates that this characteristic is very applicable to a person. In the boxplot it is too
Folder
application/
(the lowest score) indicates that this characteristic does not or hardly apply to a person. 5 (de
Machine Translated by Google
4.2 Dataframe Tool
2024-01-30
9 / 12
readme.md
Explanation
This is the home page. A CSV file can be uploaded here.
Folder
After the Python file SmmhDataframe.py has been run, a new file is created in which all data preparation steps
have been completed. This is also the basis for the visualizations.
Name Explanation
The file used from Kagggle is located at the location "data/smmh.csv".
2_Graphs.py
Location for saving CSV files
The Streamlit has 3 pages:
Below is an example of such a graph:
streamlit run .\Home.py
Location for the CSV files to be read
in the folder where Home.py is located. In the delivered code this is the application directory. Home.py is the main
page of the application. All pages under the directory pages are displayed in the sidebar by default.
1_Data_Info.py
data/
output/
The application can be started from a command line using the command:
As described earlier, an application has been developed to read a generic CSV data file and display the data in various
ways: the Dataframe Tool. This was written using Streamlit. This is a Python-based library specifically designed for
machine learning engineers to quickly build a web application.
application/pages/
This page uses the DataframeAnalyzer class to display information from 1 or more columns of
the file, such as (if numerical values): median, average, maximum value etc.
This page uses the DataframeVisualizer class to display graphs of 1 or more columns of the file.
Home.py
This folder contains the Streamlit pages (outside the main Home.py page, which is in the
application itself)
This file can be found under location "output/adjusted_smmh.csv"
Machine Translated by Google
2024-01-30
10 / 12
readme.md
4.4 Libraries used
A number of superclasses have been created:
Can be used to create some general graphs from a data frame, such as a scatter plot or
correlation matrix
Can be used to do some general analyzes based on a data frame, such as requesting
unique values of columns
DataframeCleaner
Some visualizations are created. The
mutated data is written to a new CSV file
The CSV file is read. The file is
cleaned (see 3.1).
This can be used to do some general clean-up actions on a data frame, such as
removing spaces at the beginning of a column or converting everything to uppercase to
align lowercase and uppercase letters
The file is transformed (see 3.1)
Name
DataframeVisualizer
Explanation
DataframeAnalyzer
The Dataframe class has 3 subclasses:
This class is the start of the project related to the assignment. In the 'main block' of the file the entire process is executed
from start to finish:
The Dataframe class is the superclass. This class contains general methods to, among other things, filter specific
columns from the data frame and check whether columns are numeric.
SmmhDataframe is a subclass of these 3 classes. The class itself contains the specific clean-up actions for the data
associated with the survey.
4.3 Object Oriented Programming
Machine Translated by Google
11 / 12
readme.md 2024-01-30
affects the 4 health categories. But also age, whether someone is married, or whether someone is still married
3.7.3
Different graphs
because these scored the highest.
seaborn
Version What used for
this may of course change in the future.
absolute correlation value 0.3) give a better score.
appear to have an influence, although less than the number of hours spent on social media. For Self-esteem
The model for Self-esteem can (initially) be omitted: it scores in the current one
Library
pandas
The correlation matrix initially only showed the first row. To get this working I had to
statemodels 0.14.0 qqplot
being a student or having a job have an (absolute) correlation of at least 0.3.
After developing these models, it can be assessed how well they score based on the still to be achieved
health categories. It does indeed seem that the number of hours spent on social media influences the
Reading in file
1.29.0 For the Dataframe Tool web application
Anxiety, Self-esteem and Depression, it seems useful to me to look at developing 4 separate (Machine
1.11.4 Chi-square matrix
Different graphs
The correlation matrix shows that the number of hours someone spends on social media per day is the most
I recommend starting to develop a model for ADHD and Depression respectively,
this connection is hardly present.
1.25.2 Convert type
scipy
health categories, but this is not a very clear connection. There are also other properties that
matplotlib
matlibplot be downgraded to version 3.7.3 (see https://stackoverflow.com/questions/77165100/only-the-first-row-of-annotationsdisplayed-on-seaborn-heatmap)
dataset significantly less high than the other 3 (see box plots). Keep an eye on the score though, because
0.12.2
numpy
Learning) models, namely one for each category.
Thee hypothesis was that the number of hours someone spends on social media has a major influence on their mental health
2.1.3
streamlit
Because these 5 traits show different correlations on the 4 health categories of ADHD,
determine metrics. In addition, you can check whether adding more properties (i.e. under
Citations
5 Conclusion and recommendations
Machine Translated by Google
12 / 12
readme.md 2024-01-30
Social media and mental health. (2023, 18 juli). Kaggle.
https://www.kaggle.com/datasets/souvikahmed071/social-media-and-mental-health
Machine Translated by Google