Can Machine Learning And NLP Help Predict Suicidal Risk?
From Depression To Suicide, How The Words We Use Can Predict The Way We Feel.
Category: Data Science
Author: Yoni Levine

Can Machine Learning And NLP Help Predict Suicidal Risk?

From Depression To Suicide, How The Words We Use Can Predict The Way We Feel.
Unfortunately, the number of people suffering from depression is on the rise. According to this study by the NIH, nearly 7 percent of US adults, and over 10 percent of those between the ages of 18–25, have had a major depressive episode in the last year. The CDC published another surprising and frightening statistic: less than one-third of Americans taking one antidepressant medication, and less than one-half of those taking multiple antidepressants have seen a mental health professional in the past year.
Nearly 7 % of adults and over 10 % of those between the ages of 18–25 have had a major depressive episode in the last year.
Percentage of people by age and sex who have taken anti-depressants in the last year. 2005–2008. Source: CDC.
Since many lack better alternatives, the internet has become a popular venue for people to reach out for help. All this public data provides us with a tremendous opportunity to study this population and try to understand what factors may be driving this epidemic and putting people at risk.
The goal of my project was to determine if I could predict which users of a depression forum, would eventually turn suicidal.
The goal of my project was to determine if I could predict which users of a depression forum would eventually turn suicidal. I began to scrape takethislife.com, a website that contains forums that provide support to those dealing with suicidal or depressive feelings. Scraping is a data extraction method that involves writing code to access a website and automatically obtain all the information that I needed, without needlessly copying and pasting through 25,000 pages.
I scraped the depression and suicide forums for the text of the posts along with the usernames. The next step was to actually label my data, I took all of the posts from the depression forum and labeled each post with a one or a zero indicating if that user had also posted in the suicide forum. By using Natural Language Processing to analyze the text, I hoped to learn what words were indicative of the difference in the way the two groups speak.
Data Analysis
Computers can’t actually understand words in the way that humans can.
NLP
Computers can’t actually understand words in the way that humans can. Natural Language Processing involves turning text into numbers so that they can be understood by a computer and processed mathematically. There are several different ways to do this, the most common method being a count vectorizer, which counts the number of times a word appears, on the assumption that the more it’s used the more important and the higher “score” it deserves. This assumption doesn’t hold true in many cases, and in my case I decided to go with something called a TFIDF vectorizer.
A TFIDF vectorizer uses the following rule: rank of a words appearance * number of times it’s written = score. Let me give you an example from James Joyce’s Ulysses. The word is has a rank of 20 and appears 1,435 times, which gives it a score of 28,700. The word Zurich has a rank of 29,055 and only appears 1 time, which gives it a score of 29,055, making the word Zurich more important than the word is. This makes sense, because if you were listening to a conversation, you would learn more about what the person was talking about by hearing the word Zurich once than hearing is almost 1500 times.
Predictions
Now that I had a score for each word in every post, and whether or not that user had also posted in the suicide forum I was ready to start my predictions and test out my hypothesis. Were there words that the users wrote in the depression forum that could help us predict if they would also post in the suicide forum?
Were there words that the users wrote in the depression forum that could help us predict if they would also post in the suicide forum?
To do this I used something called Machine Learning, which (to gloss over full PHD programs worth of knowledge) is when we use mathematical formulas together with usually large amounts of data to see if we could spot patterns and make predictions. It usually focuses around this question: Given some input X, what is the correct prediction Y? In this case given that a user posted words A,B, or C (our input X) can we predict if they will also post in the suicide forum (our output Y). I tried out many different ML models to see which would provide the most predictive power, and eventually went with a basic one called a logistic regression. Logistic regression follows very similar rules to a linear regression which you may vaguely remember from high-school math. Instead of giving you a quantitative output, say how much will a house cost (output Y) if it has 5 bedrooms and 3 bathrooms (input X)? it gives you the probability that an observation belongs to a specific class (in this case depressive or suicidal).
After we run our model we score it against a previously held out chunk of our original data to see how well our predictions did. The logistic regression models had a recall of 73, which means that 73 percent of the times that our user was actually suicidal our model correctly predicted so.
A moral dilemma that I ran into at this stage was the issue of weighting my classes. In medical testing models will usually be weighted more heavily to guess one side over the other in order to minimize missing any of the real occurrences of the disease. This comes at a cost though, now the model, while trying to catch all real occurrences, will also tell people who don’t have the disease that they do. While this is often a big question in medical research in general, traditionally researchers will weigh their models. To try and address this I decided to also create a model with slightly weighted classes, with the understanding that healthcare professionals could account for any potential issues. With this method I was able to get my model’s recall up to 83. I could have kept going with the weighting, but at a certain point you have to differentiate between just guessing yes for everybody and actually having a model, and I felt that this was a good balance.
A look at how the two different models classified users. Notice when the number of people correctly classified as suicidal (bottom right) went up, the people classified correctly as only depressed (top left) went down.
The words themselves tell an entire story and confirmed my belief that there is a difference in the way the two groups speak.
The Words That Tell
Since I ended up going with a logistic regression model I was able to see which words the model used to determine which class a user belonged to. The words themselves tell an entire story and confirmed my belief that there is a difference in the way the two groups speak.
The phrase hate myself for was one of the strongest indicators of a person staying in the depression class, while the term hating myself was a predictor of suicide. When I first saw these two words I assumed that they were just words that got mixed in to different classes, but when I thought about it again it made sense. The term that indicated depression (in this case a good thing) was localized, I hate myself for, a particular reason which I’m aware of, and am possibly working on. While those who just hated themselves as a global, all encompassing feeling, were not able to make that distinction and lost hope much quicker.
Another pair of words that I found interesting were God, indicative of suicide, and religious which was indicative of depression, which again I found to be confusing. My theory is as follows a person who identified as religious clearly found meaning and reason for hope in it, but a person who spoke of God, a more esoteric idea, probably used it in terms of an fear of angering God and was terribly haunted by it.
Other terms that were indicative of staying in depression were fat, with other people, seeing therapist, drama, in college, in debt, lately I’ve been, who love me, long distance, on anti-depressants, cheated on me, been diagnosed, rut, and dog. Many of these words are regular everyday stresses of life that people are having a very hard time of dealing with, and some of the others allow us to see what people use to find meaning in life, such as friends, family, religion, and pets. Some terms that were indicative of suicide were idk, have to, cutting, pain, don’t deserve to, raped, sexual abuse, the hospital, hurt so much, have no control, and ashamed of.
Words indicative of their class and their feature importance.
When you see the story that the words tell and the circumstances that lead people there it is very clear that the words people use can really tell can predict their feelings.
What Next
Moving forward in this project I plan on tagging different parts of speech to see if different users form sentences differently, using industry specific stop-words to clean up the text, and hopefully one day deploying it and use it to help people.
What do you think? Have you worked on any similar projects ? Please share in the comments and responses below. Thank you!