Mental Health and Machine Learning: CatBoost and Attempted Suicide on the Forever Lonely data set.
By: Peijin Chen
Tags: Machine Learning, Classification, Mental Hea, Data Science, Gradient Boosting

Mental Health and Machine Learning: CatBoost and Attempted Suicide on the Forever Lonely data set. Peijin Chen Follow Feb 13 · 3 min read

Photo by Kat Jayne

The Data Set: Forever Lonely. 465 responses to a survey on Reddit.

Feature to predict: attempted_suicide — YES or NO.

Algorithm: CatBoost (Python) — because of its innate ability to handle categorical features out of the box without much preprocessing.

PROBLEMATIC FEATURES:

job_description: wherein people wrote stuff like “peon” or “grunt”, or just didn’t answer. I tried cleaning this up but it might not be worth the effort. For example, I tried coding everyone that had a developer, coder or sys admin job as “IT”. I tried coding all the university students as “UNI” and all the regular students as “student”. I lumped graphic designers and artists as “artists”. I put chefs and freight workers as “manual”, while middle-management white-collar professionals became “pro”. I’m not entirely convinced this is pointless, but neither have I seen it help much either. It’s something you might want to consider if you are going for a more nuanced model. race: wherein most people wrote standard things like “Asian” but then there were people who wrote that they were mixed native American and something else, or Asian and White, or Turkish. I tried lumping the one “Turkish” person with “Middle Eastern”, and made the one “Pakistani” and “Indian” (could have called this one “desi” … ) but again, this is one of those categories where things are tricky. This seems like one case where you should try to lump any infrequent values into larger more frequent ones.

3. improve_yourself_how: wherein many people mentioned exercise and going to the gym, while others sought therapy or tried to expand their social circles. And some people claimed to have done nothing. I therefore tried to one-hot encode these categories, so that there would be, for example, a 1 or a 0 depending for a feature designating if the person was using physical exercise at all.

4. what_help_from_others: wherein many people said “I don’t want help” or something relating to dating. Therefore, I tried one-hot encoding the people that didn’t want help — thinking that people who think they are beyond help or can’t benefit from help are those that are more likely to attempt suicide. There were many people that mentioned dating — being a wingman or woman, or just getting into the dating scene. Maybe that’s a positive move that staves off suicidal ideation? I guess it depends on how those dates go. That said, this feature never seemed to be that important.

WHAT ACTUALLY MATTERED:

I ended up, for the time being, NOT using any of those problematic features.

The features that I used are the “biggies”, the things that don’t require too many subjective encoding judgments — gender, sexuality, body weight (albeit self-reported), income, education level, if they are depressed, their age, and how many friends they have.

In many of the multi-feature models I tried, “edu_level” proved to be important, however, that doesn’t mean that you can get a decent model with just that feature alone. Thus far, I have found that the features mentioned above: gender, sexuality, body weight, income, education level, depression, age, and friends — were, collectively, able to predict 18–20 of the 85 people that attempted suicide. And as for those that didn’t attempt suicide, there was just one false alarm. Of course, as humans we are more interested in not missing the suicidal, rather than false alarms among those who are not.

Can you improve on this model at all? Is there any way we can detect at least half of the attempted suicides? Is there some magical way of encoding some of the dirtier features so that we can derive some meaningful predictive information from them? Or am I missing some obvious method of feature selection and/or learning algorithm that is more effective?

A link to my Google Colab…