{ "cells": [ { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "import praw\n", "import pandas as pd\n", "from Keys import reddit_keys\n", "\n", "reddit= praw.Reddit(client_id=reddit_keys['client_id'],\t\t # your client id\n", "\t\t\t\t\tclient_secret=reddit_keys['client_secret'],\t # your client secret\n", " usernme = reddit_keys['usernme'], #profile username\n", " password = reddit_keys['password'], #profile password\n", "\t\t\t\t\tuser_agent=reddit_keys['user_agent'])\t # your user agent\n", "\n" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [], "source": [ "from praw.models import MoreComments\n", "from transformers import pipeline\n", "from transformers import DistilBertTokenizerFast\n", "classifier = pipeline(\"sentiment-analysis\", model=\"michellejieli/NSFW_text_classifier\")\n", "\n", "input_url = \"https://www.reddit.com/r/europe/comments/r0hthg/sweden_is_taking_the_lead_to_persuade_the_rest_of/\"\n", "\n", "def extract_comments(input_url):\n", " submission = reddit.submission(url=input_url)\n", " #posts_dict = {\"Post text\":[],}\n", " posts_dict = {\"Post text\":[], \"class\": []}\n", " for top_level_comment in submission.comments:\n", " if isinstance(top_level_comment, MoreComments):\n", " continue\n", " \n", " posts_dict[\"Post text\"].append(top_level_comment.body)\n", " posts_dict[\"class\"].append(classifier(top_level_comment.body)[0]['label'])\n", " df = pd.DataFrame(posts_dict)\n", " return df " ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", " | Post text | \n", "class | \n", "
---|---|---|
0 | \n", "Does… does that mean GPU’s might become afford... | \n", "SFW | \n", "
1 | \n", "But I use my mining rig for interior heating. ... | \n", "SFW | \n", "
2 | \n", "I approve.\\n\\nAlthough have they suggested how... | \n", "SFW | \n", "
3 | \n", "You can't ban cryptocurrencies. Just like you ... | \n", "SFW | \n", "
4 | \n", "But still no nuclear plants | \n", "SFW | \n", "
5 | \n", "https://twitter.com/ercwl/status/1457114531314... | \n", "NSFW | \n", "
6 | \n", "Yes, \"we\" are missing the goal. Maybe stop sub... | \n", "NSFW | \n", "
7 | \n", "here is something complete insane to think abo... | \n", "NSFW | \n", "
8 | \n", "Put a tax carbon on it, it will elimanate the ... | \n", "SFW | \n", "
9 | \n", "Banning crypto mining will only cause it to mo... | \n", "SFW | \n", "
10 | \n", "Can someone explain how this is possible? | \n", "SFW | \n", "
11 | \n", "The average knowledge on crypto in here is ver... | \n", "NSFW | \n", "
12 | \n", "Howbout u ban Facebook instead? | \n", "NSFW | \n", "
13 | \n", "Good. Blockchain is useful but so many despera... | \n", "NSFW | \n", "
14 | \n", "I don't particularly like crypto or it's fans,... | \n", "SFW | \n", "
15 | \n", "I was thinking of getting rid of my RTX 2070s,... | \n", "SFW | \n", "
16 | \n", "Total waste of time. As seen following the Chi... | \n", "SFW | \n", "
17 | \n", "It's actually shocking how uneducated people a... | \n", "NSFW | \n", "
18 | \n", "Totally doesn’t have anything to do with money... | \n", "NSFW | \n", "
19 | \n", "Sweden's biggest electric company Vattenfall s... | \n", "NSFW | \n", "
20 | \n", "BASED | \n", "SFW | \n", "
21 | \n", "Excellent news.\\n\\nIt consumes vast quantities... | \n", "NSFW | \n", "
22 | \n", "sweden want to fuck with danemark and his cryp... | \n", "NSFW | \n", "
23 | \n", "They should ban gold too, mining gold uses 10x... | \n", "NSFW | \n", "
24 | \n", "Great news. Cryptocurrencies are an embarrassi... | \n", "NSFW | \n", "
25 | \n", "This is BS, it doesn't make sense. | \n", "NSFW | \n", "
26 | \n", "I see no problem with this, the amount of ener... | \n", "SFW | \n", "
27 | \n", "[deleted] | \n", "SFW | \n", "
28 | \n", "\"I'm running my graphics card at full load so ... | \n", "SFW | \n", "
29 | \n", "The market is moving from POW to POS which is ... | \n", "SFW | \n", "
30 | \n", "And exactly how are they gonna do that?\\n\\nPir... | \n", "SFW | \n", "
31 | \n", "Yeah because that's what's the main cause of o... | \n", "SFW | \n", "
32 | \n", "Banning mining without banning transactions an... | \n", "NSFW | \n", "
33 | \n", "Crypto seems to have some side effects, who wo... | \n", "SFW | \n", "
34 | \n", "Crypto currency is the new tulips. So hot righ... | \n", "NSFW | \n", "
35 | \n", "Fuck yeah, ban them all! Easiest climate check... | \n", "NSFW | \n", "
36 | \n", "Ban fossil fuels instead. What a useless measu... | \n", "NSFW | \n", "
37 | \n", "Please no. I earn 200 euros a month from it. | \n", "SFW | \n", "
38 | \n", "Swedens state owned power company rejects this... | \n", "NSFW | \n", "
39 | \n", "Great not only we had Chinese crypto miners se... | \n", "SFW | \n", "
40 | \n", "Let's ban the currency that is somewhat indepe... | \n", "NSFW | \n", "
41 | \n", "Is that really that much of a problem with ene... | \n", "NSFW | \n", "
42 | \n", "How about instead of banning it, we make it so... | \n", "SFW | \n", "
43 | \n", "The electric footprint of crypto won't go up. ... | \n", "SFW | \n", "
44 | \n", "Lets make playing a demanding game more than t... | \n", "NSFW | \n", "
45 | \n", "How does cryptocurrency change the climate? I ... | \n", "SFW | \n", "
46 | \n", "Load ze Sweden FUD | \n", "SFW | \n", "
47 | \n", "[removed] | \n", "NSFW | \n", "
48 | \n", "Such pathetic virtue-signaling.\\n\\nIts no diff... | \n", "NSFW | \n", "
49 | \n", "shame on sweden | \n", "NSFW | \n", "
50 | \n", "-And now, NEWS FOR DUMBASSES, Bob?\\n\\n-Yes Bri... | \n", "NSFW | \n", "
51 | \n", "dumb | \n", "NSFW | \n", "
52 | \n", "Wow, Sweden is such a dictatorial shithole cou... | \n", "NSFW | \n", "
53 | \n", "talking about that, a full ban might be unlike... | \n", "NSFW | \n", "