Saravanan666/pypi-org huggingface dataset:
Package: reddit-data-collector
Reddit Data CollectorReddit Data Collector is a Python package that allows a user to collect post and comment data from Reddit. It is built on top of the Python modulePRAW, which stands for "The Python Reddit API Wrapper". It aims to make it very simple for a user to collect data from Reddit for further analysis (e.g. Natural Language Processing), without having to learn the inner workings of PRAW or the Reddit API.It is currently maintained byNico Van den Hooff.InstallationDependenciesReddit Data Collector requires Python and:pandas (>=1.3.5)praw (>=7.5.0)tqdm (>=4.62.3)User installationThe recommended way to install Reddit Data Collector is usingpip:pipinstallreddit-data-collectorHow to Use Reddit Data CollectorPlease see theexamples directoryfor step by step instructions on how to use Reddit Data Collector.DevelopmentImportant linksOfficial source code repo:https://github.com/nicovandenhooff/reddit-data-collectorDownloaded releases:https://pypi.org/project/reddit-data-collector/Issue tracker:https://github.com/nicovandenhooff/reddit-data-collector/issuesSource codeYou can check the latest sources with the command:gitclonehttps://github.com/nicovandenhooff/reddit-data-collector.gitContributingTo learn more about making a contribution to Reddit Data Collector, please see thecontributing file.Potential Ideas for ContributionAdd ability to collect images from Reddit posts that contain them.Add author information to post and comment data, currently the Reddit API is inconsistent with suspended and deleted author data, so this functionality has not been built in yet.Add plotting module that creates useful visualizations of the data that has been collectedAdd preprocessing module that cleans up the posts and/or comment data collectedTestingAfter installation, you can launch the test suite, which is contained in thetests/tests.py.  Note that you will have to havepytest>= 6.2.5 andpytest-cov>= 3.0.0 installed.  You can launch the test suite by following these steps from the projects root directory:Open uptests.pywith the following command:opentests/tests.pyComment out lines 24 to 30.  Change the values inDataCollector()in line 32 to your Reddit credentials.Run the following command to run the tests:pytesttests/tests.pyIf desired, run the following command to show test coverage:pytest--cov=srctests/tests.pyProject HistoryThe project was started in January 2022 by Nico Van den Hooff as a side project while he was completing the UBC Master of Data Science Project.  Nico wanted to obtain a sample of posts and comments from Reddit, but noticed that while PRAW existed and provided seamless access to Reddit's API, there was no package available that allowed for a simple method to collect this data.InspirationCertain sections of this README file was inspired by thescikit-learn README.
Package: reddit-dl
reddit-dlLimitless reddit downloader / No API no limitDownload from r/cats or some NFSW sub, or user profile. All supported.No api limit, make archives update them.CommitsYes please, reddit-dl needs your commit.Just before commit, e-mail address on profile.InstallationManuel Installation#clonetherepo$gitclonehttps://github.com/reddit-dl/reddit-dl.git#changetheworkingdirectorytosherlock$cdreddit-dl/#installtherequirements$python3-mpipinstall-rrequirements.txt#thencall$python3reddit_dl.py--helpInstallation with pip#install$pipinstall-Ureddit-dl#thencall$reddit-dl--helpExample usage:#downloadto`cats`subredditmedia$reddit-dlhttps://www.reddit.com/r/cats/#updatetodownloaded`cats`subredditmedia$reddit-dl--updatehttps://www.reddit.com/r/cats/`#downloadto`cats`subredditmediawith`-r`$reddit-dl-rcats#updatewithfoldernametodownloaded`cats`subredditmedia$reddit-dl--updater-cats#Notdownloadgifs$reddit-dl-u<username>--no-gifsUsage and Optionsusage: reddit_dl.py [-h] [--version] [-u [USER ...]] [-r [REDDIT ...]] [-V] [-P] [-G] [-N] [--update] [--user-agent USER_AGENT]
                    [--request-timeout N] [--max-connection-attempts N]
                    [target ...]

Download pictures, gifs, videos along with their captions and other metadata from Reddit.

Miscellaneous Options:
-h, --help            Show this help message and exit.
--version             Show version number and exit.

What to Download:
Specify a list of targets. For each of these, reddit-dl creates a folder and downloads all media content. The following targets are
supported:

target                Full url of users, subreddits or their local folder names.Exp. `https://reddit.com/r/UkrainianConflict/`
-u [USER ...], --user [USER ...]
                        Usernames to download.
-r [REDDIT ...], --reddit [REDDIT ...]
                        Subreddit names to download.

What to Download of each Post:
-V, --no-videos       Do not download videos.
-P, --no-pictures     Do not download pictures.
-G, --no-gifs         Do not download pictures.
-N, --no-nsfw         Do not download NSFW content.

Which Posts to Download:
--update              For each target, stop when encountering the first already-downloaded content.

How to Download:
--user-agent USER_AGENT
                        User Agent to use for HTTP requests.
--request-timeout N   Seconds to wait before timing out a connection request. Defaults to 300.
--max-connection-attempts N
                        Maximum number of connection attempts until a request is aborted.

https://github.com/reddit-dl/reddit-dl
Package: reddit-download
Work in progress.This software uses RedditImageGrab by Daniel Brown, which can be found athttp://github.com/dpbrown/RedditImageGrab.original “readme.txt” of RedditImageGrab:I created this script to download the latest (and greatest) wallpapers
off of image subreddits like wallpaper to keep my desktop wallpaper
fresh and interesting. The main idea is that the script would download
any JPEG or PNG formatted image that it found listed in the specified
subreddit and download them to a folder.An example of running this script to download images with a score
greater than 50 from the wallpaper sub-reddit into a folder called
wallpaper would be as follows:python redditdownload.py wallpaper wallpaper -score 50And to run the same query but only get new images you don’t already
have, run the following:python redditdownload.py wallpaper wallpaper -score 50 -updateAdvanced Examples:Retrieve last 10 pics in the ‘wallpaper’ subreddit with the word
“sunset” in the title (note: case is ignored by (?i) predicate)python redditdownload.py wallpaper sunsets -regex ‘(?i).*sunset.*’ -num 10
Package: reddit-downloader
reddit_image_downloaderDescription:The script will pull the top posts from the subreddit you give it.It will look through the posts to find images and then download them to /images.You will need a bot login for Reddit. See:https://github.com/reddit-archive/reddit/wiki/OAuth2-Quick-Start-Example#first-stepsFor praw config information see:https://praw.readthedocs.io/en/latest/getting_started/configuration/prawini.html.Required praw.ini entries are:
- [download_bot]
- client_id=
- client_secret=
- user_agent='Python:reddit_image_downloader:0.0.2 (by /u/merchantjeff)'Usageusage: main.py [-h] [-l LIMIT] [-a] [-s {new,hot,top,rising}] [-v]
               sub [sub ...]

Tool allowing the downloading of images and gifs from listed subreddit.

positional arguments:
  sub                   Input subreddit for downloading.

optional arguments:
  -h, --help            show this help message and exit
  -l LIMIT, --limit LIMIT
                        Number of submissions to be downloaded.
  -a, --album           Allow Imgur albums
  -s {new,hot,top,rising}, --sort {new,hot,top,rising}
                        Change default sort.
  -v, --verbose         More verbose logging.Requirementsas per requirements.txtpip install -r requirements.txtNotesLogging is on at debug as default, it creates a main.log file on run.
Package: reddit-edgecontext
edgecontextPython Documentation:https://reddit-edgecontext.readthedocs.io/en/latest/Go Documentation:https://pkg.go.dev/github.com/reddit/edgecontext/lib/go/edgecontextServices deep within the backend often need to know information about the
client that originated the request, such as what user is authenticated or what
country they're in. Baseplate services can get this information from the edge
context which is automatically propagated along with calls between services.This library provides a Thrift specification of an edge context payload and a
corresponding implementation of theEdgeContextFactoryinterface from
Baseplate.py.And an implementation ofecinterfacefrom Baseplate.go.UsagePythonAdd theEdgeContextFactoryto application startup:frombaseplateimportBaseplatefrombaseplate.lib.secretsimportsecrets_store_from_configfromreddit_edgecontextimportEdgeContextFactorydefmake_processor(app_config):secrets=secrets_store_from_config(app_config,timeout=60)edgecontext_factory=EdgeContextFactory(secrets)# pass edgecontext_factory to your framework's integration# for Thrift: baseplate.frameworks.thrift.baseplateify_processor# for Pyramid: baseplate.frameworks.pyramid.BaseplateConfiguratorThen read fields while handling requests:defmy_view(request):returnrequest.edge_context.user.idSeethe documentationfor all the available fields.GoUseedgecontext.Factoryto create anecinterface.Factoryimplementation
that's expected bybaseplate.New:ctx,bp,err:=baseplate.New(context.Background(),baseplate.NewArgs{ConfigPath:configPath,ServiceCfg:&cfg,// or nil if you don't have additional config to parseEdgeContextFactory:edgecontext.Factory(edgecontext.Config{Logger:log.ErrorWithSentryWrapper(),}),})When using it, get the*EdgeRequestContextobject out of context:ifec,ok:=edgecontext.GetEdgeContext(ctx);ok{user:=ec.User()loid,ok:=user.LoID()// Do something with loid}DevelopmentA Dockerfile is provided to get a development environment running. To use it,
build the base Docker image:$dockerbuild-tedgecontext.And then fire up the environment and use the provided Makefile targets to do
common tasks:$dockerrun-it-v$PWD:/src--user"$(id-u):$(id-g)"-w/srcedgecontext$makefmtThe following make targets are provided:fmt: Apply automatic formatting to the source code.thrift: Generate code from the Thrift IDL. Runfmtafter doing this.lint: Run linters on the code.test: Run the test suite.docs: Build docs.Python output can be found inlib/py/build/html/.The generated Thrift code is committed to the Git repo, so if you changeedgecontext.thriftmake sure to runmake thrift fmtand commit those
changes as well.For Go, we do the same linting checks as Baseplate.go,
so please follow Baseplate.go'sEditorguide to make sure you are doing the
same linting locally correctly.
Please also follow Baseplate.go'sStyleguide for code style.
Package: reddit-experiments
experiments.pyExperiments allow us to determine the impact of changes we make. This library
helps you run and track them in Baseplate.py services.Documentation:https://reddit-experiments.readthedocs.io/UsageInstall the library:#`reddit-v2-events`isaRedditinternalpackageusedforemittingexposureevents$pipinstallreddit-experimentsreddit-v2-events>=2.8.2Add the client to your application's Baseplate context:fromevent_utils.v2_event_utilsimportExperimentLoggerfromreddit_deciderimportdecider_client_from_configdecider=decider_client_from_config(app_config=app_config,event_logger=ExperimentLogger(),request_field_extractor=decider_field_extractor,)baseplate.add_to_context("decider",decider)and use it in request:defmy_method(request):ifrequest.decider.get_variant("foo")=="bar":passSeethe documentationfor more information (documentation builds can be foundhere)
.DevelopmentA Dockerfile is provided to get a development environment running. To use it,
build the base Docker image:$dockerbuild-texperiments.And then fire up the environment and use the provided Makefile targets to do
common tasks:$dockerrun-it-v$PWD:/src-w/srcexperiments$makefmtThe following make targets are provided:fmt: Apply automatic formatting to the source code.lint: Run linters on the code.test: Run the test suite.docs: Build the docs. Output can be found inbuild/html/.Note: some tests are skipped by default locally because they are quite slow.
Enable these by setting CI=true in the environment:CI=true make test.
Package: reddit-extract
Reddit Comments AnalyzerGeneral package for reddit comments analysis, data manipulation, and other areasInstall Instructionspip install .or for released version:pip install reddit-extractHow to UseRequired:Reddit Client IDReddit Client SecretReddit User AgentSubredditList of Thread IDs for bulk extract of multiple Reddit threadsDictionary (Should match your regex search pattern, otherwise your headers will not match the data retrieved, for example: a defined copy/paste form for a reddit thread that users reply to, aka "Megathreads")Regex Pattern (for csv)Extracting all comments for list of threads to csv with defined headers and search pattern:import reddit_extract
reddit_extract.extract_comments_csv_bulk(<Reddit ClientID>, <Reddit ClientSecret>, <Reddit User Agent>, <Subreddit>, <Thread IDs>, <Dictionary Headers>)Extracting all comments for list of threads to txt:import reddit_extract
reddit_extract.extract_comments_txt_bulk(<Reddit ClientID>, <Reddit ClientSecret>, <Reddit User Agent>, <Subreddit>, <Thread IDs>)Example:import reddit_extract
threads = ['aw79c5', 'b7x7n1', 'am5uk7', 'bji681', 'abv2gl', '9klf8e']
search_pattern = r'Form: (.*)\n*Entity: (.*)\n*Pending: (.*)\n*Approved: (.*)\n*Standardized wait: (.*)\n*STATE: (.*)'
headers = {'Form': 'Form', 'Entity': 'Entity', 'Pending': 'Pending', 'Approved': 'Approved', 'Standardized Wait': 'Standardized Wait', 'STATE': 'STATE'}
reddit_extract.extract_comments_csv_bulk(<client_id>, <client_secret>, <user_agent>, 'nfa', threads, headers, search_pattern)Testspython setup.py test
Package: reddit-gen
Reddit Account GeneratorRequirementsGoogle ChromeChrome DriverOptional, but recommended:MongoDB(get a free databasehere)Getting startedStep 1: Install the packagepipinstall-Ureddit-genStep 2: Configure your environmentreddit-gen--configureUsageusage: reddit-gen [-h] [-d] [-s] [-i] [-j] [-p] [-n CREATE_N_ACCOUNTS]
                  [-c CONFIG_FILE] [-D] [-U] [--configure]
                  [--experimental-use-vpn]
                  [--check-subreddit-ban CHECK_SUBREDDIT_BAN] [-v]

options:
  -h, --help            show this help message and exit
  -d, --disable-headless
                        Disable headless mode
  -s, --solve-manually  Solve the captcha manually
  -i, --ip-rotated      The public IP address was changed by the user since
                        the last created account (to bypass the cooldown)
  -j, --use-json        Read from the local JSON database (pass if you're not
                        using MongoDB). A new local database will be created
                        if not found
  -p, --show-local-database-path
                        Prints the path to the local database, if exists
  -n CREATE_N_ACCOUNTS, --create-n-accounts CREATE_N_ACCOUNTS
                        Number of accounts to create (default: 1)
  -c CONFIG_FILE, --config-file CONFIG_FILE
                        Path to the config file. Defaults to
                        /Users/$USER/.redditgen.env
  -D, --debug           Debug mode
  -U, --update-database
                        Update accounts metadata (MongoDB-only)
  --configure           Configure your environment
  --experimental-use-vpn
                        Experimental feature (unstable)
  --check-subreddit-ban CHECK_SUBREDDIT_BAN
                        Check if your accounts are banned from a specific
                        subreddit (MongoDB-only)
  -v, --verbose         Print more logsExample$reddit-gen# ───────────────────────────────── Starting... ─────────────────────────────────# 2022-06-20 17:22:42.739 | INFO     | reddit_gen.generator:_signup_info:65 - Your account\'s email address: some_random_username@example.com# 2022-06-20 17:22:42.739 | INFO     | reddit_gen.generator:_signup_info:67 - Username: some_random_username# 2022-06-20 17:22:45.976 | DEBUG    | reddit_gen.generator:generate:196 - Solving captcha...# 2022-06-20 17:24:12.579 | DEBUG    | reddit_gen.generator:generate:200 - Solved!# 2022-06-20 17:24:38.841 | DEBUG    | reddit_gen.generator:generate:263 - Checking account info...# 2022-06-20 17:24:39.069 | DEBUG    | reddit_gen.generator:generate:266 - Passed!# 2022-06-20 17:24:39.069 | INFO     | reddit_gen.generato──r:generate:274 - Account verified!# ───────────────────────────────────── Done! ───────────────────────────────────
Package: redditmemeapi
redditmemeapiA pythonic API wrapper to fetch memes fromhttps://meme-api.comInstallationUse the package managerpipto install foobar.pipinstallredditmemeapiUsagefromredditmemeapiimportgimme# get a random meme from 'memes', 'dankmemes', 'me_irl' subredditsprint(gimme())fromredditmemeapiimportgimmenum# get specified number of memesx=gimmenum(2)#max value 50foriinx['memes']:print(i['url'])fromredditmemeapiimportspecifysub# get memes from a specified subredditprint(specifysub('okbuddyretard'))#max value 1fromredditmemeapiimportsubnum# get specified number of memes from a specified subredditx=subnum('okbuddyretard',3)#max one sub, max 50 memesforiinx['memes']print(i['url'])ContributingPull requests are welcome. For major changes, please open an issue first
to discuss what you would like to change.Please make sure to update tests as appropriate.LicenseMIT
Package: redditmirror
Reddit MirrorPython module that takes links from your "saved" and x-posts them to a subreddit of your choice.The best way to install is usingpip:pip install redditmirrorFor the script to know your authentication details you need to create apraw.inifile, seehere.RunningThere are a few command line arguments which can be used to skip the prompts and even run in non-interactive mode. You can use-hor--helpto get help.Examples:python-mredditmirror-a# run in automatic mode; tries to create a private subreddit# called r/username if it does not existpython-mredditmirror-asexample# use subreddit r/example instead of r/usernamepython-mredditmirror-kn# 'k'eep saved posts, reprompt instead of creating new subredditpython-mredditmirror-an# crash and burn if r/username does not exist
Package: redditmon
RedditMonRedditMon is a dumb and janky little package I threw together so that I could learn how to package a python project.Installationpip install redditmon????Profit!!!UsageTo use this module (assuming you have it installed), create a config file in your home directory with the nameredditmon.config.inithat contains the details of your reddit API account. Look atthis exampleto see how to structure the file. Once you have your config file in place, you can launch the module directly by issuingredditmon <subredditname>. Alternatively, you can launch it as a python module usingpython3 -m redditmon <subredditname>. Useredditmon --helpfor more information.
Package: reddit-multimodal-crawler
Reddit Multimodal CrawlerThis is a wrapper to thePRAWpackage to scrape content from image in the form ofcsv,json,tsv,sqlfiles.This repository will help you scrape various subreddits, and will return to you multi-media attributes.You can pip install this to integrate with some other application, or use it as an commandline application.PyPI Link:https://pypi.org/project/reddit-multimodal-crawler/pip install reddit-multimodal-crawlerHow to use the repository?Before running the code, you should have registered with the Reddit API and create a sample project to run the code and obtain theclient_id,client_secretand make auser_agent. Then pass them in the arguements.Although, the easier way is to use thepip install reddit-multimodal-crawler.FunctionalitiesThis will help you scrape multiple subreddits just likePRAWbut, will also return and save datasets for the same. Will scrape the posts and the comments as well.Sample Codeimportnltkfromreddit_multimodal_crawler.crawlerimportCrawlerimportargparsenltk.download("vader_lexicon")if__name__=="__main__":parser=argparse.ArgumentParser()parser.add_argument("--subreddit_file_path","A path to the file which contains the subreddits to scrape from.",type=str,)parser.add_argument("--limit","The limit to number of articles to scrape.",type=int)parser.add_argument("--client_id","The Client ID provided by Reddit.",type=str)parser.add_argument("--client_secret","The Secret ID provided by the Reddit.",type=str)parser.add_argument("--user_agent","The User Agent in the form of <APP_NAME> <VERSION> by /u/<REDDIT_USERNAME>",type=str,)parser.add_argument("--posts","A boolean variable to parse through the posts or not.",type=bool)parser.add_argument("--comments","A boolean variable to parse through the comments of the top posts of subreddit",type=bool,)args=parser.parse_args()client_id=args["client_id"]client_secret=args["client_secret"]user_agent=args["user_agent"]file_path=args["subreddit_file_path"]limit=args["limit"]r=Crawler(client_id=client_id,client_secret=client_secret,user_agent=user_agent)subreddit_list=open(file_path,"r").readlines().split()print(subreddit_list)ifargs["posts"]:r.get_posts(subreddit_names=subreddit_list,sort_by="top",limit=limit)ifargs["comments"]:r.get_comments(subreddit_names=subreddit_list,sort_by="top",limit=limit)
Package: redditor
redditorcommand line interface for reddit
or use reddit like a dogeusageSyntax:
redditor[OPTIONS]COMMAND[ARGS]...

Options:--versionShowtheversionandexit.--helpShowthismessageandexit.

Commands:feedMuchwow,suchinteresting.subredditShowthedogylist.installationpipinstallredditorto dounlimited queryinstall-able thru pip like a dogebeautiful .svg
Package: reddit-post-scraping-tool
NoteUseKnew Karmafor more advanced and improved features.RPST (Reddit Post Scraping Tool)RetrieveRedditposts that contain the specifiedkeywordfrom a specifiedsubreddit.✅ FeaturesGUIDark mode (Right-click>Settings>Dark Mode).Saves results to a JSON/CSV file (Right-click>Settings>Save posts>to JSON/to CSV).Logs errors to a file.In-App feature to check for Updates.CLISaves results to JSON (specifiy--json).Saves results to CSV (specify--csv).Automatically checks for new updates, and notifies user if updates were found.📃 TODOGUIMake it installable with a setup.exe/setup.msi file.🖥️ Tested environmentsGUIMicrosoft Windows 11CLIAndroid TermuxMicrosoft Windows 11Ubuntu 22.04 - latest versions📖 DocumentationRefer to the Wikifor installation instructions, in
addition to all other documentation.
Package: reddit_py
Toy python wrapper as I experiment with the [Official Reddit API](http://www.reddit.com/dev/api)
It’s not intended to compete with or be as comprehensive as [PRAW](https://praw.readthedocs.org/en/latest/)[PRAW]UsageThe package is broken down into modules following the structure of the API.
Brief overview:Accountslogin, create and delete user accounts:client = user_login('your_username', 'your_password')

your_overview = current_account_overview(client)UsersRetrieve the activity of a given Reddit account, options include: overview, submissions, comments and likes:user_submissions = get_user_activity('submitted', 'valid_username', 'top', 100, 'year')SubredditsRetrieve info about a given, or all subreddits of a logged in client. More to come:python_overview = about_subreddit('python')
Package: reddit-radio
Reddit radioInstallpipinstallreddit-radioConfigBefore running you need to setup the config file, there is a sample inconfig.sample.inifor your client_id and client_secret you need to go to this
url:https://www.reddit.com/prefs/appsand create a new app. Then move the file to~/.config/reddit_radio.ini.Usagepython-mreddit_radioload-dataplayDependenciesMPV