Saravanan666/pypi-org huggingface dataset:
Package: twitterspawn
# TwitterspawnAsynchronous, concurrent requests to the Twitter REST API, that respect Twitter's rate limits, using [gevent](http://www.gevent.org/) and [requests](http://docs.python-requests.org/).See [example.py](https://github.com/swinton/twitterspawn/blob/develop/example.py) for a working example.## UsageBasically:```pythonimport twitterspawn# Define callback (can define 1 per request)def callback(response, worker):print "Got", response, "from", worker# Add requests + callbackstwitterspawn.add_request("https://api.twitter.com/1/users/show.json",dict(params=dict(screen_name="steveWINton")),callback)twitterspawn.add_request("https://api.twitter.com/1/users/show.json",dict(params=dict(screen_name="twitter")),callback)twitterspawn.add_request("https://api.twitter.com/1/users/show.json",dict(params=dict(screen_name="catbinlady")),callback)# Add workerstwitterspawn.add_worker(access_token="YOUR_FIRST_ACCESS_TOKEN",access_token_secret="YOUR_FIRST_ACCESS_TOKEN_SECRET",consumer_key="YOUR_CONSUMER_KEY",consumer_secret="YOUR_CONSUMER_SECRET")twitterspawn.add_worker(access_token="YOUR_NEXT_ACCESS_TOKEN",access_token_secret="YOUR_NEXT_ACCESS_TOKEN_SECRET",consumer_key="YOUR_CONSUMER_KEY",consumer_secret="YOUR_CONSUMER_SECRET")# ...add as many more workers as required...twitterspawn.add_worker(access_token="YOUR_LAST_ACCESS_TOKEN",access_token_secret="YOUR_LAST_ACCESS_TOKEN_SECRET",consumer_key="YOUR_CONSUMER_KEY",consumer_secret="YOUR_CONSUMER_SECRET")# Go!twitterspawn.go()```See also [example.py](https://github.com/swinton/twitterspawn/blob/develop/example.py) for a working example.## InstallationSimply:$ pip install twitterspawn## Contact@[steveWINton](https://twitter.com/steveWINton).
Package: twittersphere
TwittersphereTwittersphere is a tool for ingesting tweets and applying content based
filtering rules for tweets and user profiles. Twittersphere uses simple
inclusion/exclusion rules based on words and phrases to include or exclude
user profiles or tweets as belonging to part of a group.FunctionalityTwittersphere exposes a command line interface and a Python library for:Extracting relevant tweet entities from Twitter V2 API JSONCreating a structured relational database in SQLite for further analyticsApplying rule based filters to select user profiles based on their contentIteratively create or update these rule based filtersCommand Line UsageCreating a databaseIn the first instance Twittersphere can be used to create a local relational
database from files containing V2 API Twitter JSON data collected via twarc.
Any tweet or user JSON data collected via the Twitter API including search
and streaming endpoints should work. Note that this process safely
deduplicates items: you can insert the same file more than once and not see
the same tweet twice. This database can be queried directly from most
programming languages, or after installing anODBC connectorcan be connected to tools like Excel or Tableau.twittersphere prepare FILE1.json FILE2.json ... FILEN.json processed.dbRule Based User FilteringAn existing ruleset (such as ... this not yet public Australian Twittersphere
rules ...) can be applied as follows:twittersphere filter-users rules.csv processed.dbThis will populate theuser_matching_rulesettable with theuser_id's of
profiles that have matched that ruleset, along with the name of the filename
of the rules for later reference.Updating RulesAfter applying a ruleset, you can generate an updated list of rules with new
candidate rules to expand the existing matching population. Note that the first
time you run this command will take longer, as this is when initial statistics
about ngrams are created.The following command wiltwittersphere refine-user-rules processed.db RULESET_NAME candidate_rules.csvNote that the following will show you which rules have already been applied
and are valid rules forRULESET_NAME:twittersphere list-user-rules processed.dbCreating RulesTo create a new rules you will need to generate an initial seed rule set, or
alternatively an initial seed population set.LimitationsNote that Twittersphere does not support Twitter V1.1 data at all.Data collected with tools other than twarc, collected with twarc metadata
turned off, or collected with limited fields included in the output will not
be well supported.
Package: twitter-stream.py
:snake: Python Client For Twitter API v2:rocket: Why Twitter Stream ?Twitter-Stream.pya python API client for Twitter API v2 now supportsFilteredStream,SampledStream,RecentSearch,TweetLookUp, andUserLookUp. It makes it easier to get started with Twitter's New API. Let's see an example of howtwitter-stream.pyhandlesSampledStream. Sampled Stream delivers about 1% of Twitter's publicly available tweets in real-time and paints a picture of general sentiments, recent trends, and global events.# sampled_stream.pyimportjsonfromtwitter_streamimportSampledStreamclassStream(SampledStream):user_fields=['name','location','public_metrics']expansions=['author_id']tweet_fields=['created_at']stream=Stream()fortweetinstream.connect():print(json.dumps(tweet,indent=4))Is this all you have to do to start streaming? Yes. Are these all the data points available to you? No. Let's discussline number 5-7.Twitter's Official Documentationlists an elaborate set of query parameters. You can use these queries to get the data you need. We are subclassingSampledStreamand carefully constructing clear and eloquent queries inline 5-7. And you can do this for all the query parameters listed in theSampledStreamAPI Reference.To get more insights into other API endpoints. Visit theexamplesfolder and our documentationstwitivity.dev.Installation~$ pip3 install twitter-stream.pyCredentialsStore Twitter credentials at~/.twitter-keys.yaml.~$vim~/.twitter-keys.yamlkeys:consumer_key:CONSUMER_KEYconsumer_secret:CONSUMER_SECRETaccess_token:ACCESS_TOKENaccess_token_secret:ACCESS_TOKEN_SECRETbearer_token:BEARER_TOKEN
Package: twitter-sync
TwitterStatic aka twitter-syncPython command-line script to automatically download and create markdown copy of all or subset of tweets (and keep in sync going forward). Ideal for making a replicate on a static site.
Uses combination of Twitter's own API and Nitter.net to iterate through and parse clean body of messages.Available as a direct install, e.g.pip install twitter-syncDependency on Twitter library - if not doing a full package installation above, make sure to install the dependencyhttps://github.com/bear/python-twittervia "pip install python-twitter".Once installed, use sample fromhttps://github.com/ILAsoft/TwitterStatic/blob/master/TwitterSyncSettings.sampleto create TwitterSyncSettings.ini one folder above your site's folder and then run "TwitterSync.py" from the website's folder itself. Feel free to add appropriate styles to your website if you want to adjust the look and feel of the "tweets".
Package: twittertail
twittertailThis python app monitors a Twitter account for tweet activity, using the asyncio library to create an event loop for the life of the running app. On execution, the 5 most recent tweets are displayed (liketail, so oldest to newest), and every 10 minutes new tweet activity is displayed. The app also includes an inbuilt HTTP API to retrieve all displayed tweets so far.setupThis demo requires Python 3 (tested on Python 3.7.5+). To install:pip install twittertailcliThe cli program can be run like so:twittertail -u TWITTER_USERNAMESee-hfor more options.HTTPBy passing in a-wargument, a simple HTTP server will run, allowing you to query the tweets collected so far, which is returned in aapplication/jsonresponse as a JSON object.twittertail -u TWITTER_USERNAME -wGet tweets via:curl http://127.0.0.1:9000/tweetsRun as a Docker containerTo run this as a Docker container, first build the image from the app's root directory:sudo docker build -t twittertail .And then run the container, passing in the Twitter username you want to monitor as the first argument:sudo docker run --rm -p 9000:9000 twittertail tinycarebotThis also runs the HTTP server on port 9000.
Package: twitter-text-parser
twitter-text-pythonThis is a Python port of thetwitter/twitter-textlibraries, fully compliant with theofficial conformance test suite.FeaturesThis library calculates length of a tweet message according tothe documentation from Twitter Developers,
so that you can validate the tweet without calling the Web API at all.
Although counting characters might seem an easy task, in actual fact it is very complicated, especially when the text contains CJK characters, URLs, or emojis.The original twitter-text libraries havehit-highlightingandauto-linkingfeatures as well,
however they are not yet supported by this Python port.UsageInstallation$pipinstalltwitter-text-parserExamplesSeethe API referencefor more details.fromtwitter_textimportparse_tweet,extract_emojis_with_indices,extract_urls_with_indicestext='english text 日本語 😷 https://example.com'assertparse_tweet(text).asdict()=={'weightedLength':46,'valid':True,'permillage':164,'validRangeStart':0,'validRangeEnd':38,'displayRangeStart':0,'displayRangeEnd':38}assertextract_urls_with_indices(text)==[{'url':'https://example.com','indices':[19,38]}]assertextract_emojis_with_indices(text)==[{'emoji':'😷','indices':[17,18]}]Related Linkstwitter/twitter-text: The original, official twitter-text implementations for Java, Ruby, JavaScript and Objective-Ctwitter-text Parser – Twitter Developers: A brief overview of the twitter-text librariesCounting characters – Twitter Developers: An introduction to how to count characters in Twitter textsedmondburnett/twitter-text-python: Another python port of twitter-text, which is not compliant with theofficial conformance test suite
Package: twitter-threader
twitter-threaderIntroductionTwitter Thread is a library to manage threads on Twitter. You can GET and POST threads on Twitter.Installationpipinstalltwitter_threaderUsageGET a thread on Twitterfromtwitter_threader.threaderimportconnect_api,Threadconsumer_key='CONSUMER_KEY'consumer_secret='CONSUMER_SECRET'access_token_key='ACCESS_TOKEN'access_token_secret='ACCESS_TOKEN_SECRET'api=connect_api(consumer_key,consumer_secret,access_token_key,access_token_secret)thread=Thread(api)thread.convert_to_post('1247616218153902080')POST a thread on Twitterfromtwitter_threader.threaderimportconnect_api,Threadconsumer_key='CONSUMER_KEY'consumer_secret='CONSUMER_SECRET'access_token_key='ACCESS_TOKEN'access_token_secret='ACCESS_TOKEN_SECRET'api=connect_api(consumer_key,consumer_secret,access_token_key,access_token_secret)thread=Thread(api)username='YOUR USERNAME'thread.post_thread('Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Metus dictum at tempor commodo ullamcorper a lacus. Amet justo donec enim diam vulputate. Sit amet justo donec enim diam vulputate ut pharetra sit. Leo duis ut diam quam. At ultrices mi tempus imperdiet. Mauris augue neque gravida in fermentum. Fermentum posuere urna nec tincidunt praesent semper feugiat nibh. Placerat vestibulum lectus mauris ultrices eros in cursus turpis massa. In aliquam sem fringilla ut morbi tincidunt augue.',username)
Package: twitter-user-emulator
twitter_user_emulatorA simple Python package that wrapsscweetandgpt-2-simpleto allow easy tweet generation based on any existing Twitter account.The idea of this project is to let anyone unfamiliar with GPT-2 to create their own Twitter bot emulating an existing account (e.g.,Deep Leffen,Deep Slacks Bot,Deep Bumpaah Bot) in a single API call. The library handles all boilerplate code and basic hyperparameter tuning involved, allowing the user to spend that time curating tweets they want instead.Installtwitter_user_emulator can be installedvia PyPi:pip3 install twitter_user_emulatorTo enable GPU usage for faster training (recommended), a version of Tensorflow 2.X (min 2.5.1) must be installed on your system.UsageAn example API call:from twitter_user_emulator import emulator

emulator.emulate_user(user="bumpaah", start_date="2020-01-01", end_date="2021-01-01", out_tweets=2000)This piece of code will download and train a gpt-2 model to generate 2000 tweets based on all the tweets user @bumpaah made between 2020-01-01 and 2021-01-01.The model will fail to train if there are not enough tweets as input. Please try to pick a time range with at least 500 tweets.Colab NotebookA notebook tutorial that goes over the underlying logic of this implementation can be foundhere.LicenseMITDisclaimerThis repo has no affiliation with OpenAI or Twitter.
Package: twitterUsernameviaUserID
No authentication. No API. No limits.twitterUsernameviaUserIDis an advanced Twitter scraping tool written in Python and Selenium that allows for scraping tweet usernames from the twitter id's,withoutusing Twitter's API.tl;dr BenefitsSome of the benefits of using twitterUsernameviaUserID vs Twitter API:Can fetchalltweet usernames from their id's (Twitter API limits to last 3200 Tweets only);Fast initial setup;Can be used anonymously and without Twitter sign up;No rate limitations.RequirementsPython >= 3.6SeleniumWebDriver ManagerInstallingPip:-pip3installtwitterUsernameviaUserID
-Voila!Example:from twitterUsernameviaUserID import getHandles as gH
    # A dictionary with mapped user ids to their usernames
    temp= gH.getHandles([<list of user ids>], <delay time between request>)
Package: twitter-utils
Twitter toolsDocumentation:https://namuan.github.io/twitter-utilsSource Code:https://github.com/namuan/twitter-utilsPyPI:https://pypi.org/project/twitter-utils/Collection of twitter utilities.Installationpipinstalltwitter-utilsExample UsageAll commands intwitter-utilswork better with an authenticated user otherwise Twitter bombards you with crap.
The following command is tested on MacOS and will create a symlink to your Firefox profile.
Please replace the source path with your own.ln-s"~/Library/Application\ Support/Firefox/Profiles/.."$(pwd)/fireprofileAll commands take an argument to specify the output directory.
Each tweet captured will be saved in the output directory to a file with the file name as the tweet id.Grab tweets of an account between two datestweets-between--account<<account>>--since2020-04-10--until2020-04-25-otemp-dirGrab tweets on a pagetweets-thread-a<<account>>-t<<tweet-id>>-otemp-dirUsing twarc to capture tweet data using Twitter APIYou can usetwarcto get tweets given a list of ids.
Twarc uses Twitter API so you'll need those credentials.Twarc hydratecommand that will grab tweet data given a list of ids.# Depending on your output directory, here is a command to get the list of file name (tweet ids)find<output-folder>/*.html|whilereadfile;dobasename$file".html";done><target-file-with-list-of-tweet-ids># These feed this list of ids to twarctwarchydrate<target-file-with-list-of-tweet-ids>><file-with-all-tweets-data>.jsonlSelecting WebDriverAll commands intwitter-utilsuses Firefox by default, but it can be overridden by providing an alternate browser.
Supporting Firefox, Safari and Chrome.To use Safari instead of Firefox, pass the--browseroption before specifying the sub-command.<cmd> --browser safari ...And to use Chrome<cmd> --browser chrome ...DevelopmentClone this repositoryRequirements:PoetryPython 3.7+Create a virtual environment and install the dependenciespoetryinstallActivate the virtual environmentpoetryshellValidating buildmakebuildRelease processA release is automatically published when a new version is bumped usingmake bump.
See.github/workflows/build.ymlfor more details.
Once the release is published,.github/workflows/publish.ymlwill automatically publish it to PyPI.
Package: twitter-video-tools
This project is deprecated. Please look athttps://github.com/code-yeongyu/twitter_video_tools_v2Twitter Video ToolsA multi-processing supported video downloadersupports downloading videos from twitter (or specific user from twitter) && monsnode.Installwith PIPpipinstalltwitter-video-toolswith Poetrypoetryaddtwitter-video-toolsUsageCommand linepython3-mtwitter_video_tools[link]Supported link types:Video tweet:https://twitter.com/twtvtOfficial/status/1599748329927499777Video frommonsnode:https://monsnode.com/v1506575871309589251Specific user's uploaded videos:https://twitter.com/twtvtOfficial/mediaSpecific user's liked videos:https://twitter.com/twtvtOfficial/likesPython Embeddingfromtwitter_video_toolsimportTwitterVideoToolswithTwitterVideoTools()astwitter_video_tools:twitter_video_tools.download_from_user('twtvtOfficial')ContributionPrerequisitesPython 3.9poetrycode editor (vscode recommended)Overview of Development EnvironmentsLocalvscode ready (launching, debugging, formatting)strict type checking usingmypy&pyrighttype hint generatormonkeytypealso includedamazing linters & formatters (yapf,pylint,isort)unifyfor forcing single-quoteunit test usingpytestGitHub ActionsAll PRs are statically analyzed & checked byyapf,pylint,pyright,mypyAll PRs are tested withpytestCan be released with Github Action when creating GitHub ReleasesAll-in-oneghrepoclonecode-yeongyu/twitter_video_tools
python3-mpipinstallpoetry
poetryinstall# install dependenciescode--install-extensionemeraldwalk.RunOnSave# to force single quotecode--install-extensiontamasfe.even-better-toml# for handling tomlDone!Testpoetryshell
invtestInspirationsyt-dlpInspired me to start this project. yt-dlp is a fork project of youtube-dl.Since the cookie option of yt-dlp's twitter extractor is not working, I decided to make my own project, using browser automation.playwrightMicrosoft's browser automation module.Another major project to made me to start this project. I made up my mind to make TwitterVideoTools to experience playwright.It would be so painful to imagine making this project with selenium, but I enjoyed a lot while writing the twitter crawler part thanks to playwright.typerEver since I started this project, I always wanted to support CLI with typer's awesome development experience.TwitterVideoTools' CLI is written with typer, and it is so beautiful and easy to use at the same time.pyright&mypy&monkeytypeThese three tools helped me to write fully-typed python code.I won't start my python project without these tools.my python project templateI made this template to make my python project development experience better.Safe & Convient development environmentStrict type checkingAmazing linters & formattersUnit test supportedThis project is also based on this template.
Package: twittier
Twittier一些推特的实用小工具，方便用于推特抽奖等。包含status_dumper和lucky_draw两个工具。status_dumper用于导出推文的评论、喜欢、引用、转推和动态（包含所有的）。lucky_draw用于抽奖，可以从status_dumper产生的动态（包含所有的）中抽取指定数量的用户。安装pipinstalltwittier用法status_dumper使用浏览器（Chrome/Edge等）打开你的推文，按下F12打开开发者工具，切换到Network/网络选项卡后并刷新页面。将页面一直往下滚动，直到所有的回复都显示出来点开XXX则转推和XXX个喜欢和XXX引用，在打开的页面一直往下滚动，直到所有显示出来。点击开发者工具的下箭头图表，将所有的请求导出为HAR文件使用命令行命令python -m twittier status_dumper <HAR文件路径>运行程序程序会在当前目录下生成comments.json、likes.json、quotes.json、retweets.json和dynamic.txt文件，分别对应评论、喜欢、引用、转推和动态（包含所有的）。lucky_draw使用命令行命令python -m twittier draw <动态文件路径> -p [抽取数量列表]运行程序程序将会输出抽取的用户列表以及抽到的用户名单。例如：python -m twittier draw dynamic.txt -p 1 2 3，将会从dynamic.txt中抽取 1 个、2 个和 3 个用户。也可以添加-r来限定抽取的范围。默认为评论+喜欢+转推例如：python -m twittier draw dynamic.txt -p 1 2 3 -r like，将会从dynamic.txt中抽取 1 个、2 个和 3 个喜欢了推文的用户。例如：python -m twittier draw dynamic.txt -p 1 2 3 -r like retweet，将会从dynamic.txt中抽取 1 个、2 个和 3 个喜欢了推文并且转推了推文的用户。
Package: twitwi
TwitwiA collection of Twitter-related helper functions for python.InstallationYou can installtwitwiwith pip with the following command:pip install twitwiUsageNormalization functionsnormalize_tweets_payload_v2Formatting functionstransform_tweet_into_csv_dicttransform_user_into_csv_dictformat_tweet_as_csv_rowUseful constants (under twitwi.constants)TWEET_FIELDSUSER_FIELDSnormalize_tweets_payload_v2Function taking an entire tweets payload from the v2 API and returning a list of the contained tweets normalized and structured in a way that makes further analysis of the data convenient.fromtwitwiimportnormalize_tweets_payload_v2# Normalizing an entire tweets payload to extract a list of tweetsnormalize_tweets_payload_v2(payload)# Normalizing an entire tweets payload to extract a list of tweets# as well as the referenced tweets (quoted, retweeted, etc.)normalize_tweets_payload_v2(payload,extract_referenced_tweets=True)# Converting found dates to a chosen timezonefrompytzimporttimezoneparis_tz=timezone('Europe/Paris')normalize_tweets_payload_v2(payload,locale=paris_tz)Argumentspayload(dict): tweets payload coming from Twitter API v2.locale(pytz.timezone, optional): timezone used to convert dates. If not given, will default to UTC.extract_referenced_tweets(bool, optional): whether to keep referenced tweets (retweeted, quoted etc.) in the output. Defaults toFalse.collection_source*(string, optional): An optional information to add to the tweets to indicate whence you collected them.transform_tweet_into_csv_dictFunction transforming (i.e. mutating, so beware) a given normalized tweet into a suitable dict able to be written by acsv.DictWriteras a row.fromtwitwiimporttransform_tweet_into_csv_dict# The function returns nothing, `normalized_tweet` has been mutatedtransform_tweet_into_csv_dict(normalized_tweet)transform_user_into_csv_dictFunction transforming (i.e. mutating, so beware) a given normalized Twitter user into a suitable dict able to be written by acsv.DictWriteras a row.fromtwitwiimporttransform_user_into_csv_dict# The function returns nothing, `normalized_user` has been mutatedtransform_user_into_csv_dict(normalized_user)format_tweet_as_csv_rowFunction formatting the given normalized tweet as a list able to be written by acsv.writeras a row.fromtwitwiimportformat_tweet_as_csv_rowrow=format_tweet_as_csv_row(normalized_tweet)format_user_as_csv_rowFunction formatting the given normalized Twitter user as a list able to be written by acsv.writeras a row.fromtwitwiimportformat_user_as_csv_rowrow=format_user_as_csv_row(normalized_user)TWEET_FIELDSList of tweet field names. Useful to declare headers with csv writers:fromtwitwi.constantsimportTWEET_FIELDS# Using csv.writerw=csv.writer(f)w.writerow(TWEET_FIELDS)# Using csv.DictWriterw=csv.DictWriter(f,fieldnames=TWEET_FIELDS)w.writeheader()USER_FIELDSfromtwitwi.constantsimportUSER_FIELDS# Using csv.writerw=csv.writer(f)w.writerow(USER_FIELDS)# Using csv.DictWriterw=csv.DictWriter(f,fieldnames=USER_FIELDS)w.writeheader()
Package: twix
# twix #### Intro ###Simple project that extracts user tweets to csv / json <br>based on the gist of [yanofsky](https://gist.github.com/yanofsky/5436496)### Installation ###in your terminal:> pip install twix### Twitter API Keys ###1. Go to [twitter developers](https://dev.twitter.com/)2. Click on "Sign In" in the upper-right corner.* There is a link to sign up under the username field if you do not already have a Twitter account.* If you sign up for a new account, you'll have to confirm your email before you can get an API key.3. Enter your credentials and sign in.4. Back at [twitter developers](https://dev.twitter.com/), click on your avatar in the upper-right corner, then My Applications.5. Click on "Create a new application".6. Fill out the information, agree to the Rules of the Road, do the captcha, and click on "Create your Twitter application".7. In the application page that comes up next, copy down the **Consumer key** and **Consumer secret**. This is half of the key info.8. Click on "Create my access token" at the bottom of the application page, under "Your access token".9. (You may have to refresh the page first) Now under "Your access token", copy down your **Access token** and **Access token secret**.### Examples ###```pythonfrom twix.tweets_extractor import TweetsExtractorte = TweetsExtractor('your_consumer_key', 'your_consumer_secret', 'your_access_token', 'your_access_token_secret')te.get_all_tweets(screen_name='netanyahu', fmt='csv', path='../bb.csv')```### Contact ###[Tal Peretz](https://www.linkedin.com/in/tal-per/)