Repository: osome-iu/osometweet (language: python)

Issue:
Add example scripts
Create scripts that can be used to gather data using the available endpoints.

Here are the endpoints available in `osometweet` at the moment:
- [x] `tweet_lookup`
- [x] `user_lookup_ids`
- [x] `user_lookup_usernames` 
- [x] `get_tweet_timeline` 
- [x] `get_mentions_timeline` 
- [ ] `get_followers`
- [ ] `get_following`
- [ ] `search`
- [x] `sampled_stream`
- [x] `filtered_stream`

Please feel free to claim one of these scripts if you're interested! 😄 

See the [`examples/`](https://github.com/osome-iu/osometweet/tree/streams/examples) folder for what is available.

---

> Note: The original comment for this issue has been completely edited from it's initial form b/c it was originally opened as a question/discussion about whether we should do this. Now that we've decided to make this standard practice, I wanted to clean it up.


Discussion / hints:
Good idea! I can take one or two, but maybe after you finish one case so I can see the standard.
Awesome, thanks! I'll post updates on this issue. 

Fix patch:
diff --git a/README.md b/README.md
index 55e7e70..411f96a 100644
--- a/README.md
+++ b/README.md
@@ -74,4 +74,6 @@ which returns a list of dictionaries, where each dictionary contains the request
 ]
 ```
 
+We offer [examples](examples) for working with different endpoints.
+
 For detailed instructions, check out our [wiki](https://github.com/osome-iu/osometweet/wiki).
diff --git a/examples/README.md b/examples/README.md
index 4403c61..45584af 100644
--- a/examples/README.md
+++ b/examples/README.md
@@ -1,8 +1,13 @@
-### Examples
+# Introduction
 
-> This folder is intended to house example scripts which are actually useful.
+This folder is intended to house example scripts that are actually useful.
 
+For every endpoint, we provide an example script to demonstrate how it works.
 
-##### Scripts
-* `user-lookup-ids.py` : A command-line script for scraping many users account information. This script takes a file containing user IDs on each line as an input. You can test this script with the file [`test_data/test_user_ids.txt`](https://github.com/truthy/osometweet/blob/master/examples/test_data/test_user_ids.txt) which is a random collection of user IDs, one per row. If you'd like to use this script on a different set of user IDs, the input file of new user IDs must follow the same format.
-* `user-lookup-usernames.py` : A command-line script for scraping many users account information. This script takes a file containing usernames on each line as an input. You can test this script with the file [`test_data/test_user_names.txt`](https://github.com/osome-iu/osometweet/blob/documentation/examples/test_data/test_user_names.txt) which is a random collection of usernames, one per row. If you'd like to use this script on a different set of usernames, the input file of new usernames must follow the same format.
\ No newline at end of file
+
+
+| Endpoint type  | Endpoint | Note | Command |
+|----------------|----------|------|---------|
+| Tweet endpoint | Tweets lookup | Scraping tweets with tweet ids | `python tweet-lookup.py -f test_data/test_tweet_ids.txt` |
+| User endpoint  | User lookup with ids | Scraping many users account information with ids | `python user-lookup-ids.py -f test_data/test_user_ids.txt` |
+| User endpoint  | User lookup with usernames | Scraping many users account information with usernames | `python user-lookup-usernames.py -f test_data/test_user_names.txt` |
diff --git a/examples/tweet-lookup.py b/examples/tweet-lookup.py
new file mode 100644
index 0000000..f5b6cf5
--- /dev/null
+++ b/examples/tweet-lookup.py
@@ -0,0 +1,157 @@
+#!/usr/bin/env python3
+
+# Script Information
+#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+"""
+PURPOSE:
+    -Script to scrape tweets with the Twitter V2 get tweets endpoint
+
+INPUT:
+    - A file of tweet IDs where each line contains one id
+
+OUTPUT:
+    - tweet_data--{todays-date}.json : a file where each line
+    represents one tweet
+
+    - tweet_errors--{todays-date}.json : a file which records any
+    errors received (one per line). You can then learn why certain ids
+    were not returned.
+
+Author: Matthew R. DeVerna, Kaicheng Yang
+"""
+
+
+# Import packages
+#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+import sys
+import argparse
+import os
+import osometweet
+from osometweet.utils import chunker
+import json
+from datetime import datetime as dt
+
+
+# Create Functions.
+#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+def parse_cl_args():
+    """Set CLI Arguments."""
+
+    # Initiate the parser
+    parser = argparse.ArgumentParser(
+        description="Script to scrape tweet information."
+    )
+    # Add optional arguments
+    parser.add_argument(
+        "-f", "--file",
+        metavar='File',
+        help="Full path to the file containing the USER IDS you would like to scrape.",
+        required=True
+    )
+
+    # Read parsed arguments from the command line into "args"
+    args = parser.parse_args()
+
+    # Assign the file name to a variable and return it
+    ids_file = args.file
+    return ids_file
+
+def load_tweet_ids(ids_file):
+    # Load all tweet ids, returning a list of lists, each 100
+    # users long.
+
+    # This allows us to iterate through a long list of tweet ids
+    # 100 tweets at a time (which is the maximum number of ids
+    # we can query Twitter for in one call).
+
+    with open(ids_file, 'r') as f:
+        tweet_ids = [x.strip('\n') for x in f.readlines()]
+
+    max_query_length = 100
+
+    chunked_list = chunker(
+        seq = tweet_ids,
+        size = max_query_length
+        )
+    return chunked_list
+
+def load_bearer_token():
+    """Load Twitter Keys from Local Environment."""
+
+    # To set your enviornment variables in your terminal execute a command like the
+    # one that you see below.
+
+    # Example:
+    # export 'TWITTER_BEARER_TOKEN'='<your_twitter_bearer_token>'
+
+    # Do this for all of your tokens, and then load them with the commands below,
+    # matching the string in the .get("string") to the name you chosen to the
+    # left of the equal sign above.
+
+    # Set Twitter tokens/keys.
+    #~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+    bearer_token = os.environ.get("TWITTER_BEARER_TOKEN")
+
+    return bearer_token
+
+def gather_data(
+    bearer_token,
+    chunked_list,
+    ids_file
+    ):
+    print("Gathering Data...")
+
+    oauth2 = osometweet.OAuth2(bearer_token=bearer_token)
+    ot = osometweet.OsomeTweet(oauth2)
+
+    # Add all tweet fields
+    all_tweet_fields = osometweet.TweetFields(everything = True)
+
+    # Get today's date
+    today = dt.strftime(dt.today(), "%Y-%m-%d_%H-%M")
+
+    # Open two files. One for good data, the other for tweet errors.
+    with open(f"tweet_data--{today}.json", 'w') as data_file, open(f"tweet_errors--{today}.json", 'w') as error_file:
+
+        # Iterate through the list of lists
+        for one_hundred_tweets in chunked_list:
+            response = ot.tweet_lookup(
+                tids=one_hundred_tweets,
+                fields=all_tweet_fields
+            )
+
+            # Get data and errors
+            data = response["data"]
+            errors = response["errors"]
+
+            # No matter what `data` and `errors` will return something, however,
+            # they may return `None` (i.e. no data/errors), which will throw a TypeError.
+            try:
+                data_file.writelines(f"{json.dumps(line)}\n" for line in data)
+            except TypeError:
+                print(
+                    "No data found in this set of tweets, skipping to the next set.")
+                pass
+
+            try:
+                error_file.writelines(f"{json.dumps(line)}\n" for line in errors)
+            except TypeError:
+                print(
+                    "No problematic tweets found in this set of tweets, skipping to the next set.")
+                pass
+
+
+# Exectue the program
+#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+if __name__ == "__main__":
+    ids_file = parse_cl_args()
+    chunked_list = load_tweet_ids(ids_file)
+    bearer_token = load_bearer_token()
+
+    gather_data(
+        bearer_token = bearer_token,
+        chunked_list = chunked_list,
+        ids_file = ids_file,
+    )
+
+    print("Data pull complete.")
\ No newline at end of file


Test patch:
diff --git a/examples/test_data/test_tweet_ids.txt b/examples/test_data/test_tweet_ids.txt
new file mode 100644
index 0000000..091d9c6
--- /dev/null
+++ b/examples/test_data/test_tweet_ids.txt
@@ -0,0 +1,500 @@
+1052106874160590848
+1052106874160644097
+1052106874164633601
+1052106874168983553
+1052106874169032704
+1052106874173177858
+1052106874181554176
+1052106874181562368
+1052106874181566464
+1052106874181574657
+1052106874185609216
+1052106874185646080
+1052106874189959170
+1052106874198380549
+1052106874198392832
+1052106874198401024
+1052106878354907136
+1052106878367485952
+1052106878367518720
+1052106878371721216
+1052106878375944193
+1052106878380007425
+1052106878380109825
+1052106878380113921
+1052106878384136192
+1052106878388441088
+1052106882553241600
+1052106882553454592
+1052106882561830912
+1052106882565849089
+1052106882565967872
+1052106882570211328
+1052106882574413825
+1052106882578436096
+1052106882578571265
+1052106882582626304
+1052106882582753286
+1052106882582765568
+1052106882586824704
+1052106882586955776
+1052106886751780865
+1052106886751944704
+1052106886764326912
+1052106886764478464
+1052106886768676864
+1052106886772903936
+1052106890942062593
+1052106890950389760
+1052106890954473472
+1052106890954575872
+1052106890958778368
+1052106890958786560
+1052106890962972672
+1052106890967015424
+1052106890967166976
+1052106890967171072
+1052106890967175168
+1052106890975559681
+1052106890975596544
+1052106890975596545
+1052106895144693760
+1052106895148822528
+1052106895148892161
+1052106895148929024
+1052106895153074176
+1052106895153135616
+1052106895153135617
+1052106895161356293
+1052106895161479169
+1052106895161520129
+1052106895165706245
+1052106899326414848
+1052106899326464000
+1052106899330461696
+1052106899330498560
+1052106899338932225
+1052106899338989568
+1052106899339038720
+1052106899339051009
+1052106899343036416
+1052106899343081472
+1052106899343204353
+1052106899347333121
+1052106899347439616
+1052106899355779072
+1052106899359973376
+1052106899360010240
+1052106903520591873
+1052106903533314050
+1052106903537537024
+1052106903537537025
+1052106903554146304
+1052106903554314240
+1052106907723411457
+1052106907727605760
+1052106907744391168
+1052106907748458496
+1052106907748577280
+1052106907748589570
+1052106907752656896
+1052106911913521155
+1052106911917649920
+1052106911921958913
+1052106911930351616
+1052106911934484481
+1052106911942762496
+1052106911946956800
+1052106911947067394
+1052106911947075585
+1052106911947075586
+1052106916103639040
+1052106916103680000
+1052106916103684096
+1052106916107902976
+1052106916112019456
+1052106916116148226
+1052106916124594177
+1052106916128804864
+1052106916133052416
+1052106916137062400
+1052106916137062401
+1052106916137234432
+1052106920297811968
+1052106920302133248
+1052106920306196480
+1052106920306364416
+1052106920306372608
+1052106920310571008
+1052106920314789888
+1052106920318754817
+1052106920318758912
+1052106920318898181
+1052106920318947328
+1052106920323178497
+1052106920327139328
+1052106920327172096
+1052106920327372800
+1052106924496510976
+1052106924500619264
+1052106924500676608
+1052106924500705280
+1052106924504694784
+1052106924517466113
+1052106924525637633
+1052106924525793280
+1052106928694992896
+1052106928703168517
+1052106928707399681
+1052106928707448833
+1052106928707518464
+1052106928724176896
+1052106932885090304
+1052106932889169920
+1052106932889296896
+1052106932893376512
+1052106932893478912
+1052106932897624064
+1052106932897685504
+1052106932901822465
+1052106932901826560
+1052106932901859328
+1052106932905893888
+1052106932905893889
+1052106932906012673
+1052106932918603776
+1052106937074995200
+1052106937075040256
+1052106937075154944
+1052106937079406592
+1052106937083551747
+1052106937096130560
+1052106937100324867
+1052106937108709376
+1052106937108758528
+1052106941277728768
+1052106941282045953
+1052106941282045955
+1052106941294620673
+1052106941307052032
+1052106945467805696
+1052106945467965440
+1052106945472208896
+1052106945480597504
+1052106945484791808
+1052106945493118976
+1052106945497362433
+1052106949658066944
+1052106949666299904
+1052106949674717185
+1052106949683101696
+1052106949683220480
+1052106949683236865
+1052106949687484416
+1052106949687484417
+1052106949691490304
+1052106949695852544
+1052106953856446464
+1052106953860767744
+1052106953864802305
+1052106953873227776
+1052106958046560258
+1052106958046662656
+1052106958050750466
+1052106958055112704
+1052106958063452160
+1052106958067519488
+1052106958071828480
+1052106958076084224
+1052106958080229376
+1052106958084308992
+1052106958084460544
+1052106958084468736
+1052106962241036289
+1052106962245042176
+1052106962245177344
+1052106962245230592
+1052106962249371648
+1052106962257801216
+1052106962278785024
+1052106966435160064
+1052106966439473152
+1052106966439477248
+1052106966460452865
+1052106966464679937
+1052106966464700416
+1052106970633654272
+1052106970633826304
+1052106970637979649
+1052106970646204416
+1052106970650632192
+1052106970654756864
+1052106970654756865
+1052106970663149568
+1052106970667323392
+1052106974823878657
+1052106974823964672
+1052106974832316416
+1052106974840545281
+1052106974840664064
+1052106974840672256
+1052106974844911616
+1052106974848929792
+1052106974848933888
+1052106974849101824
+1052106974853300225
+1052106974857453568
+1052106974861570049
+1052106974861643776
+1052106979018231809
+1052106979018240000
+1052106979022376960
+1052106979026460672
+1052106979030618112
+1052106979035045889
+1052106979039154176
+1052106979043364864
+1052106979043364865
+1052106979055886336
+1052106983216689152
+1052106983229120512
+1052106983229124608
+1052106983246057473
+1052106983250255873
+1052106983250305025
+1052106987411070977
+1052106987415040000
+1052106987415040001
+1052106987419373568
+1052106987427643392
+1052106987427848192
+1052106987431845888
+1052106987436105728
+1052106987436167168
+1052106987440357378
+1052106991609372672
+1052106991609425920
+1052106991617957888
+1052106991626145792
+1052106991626268672
+1052106991630307328
+1052106991630503936
+1052106991630524416
+1052106991634669568
+1052106991634698240
+1052106991638700032
+1052106991638728704
+1052106995795456000
+1052106995795456001
+1052106995803787265
+1052106995803795456
+1052106995803873280
+1052106995812057088
+1052106995812118529
+1052106995816267776
+1052106995820584960
+1052106995824648193
+1052106995824828416
+1052106995833167872
+1052106995833217026
+1052106999989702656
+1052106999989772288
+1052107000010534912
+1052107000010735616
+1052107000010743808
+1052107000014884866
+1052107000027332609
+1052107000027500544
+1052107000027512832
+1052107004183859200
+1052107004183867392
+1052107004200640512
+1052107004200669185
+1052107004200792064
+1052107004205027329
+1052107004205035520
+1052107004213264384
+1052107004213374977
+1052107004221841408
+1052107008382365696
+1052107008382521344
+1052107008382554112
+1052107008382566400
+1052107008386707456
+1052107008386711552
+1052107008386756608
+1052107008394973184
+1052107008399142913
+1052107008403484675
+1052107008403488768
+1052107008407560192
+1052107008411746304
+1052107008411926528
+1052107008416002049
+1052107008416075776
+1052107012576661504
+1052107012576825345
+1052107012593475589
+1052107012601835520
+1052107012602023936
+1052107012610412544
+1052107012610424832
+1052107016766967809
+1052107016766980096
+1052107016775319557
+1052107016783650816
+1052107016783695872
+1052107016783708160
+1052107016800489473
+1052107016800534528
+1052107020961239040
+1052107020961239041
+1052107020965306368
+1052107020969672704
+1052107020973658113
+1052107020973662208
+1052107020982263808
+1052107020990439424
+1052107020990668800
+1052107020994666496
+1052107020994834432
+1052107020994838528
+1052107025159782400
+1052107025176518657
+1052107025176584192
+1052107025180692480
+1052107025180762112
+1052107025184944129
+1052107025188978688
+1052107025193361408
+1052107029362421760
+1052107029366460418
+1052107029366493187
+1052107029370757122
+1052107029379076096
+1052107029379198976
+1052107029387640833
+1052107029387665409
+1052107033548218368
+1052107033548390400
+1052107033560805376
+1052107033560928256
+1052107033569361920
+1052107033573543938
+1052107033573552128
+1052107033581883392
+1052107033581899776
+1052107037738455040
+1052107037742718976
+1052107037755285504
+1052107037759303680
+1052107037759410178
+1052107037763616768
+1052107037772054529
+1052107037776248832
+1052107041937022976
+1052107041945178112
+1052107041953714177
+1052107041957957632
+1052107041962045441
+1052107041966358528
+1052107046126903296
+1052107046126936064
+1052107046131326976
+1052107046131326978
+1052107046139527168
+1052107046143909888
+1052107046156423168
+1052107046164803584
+1052107050321412096
+1052107050325442560
+1052107050329694209
+1052107050329808896
+1052107050334019584
+1052107050354790401
+1052107050354954240
+1052107050359111680
+1052107050359156737
+1052107054519869441
+1052107054519898113
+1052107054540775425
+1052107058709852160
+1052107058722422784
+1052107058722435072
+1052107058726596609
+1052107058735132672
+1052107058735140864
+1052107058739376128
+1052107058743525381
+1052107058743599104
+1052107062912536577
+1052107062929428480
+1052107062929428481
+1052107062933630976
+1052107062937817088
+1052107067102781440
+1052107067102830593
+1052107067111165952
+1052107067115204608
+1052107067119546368
+1052107067127943168
+1052107067136327680
+1052107067136401408
+1052107071301140480
+1052107071301320704
+1052107071305469952
+1052107071305515008
+1052107071309717504
+1052107071309733888
+1052107071313899520
+1052107071313928192
+1052107071322288128
+1052107071330619392
+1052107071330705408
+1052107075487129602
+1052107075487129604
+1052107075491225600
+1052107075495415809
+1052107075499646976
+1052107075499806720
+1052107075504013313
+1052107075516592128
+1052107075520749568
+1052107075524878337
+1052107079685681152
+1052107079685681153
+1052107079698153473
+1052107079702462464
+1052107079710908416
+1052107079719227397
+1052107079719297024
+1052107083875831808
+1052107083888377856
+1052107083892580352
+1052107083905155072
+1052107083909406721
+1052107083913551873
+1052107088070172672
+1052107088082563072
+1052107088086761472
+1052107088095301634
+1052107088099467264
+1052107088099467265
+1052107092268527616
+1052107092272664576
+1052107092276826112
+1052107092277043200
+1052107092281065473
+1052107092285419520
+1052107092293705728
+1052107092293767168
+1052107092297830401
+1052107092297957376
+1052107092302000128
+1052107096458706944
+1052107096467091458
+1052107096467095552
+1052107096471277569
+1052107096471339008
+1052107096483930113
+1052107096492318721