How to Collect Data from TikTok (Tutorial) | by Jack Bandy | Towards Data Science

As The Information put it, TikTok has “taken off like a rocket ship in the U.S. and around the world, creating a new mobile video experience that has left YouTube and Facebook scrambling to keep up.”
Just looking at U.S. users over the age of 18, TikTok went from 22.2 million unique visitors in January, to 39.2 million in April, according to Comscore data provided to Adweek.
The platform is more than just a cool new app: as recent events have shown, its algorithms spread videos that can have real-world consequences. To name a few prominent examples,
K-pop fans used TikTok to prank Trump’s rally in Tulsa, purchasing tickets and then not showing up
Teens on TikTok organized “shopping cart abandonment” on Trump’s merchandise website in attempt to hide inventory from others
Some users are encouraging Trump’s opposition to click on ads for Trump in order to drive up the campaign’s advertising costs
The app has spread manipulated videos of Joe Biden
In short, TikTok and its driving algorithms now have a sizeable real-world influence, especially considering that a typical user spends almost an hour per day watching videos on the platform. With this in mind, it is important to understand what TikTok shows to millions of eyeballs every day, and to do that, we will need some data.
Below, I have included code for how to collect TikTok data in a variety of ways. I tried to keep it general and helpful for most use cases, but you will probably need to tweak it depending on what you are doing. The remainder of this post covers how to do the following:
👤 Collect videos posted by a user❤️ Collect videos liked by a user⛄️ Snowball a list of users📈 Collect trending videos
👤 Collect videos posted by a user
❤️ Collect videos liked by a user
⛄️ Snowball a list of users
📈 Collect trending videos
(A couple notes for the nerds: if you plan to make more than a few dozen requests, I suggest setting up a proxy. I have not yet tested it, but the API I demo here should integrate with your proxy quite easily. Second, if you want to track popularity over time, you will want to add timestamps to the statistics.)
A good place to start is collecting videos from a given user. I will be using TikTok-Api by David Teather (run pip3 install TikTokApi to get the package).
To collect videos from the Washington Post TikTok account (one of my favorites), here is all you need to do in Python:
from TikTokApi import TikTokApiapi = TikTokApi()n_videos = 100username = 'washingtonpost'user_videos = api.byUsername(username, count=n_videos)
The user_videos object is now a list of 100 video dictionaries (an example dictionary is here). You will probably be mostly interested in just a few stats, which you can extract from the full dictionary with the following function:
def simple_dict(tiktok_dict):  to_return = {}  to_return['user_name'] = tiktok_dict['author']['uniqueId']  to_return['user_id'] = tiktok_dict['author']['id']  to_return['video_id'] = tiktok_dict['id']  to_return['video_desc'] = tiktok_dict['desc']  to_return['video_time'] = tiktok_dict['createTime']  to_return['video_length'] = tiktok_dict['video']['duration']  to_return['video_link'] = 'https://www.tiktok.com/@{}/video/{}?lang=en'.format(to_return['user_name'], to_return['video_id'])  to_return['n_likes'] = tiktok_dict['stats']['diggCount']  to_return['n_shares'] = tiktok_dict['stats']['shareCount']  to_return['n_comments'] = tiktok_dict['stats']['commentCount']  to_return['n_plays'] = tiktok_dict['stats']['playCount']  return to_return
Then, we can go from the API-outputteduser_videos list to a nice, clean table (i.e. Pandas data frame) with just three lines:
user_videos = [simple_dict(v) for v in user_videos]user_videos_df = pd.DataFrame(user_videos)user_videos_df.to_csv('{}_videos.csv'.format(username),index=False)
Here is what the output file looks like (I removed some rows and columns to make it readable on here):
In this case, you may be interested in the videos “liked” by a given user. This is pretty straightforward to collect. Let’s see what videos the official TikTok account has liked recently:
username = 'tiktok'n_videos = 10liked_videos = api.userLikedbyUsername(username, count=n_videos)liked_videos = [simple_dict(v) for v in liked_videos]liked_videos_df = pd.DataFrame(liked_videos)liked_videos_df.to_csv('{}_liked_videos.csv'.format(username), index=False)
And the output file looks similar to the one from last time, since it also saves a list of videos:
Say you wanted to create a large list of users from which you could collect videos they post and videos they like. You could use the 50 most-followed TikTok accounts, but 50 may not generate a wide enough sample.
An alternative approach is to use the suggested users to snowball a list of users from just one user. First, we will do this for four different accounts:
tiktok is the app’s official account
washingtonpost is one of my favorite accounts
charlidamelio is the most-followed account on TikTok
chunkysdead leads a self-proclaimed “cult” on the app
Here is the code I used:
seed_users = ['tiktok', 'washingtonpost', 'charlidamelio', 'chunkysdead']seed_ids = [api.getUser(user_name)['userInfo']['user']['id'] for user_name in seed_users]suggested = [api.getSuggestedUsersbyID(count=20, startingId=s_id) for s_id in seed_ids]
And here are the suggested users:
Notably, the list of recommendations for washingtonpost and chunkysdead were identical, and there is a lot of overlap between the other recommendations, so this approach may not give you what you need.
Another method to create a large list of users it to use the getSuggestedUsersbyIDCrawler to keep the snowball rolling, so to speak. To create a list of 100 suggested accounts using tiktok as the seed account, you just need the following code:
tiktok_id = api.getUser('tiktok')['userInfo']['user']['id']suggested_100 = api.getSuggestedUsersbyIDCrawler(count=100, startingId=tiktok_id)
This creates a list which contains a variety of different celebrity accounts, here are a few:
@lizzo (lizzo, 8900000 fans)@wizkhalifa (Wiz Khalifa, 1800000 fans)@capuchina114 (Capuchina❗️👸🏼, 32600 fans)@silviastephaniev (Silvia Stephanie💓, 27600 fans)@theweeknd (The Weeknd, 1400000 fans)@theawesometalents (Music videos, 33400 fans)...
From what I observed, the getSuggestedUsersbyIDCrawler method starts to branch out and find smaller, more niche accounts, which have tens of thousands of followers rather than hundreds of thousands or millions. This is good news if you want a representative dataset.
If you want to collect a wide sample of data from TikTok, I advise starting with the suggested users crawler.
Finally, maybe you simply want to collect trending videos for a simple content analysis, or just to keep up 🙂. The API makes that pretty simple, as follows:
n_trending = 20trending_videos = api.trending(count=n_trending)trending_videos = [simple_dict(v) for v in trending_videos]trending_videos_df = pd.DataFrame(trending_videos)trending_videos_df.to_csv('trending.csv',index=False)
And here is the output file for trending videos on Thursday afternoon (July 2nd, 2020):
That will be all for this tutorial, thank you for reading! Here is a file with all the code I used.