Zero to SaaS - Bookmarksy.io Working around Twitter API Limitations
Tags: webdev, javascript, beginners, programming
Published: 2022-11-29 00:37:21

One of the most important features of [bookmarksy.io](http://bookmarksy.io) is it’s ability to reach out to Twitters API and retrieve a given users bookmarks.

Getting a list of users bookmarks is the easy bit. You can make a call to 

```jsx
https://api.twitter.com/2/users/:id/bookmarks
```

and what is returned is a list that looks something like this:

```jsx
{
    "data": [
        {
            "created_at": "2022-11-11T18:18:36.000Z",
            "edit_controls": {
                "edits_remaining": 5,
                "is_edit_eligible": false,
                "editable_until": "2022-11-11T18:48:36.000Z"
            },
            "text": "Build a $50 info-product in one day using this process:\n\n(Probably shouldn't give this away for free bc later I'm going to sell a guide to this same process for ~$50.)",
            "public_metrics": {
                "retweet_count": 242,
                "reply_count": 85,
                "like_count": 1171,
                "quote_count": 29
            },
            "source": "Tweet Hunter Pro",
            "conversation_id": "1591133306069843979",
            "lang": "en",
            "id": "1591133306069843979",
            "reply_settings": "everyone",
            "possibly_sensitive": false,
            "edit_history_tweet_ids": [
                "1591133306069843979"
            ]
        }
    ],
    "includes": {
        "users": [
            {
                "name": "Rob Lennon 🗯 | Audience Growth",
                "username": "thatroblennon"
            }
        ],
        "tweets": [
            {
                "created_at": "2022-11-11T18:18:36.000Z",
                "edit_controls": {
                    "edits_remaining": 5,
                    "is_edit_eligible": false,
                    "editable_until": "2022-11-11T18:48:36.000Z"
                },
                "text": "Build a $50 info-product in one day using this process:\n\n(Probably shouldn't give this away for free bc later I'm going to sell a guide to this same process for ~$50.)",
                "public_metrics": {
                    "retweet_count": 242,
                    "reply_count": 85,
                    "like_count": 1171,
                    "quote_count": 29
                },
                "source": "Tweet Hunter Pro",
                "conversation_id": "1591133306069843979",
                "lang": "en",
                "id": "1591133306069843979",
                "reply_settings": "everyone",
                "possibly_sensitive": false,
                "edit_history_tweet_ids": [
                    "1591133306069843979"
                ]
            }
        ]
    },
    "meta": {
        "result_count": 1,
        "next_token": "zldjwdz3w6sba13nbs0z2lzml73coscn3mc18c0avxs"
    }
}
```

And that’s all fine and well. But if I’m dealing with a thread, which most bookmarks tend to be, how do I retrieve all the related tweets?

The twitter documentation speaks of use of recent search or archive search for tweets. This isn’t viable for two reasons:

1. recent search only works on tweets less than seven days old
2. archive search is enrollment based and EXPENSIVE!.

So what’s the solution?  Here’s what I’ve implemented to work around this:

assumptions:

- A bookmark is either a single tweet, or a collection (conversation: twitter language) of tweets.
- If a bookmark is a collection of tweets, it can be assumed that this bookmark is a thread.
- Thread tweets are typically created around the same time (roughly < 10 seconds between tweets).

Solution:

Leverage the 

```jsx
https://api.twitter.com/2/users/:id/tweets
```

endpoint with a request body as such:

```jsx
{
	since_id: <ID OF BOOKMARKED TWEET>,
	end_time: <CREATED_AT OF BOOKMARKED TWEET + 5 MINS>,
	limit: 100
}
```

this will retrieve the first 100 tweets published within the first five minutes after the initial tweet.

Then we can filter on those tweets by `conversation_id` to retrieve the ones associated with our bookmark! 

With this solution, we work around the 7 day and archive restrictions. Is it ideal? of course not, but it’s been reliable so far! And unless your bookmark is a thread of over one hundred tweets, there doesn’t appear to be any data loss.

The only drawback is that I will likely end up querying redundant tweets which would eat into my monthly query limit, but we will cross that bridge when we get to it! 

[https://bookmarksy.io](https://bookmarksy.io) is coming along nicely so far. Hit a couple of snag’s but I will be looking for beta testers soon! 

Follow the journey on twitter [bookmarksy_io](https://twitter.com/bookmarksy_io) or [brandonkpbailey](https://twitter.com/brandonkpbailey) and join the waitlist! [https://brandonkpbailey.substack.com/](https://brandonkpbailey.substack.com/)