Hacking together your own Youtube Suggest API
Tags: youtube, api, search, react
Published: 2020-01-13 12:02:18

---
title: Hacking together your own Youtube Suggest API
published: true
description: Learn how to leverage hidden APIs for your web app using simple Chrome Dev Tool techniques
tags: youtube, apis, search, react
series: Audio Streaming Progressive Web App
---

##Premise

I had some extra time last weekend and decided to work on a small project that would solve my own problem. It’s suppose to be a really simple mobile PWA that only streams Youtube audio and loops repeatedly. There’s a bunch of them out there so it’s not exactly an original idea, but I wanted something simple without all the extra cruft like ads, comments, or authentication.

##Feature to Implement
Right off the bat, there was a core functionality I needed to implement. A “search box” for querying the YT api for videos. The API for video search is straightforward enough: https://developers.google.com/youtube/v3/docs/search/list. Just sign up for the Youtube Data API key and we have access to all the power of Youtube search.

Now, we could just build one that the user enters in the query and then taps a button to search, but it would be much more user friendly if there was autocomplete suggest like the actual Youtube site. Auto suggest would make it so much easier to use. Here is an image of desire functionality from youtube.

![Alt Text](https://thepracticaldev.s3.amazonaws.com/i/86ui8dg9eknjwv15qeqt.png)

##No API 😞
But, alas! After a couple hours of searching on both Google and Youtube API docs, it turns out Google does not provide an official auto-suggest API. Or it seems, they provided the API previously, but decided to shut it down for some reason. Now at this point, I could just go ahead with the core search functionality and forget about the auto suggestions… but let’s have a look at Youtube first just out of curiosity.

##Looking underneath the hood
Over at Youtube, if we start typing in the search bar, and open up the Chrome dev tools, we see network requests being made that points to an undocumented API endpoint: `https://clients1.google.com/complete/search?client=``youtube``&hl=en&gl=sg&gs_rn=64&gs_ri=youtube&tok=h3yTGb1h3-yuCBwsAaQpxQ&ds=yt&cp=3&gs_id=2u&q=jaz&callback=``google.sbox.p50``&gs_gbg=0l0MjG05RWnWBe9WcipQbsy`

After playing around with the parameters, it turns out that most of the params are not really needed. The main ones that matter to our use case is:

- client: forces json response, we want to use `youtube` here
- ds: google site properties, use `yt` to restrict to Youtube
- hl: the culture or language. use for localization. Default is usually `en`
- callback: this is the jsonp callback
- q: term query you want to search for

At this point, the endpoint works. If you tried it in the browser now, you would download a text. It’s strange text file with numbers and gibberish, but inside it we clearly see the data we need to implement our autocomplete search. Hooray!


```
// contents of autosuggest endpoint
google.sbox.p50 && google.sbox.p50(["jazz ",[["jazz music",0],["jazz piano",0],["jazz songs",0],["jazz dance",0,[131]],["jazz music best songs",0],["jazz instrumental",0],["jazz guitar",0],["jazz relaxing music",0,[131]],["jazz jennings",0],["jazz for work",0,[131]]],{"a":"FCwlE6frPjfCHAJSPzskH5xxMxJia3UhfNxNRVG6aehsz7iBn4XxJQ6ACUGMVuaAl5f1LHrO2ErGn7t4d6mIXg965Zxp3bENM4iS00nEvwhiiSe8Bi39NZsbdj2BHz3FD0C","j":"32","k":1,"q":"8KKe7s-xREtd_veunmBB7oKGghg"}])
```

You might recognized this as `jsonp` and if not, then a few google searches, and we have the answer! The `google.sbox.p50` is the callback function which we’ll pass in ourselves.

In my side project, I’m using `axios`, and we can find a `jsonp` adaptor for `axios` here. The basic request logic looks like this:


```javascript
    export const suggest = (term: string) => {
      const GOOGLE_AC_URL: string = `https://clients1.google.com/complete/search`;
      return axios({
        // A YT undocumented API for auto suggest search queries
        url: GOOGLE_AC_URL,
        adapter: jsonpAdapter,
        params: {
          client: "youtube",
          hl: "en",
          ds: "yt",
          q: term,
        }
      })
      .then((res: AxiosResponse) => {
        console.log("jsonp results >> ", res);
        if (res.status !== 200) {
          throw Error("Suggest API not 200!");
        }
        return res.data[1].map((item: any[]) => item[0]);
      })
    }
```

Now, we just need to connect the results to a react component input element and wire up the state and change handlers. I have used `styled-components` components to rename and style many of the html elements. The bare bones implementation is show here:

```javascript

    /* search-box component */
    import * as React from "react";
    import styled from "styled-components";
    import * as _ from "lodash";
    
    import {youtubeSearch, suggest} from "./services/youtube";
    
    export default class SearchBox extends React.PureComponent<any, any> {
      public state = {
        suggestions: null,
      };
    
      constructor(props: any) {
        super(props);
        this.onTypeSuggest = _.debounce(this.onTypeSuggest, 500, { leading: true });
      }
    
      public render() {
        return (
          <>
            <Container>
              <SearchBox
                onChange={e => this.onTypeSuggest(e.target.value)}
                placeholder="Search for music, songs, podcasts"
                type="search"
              />
            </Container>
            <Suggestions onSearch={this.onSearch} items={this.state.suggestions} />
          </>
        )
      }
    
      private onTypeSuggest = async (
        queryString: string,
      ) => {
        if (queryString.length < 5) {
          // search only after 5 chars
          return null;
        }
        const list = await suggest(queryString);
        return this.setState({ suggestions: list });
      }
      
      // Search the term when selected using the official youtube search api:
      // https://www.googleapis.com/youtube/v3/search
      private onSearch = async (queryString: string, platform: MediaPlatforms) => {
        if (platform === MediaPlatforms.Youtube) {
          const platformSearchResults = await youtubeSearch(queryString);
          this.setState({ suggestions: null });
          this.props.update(platformSearchResults);
        }
      }
    }
```

We also want to show the suggestions that we get from our hand crafted youtube autosuggest api. A simple list of items will do. Each suggestion is passed the onSearch function which takes the autosuggestion selected and queries the official youtube search api above.

```javascript
function Suggestions({ onSearch, items }: any) {
  if (!items || !items.length) {
    return null;
  }
  return (
    <Section>
      {items.map((item: string, key: number) => {
        return <SuggestionRow onSearch={onSearch} key={key} text={item} />;
      })}
    </Section>
  );
}

function SuggestionRow({ onSearch, text }: any) {
  return (
    <SuggestionSpan onClick={() => onSearch(text, MediaPlatforms.Youtube)}>
      {text}
    </SuggestionSpan>
  );
}
```

And add some styling to pretty things up a bit.

```javascript
const Container = styled("div")`
  position: relative;
  width: 100%;
  display: flex;
  flex-flow: row nowrap;
  align-items: center;
  justify-content: flex-start;
`;
const SearchBox = styled("input")`
  box-sizing: border-box;
  height: 2.9rem;
  width: 100%;
  padding: 0.5rem;
  padding-left: 1rem;
  border-radius: 0.2rem;
  border: 2px solid #aaa;
  max-width: 800px;
  font-size: 1rem;
`;
const CSearchIcon = styled(SearchIcon)`
  cursor: pointer;
  margin-left: -50px;
`;
const Section = styled("section")`
  width: 95%;
  min-height: 18rem;
  height: auto;
  border: 1px solid #ddd;
  border-top: none;
  border-radius: 5px;
  margin-top: 1rem;
  padding: 0.5rem;
  box-shadow: 1px 1px 1px #ddd;
  z-index: 1000;
`;
const SuggestionSpan = styled("span")`
  display: inline-block;
  width: 100%;
  color: #9c27b0;
  font-weight: 800;
  margin-bottom: 0.5rem;
  margin-left: 0.5rem;
  cursor: pointer;
  z-index: 1000;
`;
```

Awesome. Looks like it works as expected!

![Autosuggest search demo 1](https://thepracticaldev.s3.amazonaws.com/i/x3ugyobqwcu8bmohyypn.png)
![Autosuggest demo 2](https://thepracticaldev.s3.amazonaws.com/i/agx028t43onaj2qfrlnj.png)


There might be many creative uses of the autosuggest api, not just for the obvious search suggestions. Would be interesting to hear some others. But make sure to use responsibly!

## Finishing up and next steps

Now that the basic functionality works. We could package it up nicely into a react hook so it’s easily usable in any component or by anyone with an `npm install`

If you want to reverse engineer the Youtube stream, that’s an another level of complexity and very interesting topic to explore. A good starting point would be here: `https://tyrrrz.me/blog/reverse-engineering-youtube` and checkout out the `node-ytdl` source or just use the package!

I may write follow-up post on Node.js audio streaming once I learn more about the topic. Also, I would like to write about the intricacies of audio playback for mobile browsers such as Chrome and Safari as it relates to PWAs in a future post.

Cheers.