How to make a CLI based movie scraper using NodeJS
Tags: javascript, scraper, node, tutorial
Published: 2022-05-02 02:03:18

#How to make a CLI based movie scraper using NodeJS

This guide will instruct you on how to make a CLI application that will ask the user for movie, and present info about that movie in return. To be most successful you should have at least a beginers understanding of Node and NPM. You should also have a basic understanding of using bash to navigate files. Although you are free to use a GUI application if you wish, only terminal examples will be provided.


## Prerequsites  

- A MovieDB API Key


	You will need a free acount with TheMovieDB to obtain and API key. You can sign up for one [here](https://www.themoviedb.org/signup) 
 Once you have a key, you can consult the [API Documentation](https://www.themoviedb.org/documentation/api)
 
- Node JS

	You will need to have Node JS installed. Instruction for various platforms can be found [here](https://nodejs.dev/learn/how-to-install-nodejs)

- A text editor or IDE of your choice.

- A terminal that is capable of running bash commands.




##Part One (Setup Folder & File Structure, Install NPM Packages)

First, create the root folder for the project. This guide will call it `movie-scraper`. Then navigate into the newly created folder.

```bash
mkdir movie-scraper && cd movie-scraper
```


Within the `movie-scraper` directory run the following command.

```bash
npm init -y
```


This will create the `package.json` file. You need to make two modifications to this file.
Add `  "type": "module"` to the top level, and `"start": "node src/index.js"` to the scripts section.

It should look like this. (You may have some slight differences depending on your pesoral npm settings, just focus on the changes needed above) 

```json
{
  "name": "movie-scraper",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "type": "module",
  "scripts": {
    "start": "node src/index.js",
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "keywords": [],
  "author": "",
  "license": "ISC",
  "dependencies": {
  }
}
```

Run this next command in your terminal to install the npm packages `axios` `dotenv` and `inquirer`

```bash
npm i axios dotenv inquirer
```

You should now see a `node-modules` directory in your root folder.


```bash
.
├── node_modules
├── package-lock.json
├── package.json

```
You should also see the newly installed packages listed in your `package.json` dependcies

```json
"dependencies": {
	"axios": "^0.26.1",
	"dotenv": "^16.0.0",
	"inquirer": "^8.2.2"
}
```

Now you are going to create a `.env` file that will hold your API key for TheMovieDB. If you are using git or any other version control for this project, remember to not commit this file.

Replace `PUT_YOUR_API_KEY_HERE` with your actual API key. Keep the backslashes and quotation marks.

```bash
echo TMDB_API_KEY=\"PUT_YOUR_API_KEY_HERE\" >> .env
```

You can verify the file was created using the `cat` command 

```bash
cat .env
```

You will see th following output with your API key in place of the Xs
```javascript
TMDB_API_KEY="XXXXXXXXXX"
```

Create the `src` directory that will contain the logic for the program, and navigate into it.

``` bash
mkdir src && cd src
```

Create the the following files in the src directory: 

`Movie.js`
`config.js`
`getByID.js`
`index.js`
`prompt.js`
`searchMovie.js`

You can do this in one command in your terminal

```bash
touch {Movie,config,getByID,index,prompt,searchMovie}.js
```

Your project should now have the following file stucture

```bash
.
├── .env
├── node_modules
├── package-lock.json
├── package.json
└── src
    ├── Movie.js
    ├── config.js
    ├── getByID.js
    ├── index.js
    ├── prompt.js
    └── searchMovie.js

```

##Part Two (Javascript Files)

Copy/Paste the following code to each correspodning file


###`src/config.js`

```javascript
import dotenv from "dotenv";
dotenv.config();

export default{
	tmbdkey: process.env.TMDB_API_KEY,
};
```


###`src/getByID.js`

```javascript
import config from "./config.js";
import axios from 'axios'

const getByID = async (id) => {
		const options={
			params: {
				api_key: config.tmbdkey,
				language: "en-US",
				append_to_response: `credits`
			}
	}
	
	let response = await axios.get(`https://api.themoviedb.org/3/movie/${id}`, options)
	return response.data
	
}

export default getByID;

```


###`src/index.js`

```javascript
import inquirer from 'inquirer';
import Movie from './Movie.js'
import moviePrompts from './prompt.js'

const movieResponse = await inquirer.prompt(moviePrompts)
const selectedMovie = await new Movie(movieResponse.movieID);
selectedMovie.summary()
```

###`src/Movie.js`

```javascript
import getByID from './getByID.js'

class Movie {
	constructor(id) {
		return (async () => {
			this.details = await getByID(id)
			return this
		})()
	}

	get title() {
		return this.details.title
	}

	get tagline() {
		return this.details.tagline
	}

	get overview() {
		return this.details.overview
	}

	get directors() {
		const directors = this.details.credits.crew.filter(
			(obj) => obj.job === 'Director',
		)
		return directors.map((director) => director.name)
	}

	get writers() {
		const writers = this.details.credits.crew.filter(
			(obj) => obj.job === 'Screenplay',
		)
		return writers.map((writer) => writer.name)
	}

	get cast() {
		const cast = this.details.credits.cast.slice(0, 5)
		return cast.map((castMember) => ({
			name: castMember.name,
			role: castMember.character,
		}))
	}

	summary() {
		const summary = 
`
${this.title} - ${this.tagline}
Directed By: ${this.directors}  Written By: ${this.writers}

${this.overview}

Starring:
${this.cast.map(castMember => ` ${castMember.name} as ${castMember.role}`)}
`
		console.log(summary) 
	}
}

export default Movie

```

###`src/prompt.js`

```javascript
import searchMovie from './searchMovie.js'

const moviePrompts = [	
	{
		name: "name",
		type: "input",
		message: "Enter a movie to search:"
	},
	{
		name: "movieID",
		type: "list",
		message: "Select a movie:",
		choices: (answers) => searchMovie(answers.name)
	}
]

export default moviePrompts
```


###`src/searchMovie.js`

```javascript
import config from "./config.js";
import axios from 'axios'

const searchMovie = async (movie) => {
		const options={
			params: {
				api_key: config.tmbdkey,
				language: "en-US",
				query: movie,
			}
	}
	
	const simpleList = movieObj => ({name: `${movieObj.title} (${movieObj.release_date.slice(0,4)})`, value: movieObj.id})
	
	const res = await axios.get('https://api.themoviedb.org/3/search/movie', options);
	
	const list = res.data.results.map(simpleList).slice(0,5)

	return list;
	
}

export default searchMovie;
```

##Part 3 (Execution)

At the root of your project run

```bash
npm start
```

You will be preseted with the following.

```bash
> movie-scraper@1.0.0 start
> node src/index.js

? Enter a movie to search: 
```

Follow the prompts as given by the application.