Jordan Uses Luminati
Tags: luminati, axios, javascript
Published: 2020-10-22 21:36:52

---
title: Jordan Uses Luminati
published: true
date: 2020-09-14 12:40:00 UTC
tags: Luminati,Axios,Javascript,luminati
canonical_url: https://javascriptwebscrapingguy.com/jordan-uses-luminati/?utm_source=rss&utm_medium=rss&utm_campaign=jordan-uses-luminati
---

[Demo code here](https://github.com/aarmora/jordan-uses-luminati)

_Disclaimer: While this is not a sponsored post, the links contained here are affiliate links. Luminati is a product, however, that I use currently in my production code._

Sometimes using a proxy service is worth the cost. Using Luminati is a great example of when that is the case. I don’t often use proxies. I like to investigate and reverse engineer. Learn more about Luminati [here](https://luminati.io/?affiliate=ref_5f240848c7669177abf8ef2a).

The fun in web scraping is understanding what is stopping you from automated access to the website and then creating a tool that is able to emulate perfectly the expected request. When I need to have code that is running regularly, however, I need to run it from the cloud. And sometimes, cloud ip addresses are blocked at the ip level. This is a time when using a proxy makes sense.

## Enter Luminati

![Fun gif](https://media.giphy.com/media/uFOW5cbNaoTaU/giphy.gif)

Luminati makes it very simple to sign up. Go [here](https://luminati.io/?affiliate=ref_5f240848c7669177abf8ef2a) and hit “Start Now”. They are very thorough with verification and before I could use it fully they had to verify that I had a valid business with the Secretary of State. I’m not sure what would happen if I didn’t have an actual business that was registered with the Secretary of State.

Once within Luminati, you’ll see a dashboard and a place to configure your “Zones” or different proxies.

[![luminati dashboard](https://javascriptwebscrapingguy.com/wp-content/uploads/2020/09/image-3-1024x568.png)](https://javascriptwebscrapingguy.com/wp-content/uploads/2020/09/image-3.png)

The one in which I was most interested was the rotating residential proxy. I have an example of a target website from which I gather information that when scraping from my home, it works without any problem. When scraping from the cloud, the request just times out. At one point I even set up a VNC connection and tried to access the site from a chrome browser from the GUI and it still timed out. **This is an example of when a proxy makes sense.**

[![Luminati residential credentials](https://javascriptwebscrapingguy.com/wp-content/uploads/2020/09/image-4.png)](https://javascriptwebscrapingguy.com/wp-content/uploads/2020/09/image-4.png)

Hitting the “Edit” icon near the residential settings provides the username and password that you will need to use. If you are using the sample code that I am providing, just rename `.sample.env` to `.env` and replace the sample username and password with the ones found in the above section.

When you want to use the proxy, just add the `prod=true` property to the `.env` file.

And that’s really all you need to do within Luminati to get setup.

## The code

![Fun gif](https://media.giphy.com/media/13HgwGsXF0aiGY/giphy.gif)

I have examples using [axios](https://github.com/axios/axios), request (well, [request-promise](https://github.com/request/request-promise)), and [Puppeteer](https://github.com/puppeteer/puppeteer).

Axios is the most strange. It seems that using the `proxy` option doesn’t work when calling to https sites. Luminati has a test site that gives the ip address from which we are calling.

```
async function withAxios() {
	const url = 'https://lumtest.com/myip.json';
	const options: AxiosRequestConfig = { };

	if (process.env.prod) {
		console.log('Production enabled, using Luminati');
		options.proxy = false;
		options.httpsAgent = new HttpsProxyAgent(`https://${process.env.luminatiUsername}:${process.env.luminatiPassword}@zproxy.lum-superproxy.io:22225`)
	}

	const axiosResponse = await axios.get(url, options);

	console.log('Response from axios', axiosResponse.data);

}
```

And here is the difference with and without the proxy:

[![Axios with and without Luminati](https://javascriptwebscrapingguy.com/wp-content/uploads/2020/09/image.png)](https://javascriptwebscrapingguy.com/wp-content/uploads/2020/09/image.png)

Here’s the code when using request. The proxy works as expected here.

```
async function withRequest() {
	const url = 'http://lumtest.com/myip.json';

	const options: Request.Options = {
		url: url,
		method: 'GET'
	};

	if (process.env.prod) {
		console.log('Production enabled, using Luminati');
		options.proxy = `http://${process.env.luminatiUsername}:${process.env.luminatiPassword}@zproxy.lum-superproxy.io:22225`;
	}

	const response = await requestPromise(options);

	console.log('Response from request', JSON.parse(response));

}

```

And the response with and without the proxy:

[![Request with and without Luminati](https://javascriptwebscrapingguy.com/wp-content/uploads/2020/09/image-1.png)](https://javascriptwebscrapingguy.com/wp-content/uploads/2020/09/image-1.png)

And finally, the code with Puppeteer. It’s a simple thing; just add in the proxy as an `args` option and then authenticate with the page object.

```
async function withPuppeteer() {	
	const url = 'https://lumtest.com/myip.json';

	const options = {
		args: []
	};

	if (process.env.prod) {
		options.args.push('--proxy-server=zproxy.lum-superproxy.io:22225');
	}
	
	const browser = await puppeteer.launch(options);
	const page = await browser.newPage();

	if (process.env.prod) {
		console.log('Production enabled, using Luminati');
		await page.authenticate({
			username: process.env.luminatiUsername,
			password: process.env.luminatiPassword
		});
	}

	await page.goto(url);

	const response = await page.$eval('pre', elem => elem.textContent);

	console.log('Response from puppeteer', JSON.parse(response));

	await browser.close();

}
```

And the response with and without the proxy:

[![Puppeteer with and without Luminati](https://javascriptwebscrapingguy.com/wp-content/uploads/2020/09/image-2.png)](https://javascriptwebscrapingguy.com/wp-content/uploads/2020/09/image-2.png)

## Customer service

I would be very remiss if I didn’t speak of Luminati’s customer service. As soon as I created my account, I was assigned a case manager and it included his Skype information. I had a few questions and so I reached out. He responded quickly and got me in contact with a technical rep.

Both him and the technical rep answered my questions always quickly and extremely helpfully. I just recently had a call with the technical rep to discuss a problem I was having. He spent a good amount of time helping me find a solution to a problem that was mine and not related to Luminati.

Luminati’s customer service is GREAT. I cannot recommend it enough. Luminati is truly the Rolls-Royce of proxies and I use them now and I intend to keep using them. Check out more about Luminati [here](https://luminati.io/?affiliate=ref_5f240848c7669177abf8ef2a).

[Demo code here](https://github.com/aarmora/jordan-uses-luminati)

## Looking for business leads?

Using the techniques talked about here at [javascriptwebscrapingguy.com](https://javascriptwebscrapingguy.com), we’ve been able to launch a way to access awesome web data. Learn more at [Cobalt Intelligence](https://cobaltintelligence.com)!

The post [Jordan Uses Luminati](https://javascriptwebscrapingguy.com/jordan-uses-luminati/) appeared first on [JavaScript Web Scraping Guy](https://javascriptwebscrapingguy.com).