sylvinus/node-crawler — pull request #132 by momohuri (2014-12-25 00:29:57+00:00)
Title: option to restric to certain hostnames

When automatically adding links to the queue you don't necessarily want to crawl other websites.

I have added a simple option "authorizedHostnames" that can be a string or array of string with the hostname that you want to crawl and will automatically remove link that doesn't match.

ex:
With the following code you will soon enough be crawling twitter
```javascript

var c = new Crawler({
callback: function (error, result, $) {
$('a').each(function (index, a) {
var toQueueUrl = $(a).attr('href');
c.queue(toQueueUrl);
});
}
});
c.queue('http://www.joshfire.com')
```
Just adding the option will restring to joshfire website

```javascript

var c = new Crawler({
authorizedHostnames: 'joshfire.com',
callback: function (error, result, $) {
$('a').each(function (index, a) {
var toQueueUrl = $(a).attr('href');
c.queue(toQueueUrl);
});
}
});
c.queue('http://www.joshfire.com')
```