Repository: sindresorhus/got
Pull request #516 (2018-07-11 18:26:12)

Add `beforeRequest` hook
This PR build on the leverage of being able to create got instances (thanks @knksmith57!) by adding a `finalize` option, which eases the use of HMAC request signatures, such as AWS V4.

I hope the updated example in the readme makes the motivation clear.

I explored implementing this using an event, but it works better if the hook can be an async function (also demonstrated in the readme).

Along the way, I made a couple of tweaks to other areas of the code, which I included here in separate commits. I can pull those off if they're not wanted, but I thought they were small enough that it would just be easier to propose them alongside this new feature.

szmarczak: **Note:** *creating `got` instances is still in the works.*

Seems good, but I don't see any sense of that. You could just do:

```js
const sign = ...;

const instance = got.create({
	options: got.defaults.options,
	methods: got.defaults.methods,
	handler: (url, options, next) => {
		options.headers['sign'] = sign(options);

		return next(url, options);
	}
});
```

> Along the way, I made a couple of tweaks to other areas of the code, which I included here in separate commits.

Can you do another PR for that? To make everything more clear :)
brandon93s: This is similar in objective to [interceptors](https://github.com/axios/axios#interceptors) in axios. It's also a great place to add support for plugins, or hooks. An API like the following would allow us to expand this to different hooks and provide support for _multiple_ hooks at once:

```js
const plugin = got.hooks.beforeRequest.add(async req => {

});

got.hooks.beforeRequest.remove(plugin);
```

Hooks here could be things like: 

- Authentication 
- Logging / Debugging
	- For example: `chalk()` of network activity in development mode
- Etc

I support the current implementation, but wanted to bring up the alternative above if we want to future proof the implementation. 
jstewmon: I actually ended up coming back to got for this after trying to solve the problem with axios interceptors. :-)

axios request interceptors run before their dispatchRequest method, which is where they normalize the request.

But, that's not your point...

I considered allowing the config setting an array or a function, but I decided that it was best to start the conversation around the feature by making it as simple as possible.

If we allow the setting to be an array (without providing a custom interface), I think that might provide better mechanics for anyone wanting to apply a custom merge strategy to the options for a new instance (versus the function wrapping discussed earlier).

If it's conceivable that additional hook points would be added in the future, then we might consider having a top-level `hooks` setting, which is an object where they keys are the hook names and the values are `Array|Function`.
szmarczak: @brandon93s +1
sindresorhus: @szmarczak Events notify while hooks let you transform.
jstewmon: I rebased off master and updated the interface to reflect the discussion about future-proofing the configuration interface for additional hook events.

Patch:
diff --git a/readme.md b/readme.md
index ddc07c3..5e20aaf 100644
--- a/readme.md
+++ b/readme.md
@@ -258,6 +258,24 @@ Determines if a `got.HTTPError` is thrown for error responses (non-2xx status co
 
 If this is disabled, requests that encounter an error status code will be resolved with the `response` instead of throwing. This may be useful if you are checking for resource availability and are expecting error responses.
 
+###### hooks
+
+Type: `Object<string, Array<Function>>`<br>
+Default: `{ beforeRequest: [] }`
+
+Hooks allow modifications during the request lifecycle. Hook functions may be async and are run serially.
+
+###### hooks.beforeRequest
+
+Type: `Array<Function>`<br>
+Default: `[]`
+
+Called with the normalized request options. Got will make no further changes to the request before it is sent. This is especially useful in conjunction with [`got.extend()`](#instances) and [`got.create()`](advanced-creation.md) when you want to create an API client that uses HMAC-signing.
+
+See the [AWS section](#aws) for an example.
+
+**Note**: Modifying the `body` is not recommended because the `content-length` header has already been computed and assigned.
+
 #### Streams
 
 **Note**: Progress events, redirect events and request/response events can also be used with promises.
@@ -626,47 +644,35 @@ got('http://unix:/var/run/docker.sock:/containers/json');
 got('unix:/var/run/docker.sock:/containers/json');
 ```
 
+
 ## AWS
 
-Requests to AWS services need to have their headers signed. This can be accomplished by using the [`aws4`](https://www.npmjs.com/package/aws4) package. This is an example for querying an ["Elasticsearch Service"](https://aws.amazon.com/elasticsearch-service/) host with a signed request.
+Requests to AWS services need to have their headers signed. This can be accomplished by using the [`aws4`](https://www.npmjs.com/package/aws4) package. This is an example for querying an ["API Gateway"](https://docs.aws.amazon.com/apigateway/api-reference/signing-requests/) with a signed request.
 
 ```js
-const url = require('url');
 const AWS = require('aws-sdk');
 const aws4 = require('aws4');
 const got = require('got');
-const config = require('./config');
-
-// Reads keys from the environment or `~/.aws/credentials`. Could be a plain object.
-const awsConfig = new AWS.Config({ region: config.region });
 
-function request(url, options) {
-	const awsOpts = {
-		region: awsConfig.region,
-		headers: {
-			accept: 'application/json',
-			'content-type': 'application/json'
-		},
-		method: 'GET',
-		json: true
-	};
-
-	// We need to parse the URL before passing it to `got` so `aws4` can sign the request
-	options = {
-		...url.parse(url),
-		...awsOpts,
-		...options
-	};
-
-	aws4.sign(options, awsConfig.credentials);
-
-	return got(options);
-}
-
-request(`https://${config.host}/production/users/1`);
+const credentials = await new AWS.CredentialProviderChain().resolvePromise();
+
+// Create a Got instance to use relative paths and signed requests
+const awsClient = got.extend(
+	{
+		baseUrl: 'https://<api-id>.execute-api.<api-region>.amazonaws.com/<stage>/',
+		hooks: {
+			beforeRequest: [
+				async options => {
+					await credentials.getPromise();
+					aws4.sign(options, credentials);
+				}
+			]
+		}
+	}
+);
 
-request(`https://${config.host}/production/`, {
-	// All usual `got` options
+const response = await awsClient('endpoint/path', {
+	// Request-specific options
 });
 ```
 
diff --git a/source/index.js b/source/index.js
index 1f9f4c4..f4573f8 100644
--- a/source/index.js
+++ b/source/index.js
@@ -23,6 +23,9 @@ const defaults = {
 		throwHttpErrors: true,
 		headers: {
 			'user-agent': `${pkg.name}/${pkg.version} (https://github.com/sindresorhus/got)`
+		},
+		hooks: {
+			beforeRequest: []
 		}
 	}
 };
diff --git a/source/normalize-arguments.js b/source/normalize-arguments.js
index 82a329b..9232d02 100644
--- a/source/normalize-arguments.js
+++ b/source/normalize-arguments.js
@@ -8,6 +8,7 @@ const urlToOptions = require('./url-to-options');
 const isFormData = require('./is-form-data');
 
 const retryAfterStatusCodes = new Set([413, 429, 503]);
+const knownHookEvents = ['beforeRequest'];
 
 module.exports = (url, options, defaults) => {
 	if (Reflect.has(options, 'url') || (is.object(url) && Reflect.has(url, 'url'))) {
@@ -185,5 +186,31 @@ module.exports = (url, options, defaults) => {
 		delete options.timeout;
 	}
 
+	if (is.nullOrUndefined(options.hooks)) {
+		options.hooks = {};
+	}
+	if (is.object(options.hooks)) {
+		for (const hookEvent of knownHookEvents) {
+			const hooks = options.hooks[hookEvent];
+			if (is.nullOrUndefined(hooks)) {
+				options.hooks[hookEvent] = [];
+			} else if (is.array(hooks)) {
+				hooks.forEach(
+					(hook, index) => {
+						if (!is.function_(hook)) {
+							throw new TypeError(
+								`Parameter \`hooks.${hookEvent}[${index}]\` must be a function, not ${is(hook)}`
+							);
+						}
+					}
+				);
+			} else {
+				throw new TypeError(`Parameter \`hooks.${hookEvent}\` must be an array, not ${is(hooks)}`);
+			}
+		}
+	} else {
+		throw new TypeError(`Parameter \`hooks\` must be an object, not ${is(options.hooks)}`);
+	}
+
 	return options;
 };
diff --git a/source/request-as-event-emitter.js b/source/request-as-event-emitter.js
index 38c062c..80a9ee6 100644
--- a/source/request-as-event-emitter.js
+++ b/source/request-as-event-emitter.js
@@ -241,6 +241,11 @@ module.exports = (options = {}) => {
 				options.headers['content-length'] = uploadBodySize;
 			}
 
+			for (const hook of options.hooks.beforeRequest) {
+				// eslint-disable-next-line no-await-in-loop
+				await hook(options);
+			}
+
 			get(options);
 		} catch (error) {
 			emitter.emit('error', error);


Test patch:
diff --git a/test/arguments.js b/test/arguments.js
index 964941a..ab70ab2 100644
--- a/test/arguments.js
+++ b/test/arguments.js
@@ -101,6 +101,40 @@ test('throws TypeError when `url` is passed as an option', async t => {
 	await t.throws(got({url: 'example.com'}), {instanceOf: TypeError});
 });
 
+test('throws TypeError when `hooks` is not an object', async t => {
+	await t.throws(
+		() => got(s.url, {hooks: 'not object'}),
+		{
+			instanceOf: TypeError,
+			message: 'Parameter `hooks` must be an object, not string'
+		}
+	);
+});
+
+test('throws TypeError when known `hooks` value is not an array', async t => {
+	await t.throws(
+		() => got(s.url, {hooks: {beforeRequest: {}}}),
+		{
+			instanceOf: TypeError,
+			message: 'Parameter `hooks.beforeRequest` must be an array, not Object'
+		}
+	);
+});
+
+test('throws TypeError when known `hooks` array item is not a function', async t => {
+	await t.throws(
+		() => got(s.url, {hooks: {beforeRequest: [{}]}}),
+		{
+			instanceOf: TypeError,
+			message: 'Parameter `hooks.beforeRequest[0]` must be a function, not Object'
+		}
+	);
+});
+
+test('allows extra keys in `hooks`', async t => {
+	await t.notThrows(() => got(`${s.url}/test`, {hooks: {extra: {}}}));
+});
+
 test.after('cleanup', async () => {
 	await s.close();
 });
diff --git a/test/hooks.js b/test/hooks.js
new file mode 100644
index 0000000..220bf97
--- /dev/null
+++ b/test/hooks.js
@@ -0,0 +1,105 @@
+import test from 'ava';
+import delay from 'delay';
+import {createServer} from './helpers/server';
+import got from '..';
+
+let s;
+
+test.before('setup', async () => {
+	s = await createServer();
+	const echoHeaders = (req, res) => {
+		res.statusCode = 200;
+		res.write(JSON.stringify(req.headers));
+		res.end();
+	};
+	s.on('/', echoHeaders);
+	await s.listen(s.port);
+});
+
+test('beforeRequest receives normalized options', async t => {
+	await got(
+		s.url,
+		{
+			json: true,
+			hooks: {
+				beforeRequest: [
+					options => {
+						t.is(options.path, '/');
+						t.is(options.hostname, 'localhost');
+					}
+				]
+			}
+		}
+	);
+});
+
+test('beforeRequest allows modifications', async t => {
+	const res = await got(
+		s.url,
+		{
+			json: true,
+			hooks: {
+				beforeRequest: [
+					options => {
+						options.headers.foo = 'bar';
+					}
+				]
+			}
+		}
+	);
+	t.is(res.body.foo, 'bar');
+});
+
+test('beforeRequest awaits async function', async t => {
+	const res = await got(
+		s.url,
+		{
+			json: true,
+			hooks: {
+				beforeRequest: [
+					async options => {
+						await delay(100);
+						options.headers.foo = 'bar';
+					}
+				]
+			}
+		}
+	);
+	t.is(res.body.foo, 'bar');
+});
+
+test('beforeRequest rejects when beforeRequest throws', async t => {
+	await t.throws(
+		() => got(s.url, {
+			hooks: {
+				beforeRequest: [
+					() => {
+						throw new Error('oops');
+					}
+				]
+			}
+		}),
+		{
+			instanceOf: Error,
+			message: 'oops'
+		}
+	);
+});
+
+test('beforeRequest rejects when beforeRequest rejects', async t => {
+	await t.throws(
+		() => got(s.url, {
+			hooks: {
+				beforeRequest: [() => Promise.reject(new Error('oops'))]
+			}
+		}),
+		{
+			instanceOf: Error,
+			message: 'oops'
+		}
+	);
+});
+
+test.after('cleanup', async () => {
+	await s.close();
+});