Repository: mcollina/autocannon
Pull request #260 (2020-05-18 12:54:49)

Add a new feature to support run with multiple url
This is a non-breaking-change feature

## Why we need this feature

When we bench a distributed system across continents (such as a blockchain system), there is often no load balancing. At this time, if it supports natively multi-node bench, it will be very convenient.

## How to use it

Added a compatible new feature to support multiple URLs like this

```
autocannon http://localhost:1234 http://localhost:1235
```

also programmatically supported with

```js
const autocannon = require('autocannon')

autocannon({
    url: ['http://foo.bar:3000', 'http://baz.qux:3001']
})
```



Patch:
diff --git a/.taprc b/.taprc
new file mode 100644
index 0000000..55dd4cb
--- /dev/null
+++ b/.taprc
@@ -0,0 +1,6 @@
+esm: false
+ts: false
+jsx: false
+flow: false
+timeout: 45
+coverage: true
diff --git a/README.md b/README.md
index a7dc3ee..1943b8a 100644
--- a/README.md
+++ b/README.md
@@ -240,7 +240,13 @@ Start autocannon against the given target.
     * `connectionRate`: A `Number` stating the rate of requests to make per second from each individual connection. No rate limiting by default. _OPTIONAL_
     * `overallRate`: A `Number` stating the rate of requests to make per second from all connections. `connectionRate` takes precedence if both are set. No rate limiting by default. _OPTIONAL_
     * `reconnectRate`: A `Number` which makes the individual connections disconnect and reconnect to the server whenever it has sent that number of requests. _OPTIONAL_
-    * `requests`: An `Array` of `Object`s which represents the sequence of requests to make while benchmarking. Can be used in conjunction with the `body`, `headers` and `method` params above. The `Object`s in this array can have `body`, `headers`, `method`, or `path` attributes, which overwrite those that are passed in this `opts` object. Therefore, the ones in this (`opts`) object take precedence and should be viewed as defaults. Additionally, an optional `setupRequest` `Function` may be provided that will mutate the raw `request` object, e.g. `request.method = 'GET'`. Check the samples folder for an example of how this might be used. _OPTIONAL_.
+    * `requests`: An `Array` of `Object`s which represents the sequence of requests to make while benchmarking. Can be used in conjunction with the `body`, `headers` and `method` params above. Check the samples folder for an example of how this might be used. _OPTIONAL_. Contained objects can have these attributes:
+       * `body`: When present, will override `opts.body`. _OPTIONAL_.
+       * `headers`: When present, will override `opts.headers`. _OPTIONAL_.
+       * `method`: When present, will override `opts.method`. _OPTIONAL_.
+       * `path`: When present, will override `opts.path`. _OPTIONAL_.
+       * `setupRequest`: A `Function` you may provide to mutate the raw `request` object, e.g. `request.method = 'GET'`. It takes `request` (Object) and `context` (Object) parameters, and must return the modified request. _OPTIONAL_.
+       * `onResponse`: A `Function` you may provide to process the received response. It takes `status` (Number), `body` (String) and `context` (Object) parameters. _OPTIONAL_.
     * `idReplacement`: A `Boolean` which enables the replacement of `[<id>]` tags within the request body with a randomly generated ID, allowing for unique fields to be sent with requests. Check out [an example of programmatic usage](./samples/using-id-replacement.js) can be found in the samples. _OPTIONAL_ default: `false`
     * `forever`: A `Boolean` which allows you to setup an instance of autocannon that restarts indefinitely after emiting results with the `done` event. Useful for efficiently restarting your instance. To stop running forever, you must cause a `SIGINT` or call the `.stop()` function on your instance. _OPTIONAL_ default: `false`
     * `servername`: A `String` identifying the server name for the SNI (Server Name Indication) TLS extension. _OPTIONAL_ default: `undefined`.
@@ -252,6 +258,17 @@ Start autocannon against the given target.
 
 **Returns** an instance/event emitter for tracking progress, etc. If cb omitted, the return value can also be used as a Promise.
 
+### Customizing sent requests
+
+When running, autocannon will create has many `Client` as desired connections. They will run in parallel, until the benchmark is over (duration or total number of requests).
+Each client will loop over the `requests` array, would it contain one or several requests.
+
+While going through available requests, the client will maintain a `context`: an object you can use in `onResponse` and `setupRequest` functions, to store and read some contextual data.
+Please check the `request-context.js` file in samples.
+
+Note that `context` object will be cleared when restarting to the first available request, ensuring similar runs.
+
+
 ### autocannon.track(instance[, opts])
 
 Track the progress of your autocannon, programmatically.
@@ -286,7 +303,7 @@ autocannon.track(instance, {renderProgressBar: false})
 
 Checkout [this example](./samples/track-run.js) to see it in use, as well.
 
-### autocannon events
+### Autocannon events
 
 Because an autocannon instance is an `EventEmitter`, it emits several events. these are below:
 
@@ -301,7 +318,7 @@ Because an autocannon instance is an `EventEmitter`, it emits several events. th
 * `reqError`: Emitted in the case of a request error e.g. a timeout.
 * `error`: Emitted if there is an error during the setup phase of autocannon.
 
-### results
+### Results
 
 The results object emitted by `done` and passed to the `autocannon()` callback has these properties:
 
diff --git a/autocannon.js b/autocannon.js
index 0ecd8cf..9c4dc0f 100755
--- a/autocannon.js
+++ b/autocannon.js
@@ -15,6 +15,7 @@ const hasAsyncHooks = require('has-async-hooks')
 const help = fs.readFileSync(path.join(__dirname, 'help.txt'), 'utf8')
 const run = require('./lib/run')
 const track = require('./lib/progressTracker')
+const { checkURL, ofURL } = require('./lib/url')
 
 if (typeof URL !== 'function') {
   console.error('autocannon requires the WHATWG URL API, but it is not available. Please upgrade to Node 6.13+.')
@@ -79,7 +80,7 @@ function parseArguments (argvs) {
     '--': true
   })
 
-  argv.url = argv._[0]
+  argv.url = argv._.length > 1 ? argv._ : argv._[0]
 
   if (argv.onPort) {
     argv.spawn = argv['--']
@@ -97,7 +98,7 @@ function parseArguments (argvs) {
     return
   }
 
-  if (!argv.url || argv.help) {
+  if (!checkURL(argv.url) || argv.help) {
     console.error(help)
     return
   }
@@ -106,28 +107,33 @@ function parseArguments (argvs) {
   // this allows doing:
   //     0x --on-port 'autocannon /path' -- node server.js
   if (process.env.PORT) {
-    argv.url = new URL(argv.url, `http://localhost:${process.env.PORT}`).href
+    argv.url = ofURL(argv.url).map(url => new URL(url, `http://localhost:${process.env.PORT}`).href)
   }
   // Add http:// if it's not there and this is not a /path
-  if (argv.url.indexOf('http') !== 0 && argv.url[0] !== '/') {
-    argv.url = `http://${argv.url}`
-  }
+  argv.url = ofURL(argv.url).map(url => {
+    if (url.indexOf('http') !== 0 && url[0] !== '/') {
+      url = `http://${url}`
+    }
+    return url
+  })
 
   // check that the URL is valid.
-  try {
-    // If --on-port is given, it's acceptable to not have a hostname
-    if (argv.onPort) {
-      new URL(argv.url, 'http://localhost') // eslint-disable-line no-new
-    } else {
-      new URL(argv.url) // eslint-disable-line no-new
+  ofURL(argv.url).map(url => {
+    try {
+      // If --on-port is given, it's acceptable to not have a hostname
+      if (argv.onPort) {
+        new URL(url, 'http://localhost') // eslint-disable-line no-new
+      } else {
+        new URL(url) // eslint-disable-line no-new
+      }
+    } catch (err) {
+      console.error(err.message)
+      console.error('')
+      console.error('When targeting a path without a hostname, the PORT environment variable must be available.')
+      console.error('Use a full URL or set the PORT variable.')
+      process.exit(1)
     }
-  } catch (err) {
-    console.error(err.message)
-    console.error('')
-    console.error('When targeting a path without a hostname, the PORT environment variable must be available.')
-    console.error('Use a full URL or set the PORT variable.')
-    process.exit(1)
-  }
+  })
 
   if (argv.input) {
     argv.body = fs.readFileSync(argv.input)
diff --git a/lib/httpClient.js b/lib/httpClient.js
index 9216337..0aafade 100644
--- a/lib/httpClient.js
+++ b/lib/httpClient.js
@@ -18,6 +18,7 @@ function Client (opts) {
 
   this.opts.setupClient = this.opts.setupClient || noop
   this.opts.pipelining = this.opts.pipelining || 1
+  const pipelining = this.opts.pipelining
   this.opts.port = this.opts.port || 80
   this.opts.expectBody = this.opts.expectBody || null
   this.timeout = (this.opts.timeout || 10) * 1000
@@ -44,8 +45,8 @@ function Client (opts) {
   // used for forcing reconnects
   this.reconnectRate = this.opts.reconnectRate
 
-  this.resData = new Array(this.opts.pipelining)
-  for (let i = 0; i < this.opts.pipelining; i++) {
+  this.resData = new Array(pipelining)
+  for (let i = 0; i < pipelining; i++) {
     this.resData[i] = {
       bytes: 0,
       headers: {},
@@ -88,26 +89,30 @@ function Client (opts) {
 
   this.parser[HTTPParser.kOnBody] = (body, start, len) => {
     this.emit('body', body)
+    const bodyString = '' + body.slice(start, start + len)
 
     if (this.opts.expectBody) {
-      const bodyString = '' + body.slice(start, start + len)
       if (this.opts.expectBody !== bodyString) {
         return this.emit('mismatch', bodyString)
       }
     }
+
+    const resp = this.resData[this.cer]
+    this.requestIterator.recordBody(resp.req, resp.headers.statusCode, bodyString)
   }
 
   this.parser[HTTPParser.kOnMessageComplete] = () => {
-    const end = process.hrtime(this.resData[this.cer].startTime)
-    const responseTime = end[0] * 1e3 + end[1] / 1e6
-    this.emit('response', this.resData[this.cer].headers.statusCode, this.resData[this.cer].bytes, responseTime)
-    this.resData[this.cer].bytes = 0
+    const resp = this.resData[this.cer]
+    const end = process.hrtime(resp.startTime)
+    resp.duration = end[0] * 1e3 + end[1] / 1e6
 
     if (!this.destroyed && this.reconnectRate && this.reqsMade % this.reconnectRate === 0) {
       return this._resetConnection()
     }
 
-    this.cer = this.cer === opts.pipelining - 1 ? 0 : this.cer++
+    this.emit('response', resp.headers.statusCode, resp.bytes, resp.duration)
+    resp.bytes = 0
+    this.cer = this.cer === pipelining - 1 ? 0 : this.cer++
     this._doRequest(this.cer)
   }
 
@@ -157,8 +162,12 @@ Client.prototype._doRequest = function (rpi) {
       return this.destroy()
     }
     this.emit('request')
+    if (this.reqsMade > 0) {
+      this.requestIterator.nextRequest()
+    }
+    this.resData[rpi].req = this.requestIterator.currentRequest
     this.resData[rpi].startTime = process.hrtime()
-    this.conn.write(this.requestIterator.move())
+    this.conn.write(this.getRequestBuffer())
     this.timeoutTicker.reschedule(this.timeout)
     this.reqsMade++
   } else {
@@ -188,7 +197,7 @@ Client.prototype.destroy = function () {
   }
 }
 
-Client.prototype.getRequestBuffer = function (newHeaders) {
+Client.prototype.getRequestBuffer = function () {
   return this.requestIterator.currentRequest.requestBuffer
 }
 
diff --git a/lib/httpRequestBuilder.js b/lib/httpRequestBuilder.js
index d075f5f..a4ee86d 100644
--- a/lib/httpRequestBuilder.js
+++ b/lib/httpRequestBuilder.js
@@ -20,8 +20,9 @@ function requestBuilder (defaults) {
   defaults = Object.assign(builderDefaults, defaults)
 
   // buildRequest takes an object, and turns it into a buffer representing the
-  // http request
-  return function buildRequest (reqData) {
+  // http request.
+  // second parameter is passed to setupRequest, when relevant
+  return function buildRequest (reqData, context) {
     // below is a hack to enable deep extending of the headers so the default
     // headers object isn't overwritten by accident
     reqData = reqData || {}
@@ -29,7 +30,7 @@ function requestBuilder (defaults) {
 
     reqData = Object.assign({}, defaults, reqData)
 
-    reqData = reqData.setupRequest(reqData)
+    reqData = reqData.setupRequest(reqData, context)
 
     // for some reason some tests fail with method === undefined
     // the reqData.method should be set to SOMETHING in this case
diff --git a/lib/requestIterator.js b/lib/requestIterator.js
index b968876..f647d19 100644
--- a/lib/requestIterator.js
+++ b/lib/requestIterator.js
@@ -9,6 +9,7 @@ function RequestIterator (opts) {
     return new RequestIterator(opts)
   }
 
+  this.context = {}
   this.reqDefaults = opts
   this.requestBuilder = requestBuilder(opts)
   this.setRequests(opts.requests)
@@ -18,8 +19,16 @@ inherits(RequestIterator, Object)
 
 RequestIterator.prototype.nextRequest = function () {
   ++this.currentRequestIndex
-  this.currentRequestIndex = this.currentRequestIndex < this.requests.length ? this.currentRequestIndex : 0
+  // when looping over available request, clear context for a fresh start
+  if (this.currentRequestIndex === this.requests.length) {
+    this.context = {}
+    this.currentRequestIndex = 0
+  }
   this.currentRequest = this.requests[this.currentRequestIndex]
+  // only builds if it has dynamic setup
+  if (this.reqDefaults.idReplacement || typeof this.currentRequest.setupRequest === 'function') {
+    this.rebuildRequest()
+  }
   return this.currentRequest
 }
 
@@ -29,26 +38,17 @@ RequestIterator.prototype.nextRequestBuffer = function () {
   return this.currentRequest.requestBuffer
 }
 
-RequestIterator.prototype.move = function () {
-  // get the current buffer and proceed to next request
-  const ret = this.currentRequest.requestBuffer
-  this.nextRequest()
-  return this.reqDefaults.idReplacement
-    ? Buffer.from(ret.toString().replace(/\[<id>\]/g, hyperid()))
-    : ret
-}
-
 RequestIterator.prototype.setRequests = function (newRequests) {
   this.requests = newRequests || [{}]
   this.currentRequestIndex = 0
-  this.currentRequest = this.requests[0]
-  this.rebuildRequests()
-}
-
-RequestIterator.prototype.rebuildRequests = function () {
-  this.requests.forEach((request) => {
-    request.requestBuffer = this.requestBuilder(request)
+  // build all request which don't have dynamic setup, except if it's the first one
+  this.requests.forEach((request, i) => {
+    this.currentRequest = request
+    if (i === 0 || typeof request.setupRequest !== 'function') {
+      this.rebuildRequest()
+    }
   })
+  this.currentRequest = this.requests[0]
 }
 
 RequestIterator.prototype.setHeaders = function (newHeaders) {
@@ -73,8 +73,18 @@ RequestIterator.prototype.setRequest = function (newRequest) {
 }
 
 RequestIterator.prototype.rebuildRequest = function () {
-  this.currentRequest.requestBuffer = this.requestBuilder(this.currentRequest)
-  this.requests[this.currentRequestIndex] = this.currentRequest
+  if (this.currentRequest) {
+    const data = this.requestBuilder(this.currentRequest, this.context)
+    this.currentRequest.requestBuffer = this.reqDefaults.idReplacement
+      ? Buffer.from(data.toString().replace(/\[<id>\]/g, hyperid()))
+      : data
+  }
+}
+
+RequestIterator.prototype.recordBody = function (request, status, body) {
+  if (request && typeof request.onResponse === 'function') {
+    request.onResponse(status, body, this.context)
+  }
 }
 
 module.exports = RequestIterator
diff --git a/lib/run.js b/lib/run.js
index 938e79c..a105415 100644
--- a/lib/run.js
+++ b/lib/run.js
@@ -9,6 +9,7 @@ const DefaultOptions = require('./defaultOptions')
 const multipart = require('./multipart')
 const histUtil = require('hdr-histogram-percentiles-obj')
 const reInterval = require('reinterval')
+const { ofURL, checkURL } = require('./url')
 const histAsObj = histUtil.histAsObj
 const addPercentiles = histUtil.addPercentiles
 
@@ -79,9 +80,6 @@ function _run (opts, cb, tracker) {
   // is done
   tracker.opts = opts
 
-  if (opts.url.indexOf('http') !== 0) opts.url = 'http://' + opts.url
-  const url = URL.parse(opts.url) // eslint-disable-line node/no-deprecated-api
-
   if (opts.overallRate && (opts.overallRate < opts.connections)) opts.connections = opts.overallRate
 
   let counter = 0
@@ -109,22 +107,34 @@ function _run (opts, cb, tracker) {
     }
   }
 
-  // copy over fields so that the client
-  // performs the right HTTP requests
-  url.pipelining = opts.pipelining
-  url.method = opts.method
-  url.body = form ? form.getBuffer() : opts.body
-  url.headers = form ? Object.assign({}, opts.headers, form.getHeaders()) : opts.headers
-  url.setupClient = opts.setupClient
-  url.timeout = opts.timeout
-  url.requests = opts.requests
-  url.reconnectRate = opts.reconnectRate
-  url.responseMax = amount || opts.maxConnectionRequests || opts.maxOverallRequests
-  url.rate = opts.connectionRate || opts.overallRate
-  url.idReplacement = opts.idReplacement
-  url.socketPath = opts.socketPath
-  url.servername = opts.servername
-  url.expectBody = opts.expectBody
+  opts.url = ofURL(opts.url).map((url) => {
+    if (url.indexOf('http') !== 0) return 'http://' + url
+    return url
+  })
+
+  const urls = ofURL(opts.url, true).map(url => {
+    if (url.indexOf('http') !== 0) url = 'http://' + url
+    url = URL.parse(url) // eslint-disable-line node/no-deprecated-api
+
+    // copy over fields so that the client
+    // performs the right HTTP requests
+    url.pipelining = opts.pipelining
+    url.method = opts.method
+    url.body = form ? form.getBuffer() : opts.body
+    url.headers = form ? Object.assign({}, opts.headers, form.getHeaders()) : opts.headers
+    url.setupClient = opts.setupClient
+    url.timeout = opts.timeout
+    url.requests = opts.requests
+    url.reconnectRate = opts.reconnectRate
+    url.responseMax = amount || opts.maxConnectionRequests || opts.maxOverallRequests
+    url.rate = opts.connectionRate || opts.overallRate
+    url.idReplacement = opts.idReplacement
+    url.socketPath = opts.socketPath
+    url.servername = opts.servername
+    url.expectBody = opts.expectBody
+
+    return url
+  })
 
   let clients = []
   initialiseClients(clients)
@@ -217,6 +227,7 @@ function _run (opts, cb, tracker) {
 
   function initialiseClients (clients) {
     for (let i = 0; i < opts.connections; i++) {
+      const url = urls[i % urls.length]
       if (!amount && !opts.maxConnectionRequests && opts.maxOverallRequests) {
         url.responseMax = distributeNums(opts.maxOverallRequests, i)
       }
@@ -299,7 +310,7 @@ function _run (opts, cb, tracker) {
 
   // will return true if error with opts entered
   function checkOptsForErrors () {
-    if (!opts.url && !opts.socketPath) {
+    if (!checkURL(opts.url) && !opts.socketPath) {
       errorCb(new Error('url or socketPath option required'))
       return true
     }
diff --git a/lib/url.js b/lib/url.js
new file mode 100644
index 0000000..ce438bd
--- /dev/null
+++ b/lib/url.js
@@ -0,0 +1,34 @@
+'use strict'
+
+/**
+ * check the url is not an empty string or empty array
+ * @param url
+ */
+function checkURL (url) {
+  return (typeof url === 'string' && url) ||
+    (Array.isArray(url) && url.length > 0)
+}
+
+/**
+ *
+ * @param url
+ * @param asArray
+ * @returns
+ */
+function ofURL (url, asArray) {
+  if (Array.isArray(url)) return url
+
+  if (typeof url === 'string') {
+    return {
+      map (fn) {
+        if (asArray) return [fn(url)]
+        return fn(url)
+      }
+    }
+  }
+
+  throw new Error('url should only be a string or an array of string')
+}
+
+exports.checkURL = checkURL
+exports.ofURL = ofURL
diff --git a/package.json b/package.json
index fcdd846..faa3324 100644
--- a/package.json
+++ b/package.json
@@ -1,13 +1,13 @@
 {
   "name": "autocannon",
-  "version": "4.6.0",
+  "version": "5.0.1",
   "description": "Fast HTTP benchmarking tool written in Node.js",
   "main": "autocannon.js",
   "bin": {
     "autocannon": "autocannon.js"
   },
   "scripts": {
-    "test": "standard && tap --no-esm --no-jsx --timeout 45 test/serial/*.test.js test/*.test.js",
+    "test": "standard && tap test/serial/*.test.js test/*.test.js",
     "standard:fix": "standard --fix"
   },
   "pre-commit": [
diff --git a/samples/request-context.js b/samples/request-context.js
new file mode 100644
index 0000000..3edbe6a
--- /dev/null
+++ b/samples/request-context.js
@@ -0,0 +1,53 @@
+'use strict'
+
+const http = require('http')
+const autocannon = require('autocannon')
+
+const server = http.createServer(handle)
+
+server.listen(0, startBench)
+
+function handle (req, res) {
+  const body = []
+  // this route handler simply returns whatever it gets from response
+  req
+    .on('data', chunk => body.push(chunk))
+    .on('end', () => res.end(Buffer.concat(body)))
+}
+
+function startBench () {
+  const url = 'http://localhost:' + server.address().port
+
+  autocannon({
+    url: url,
+    requests: [
+      {
+        // let's create a new user
+        method: 'POST',
+        path: '/users',
+        body: JSON.stringify({ firstName: 'Jane', id: 10 }),
+        onResponse: (status, body, context) => {
+          if (status === 200) {
+            context.user = JSON.parse(body)
+          } // on error, you may abort the benchmark
+        }
+      },
+      {
+        // now we'll give them a last name
+        method: 'PUT',
+        setupRequest: (req, context) => ({
+          ...req,
+          path: `/user/${context.user.id}`,
+          body: JSON.stringify({
+            ...context.user,
+            lastName: 'Doe'
+          })
+        })
+      }
+    ]
+  }, finishedBench)
+
+  function finishedBench (err, res) {
+    console.log('finished bench', err, res)
+  }
+}


Test patch:
diff --git a/test/argumentParsing.test.js b/test/argumentParsing.test.js
index 6197471..241c9a0 100644
--- a/test/argumentParsing.test.js
+++ b/test/argumentParsing.test.js
@@ -147,3 +147,14 @@ test('parse argument not correctly formatted header', (t) => {
     ])
   }, /An HTTP header was not correctly formatted/)
 })
+
+test('parse argument with multiple url', (t) => {
+  t.plan(2)
+  var args = Autocannon.parseArguments([
+    'localhost/foo/bar',
+    'http://localhost/baz/qux'
+  ])
+
+  t.equal(args.url[0], 'http://localhost/foo/bar')
+  t.equal(args.url[1], 'http://localhost/baz/qux')
+})
diff --git a/test/helper.js b/test/helper.js
index dc0679c..8b4361b 100644
--- a/test/helper.js
+++ b/test/helper.js
@@ -20,7 +20,7 @@ function startServer (opts) {
 
   function handle (req, res) {
     res.statusCode = statusCode
-    res.end(opts.body || 'hello world')
+    res.end(typeof opts.body === 'function' ? opts.body(req) : (opts.body || 'hello world'))
   }
 
   server.unref()
diff --git a/test/httpClient.test.js b/test/httpClient.test.js
index 0cf9f2e..f089b96 100644
--- a/test/httpClient.test.js
+++ b/test/httpClient.test.js
@@ -12,6 +12,23 @@ const tlsServer = helper.startTlsServer()
 const trailerServer = helper.startTrailerServer()
 const bl = require('bl')
 
+const makeResponseFromBody = ({ server, method, body, headers = {} }) => {
+  const sentHeaders = {
+    Connection: 'keep-alive',
+    ...headers
+  }
+  if (!sentHeaders['Content-Length'] && body) {
+    sentHeaders['Content-Length'] = body.length
+  }
+  return `${method} / HTTP/1.1\r\nHost: localhost:${
+    server.address().port
+  }\r\n${
+    Object.keys(sentHeaders).map(name => `${name}: ${sentHeaders[name]}`).join('\r\n')
+  }\r\n${
+    body ? `\r\n${body}` : '\r\n'
+  }`
+}
+
 test('client calls a server twice', (t) => {
   t.plan(4)
 
@@ -266,15 +283,12 @@ test('client supports changing the body', (t) => {
 
   const client = new Client(opts)
 
-  t.same(client.getRequestBuffer(),
-    Buffer.from(`POST / HTTP/1.1\r\nHost: localhost:${server.address().port}\r\nConnection: keep-alive\r\nContent-Length: 11\r\n\r\nhello world`),
-    'request is okay before modifying')
+  t.same(client.getRequestBuffer().toString(), makeResponseFromBody({ server, ...opts }), 'request is okay before modifying')
 
-  client.setBody('modified')
+  const body = 'modified'
+  client.setBody(body)
 
-  t.same(client.getRequestBuffer(),
-    Buffer.from(`POST / HTTP/1.1\r\nHost: localhost:${server.address().port}\r\nConnection: keep-alive\r\nContent-Length: 8\r\n\r\nmodified`),
-    'body changes updated request')
+  t.same(client.getRequestBuffer().toString(), makeResponseFromBody({ server, ...opts, body }), 'body changes updated request')
   client.destroy()
 })
 
@@ -285,17 +299,12 @@ test('client supports changing the headers', (t) => {
   opts.method = 'POST'
 
   const client = new Client(opts)
-  t.same(client.getRequestBuffer(),
-    Buffer.from(`POST / HTTP/1.1\r\nHost: localhost:${server.address().port}\r\nConnection: keep-alive\r\n\r\n`),
-    'request is okay before modifying')
+  t.same(client.getRequestBuffer().toString(), makeResponseFromBody({ server, ...opts }), 'request is okay before modifying')
 
-  client.setHeaders({
-    header: 'modified'
-  })
+  const headers = { header: 'modified' }
+  client.setHeaders(headers)
 
-  t.same(client.getRequestBuffer(),
-    Buffer.from(`POST / HTTP/1.1\r\nHost: localhost:${server.address().port}\r\nConnection: keep-alive\r\nheader: modified\r\n\r\n`),
-    'header changes updated request')
+  t.same(client.getRequestBuffer().toString(), makeResponseFromBody({ server, ...opts, headers }), 'header changes updated request')
   client.destroy()
 })
 
@@ -308,18 +317,14 @@ test('client supports changing the headers and body', (t) => {
 
   const client = new Client(opts)
 
-  t.same(client.getRequestBuffer(),
-    Buffer.from(`POST / HTTP/1.1\r\nHost: localhost:${server.address().port}\r\nConnection: keep-alive\r\nContent-Length: 11\r\n\r\nhello world`),
-    'request is okay before modifying')
+  t.same(client.getRequestBuffer().toString(), makeResponseFromBody({ server, ...opts }), 'request is okay before modifying')
 
-  client.setBody('modified')
-  client.setHeaders({
-    header: 'modifiedHeader'
-  })
+  const body = 'modified'
+  const headers = { header: 'modifiedHeader' }
+  client.setBody(body)
+  client.setHeaders(headers)
 
-  t.same(client.getRequestBuffer(),
-    Buffer.from(`POST / HTTP/1.1\r\nHost: localhost:${server.address().port}\r\nConnection: keep-alive\r\nheader: modifiedHeader\r\nContent-Length: 8\r\n\r\nmodified`),
-    'changes updated request')
+  t.same(client.getRequestBuffer().toString(), makeResponseFromBody({ server, ...opts, headers, body }), 'changes updated request')
   client.destroy()
 })
 
@@ -332,17 +337,13 @@ test('client supports changing the headers and body together', (t) => {
 
   const client = new Client(opts)
 
-  t.same(client.getRequestBuffer(),
-    Buffer.from(`POST / HTTP/1.1\r\nHost: localhost:${server.address().port}\r\nConnection: keep-alive\r\nContent-Length: 11\r\n\r\nhello world`),
-    'request is okay before modifying')
+  t.same(client.getRequestBuffer().toString(), makeResponseFromBody({ server, ...opts }), 'request is okay before modifying')
 
-  client.setHeadersAndBody({
-    header: 'modifiedHeader'
-  }, 'modified')
+  const body = 'modified'
+  const headers = { header: 'modifiedHeader' }
+  client.setHeadersAndBody(headers, body)
 
-  t.same(client.getRequestBuffer(),
-    Buffer.from(`POST / HTTP/1.1\r\nHost: localhost:${server.address().port}\r\nConnection: keep-alive\r\nheader: modifiedHeader\r\nContent-Length: 8\r\n\r\nmodified`),
-    'changes updated request')
+  t.same(client.getRequestBuffer().toString(), makeResponseFromBody({ server, ...opts, headers, body }), 'changes updated request')
   client.destroy()
 })
 
@@ -355,16 +356,12 @@ test('client supports changing the headers and body with null values', (t) => {
 
   const client = new Client(opts)
 
-  t.same(client.getRequestBuffer(),
-    Buffer.from(`POST / HTTP/1.1\r\nHost: localhost:${server.address().port}\r\nConnection: keep-alive\r\nContent-Length: 11\r\n\r\nhello world`),
-    'request is okay before modifying')
+  t.same(client.getRequestBuffer().toString(), makeResponseFromBody({ server, ...opts }), 'request is okay before modifying')
 
   client.setBody(null)
   client.setHeaders(null)
 
-  t.same(client.getRequestBuffer(),
-    Buffer.from(`POST / HTTP/1.1\r\nHost: localhost:${server.address().port}\r\nConnection: keep-alive\r\n\r\n`),
-    'changes updated request')
+  t.same(client.getRequestBuffer().toString(), makeResponseFromBody({ server, method: 'POST' }), 'changes updated request')
   client.destroy()
 })
 
@@ -377,15 +374,11 @@ test('client supports changing the headers and body together with null values',
 
   const client = new Client(opts)
 
-  t.same(client.getRequestBuffer(),
-    Buffer.from(`POST / HTTP/1.1\r\nHost: localhost:${server.address().port}\r\nConnection: keep-alive\r\nContent-Length: 11\r\n\r\nhello world`),
-    'request is okay before modifying')
+  t.same(client.getRequestBuffer().toString(), makeResponseFromBody({ server, ...opts }), 'request is okay before modifying')
 
   client.setHeadersAndBody(null, null)
 
-  t.same(client.getRequestBuffer(),
-    Buffer.from(`POST / HTTP/1.1\r\nHost: localhost:${server.address().port}\r\nConnection: keep-alive\r\n\r\n`),
-    'changes updated request')
+  t.same(client.getRequestBuffer().toString(), makeResponseFromBody({ server, method: 'POST' }), 'changes updated request')
   client.destroy()
 })
 
@@ -398,21 +391,18 @@ test('client supports updating the current request object', (t) => {
 
   const client = new Client(opts)
 
-  t.same(client.getRequestBuffer(),
-    Buffer.from(`POST / HTTP/1.1\r\nHost: localhost:${server.address().port}\r\nConnection: keep-alive\r\nContent-Length: 11\r\n\r\nhello world`),
-    'request is okay before modifying')
+  t.same(client.getRequestBuffer().toString(), makeResponseFromBody({ server, ...opts }), 'request is okay before modifying')
 
-  client.setRequest({
+  const newReq = {
     headers: {
       header: 'modifiedHeader'
     },
     body: 'modified',
     method: 'GET'
-  })
+  }
+  client.setRequest(newReq)
 
-  t.same(client.getRequestBuffer(),
-    Buffer.from(`GET / HTTP/1.1\r\nHost: localhost:${server.address().port}\r\nConnection: keep-alive\r\nheader: modifiedHeader\r\nContent-Length: 8\r\n\r\nmodified`),
-    'changes updated request')
+  t.same(client.getRequestBuffer().toString(), makeResponseFromBody({ server, ...newReq }), 'changes updated request')
   client.destroy()
 })
 
@@ -422,25 +412,21 @@ test('client customiseRequest function overwrites the headers and body', (t) =>
   const opts = server.address()
   opts.body = 'hello world'
   opts.method = 'POST'
+  const body = 'modified'
+  const headers = { header: 'modifiedHeader' }
   opts.setupClient = (client) => {
     t.ok(client.setHeadersAndBody, 'client had setHeadersAndBody method')
     t.ok(client.setHeaders, 'client had setHeaders method')
     t.ok(client.setBody, 'client had setBody method')
 
-    client.setHeadersAndBody({
-      header: 'modifiedHeader'
-    }, 'modified')
+    client.setHeadersAndBody(headers, body)
   }
 
   const client = new Client(opts)
 
-  t.same(client.getRequestBuffer(),
-    Buffer.from(`POST / HTTP/1.1\r\nHost: localhost:${server.address().port}\r\nConnection: keep-alive\r\nheader: modifiedHeader\r\nContent-Length: 8\r\n\r\nmodified`),
-    'changes updated request')
+  t.same(client.getRequestBuffer().toString(), makeResponseFromBody({ server, ...opts, headers, body }), 'changes updated request')
 
-  t.notSame(client.getRequestBuffer(),
-    Buffer.from(`POST / HTTP/1.1\r\nHost: localhost:${server.address().port}\r\nConnection: keep-alive\r\nContent-Length: 11\r\n\r\nhello world`),
-    'changes updated request')
+  t.notSame(client.getRequestBuffer().toString(), makeResponseFromBody({ server, ...opts }), 'changes updated request')
 
   client.destroy()
 })
@@ -501,10 +487,10 @@ test('client should have 2 different requests it iterates over', (t) => {
   t.plan(3)
   const server = helper.startServer()
   const opts = server.address()
-  opts.method = 'POST'
 
   const requests = [
     {
+      method: 'POST',
       body: 'hello world again'
     },
     {
@@ -521,16 +507,126 @@ test('client should have 2 different requests it iterates over', (t) => {
     switch (number) {
       case 1:
       case 3:
-        t.same(client.getRequestBuffer(),
-          Buffer.from(`POST / HTTP/1.1\r\nHost: localhost:${server.address().port}\r\nConnection: keep-alive\r\nContent-Length: 17\r\n\r\nhello world again`),
-          'request was okay')
+        t.same(client.getRequestBuffer().toString(), makeResponseFromBody({ server, ...requests[0] }), 'request was okay')
+        break
+      case 2:
+        t.same(client.getRequestBuffer().toString(), makeResponseFromBody({ server, ...requests[1] }), 'body changes updated request')
+        break
+      case 4:
+        client.destroy()
+        t.end()
+        break
+    }
+  })
+})
+
+test('client exposes response bodies and statuses', (t) => {
+  const server = helper.startServer({
+    body: ({ method }) => method === 'POST' ? 'hello!' : 'world!'
+  })
+  const opts = server.address()
+  opts.requests = [
+    {
+      method: 'POST',
+      body: 'hello world!',
+      onResponse: (status, body) => responses.push({ status, body })
+    },
+    {
+      method: 'POST',
+      body: 'hello world again'
+    },
+    {
+      method: 'GET',
+      onResponse: (status, body) => responses.push({ status, body })
+    }
+  ]
+  const responses = []
+
+  const client = new Client(opts)
+  let number = 0
+  client.on('response', (statusCode, length) => {
+    number++
+    switch (number) {
+      case 1:
+        t.same(client.getRequestBuffer().toString(), makeResponseFromBody({ server, ...opts.requests[0] }), 'first request')
+        t.deepEqual(responses, [{
+          status: 200,
+          body: 'hello!'
+        }])
         break
       case 2:
-        t.same(client.getRequestBuffer(),
-          Buffer.from(`GET / HTTP/1.1\r\nHost: localhost:${server.address().port}\r\nConnection: keep-alive\r\nContent-Length: 8\r\n\r\nmodified`),
-          'body changes updated request')
+        t.same(client.getRequestBuffer().toString(), makeResponseFromBody({ server, ...opts.requests[1] }), 'second request')
+        t.deepEqual(responses, [{
+          status: 200,
+          body: 'hello!'
+        }])
+        break
+      case 3:
+        t.same(client.getRequestBuffer().toString(), makeResponseFromBody({ server, ...opts.requests[2] }), 'third request')
+        t.deepEqual(responses, [{
+          status: 200,
+          body: 'hello!'
+        }, {
+          status: 200,
+          body: 'world!'
+        }])
         break
       case 4:
+        t.same(client.getRequestBuffer().toString(), makeResponseFromBody({ server, ...opts.requests[0] }), 'first request')
+        t.deepEqual(responses, [{
+          status: 200,
+          body: 'hello!'
+        }, {
+          status: 200,
+          body: 'world!'
+        }, {
+          status: 200,
+          body: 'hello!'
+        }])
+        client.destroy()
+        t.end()
+        break
+    }
+  })
+})
+
+test('client keeps context and reset it when looping on requests', (t) => {
+  const server = helper.startServer()
+  const opts = server.address()
+  let number = 0
+  const expectedResponse = 'hello world'
+
+  opts.requests = [
+    {
+      method: 'POST',
+      body: 'hello world again',
+      onResponse: (status, body, context) => {
+        if (number < 3) {
+          t.deepEqual(context, {}, 'context was supposed to be null')
+          context.previousRes = body
+        }
+      }
+    },
+    {
+      method: 'PUT',
+      setupRequest: (req, context) => {
+        if (number < 3) {
+          t.deepEqual(context, { previousRes: expectedResponse }, 'context was supposed to contain previous response')
+        }
+        return Object.assign({}, req, { body: context.previousRes })
+      }
+    }
+  ]
+
+  const client = new Client(opts)
+  client.on('response', (statusCode, length) => {
+    number++
+    switch (number) {
+      case 1:
+        t.same(client.getRequestBuffer().toString(), makeResponseFromBody({ server, ...opts.requests[0] }), 'hard-coded body')
+        break
+      case 2:
+        t.same(client.getRequestBuffer().toString(), makeResponseFromBody({ server, method: 'PUT', body: expectedResponse }), 'dynamic body')
         client.destroy()
         t.end()
         break
@@ -590,3 +686,83 @@ test('client calls twice using socket on secure server', (t) => {
     }
   })
 })
+
+test('client emits mistmatch when expectBody doesn\'t match actual body', (t) => {
+  const responses = ['hello...', 'world!']
+  const server = helper.startServer({
+    body: ({ method }) => responses[method === 'POST' ? 0 : 1]
+  })
+  const opts = server.address()
+  opts.requests = [
+    {
+      method: 'POST',
+      body: 'hi there!'
+    },
+    {
+      method: 'GET'
+    }
+  ]
+  opts.expectBody = responses[0]
+
+  const client = new Client(opts)
+  client.on('mismatch', (body) => {
+    // we expect body mismatch on second request
+    t.same(body, responses[1])
+    client.destroy()
+    t.end()
+  })
+})
+
+test('client invokes appropriate onResponse when using pipelining', (t) => {
+  const server = helper.startServer({
+    body: ({ method }) => method
+  })
+  const opts = server.address()
+  opts.pipelining = 2
+  const responses = []
+  const onResponse = (status, body) => responses.push(body)
+  opts.requests = [
+    {
+      method: 'POST',
+      onResponse
+    },
+    {
+      method: 'GET',
+      onResponse
+    },
+    {
+      method: 'PUT',
+      onResponse
+    }
+  ]
+
+  const client = new Client(opts)
+  let number = 0
+  client.on('response', (statusCode, length) => {
+    number++
+    switch (number) {
+      case 1:
+        // 1st & 2nd were sent, receiving 1st
+        t.same(client.getRequestBuffer().toString(), makeResponseFromBody({ server, method: 'GET' }), 'current should be second request')
+        t.deepEqual(responses, ['POST'])
+        break
+      case 2:
+        // 3rd was sent as 1st is finished, receiving 2st
+        t.same(client.getRequestBuffer().toString(), makeResponseFromBody({ server, method: 'PUT' }), 'current should be third request')
+        t.deepEqual(responses, ['POST', 'GET'])
+        break
+      case 3:
+        // 1st was resent, receiving 3st
+        t.same(client.getRequestBuffer().toString(), makeResponseFromBody({ server, method: 'POST' }), 'current should be first request')
+        t.deepEqual(responses, ['POST', 'GET', 'PUT'])
+        break
+      case 4:
+        // 2st was resent, receiving 1st
+        t.same(client.getRequestBuffer().toString(), makeResponseFromBody({ server, method: 'GET' }), 'current should be second request')
+        t.deepEqual(responses, ['POST', 'GET', 'PUT', 'POST'])
+        client.destroy()
+        t.end()
+        break
+    }
+  })
+})
diff --git a/test/requestIterator.test.js b/test/requestIterator.test.js
index d398e1f..e598550 100644
--- a/test/requestIterator.test.js
+++ b/test/requestIterator.test.js
@@ -163,7 +163,7 @@ test('request iterator should not replace any [<id>] tags with generated IDs whe
   opts.requests = [{}]
 
   const iterator = new RequestIterator(opts)
-  const result = iterator.move().toString().trim()
+  const result = iterator.currentRequest.requestBuffer.toString().trim()
 
   const contentLength = result.split('Content-Length: ')[1].slice(0, 1)
   t.equal(contentLength, '6', 'Content-Length was incorrect')
@@ -173,7 +173,7 @@ test('request iterator should not replace any [<id>] tags with generated IDs whe
 })
 
 test('request iterator should replace all [<id>] tags with generated IDs when calling move with idReplacement enabled', (t) => {
-  t.plan(2)
+  t.plan(4)
 
   const opts = server.address()
   opts.method = 'POST'
@@ -182,10 +182,49 @@ test('request iterator should replace all [<id>] tags with generated IDs when ca
   opts.idReplacement = true
 
   const iterator = new RequestIterator(opts)
-  const result = iterator.move().toString().trim()
+  const first = iterator.currentRequest.requestBuffer.toString().trim()
+
+  t.equal(first.includes('[<id>]'), false, 'One or more [<id>] tags were not replaced')
+  t.equal(first.slice(-1), '0', 'Generated ID should end with request number')
+
+  iterator.nextRequest()
+  const second = iterator.currentRequest.requestBuffer.toString().trim()
+
+  t.equal(second.includes('[<id>]'), false, 'One or more [<id>] tags were not replaced')
+  t.equal(second.slice(-1), '1', 'Generated ID should end with a unique request number')
+})
+
+test('request iterator should invoke onResponse callback when set', (t) => {
+  t.plan(9)
+
+  const opts = server.address()
+  opts.requests = [
+    {
+      onResponse: (status, body, context) => {
+        t.same(status, 200)
+        t.same(body, 'ok')
+        t.deepEqual(context, {})
+      }
+    },
+    {},
+    {
+      onResponse: (status, body, context) => {
+        t.same(status, 201)
+        t.same(body, '')
+        t.deepEqual(context, {})
+      }
+    }
+  ]
 
-  t.equal(result.includes('[<id>]'), false, 'One or more [<id>] tags were not replaced')
-  t.equal(result.slice(-1), '0', 'Generated ID should end with request number')
+  const iterator = new RequestIterator(opts)
+  iterator.recordBody(iterator.currentRequest, 200, 'ok')
+  iterator.nextRequest()
+  iterator.recordBody(iterator.currentRequest, 500, 'ignored')
+  iterator.nextRequest()
+  iterator.recordBody(iterator.currentRequest, 201, '')
+  // will reset the iterator
+  iterator.nextRequest()
+  iterator.recordBody(iterator.currentRequest, 200, 'ok')
 })
 
 test('request iterator should properly mutate requests if a setupRequest function is located', (t) => {
@@ -196,7 +235,7 @@ test('request iterator should properly mutate requests if a setupRequest functio
 
   let i = 0
 
-  const requests1 = [
+  opts.requests = [
     {
       body: 'hello world',
       setupRequest: req => {
@@ -217,10 +256,9 @@ test('request iterator should properly mutate requests if a setupRequest functio
   const request1Res = Buffer.from(`POST / HTTP/1.1\r\nHost: localhost:${server.address().port}\r\nConnection: keep-alive\r\nContent-Length: 12\r\n\r\nhello world0`)
   const request2Res = Buffer.from(`POST / HTTP/1.1\r\nHost: localhost:${server.address().port}\r\nConnection: keep-alive\r\nContent-Length: 9\r\n\r\nmodified1`)
   const request3Res = Buffer.from(`GET / HTTP/1.1\r\nHost: localhost:${server.address().port}\r\nConnection: keep-alive\r\nContent-Length: 8\r\n\r\nmodified`)
-  const request4Res = Buffer.from(`POST / HTTP/1.1\r\nHost: localhost:${server.address().port}\r\nConnection: keep-alive\r\nheader: modifiedHeader\r\nContent-Length: 9\r\n\r\nmodified2`)
-  const request5Res = Buffer.from(`POST / HTTP/1.1\r\nHost: localhost:${server.address().port}\r\nConnection: keep-alive\r\n\r\n`)
-
-  opts.requests = requests1
+  const request4Res = Buffer.from(`POST / HTTP/1.1\r\nHost: localhost:${server.address().port}\r\nConnection: keep-alive\r\nContent-Length: 9\r\n\r\nmodified2`)
+  const request5Res = Buffer.from(`POST / HTTP/1.1\r\nHost: localhost:${server.address().port}\r\nConnection: keep-alive\r\nheader: modifiedHeader\r\nContent-Length: 9\r\n\r\nmodified3`)
+  const request6Res = Buffer.from(`POST / HTTP/1.1\r\nHost: localhost:${server.address().port}\r\nConnection: keep-alive\r\n\r\n`)
 
   const iterator = new RequestIterator(opts)
   t.same(iterator.currentRequest.requestBuffer, request1Res, 'request was okay')
@@ -229,11 +267,47 @@ test('request iterator should properly mutate requests if a setupRequest functio
   iterator.nextRequest() // verify it didn't affect the other request
   t.same(iterator.currentRequest.requestBuffer, request3Res, 'request was okay')
   iterator.nextRequest()
-  t.same(iterator.currentRequest.requestBuffer, request2Res, 'request was okay')
-  iterator.setHeaders({ header: 'modifiedHeader' })
   t.same(iterator.currentRequest.requestBuffer, request4Res, 'request was okay')
-  iterator.setRequest() // this should build default request
+  iterator.setHeaders({ header: 'modifiedHeader' })
   t.same(iterator.currentRequest.requestBuffer, request5Res, 'request was okay')
+  iterator.setRequest() // this should build default request
+  t.same(iterator.currentRequest.requestBuffer, request6Res, 'request was okay')
+})
+
+test('request iterator should maintain context while looping on requests', (t) => {
+  t.plan(4)
+
+  const opts = server.address()
+  opts.requests = [
+    {
+      setupRequest: (req, context) => {
+        t.deepEqual(context, {}, 'context was supposed to be empty for first request')
+        context.num = 1
+        context.init = true
+        return req
+      }
+    },
+    {
+      setupRequest: (req, context) => {
+        t.deepEqual(context, { num: 1, init: true }, 'context was supposed to be initialized for second request')
+        context.num++
+        return req
+      }
+    },
+    {
+      setupRequest: (req, context) => {
+        t.deepEqual(context, { num: 2, init: true }, 'context was supposed to be initialized for third request')
+        context.num++
+        return req
+      }
+    }
+  ]
+
+  const iterator = new RequestIterator(opts)
+  iterator.nextRequest()
+  iterator.nextRequest()
+  // will reset the iterator
+  iterator.nextRequest()
 })
 
 test('request iterator should return instance of RequestIterator', t => {
diff --git a/test/runMultiServer.test.js b/test/runMultiServer.test.js
new file mode 100644
index 0000000..d76a47a
--- /dev/null
+++ b/test/runMultiServer.test.js
@@ -0,0 +1,40 @@
+const test = require('tap').test
+const run = require('../lib/run')
+const helper = require('./helper')
+
+const server1 = helper.startServer({ body: 'from server1' })
+const server2 = helper.startServer({ body: 'from server2' })
+const server3 = helper.startServer({ body: 'from server3' })
+
+test('should receive the message from different server', (t) => {
+  t.plan(3)
+
+  const instance = run({
+    url: [
+      server1,
+      server2,
+      server3
+    ].map(server => `http://localhost:${server.address().port}`),
+    duration: 1,
+    connections: 3
+  })
+
+  let receivedServer1 = false
+  let receivedServer2 = false
+  let receivedServer3 = false
+
+  instance.on('response', (client) => {
+    if (!receivedServer1 && client.parser.chunk.toString().includes('from server1')) {
+      receivedServer1 = true
+      t.pass()
+    }
+    if (!receivedServer2 && client.parser.chunk.toString().includes('from server2')) {
+      receivedServer2 = true
+      t.pass()
+    }
+    if (!receivedServer3 && client.parser.chunk.toString().includes('from server3')) {
+      receivedServer3 = true
+      t.pass()
+    }
+  })
+})