Repository: elastic/synthetics (ts)

Issue:
Synthetics Doesn’t Handle Popups (new tab or new window)

The Synthetic Agent is capturing the performance data and screenshots for the main tab/page of the browser.

If a user journey causes a new tab or window (popup) to be opened, the metrics and screen shots are still taken from the initial (main) page, so there are no new metrics for the new tab/page, and the screenshot still reflects the original page.

This can be recreated by extending the todos folder/app

Create `./examples/todos/app/popup1.html`
```
<html>
<head>
</head>
<body>
Test popup page 1
<p><a href="popup2.html" onclick="return !window.open(this.href, 'Test Page', 'width=500,height=500')" target="_blank" id="linkWithPopup">Click</a> here (id=linkWithPopup) to open - Test Page - popup window</p>
<p><a href="popup2.html" target="_blank" id="linkWithPopupNoNewWindow">Click</a> here (id=linkWithPopupNoNewWindow) to open - Test Page - new tab (not new window)</p>
</body>
</html>
```
Create `./examples/todos/app/popup2.html`
```
<html>
<head>
</head>
<body>
Test popup page 2
</body>
</html>
```
Create `./examples/todos/popup.journey.ts`
```
import { journey, step } from '@elastic/synthetics';
import { deepStrictEqual } from 'assert';
import { join } from 'path';

journey('Popup Test', async ({ page }) => {
  step('Popup Landing Page', async () => {
    const path = 'file://' + join(__dirname, 'app', 'popup1.html');
    await page.goto(path);
    await page.waitForSelector('text=Test popup page 1');
  });

  step('Open Popup', async () => {
    // Test 1 - new window (popup)
    await page.click('#linkWithPopup');

    // Test 2 - new tab (not new popup window)
    //await page.click('#linkWithPopupNoNewWindow');

    const [popup] = await Promise.all([
      page.waitForEvent('popup'),
      page.click('#linkWithPopup')
    ]);
    await popup.waitForSelector('text=Test popup page 2');
  });
});
```

**Note** that there are two test cases (both will fail in the same way)
* Test 1 opens a new popup window
* Test 2 (commented out) opens a new tab (not a separate popup window)

We can act on the new/popup page, for example with `popup.waitForSelector
` as shown in [this example](https://playwright.dev/docs/api/class-page/#pagewaitforloadstatestate-options), however the Synthetic Agent is still connected to the CDP from the original window.

When running this, you will notice:
* The screenshot for step 2 still shows page 1 (the original window)
* There is no waterfall chart (no timing metrics) for step 2

Pull request description:
feat: use network events from pw
+ bring back the original PR along with couple of improvements #372 #398
+ fixes #253 
+ We use the network events from the Playwright context instead of events from CDP which can only listen for the first page request and ignores all subsequent requests. With this change, all requests inside all frames, windows gets captured with this change and also we get header and body size for each individual request.
+ Timing calculation becomes a lot simpler to deal with and don't require much understanding of the chrome devtools code which we relied on before.
+ Changed logic around emulating the subsequent pages for network throttling as the previous implementation had bug and did not account for pages that are closed during the navigation. 
+ Barriers are added for all the async operations that happens while observing for network events. 

### Missing data
- We don't have a **queuing** time exposed from the PW API as Queueing time is pretty specific to chromium and not exposed for other browsers like FF and Webkit and its in the order of 1-2ms max when tested - 
- Bare **uncompressed resource size** is not exposed for each request instead we have **header and body sizes** for both request and responses. 
- `http.version` data is not exposed, but we can add that as a follow up once PW adds support for it and we are not using it on the waterfall UI. 


Patch:
diff --git a/NOTICE.txt b/NOTICE.txt
index e41cbb7..844921e 100644
--- a/NOTICE.txt
+++ b/NOTICE.txt
@@ -219,32 +219,3 @@ This product relies on `expect`
  *
  */
 ---------------------------------------------------------------------------
-This product relies on `ChromeDevTools/devtools-frontend`
-
-// Copyright 2014 The Chromium Authors. All rights reserved.
-//
-// Redistribution and use in source and binary forms, with or without
-// modification, are permitted provided that the following conditions are
-// met:
-//
-//    * Redistributions of source code must retain the above copyright
-// notice, this list of conditions and the following disclaimer.
-//    * Redistributions in binary form must reproduce the above
-// copyright notice, this list of conditions and the following disclaimer
-// in the documentation and/or other materials provided with the
-// distribution.
-//    * Neither the name of Google Inc. nor the names of its
-// contributors may be used to endorse or promote products derived from
-// this software without specific prior written permission.
-//
-// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
-// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
-// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
-// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
-// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
-// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
-// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
-// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
-// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
-// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
-// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
diff --git a/src/common_types.ts b/src/common_types.ts
index dbbb149..589ea49 100644
--- a/src/common_types.ts
+++ b/src/common_types.ts
@@ -31,7 +31,6 @@ import {
   ChromiumBrowserContext,
   Page,
 } from 'playwright-chromium';
-import { Protocol } from 'playwright-core/types/protocol';
 import { Step } from './dsl';
 import { Reporter, reporters } from './reporters';
 
@@ -130,11 +129,9 @@ export type Request = {
 
 export type Response = {
   url?: string;
-  protocol?: string;
   status: number;
   statusText?: string;
-  mimeType: string;
-  httpVersion?: string;
+  mimeType?: string;
   headers: Record<string, string>;
   // Total size in bytes of the response (body and headers)
   bytes?: number;
@@ -148,7 +145,6 @@ export type Response = {
   remoteIPAddress?: string;
   remotePort?: number;
   fromServiceWorker?: boolean;
-  timing?: Protocol.Network.ResourceTiming;
 };
 
 export type NetworkInfo = {
@@ -156,19 +152,17 @@ export type NetworkInfo = {
   browser: BrowserInfo;
   type: string;
   request: Request;
-  response?: Response;
+  response: Response;
   isNavigationRequest: boolean;
   requestSentTime: number;
   loadEndTime: number;
   responseReceivedTime: number;
   resourceSize: number;
   transferSize: number;
-  timings?: {
+  timings: {
     blocked: number;
-    queueing: number;
     dns: number;
     ssl: number;
-    proxy: number;
     connect: number;
     send: number;
     wait: number;
diff --git a/src/core/gatherer.ts b/src/core/gatherer.ts
index f802008..f745728 100644
--- a/src/core/gatherer.ts
+++ b/src/core/gatherer.ts
@@ -57,7 +57,7 @@ export class Gatherer {
       ...playwrightOptions,
       userAgent: await Gatherer.getUserAgent(playwrightOptions?.userAgent),
     });
-    await Gatherer.setNetworkConditions(context, networkConditions);
+    Gatherer.setNetworkConditions(context, networkConditions);
 
     const page = await context.newPage();
     const client = await context.newCDPSession(page);
@@ -73,18 +73,26 @@ export class Gatherer {
     return userAgent + syntheticsIdentifier;
   }
 
-  static async setNetworkConditions(
+  static setNetworkConditions(
     context: BrowserContext,
     networkConditions: RunOptions['networkConditions']
   ) {
     if (networkConditions) {
-      context.on('page', async page => {
+      context.on('page', page => {
         const context = page.context();
-        const client = await context.newCDPSession(page);
-        await client.send(
-          'Network.emulateNetworkConditions',
-          networkConditions
-        );
+        const emulatePromise = context
+          .newCDPSession(page)
+          .then(client =>
+            client.send('Network.emulateNetworkConditions', networkConditions)
+          );
+        /**
+         * Guard against pages that gets closed before the emulation kicks to capture
+         * unhandled rejections from accessing the CDP session of closed page
+         */
+        Promise.race([
+          new Promise<void>(resolve => page.on('close', () => resolve())),
+          emulatePromise,
+        ]);
       });
     }
   }
diff --git a/src/core/runner.ts b/src/core/runner.ts
index df1394b..89502da 100644
--- a/src/core/runner.ts
+++ b/src/core/runner.ts
@@ -332,7 +332,7 @@ export default class Runner extends EventEmitter {
   ) {
     const end = monotonicTimeInSeconds();
     const { pluginManager, start, status, error } = result;
-    const pluginOutput = pluginManager.output();
+    const pluginOutput = await pluginManager.output();
     this.emit('journey:end', {
       journey,
       status,
diff --git a/src/plugins/network.ts b/src/plugins/network.ts
index 53d40b5..53965f6 100644
--- a/src/plugins/network.ts
+++ b/src/plugins/network.ts
@@ -23,360 +23,284 @@
  *
  */
 
-import { Protocol } from 'playwright-core/types/protocol';
+import { Page, Request, Response } from 'playwright-chromium';
 import { NetworkInfo, BrowserInfo, Driver } from '../common_types';
 import { Step } from '../dsl';
 import { getTimestamp } from '../helpers';
 
+/**
+ * Kibana UI expects the requestStartTime and loadEndTime to be baseline
+ * in seconds as they have the logic to convert it to milliseconds before
+ * using for offset calculation
+ */
+function epochTimeInSeconds() {
+  return getTimestamp() / 1e6;
+}
+
+/**
+ * Used as a key in each Network Request to identify the
+ * associated request across distinct lifecycle events
+ */
+export const NETWORK_ENTRY_SUMBOL = Symbol.for('NetworkEntry');
+
+type RequestWithEntry = Request & {
+  NETWORK_ENTRY_SUMBOL?: symbol;
+};
+
 export class NetworkManager {
   private _browser: BrowserInfo;
-  private _InflightRequestMap = new Map<string, NetworkInfo>();
+  private _barrierPromises = new Set<Promise<void>>();
   results: Array<NetworkInfo> = [];
   _currentStep: Partial<Step> = null;
 
   constructor(private driver: Driver) {}
 
+  /**
+   * Adds a protection barrier aganist all asynchronous extract operations from
+   * request/response object that are happening during page lifecycle, If the
+   * page is closed during the extraction, the barrier enforces those operations
+   * to not result in exception
+   */
+  private _addBarrier(page: Page, promise: Promise<void>) {
+    if (!page) return;
+    const race = Promise.race([
+      new Promise<void>(resolve =>
+        page.on('close', () => {
+          this._barrierPromises.delete(race);
+          resolve();
+        })
+      ),
+      promise,
+    ]);
+    this._barrierPromises.add(race);
+    race.then(() => this._barrierPromises.delete(race));
+  }
+
   async start() {
-    const { client } = this.driver;
+    const { client, context } = this.driver;
     const { product } = await client.send('Browser.getVersion');
     const [name, version] = product.split('/');
     this._browser = { name, version };
     await client.send('Network.enable');
     /**
-     * Listen for all network events
+     * Listen for all network events from PW context
      */
-    client.on(
-      'Network.requestWillBeSent',
-      this._onRequestWillBeSent.bind(this)
-    );
-    client.on(
-      'Network.requestWillBeSentExtraInfo',
-      this._onRequestWillBeSentExtraInfo.bind(this)
-    );
-    client.on('Network.dataReceived', this._onDataReceived.bind(this));
-    client.on('Network.responseReceived', this._onResponseReceived.bind(this));
-    client.on('Network.loadingFinished', this._onLoadingFinished.bind(this));
-    client.on('Network.loadingFailed', this._onLoadingFailed.bind(this));
+    context.on('request', this._onRequest.bind(this));
+    context.on('response', this._onResponse.bind(this));
+    context.on('requestfinished', this._onRequestCompleted.bind(this));
+    context.on('requestfailed', this._onRequestCompleted.bind(this));
   }
 
-  _findNetworkRecord(requestId: string) {
-    return this._InflightRequestMap.get(requestId);
+  private _findNetworkEntry(
+    request: RequestWithEntry
+  ): NetworkInfo | undefined {
+    return request[NETWORK_ENTRY_SUMBOL];
   }
 
-  _onRequestWillBeSent(event: Protocol.Network.requestWillBeSentPayload) {
-    const { url, method, postData, headers } = event.request;
+  private _onRequest(request: Request) {
+    const url = request.url();
     /**
      * Data URI should not show up as network requests
      */
     if (url.startsWith('data:')) {
       return;
     }
-    const { requestId, timestamp, type, loaderId } = event;
-    const isNavigationRequest = requestId == loaderId && type === 'Document';
-    const record = this._findNetworkRecord(requestId);
-    /**
-     * On redirects, another `requestWillBeSent` event will be fired for the
-     * same requestId. We calculate the timings for the redirect request using
-     * the `redirectedResponse` from the redirected request.
-     */
-    if (record) {
-      if (event.redirectResponse) {
-        record.response.redirectURL = url;
-        const response = event.redirectResponse;
-        const data = Object.assign(event, {
-          type: event.type,
-          response,
-          encodedDataLength: response.encodedDataLength,
-          hasExtraInfo: false,
-        });
-        this._onResponseReceived(data);
-        this._onLoadingFinished(data);
-      } else {
-        /**
-         * Edge case, we handle it to proceed with next navigation
-         */
-        this._onLoadingFailed(
-          Object.assign(event, {
-            type: event.type,
-            errorText: 'redirectResponse data is not available',
-          })
-        );
-      }
-      const redirectedRecord = this._findNetworkRecord(requestId);
-      /**
-       * Rewrite the map with new redirect id to not reset
-       * the redirect request with original request
-       */
-      this._InflightRequestMap.delete(requestId);
-      this._InflightRequestMap.set(
-        `redirect:${timestamp}:${requestId}`,
-        redirectedRecord
-      );
-    }
 
-    this._InflightRequestMap.set(requestId, {
+    const timestamp = getTimestamp();
+    const networkEntry: NetworkInfo = {
       browser: this._browser,
       step: this._currentStep,
-      timestamp: getTimestamp(),
+      timestamp,
       url,
-      type,
+      type: request.resourceType(),
       request: {
         url,
-        method,
-        headers,
-        bytes: postData?.length || 0,
+        method: request.method(),
+        headers: {},
+        body: {
+          bytes: request.postDataBuffer()?.length || 0,
+        },
       },
       response: {
         status: -1,
-        mimeType: 'x-unknown',
         headers: {},
-        timing: null,
+        mimeType: 'x-unknown',
+        redirectURL: '',
       },
-      isNavigationRequest,
-      requestSentTime: timestamp,
-      responseReceivedTime: -1,
+      isNavigationRequest: request.isNavigationRequest(),
+      requestSentTime: epochTimeInSeconds(),
       loadEndTime: -1,
+      responseReceivedTime: -1,
       resourceSize: 0,
       transferSize: 0,
       timings: null,
-    });
-  }
-
-  _onRequestWillBeSentExtraInfo(
-    event: Protocol.Network.requestWillBeSentExtraInfoPayload
-  ) {
-    const { requestId, headers } = event;
-    const record = this._findNetworkRecord(requestId);
-    if (!record) {
-      return;
-    }
-    /**
-     * Enhance request headers with additional information
-     */
-    record.request.headers = {
-      ...record.request.headers,
-      ...headers,
     };
-    record.request.referrer = record.request.headers?.Referer;
-  }
-
-  _onDataReceived(event: Protocol.Network.dataReceivedPayload) {
-    const { requestId, dataLength, encodedDataLength } = event;
-    const record = this._findNetworkRecord(requestId);
-    if (!record) {
-      return;
-    }
-    record.resourceSize += dataLength;
-    if (encodedDataLength >= 0) record.transferSize += encodedDataLength;
-  }
 
-  _onResponseReceived(event: Protocol.Network.responseReceivedPayload) {
-    const { requestId, response, timestamp } = event;
-    const record = this._findNetworkRecord(requestId);
-    if (!record) {
-      return;
+    if (request.redirectedFrom()) {
+      const fromEntry = this._findNetworkEntry(request.redirectedFrom());
+      if (fromEntry) fromEntry.response.redirectURL = request.url();
     }
 
-    record.responseReceivedTime = timestamp;
-    record.transferSize = response.encodedDataLength;
-    record.response = {
-      url: response.url,
-      status: response.status,
-      protocol: response.protocol,
-      statusText: response.statusText,
-      headers: response.headers,
-      mimeType: response.mimeType,
-      remoteIPAddress: response.remoteIPAddress,
-      remotePort: response.remotePort,
-      fromServiceWorker: response.fromServiceWorker,
-      redirectURL: record.response.redirectURL,
-      timing: response.timing,
-    };
-    if (response.securityDetails) {
-      record.response.securityDetails = {
-        protocol: response.securityDetails.protocol,
-        subjectName: response.securityDetails.subjectName,
-        issuer: response.securityDetails.issuer,
-        validFrom: response.securityDetails.validFrom,
-        validTo: response.securityDetails.validTo,
-      };
-    }
-  }
-
-  _onLoadingFinished(event: Protocol.Network.loadingFinishedPayload) {
-    const { requestId, timestamp, encodedDataLength } = event;
-    this._requestCompleted(requestId, timestamp, encodedDataLength);
+    request[NETWORK_ENTRY_SUMBOL] = networkEntry;
+    this.results.push(networkEntry);
   }
 
-  _onLoadingFailed(event: Protocol.Network.loadingFailedPayload) {
-    const { requestId, timestamp } = event;
-    this._requestCompleted(requestId, timestamp);
-  }
+  private async _onResponse(response: Response) {
+    const request = response.request();
+    const networkEntry = this._findNetworkEntry(request);
+    if (!networkEntry) return;
 
-  _requestCompleted(
-    requestId: string,
-    endTime: number,
-    encodedDataLength?: number
-  ) {
-    const record = this._findNetworkRecord(requestId);
-    if (!record) {
-      return;
-    }
-    if (encodedDataLength >= 0) {
-      record.transferSize = encodedDataLength;
-      record.response.body = {
-        bytes: encodedDataLength,
-      };
-    }
-    record.loadEndTime = endTime;
-    record.timings = calculateTimings(record);
-    this.results.push(record);
-    /**
-     * When there is DNS failure, browsers keeps retrying requests which
-     * might result in incorrect timings so we get remove them to measure only once
-     */
-    this._InflightRequestMap.delete(requestId);
-  }
+    networkEntry.responseReceivedTime = epochTimeInSeconds();
+    networkEntry.response = {
+      url: response.url(),
+      status: response.status(),
+      statusText: response.statusText(),
+      headers: {},
+      mimeType: 'x-unknown',
+      redirectURL: networkEntry.response.redirectURL,
+    };
 
-  /**
-   * Account for missing response received event and also adjust the
-   * response received event based on when first byte event was recorded
-   */
+    const page = request.frame().page();
+    this._addBarrier(
+      page,
+      request.allHeaders().then(reqHeaders => {
+        networkEntry.request.headers = reqHeaders;
+        networkEntry.request.referrer = reqHeaders?.referer;
+      })
+    );
+    this._addBarrier(
+      page,
+      response.serverAddr().then(server => {
+        networkEntry.response.remoteIPAddress = server?.ipAddress;
+        networkEntry.response.remotePort = server?.port;
+      })
+    );
+    this._addBarrier(
+      page,
+      response.securityDetails().then(details => {
+        if (details) networkEntry.response.securityDetails = details;
+      })
+    );
+    this._addBarrier(
+      page,
+      response.allHeaders().then(resHeaders => {
+        networkEntry.response.headers = resHeaders;
 
-  stop() {
-    return this.results;
+        const contentType = resHeaders['content-type'];
+        if (contentType)
+          networkEntry.response.mimeType = contentType.split(';')[0];
+      })
+    );
   }
-}
 
-function getResponseReceivedTime(
-  timing: Protocol.Network.Response['timing'],
-  responseReceivedTime: number
-) {
-  if (timing == null) {
-    return responseReceivedTime;
-  }
-  const startTime = timing.requestTime;
-  const headersReceivedTime = startTime + timing.receiveHeadersEnd / 1000;
-  if (responseReceivedTime < 0 || responseReceivedTime > headersReceivedTime) {
-    responseReceivedTime = headersReceivedTime;
-  }
-  if (startTime > responseReceivedTime) {
-    responseReceivedTime = startTime;
-  }
-  return responseReceivedTime;
-}
+  private async _onRequestCompleted(request: Request) {
+    const networkEntry = this._findNetworkEntry(request);
+    if (!networkEntry) return;
 
-/**
- * The timing calculations are based on the chrome devtools frontend
- * https://github.com/ChromeDevTools/devtools-frontend/blob/7f5478d8ceb7586f23f4073ab5c2085dac1ec26a/front_end/network/RequestTimingView.js#L98-L193
- */
-export function calculateTimings(record: NetworkInfo) {
-  const result: NetworkInfo['timings'] = {
-    blocked: -1,
-    queueing: -1,
-    proxy: -1,
-    dns: -1,
-    ssl: -1,
-    connect: -1,
-    send: -1,
-    wait: -1,
-    receive: -1,
-    total: -1,
-  };
+    networkEntry.loadEndTime = epochTimeInSeconds();
+    const timing = request.timing();
+    const { loadEndTime, requestSentTime } = networkEntry;
+    networkEntry.timings = {
+      blocked: -1,
+      dns: -1,
+      ssl: -1,
+      connect: -1,
+      send: -1,
+      wait: -1,
+      receive: -1,
+      total: -1,
+    };
 
-  const toMilliseconds = (time: number) => (time === -1 ? -1 : time * 1000);
-  const calculateDiffInMs = (
-    name: keyof NetworkInfo['timings'],
-    start: number,
-    end: number
-  ) => {
-    if (start < Number.MAX_VALUE && start <= end) {
-      result[name] = toMilliseconds(end - start);
-    }
-  };
-  const firstPositive = (numbers: number[]) => {
-    for (let i = 0; i < numbers.length; ++i) {
-      if (numbers[i] > 0) {
-        return numbers[i];
+    const firstPositive = (numbers: number[]) => {
+      for (let i = 0; i < numbers.length; ++i) {
+        if (numbers[i] > 0) {
+          return numbers[i];
+        }
       }
-    }
-    return null;
-  };
-  /**
-   * requestTime is baseline in seconds, rest of the timing data are ticks in milliseconds
-   * from the requestTime, so we calculate the offset from that
-   */
-  const addOffsetRange = (
-    name: keyof NetworkInfo['timings'],
-    start: number,
-    end: number
-  ) => {
-    if (start >= 0 && end >= 0) {
-      calculateDiffInMs(name, startTime + start / 1000, startTime + end / 1000);
-    }
-  };
-  const { requestSentTime, loadEndTime, responseReceivedTime, response } =
-    record;
-  const timing = response.timing;
-  const actResRcvdTime = getResponseReceivedTime(timing, responseReceivedTime);
-  const issueTime = requestSentTime;
-  const startTime = timing == null ? -1 : timing.requestTime;
-  const endTime = firstPositive([loadEndTime, actResRcvdTime]) || startTime;
+      return null;
+    };
+    const roundMilliSecs = (value: number): number => {
+      return Math.floor(value * 1000) / 1000;
+    };
 
-  if (timing == null) {
-    const start =
-      issueTime !== -1 ? issueTime : startTime !== -1 ? startTime : 0;
-    const middle = actResRcvdTime === -1 ? Number.MAX_VALUE : actResRcvdTime;
-    const end = endTime === -1 ? Number.MAX_VALUE : endTime;
-    calculateDiffInMs('total', start, end);
-    calculateDiffInMs('blocked', start, middle);
-    calculateDiffInMs('receive', middle, end);
-    // Check if the request is blocked/stalled for the whole timeframe and
-    // calculate timings appropriately for that request
-    if (!isFinite(result['blocked'])) {
-      result['blocked'] = result['total'];
+    if (timing.startTime === 0) {
+      // Convert to milliseconds before round off
+      const total = roundMilliSecs((loadEndTime - requestSentTime) * 1000);
+      networkEntry.timings.total = total;
+      return;
     }
-    return result;
-  }
-  calculateDiffInMs(
-    'total',
-    issueTime < startTime ? issueTime : startTime,
-    endTime
-  );
-  if (issueTime < startTime) {
-    calculateDiffInMs('queueing', issueTime, startTime);
-  }
-  const responseReceived = toMilliseconds(actResRcvdTime - startTime);
 
-  if (response.fromServiceWorker) {
-    addOffsetRange('blocked', 0, timing.workerStart);
-    addOffsetRange('wait', timing.sendEnd, responseReceived);
-  } else if (!timing.pushStart) {
-    const blockingEnd =
-      firstPositive([
-        timing.dnsStart,
-        timing.connectStart,
-        timing.sendStart,
-        responseReceived,
-      ]) || 0;
-    addOffsetRange('blocked', 0, blockingEnd);
-    addOffsetRange('proxy', timing.proxyStart, timing.proxyEnd);
-    addOffsetRange('dns', timing.dnsStart, timing.dnsEnd);
-    addOffsetRange('connect', timing.connectStart, timing.connectEnd);
-    addOffsetRange('ssl', timing.sslStart, timing.sslEnd);
-    addOffsetRange('send', timing.sendStart, timing.sendEnd);
-    addOffsetRange(
-      'wait',
-      Math.max(
-        timing.sendEnd,
-        timing.connectEnd,
-        timing.dnsEnd,
-        timing.proxyEnd,
-        blockingEnd
-      ),
-      responseReceived
+    const blocked =
+      roundMilliSecs(
+        firstPositive([
+          timing.domainLookupStart,
+          timing.connectStart,
+          timing.requestStart,
+        ])
+      ) || -1;
+    const dns =
+      timing.domainLookupEnd !== -1
+        ? roundMilliSecs(timing.domainLookupEnd - timing.domainLookupStart)
+        : -1;
+    const connect =
+      timing.connectEnd !== -1
+        ? roundMilliSecs(timing.connectEnd - timing.connectStart)
+        : -1;
+    const ssl =
+      timing.secureConnectionStart !== -1
+        ? roundMilliSecs(timing.connectEnd - timing.secureConnectionStart)
+        : -1;
+    const wait =
+      timing.responseStart !== -1
+        ? roundMilliSecs(timing.responseStart - timing.requestStart)
+        : -1;
+    const receive =
+      timing.responseEnd !== -1
+        ? roundMilliSecs(timing.responseEnd - timing.responseStart)
+        : -1;
+    const total = [blocked, dns, connect, wait, receive].reduce(
+      (pre, cur) => (cur > 0 ? cur + pre : pre),
+      0
+    );
+    networkEntry.timings = {
+      blocked,
+      dns,
+      connect,
+      ssl,
+      wait,
+      send: 0, // not exposed via RT api
+      receive,
+      total,
+    };
+
+    const page = request.frame().page();
+    // For aborted/failed requests sizes will not be present
+    this._addBarrier(
+      page,
+      request.sizes().then(sizes => {
+        networkEntry.request.bytes =
+          sizes.requestHeadersSize + sizes.requestBodySize;
+        networkEntry.request.body = {
+          bytes: sizes.requestBodySize,
+        };
+        networkEntry.response.bytes =
+          sizes.responseHeadersSize + sizes.responseBodySize;
+        networkEntry.response.body = {
+          bytes: sizes.responseBodySize,
+        };
+        networkEntry.transferSize = sizes.responseBodySize;
+      })
     );
   }
-  calculateDiffInMs('receive', actResRcvdTime, endTime);
-  return result;
+
+  async stop() {
+    await Promise.all(this._barrierPromises);
+    const context = this.driver.context;
+    context.off('request', this._onRequest.bind(this));
+    context.off('response', this._onResponse.bind(this));
+    context.off('requestfinished', this._onRequestCompleted.bind(this));
+    context.off('requestfailed', this._onRequestCompleted.bind(this));
+    this._barrierPromises.clear();
+    return this.results;
+  }
 }
diff --git a/src/plugins/plugin-manager.ts b/src/plugins/plugin-manager.ts
index 0871b01..740bcb0 100644
--- a/src/plugins/plugin-manager.ts
+++ b/src/plugins/plugin-manager.ts
@@ -102,11 +102,11 @@ export class PluginManager {
     (this.get('network') as NetworkManager)._currentStep = step;
   }
 
-  output() {
+  async output() {
     const data: PluginOutput = {};
     for (const [, plugin] of this.plugins) {
       if (plugin instanceof NetworkManager) {
-        data.networkinfo = plugin.stop();
+        data.networkinfo = await plugin.stop();
       } else if (plugin instanceof BrowserConsole) {
         data.browserconsole = plugin.stop();
       }
diff --git a/src/reporters/json.ts b/src/reporters/json.ts
index bcc9036..a89407c 100644
--- a/src/reporters/json.ts
+++ b/src/reporters/json.ts
@@ -128,23 +128,8 @@ function getMetadata() {
   };
 }
 
-function formatVersion(protocol: string | undefined) {
-  if (!protocol) {
-    return;
-  }
-  if (protocol === 'h2') {
-    return 2;
-  } else if (protocol === 'http/1.1') {
-    return 1.1;
-  } else if (protocol === 'http/1.0') {
-    return 1.0;
-  } else if (protocol.startsWith('h3')) {
-    return 3;
-  }
-}
-
 function formatTLS(tls: SecurityDetails) {
-  if (!tls) {
+  if (!tls || !tls.protocol) {
     return;
   }
   const [name, version] = tls.protocol.toLowerCase().split(' ');
@@ -177,7 +162,6 @@ export function formatNetworkFields(network: NetworkInfo) {
       original: request.headers?.['User-Agent'],
     },
     http: {
-      version: formatVersion(response?.protocol),
       request,
       response,
     },


Test patch:
diff --git a/__tests__/core/gatherer.test.ts b/__tests__/core/gatherer.test.ts
index 876a21e..bd3f6ec 100644
--- a/__tests__/core/gatherer.test.ts
+++ b/__tests__/core/gatherer.test.ts
@@ -205,16 +205,33 @@ describe('Gatherer', () => {
         page.click('a'),
       ]);
       await page1.waitForLoadState();
-
       // @ts-ignore
       // Experimental browser API https://developer.mozilla.org/en-US/docs/Web/API/NetworkInformation/downlink
       const downlink = await page1.evaluate(
         () => (navigator['connection'] as any).downlink
       );
-
       expect(3.5 > downlink && downlink > 2.5).toBe(true);
       await Gatherer.dispose(driver);
       await Gatherer.stop();
     });
+
+    it('dont throw for closed popups before load', async () => {
+      const driver = await Gatherer.setupDriver({
+        wsEndpoint,
+        networkConditions,
+      });
+      const { page, context } = driver;
+      await page.goto(server.TEST_PAGE);
+      await page.setContent(
+        '<a target=_blank rel=noopener href="/popup.html">popup</a>'
+      );
+      const [page1] = await Promise.all([
+        context.waitForEvent('page'),
+        page.click('a'),
+      ]);
+      await page1.close();
+      await Gatherer.dispose(driver);
+      await Gatherer.stop();
+    });
   });
 });
diff --git a/__tests__/fixtures/networkinfo.ts b/__tests__/fixtures/networkinfo.ts
index 6411d81..4deecc5 100644
--- a/__tests__/fixtures/networkinfo.ts
+++ b/__tests__/fixtures/networkinfo.ts
@@ -48,7 +48,6 @@ export const NETWORK_INFO: Array<NetworkInfo> = [
     response: {
       url: 'http://localhost:56899/index',
       status: 200,
-      protocol: 'http/1.1',
       statusText: 'OK',
       headers: {
         'content-type': 'text/html',
@@ -62,26 +61,6 @@ export const NETWORK_INFO: Array<NetworkInfo> = [
       remotePort: 56899,
       fromServiceWorker: false,
       redirectURL: undefined,
-      timing: {
-        requestTime: 300869.367875,
-        proxyStart: -1,
-        proxyEnd: -1,
-        dnsStart: 0.25,
-        dnsEnd: 0.25,
-        connectStart: 0.25,
-        connectEnd: 0.5,
-        sslStart: -1,
-        sslEnd: -1,
-        workerStart: -1,
-        workerReady: -1,
-        workerFetchStart: -1,
-        workerRespondWithSettled: -1,
-        sendStart: 0.5,
-        sendEnd: 0.5,
-        pushStart: 0,
-        pushEnd: 0,
-        receiveHeadersEnd: 5.125,
-      },
       body: { bytes: 194 },
     },
     isNavigationRequest: true,
@@ -92,8 +71,6 @@ export const NETWORK_INFO: Array<NetworkInfo> = [
     transferSize: 194,
     timings: {
       blocked: 0.24999998277053237,
-      queueing: 0.6249999860301614,
-      proxy: -1,
       dns: 0,
       ssl: -1,
       connect: 0.2500000409781933,
@@ -126,7 +103,6 @@ export const NETWORK_INFO: Array<NetworkInfo> = [
       status: -1,
       mimeType: 'x-unknown',
       headers: {},
-      timing: null,
     },
     isNavigationRequest: false,
     requestSentTime: 300869.377,
@@ -136,8 +112,6 @@ export const NETWORK_INFO: Array<NetworkInfo> = [
     transferSize: 0,
     timings: {
       blocked: 79.75000003352761,
-      queueing: -1,
-      proxy: -1,
       dns: -1,
       ssl: -1,
       connect: -1,
diff --git a/__tests__/plugins/network.test.ts b/__tests__/plugins/network.test.ts
index 5555165..069412b 100644
--- a/__tests__/plugins/network.test.ts
+++ b/__tests__/plugins/network.test.ts
@@ -24,7 +24,7 @@
  */
 
 import { Gatherer } from '../../src/core/gatherer';
-import { NetworkManager, calculateTimings } from '../../src/plugins/network';
+import { NetworkManager } from '../../src/plugins/network';
 import { Server } from '../utils/server';
 import { wsEndpoint } from '../utils/test-config';
 
@@ -52,12 +52,29 @@ describe('network', () => {
       step: null,
       timestamp: expect.any(Number),
       url: server.TEST_PAGE,
-      request: expect.any(Object),
-      response: expect.any(Object),
-      type: 'Document',
+      request: {
+        url: server.TEST_PAGE,
+        method: 'GET',
+        body: {
+          bytes: 0,
+        },
+        bytes: expect.any(Number),
+      },
+      response: {
+        url: server.TEST_PAGE,
+        status: 200,
+        statusText: 'OK',
+        body: {
+          bytes: expect.any(Number),
+        },
+        bytes: expect.any(Number),
+      },
+      type: 'document',
       requestSentTime: expect.any(Number),
       loadEndTime: expect.any(Number),
       responseReceivedTime: expect.any(Number),
+      resourceSize: expect.any(Number),
+      transferSize: expect.any(Number),
       timings: expect.any(Object),
     });
   });
@@ -106,8 +123,8 @@ describe('network', () => {
     });
     const netinfo = await network.stop();
     expect(netinfo[0]).toMatchObject({
-      resourceSize: 10,
-      transferSize: expect.any(Number),
+      resourceSize: 0,
+      transferSize: 10,
     });
     await Gatherer.stop();
   });
@@ -121,57 +138,60 @@ describe('network', () => {
     await network.start();
 
     const delayTime = 20;
-    server.route('/delay20', async (req, res) => {
+    server.route('/abort', async (req, res) => {
       await delay(delayTime);
       res.destroy();
     });
     server.route('/index', async (_, res) => {
       res.setHeader('content-type', 'text/html');
-      res.end(`<script src=${server.PREFIX}/delay20 />`);
+      res.end(`<script src=${server.PREFIX}/abort />`);
     });
 
-    await driver.page.goto(server.PREFIX + '/index');
+    await driver.page.goto(server.PREFIX + '/index', {
+      waitUntil: 'networkidle',
+    });
     await driver.page.waitForLoadState();
     await Gatherer.stop();
     const netinfo = await network.stop();
     expect(netinfo.length).toBe(2);
     expect(netinfo[1]).toMatchObject({
-      url: `${server.PREFIX}/delay20`,
+      url: `${server.PREFIX}/abort`,
       response: {
-        headers: {},
         mimeType: 'x-unknown',
         status: -1,
-        timing: null,
+        redirectURL: '',
       },
       timings: expect.any(Object),
     });
     expect(netinfo[1].timings.total).toBeGreaterThan(delayTime);
-    expect(netinfo[1].timings.total).toEqual(netinfo[1].timings.blocked);
   });
 
   it('timings for chunked response', async () => {
     const driver = await Gatherer.setupDriver({ wsEndpoint });
+    await driver.client.send('Network.setCacheDisabled', {
+      cacheDisabled: true,
+    });
     const network = new NetworkManager(driver);
     await network.start();
 
     const delayTime = 100;
     server.route('/chunked', async (req, res) => {
+      await delay(delayTime);
       res.writeHead(200, {
         'content-type': 'application/javascript',
       });
       res.write('a');
       await delay(delayTime);
-      res.write('b');
-      await delay(delayTime);
-      return res.end('c');
+      return res.end('b');
     });
     server.route('/index', async (_, res) => {
       res.setHeader('content-type', 'text/html');
       res.end(`<script src=${server.PREFIX}/chunked />`);
     });
 
-    await driver.page.goto(server.PREFIX + '/index');
-    await driver.page.waitForLoadState();
+    await driver.page.goto(server.PREFIX + '/index', {
+      waitUntil: 'networkidle',
+    });
     await Gatherer.stop();
     const netinfo = await network.stop();
     expect(netinfo.length).toBe(2);
@@ -180,89 +200,42 @@ describe('network', () => {
       response: expect.any(Object),
       timings: expect.any(Object),
     });
-    expect(netinfo[1].timings.total).toBeGreaterThan(delayTime);
+    const timings = netinfo[1].timings;
+    expect(timings.wait).toBeGreaterThan(delayTime);
+    expect(timings.receive).toBeGreaterThan(delayTime);
+    expect(timings.total).toBeGreaterThan(timings.wait + timings.receive);
   });
 
-  describe('waterfall timing calculation', () => {
-    const getEvent = () => {
-      return {
-        response: {
-          // requestTime is in seconds, rest of the timing.* is in milliseconds
-          timing: {
-            requestTime: 1,
-            proxyStart: -1,
-            proxyEnd: -1,
-            dnsStart: 0.1,
-            dnsEnd: 26,
-            connectStart: 26,
-            connectEnd: 92.669,
-            sslStart: 40,
-            sslEnd: 92,
-            sendStart: 94,
-            sendEnd: 95,
-            receiveHeadersEnd: 2350,
-          },
-        },
-        requestSentTime: 1,
-        loadEndTime: 3,
-        responseReceivedTime: 2,
-      };
-    };
-
-    it('calculate timings for a request event', () => {
-      const record = getEvent();
-      const timings = calculateTimings(record as any);
-      expect(timings).toEqual({
-        blocked: 0.09999999999998899,
-        queueing: -1,
-        proxy: -1,
-        dns: 25.900000000000034,
-        ssl: 52.00000000000004,
-        connect: 66.66899999999987,
-        send: 0.9999999999998899,
-        wait: 905,
-        receive: 1000,
-        total: 2000,
-      });
+  it('capture network data from popups', async () => {
+    const driver = await Gatherer.setupDriver({
+      wsEndpoint,
     });
+    const { page, context } = driver;
+    const network = new NetworkManager(driver);
+    await network.start();
 
-    it('when some resource timing data is unavailable', () => {
-      const record = getEvent();
-      Object.assign(record.response.timing, {
-        connectEnd: -1,
-        dnsStart: -1,
-      });
-      const timings = calculateTimings(record as any);
-      expect(timings).toEqual({
-        blocked: 26.00000000000002,
-        connect: -1,
-        dns: -1,
-        proxy: -1,
-        queueing: -1,
-        receive: 1000,
-        send: 0.9999999999998899,
-        ssl: 52.00000000000004,
-        total: 2000,
-        wait: 905,
-      });
-    });
+    // Simulate user flow from test page -> popup page
+    await page.goto(server.TEST_PAGE);
+    await page.setContent(
+      '<a target=_blank rel=noopener href="/popup.html">popup</a>'
+    );
+    const [page1] = await Promise.all([
+      context.waitForEvent('page'),
+      page.click('a'),
+    ]);
+    await page1.waitForLoadState('load');
+    expect(await page1.textContent('body')).toEqual('Not found');
 
-    it('when complete resource timing is not available', () => {
-      const record = getEvent();
-      record.response.timing = null;
-      const timings = calculateTimings(record as any);
-      expect(timings).toEqual({
-        blocked: 1000,
-        connect: -1,
-        dns: -1,
-        proxy: -1,
-        queueing: -1,
-        receive: 1000,
-        send: -1,
-        ssl: -1,
-        total: 2000,
-        wait: -1,
-      });
-    });
+    await Gatherer.stop();
+    const netinfo = await network.stop();
+    expect(netinfo.length).toBe(2);
+    expect(netinfo).toMatchObject([
+      {
+        url: server.TEST_PAGE,
+      },
+      {
+        url: `${server.PREFIX}/popup.html`,
+      },
+    ]);
   });
 });
diff --git a/__tests__/reporters/__snapshots__/json.test.ts.snap b/__tests__/reporters/__snapshots__/json.test.ts.snap
index ddae101..524ccc3 100644
--- a/__tests__/reporters/__snapshots__/json.test.ts.snap
+++ b/__tests__/reporters/__snapshots__/json.test.ts.snap
@@ -34,35 +34,13 @@ Object {
           "keep_alive": "timeout=5",
         },
         "mime_type": "text/html",
-        "protocol": "http/1.1",
         "redirect_u_r_l": undefined,
         "remote_i_p_address": "[::1]",
         "remote_port": 56899,
         "status": 200,
         "status_text": "OK",
-        "timing": Object {
-          "connect_end": 0.5,
-          "connect_start": 0.25,
-          "dns_end": 0.25,
-          "dns_start": 0.25,
-          "proxy_end": -1,
-          "proxy_start": -1,
-          "push_end": 0,
-          "push_start": 0,
-          "receive_headers_end": 5.125,
-          "request_time": 300869.367875,
-          "send_end": 0.5,
-          "send_start": 0.5,
-          "ssl_end": -1,
-          "ssl_start": -1,
-          "worker_fetch_start": -1,
-          "worker_ready": -1,
-          "worker_respond_with_settled": -1,
-          "worker_start": -1,
-        },
         "url": "http://localhost:56899/index",
       },
-      "version": 1.1,
     },
     "tls": undefined,
     "url": "http://localhost:56899/index",
@@ -86,8 +64,6 @@ Object {
       "blocked": 0.24999998277053237,
       "connect": 0.2500000409781933,
       "dns": 0,
-      "proxy": -1,
-      "queueing": 0.6249999860301614,
       "receive": 0.12499996228143573,
       "send": 0,
       "ssl": -1,
@@ -120,9 +96,7 @@ Object {
         "headers": Object {},
         "mime_type": "x-unknown",
         "status": -1,
-        "timing": null,
       },
-      "version": undefined,
     },
     "tls": undefined,
     "url": "http://localhost:56899/delay100",
@@ -144,8 +118,6 @@ Object {
       "blocked": 79.75000003352761,
       "connect": -1,
       "dns": -1,
-      "proxy": -1,
-      "queueing": -1,
       "receive": -1,
       "send": -1,
       "ssl": -1,