Repository: elastic/synthetics
Pull request #406 (2021-11-03 04:32:01)

deprecate suite-params option over params
+ We have reverted this change - https://github.com/elastic/synthetics/pull/379 for the time being to make 7.15 work, But we added patch to the 7.x branch to add params support in HB and now that we are at the 7.16 release timeline, we can safely remove this from Synthetics side. 

apmmachine: 
## :broken_heart: Tests Failed


<!-- BUILD BADGES-->
> _the below badges are clickable and redirect to their specific view in the CI or DOCS_
[![Pipeline View](https://img.shields.io/badge/pipeline-pipeline%20-green)](https://apm-ci.elastic.co/blue/organizations/jenkins/apm-agent-rum%2Felastic-synthetics%2FPR-406/detail/PR-406/1//pipeline) [![Test View](https://img.shields.io/badge/test-test-green)](https://apm-ci.elastic.co/blue/organizations/jenkins/apm-agent-rum%2Felastic-synthetics%2FPR-406/detail/PR-406/1//tests) [![Changes](https://img.shields.io/badge/changes-changes-green)](https://apm-ci.elastic.co/blue/organizations/jenkins/apm-agent-rum%2Felastic-synthetics%2FPR-406/detail/PR-406/1//changes) [![Artifacts](https://img.shields.io/badge/artifacts-artifacts-yellow)](https://apm-ci.elastic.co/blue/organizations/jenkins/apm-agent-rum%2Felastic-synthetics%2FPR-406/detail/PR-406/1//artifacts) [![preview](https://img.shields.io/badge/docs-preview-yellowgreen)](http://synthetics_406.docs-preview.app.elstc.co/diff)  [![preview](https://img.shields.io/badge/elastic-observability-blue)](https://ci-stats.elastic.co/app/apm/services/apm-ci/transactions/view?rangeFrom=2021-11-03T04:22:15.558Z&rangeTo=2021-11-03T04:42:15.558Z&transactionName=apm-agent-rum/elastic-synthetics/PR-{number}&transactionType=unknown&latencyAggregationType=avg&traceId=48d3df22a0c4f6efa1d1a708484250c4&transactionId=d4778d37629ae43e)

<!-- BUILD SUMMARY-->
<details><summary>Expand to view the summary</summary>
<p>

#### Build stats




* Start Time: 2021-11-03T04:32:15.558+0000


* Duration: 8 min 5 sec


* Commit: 56567cc596caf8f2f6658884c7cd6cafc369f566



#### Test stats :test_tube:

| Test         | Results                         |
| ------------ | :-----------------------------: |
| Failed       | 1  |
| Passed       | 137  |
| Skipped      | 0 |
| Total        | 138   |


</p>
</details>

<!-- TEST RESULTS IF ANY-->

  ### Test errors [![1](https://img.shields.io/badge/1%20-red)](https://apm-ci.elastic.co/blue/organizations/jenkins/apm-agent-rum%2Felastic-synthetics%2FPR-406/detail/PR-406/1//tests)
<details><summary>Expand to view the tests failures</summary><p>
    
    
##### `Test / CLI screenshots with device emulation – CLI screenshots with device emulation`


<ul>
<li>no error details</li>
<details><summary>Expand to view the stacktrace</summary><p>

```
 Error: Failed: "Error SyntaxError: Unexpected end of JSON input could not parse data '{\"type\":\"screenshot/block\",\"_id\":\"71bb77ed1acf85f13f1e8a2ab3cf3401850f7b9d\",\"@timestamp\":1635914390993924.5,\"root_fields\":{\"os\":{\"platform\":\"linux\"},\"package\":{\"name\":\"@elastic/synthetics\",\"version\":\"1.0.0-beta.16\"}},\"blob\":\"/9j/2wBDAAYEBQYFBAYGBQYHBwYIChAKCgkJChQODwwQFxQYGBcUFhYaHSUfGhsjHBYWICwgIyYnKSopGR8tMC0oMCUoKSj/2wBDAQcHBwoIChMKChMoGhYaKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCj/wAARCAFyALQDASIAAhEBAxEB/8QAFQABAQAAAAAAAAAAAAAAAAAAAAj/xAAUEAEAAAAAAAAAAAAAAAAAAAAA/8QAFAEBAAAAAAAAAAAAAAAAAAAAAP/EABQRAQAAAAAAAAAAAAAAAAAAAAD/2gAMAwEAAhEDEQA/AKpAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA'"
    at Env.it (/var/lib/jenkins/workspace/nt-rum_elastic-synthetics_PR-406/src/github.com/elastic/synthetics/node_modules/jest-jasmine2/build/jasmineAsyncInstall.js:95:24)
    at it (/var/lib/jenkins/workspace/nt-rum_elastic-synthetics_PR-406/src/github.com/elastic/synthetics/node_modules/jest-jasmine2/build/jasmine/jasmineLight.js:100:21)
    at Suite.<anonymous> (/var/lib/jenkins/workspace/nt-rum_elastic-synthetics_PR-406/src/github.com/elastic/synthetics/__tests__/cli.test.ts:146:3)
    at addSpecsToSuite (/var/lib/jenkins/workspace/nt-rum_elastic-synthetics_PR-406/src/github.com/elastic/synthetics/node_modules/jest-jasmine2/build/jasmine/Env.js:444:51)
    at Env.describe (/var/lib/jenkins/workspace/nt-rum_elastic-synthetics_PR-406/src/github.com/elastic/synthetics/node_modules/jest-jasmine2/build/jasmine/Env.js:414:11)
    at describe (/var/lib/jenkins/workspace/nt-rum_elastic-synthetics_PR-406/src/github.com/elastic/synthetics/node_modules/jest-jasmine2/build/jasmine/jasmineLight.js:88:18)
    at Object.<anonymous> (/var/lib/jenkins/workspace/nt-rum_elastic-synthetics_PR-406/src/github.com/elastic/synthetics/__tests__/cli.test.ts:31:1)
    at Runtime._execModule (/var/lib/jenkins/workspace/nt-rum_elastic-synthetics_PR-406/src/github.com/elastic/synthetics/node_modules/jest-runtime/build/index.js:1299:24)
    at Runtime._loadModule (/var/lib/jenkins/workspace/nt-rum_elastic-synthetics_PR-406/src/github.com/elastic/synthetics/node_modules/jest-runtime/build/index.js:898:12)
    at Runtime.requireModule (/var/lib/jenkins/workspace/nt-rum_elastic-synthetics_PR-406/src/github.com/elastic/synthetics/node_modules/jest-runtime/build/index.js:746:10) 
 ```
</p></details>
</ul>
    
</p></details>


<!-- STEPS ERRORS IF ANY -->











### Steps errors [![1](https://img.shields.io/badge/1%20-red)](https://apm-ci.elastic.co/blue/organizations/jenkins/apm-agent-rum%2Felastic-synthetics%2FPR-406/detail/PR-406/1//pipeline)
  <details><summary>Expand to view the steps failures</summary>
  <p>


  
  
  
  
##### `Runs the tests`
<ul>
<li>Took 1 min 58 sec . View more details <a href="https://apm-ci.elastic.co//blue/rest/organizations/jenkins/pipelines/apm-agent-rum/pipelines/elastic-synthetics/pipelines/PR-406/runs/1/steps/303/log/?start=0">here</a></li>
<li>Description: <code>./node_modules/.bin/jest --ci --reporters=default --reporters=jest-junit</code></l1>
</ul>
  
  </p>
  </details>


## :robot: GitHub comments

To re-run your PR in the CI, just comment with:


- `/test` : Re-trigger the build.




<!--COMMENT_GENERATED_WITH_ID_comment.id-->

Patch:
diff --git a/src/cli.ts b/src/cli.ts
index faf8b23..0be4658 100644
--- a/src/cli.ts
+++ b/src/cli.ts
@@ -175,11 +175,7 @@ async function prepareSuites(inputs: string[]) {
    * Validate and handle configs
    */
   const config = readConfig(environment, options.config);
-  const params = merge(
-    config.params,
-    options.suiteParams || {},
-    options.params || {}
-  );
+  const params = merge(config.params, options.params || {});
   const playwrightOptions = merge(config.playwrightOptions, {
     headless: options.headless,
     chromiumSandbox: options.sandbox,
diff --git a/src/common_types.ts b/src/common_types.ts
index 7ec9478..b48baf4 100644
--- a/src/common_types.ts
+++ b/src/common_types.ts
@@ -164,10 +164,6 @@ export type CliArgs = {
   ignoreHttpsErrors?: boolean;
   params?: Params;
   throttling?: boolean | string;
-  /**
-   * @deprecated use params instead
-   */
-  suiteParams?: Params;
   richEvents?: boolean;
 };
 
diff --git a/src/core/gatherer.ts b/src/core/gatherer.ts
index 51b15ea..b73c9ed 100644
--- a/src/core/gatherer.ts
+++ b/src/core/gatherer.ts
@@ -37,6 +37,7 @@ export class Gatherer {
   static browser: ChromiumBrowser;
 
   static async setupDriver(options: RunOptions): Promise<Driver> {
+    log('Gatherer: setup driver');
     const { wsEndpoint, playwrightOptions, networkConditions } = options;
 
     if (Gatherer.browser == null) {
diff --git a/src/core/runner.ts b/src/core/runner.ts
index c9e9354..93a2838 100644
--- a/src/core/runner.ts
+++ b/src/core/runner.ts
@@ -123,6 +123,7 @@ interface Events {
     JourneyResult & {
       journey: Journey;
       options: RunOptions;
+      timestamp: number;
     };
   'journey:end:reported': unknown;
   'step:start': { journey: Journey; step: Step };
@@ -149,8 +150,12 @@ export default class Runner extends EventEmitter {
   static screenshotPath = join(CACHE_PATH, 'screenshots');
 
   static async createContext(options: RunOptions): Promise<JourneyContext> {
-    const start = monotonicTimeInSeconds();
     const driver = await Gatherer.setupDriver(options);
+    /**
+     * Do not include browser launch/context creation duration
+     * as part of journey duration
+     */
+    const start = monotonicTimeInSeconds();
     const pluginManager = await Gatherer.beginRecording(driver, options);
     /**
      * For each journey we create the screenshots folder for
@@ -339,6 +344,7 @@ export default class Runner extends EventEmitter {
     result: JourneyContext & JourneyResult,
     options: RunOptions
   ) {
+    const end = monotonicTimeInSeconds();
     const { pluginManager, start, status, error } = result;
     const pluginOutput = pluginManager.output();
     this.emit('journey:end', {
@@ -346,7 +352,8 @@ export default class Runner extends EventEmitter {
       status,
       error,
       start,
-      end: monotonicTimeInSeconds(),
+      end,
+      timestamp: getTimestamp(),
       options,
       ...pluginOutput,
       browserconsole: status == 'failed' ? pluginOutput.browserconsole : [],
@@ -378,6 +385,7 @@ export default class Runner extends EventEmitter {
     };
     this.emit('journey:end', {
       journey,
+      timestamp: getTimestamp(),
       start,
       options,
       end: monotonicTimeInSeconds(),
@@ -393,8 +401,8 @@ export default class Runner extends EventEmitter {
     const result: JourneyResult = {
       status: 'succeeded',
     };
-    log(`Runner: start journey (${journey.name})`);
     const context = await Runner.createContext(options);
+    log(`Runner: start journey (${journey.name})`);
     try {
       this.registerJourney(journey, context);
       const hookArgs = {
diff --git a/src/parse_args.ts b/src/parse_args.ts
index 16893ed..3020947 100644
--- a/src/parse_args.ts
+++ b/src/parse_args.ts
@@ -43,11 +43,6 @@ program
     'JSON object that gets injected to all journeys',
     JSON.parse
   )
-  .option(
-    '-s, --suite-params <jsonstring>',
-    'DEPRECATED: Use --params instead',
-    JSON.parse
-  )
   .addOption(
     new Option('--reporter <value>', `output repoter format`).choices(
       Object.keys(reporters)
diff --git a/src/reporters/json.ts b/src/reporters/json.ts
index 1dfa277..e484499 100644
--- a/src/reporters/json.ts
+++ b/src/reporters/json.ts
@@ -435,6 +435,7 @@ export default class JSONReporter extends BaseReporter {
       'journey:end',
       async ({
         journey,
+        timestamp,
         start,
         end,
         networkinfo,
@@ -501,6 +502,7 @@ export default class JSONReporter extends BaseReporter {
         this.writeJSON({
           type: 'journey/end',
           journey,
+          timestamp,
           error,
           payload: {
             start,


Test patch:
diff --git a/__tests__/cli.test.ts b/__tests__/cli.test.ts
index 5823a2f..65dc646 100644
--- a/__tests__/cli.test.ts
+++ b/__tests__/cli.test.ts
@@ -54,22 +54,6 @@ describe('CLI', () => {
     expect(await cli.exitCode).toBe(0);
   });
 
-  // TODO: Delete this once --suite-params is removed
-  it('runs with legacy suite-params', async () => {
-    const cli = new CLIMock()
-      .stdin(
-        `step('check h2', async () => {
-        await page.goto(params.url, { timeout: 1500 });
-        const sel = await page.waitForSelector('h2.synthetics', { timeout: 1500 });
-        expect(await sel.textContent()).toBe("Synthetics test page");
-      })`
-      )
-      .args(['--inline', '--suite-params', JSON.stringify(serverParams)])
-      .run();
-    await cli.waitFor('Journey: inline');
-    expect(await cli.exitCode).toBe(0);
-  });
-
   it('run suites and exit with 0', async () => {
     const cli = new CLIMock()
       .args([join(FIXTURES_DIR, 'fake.journey.ts')])
diff --git a/__tests__/reporters/base.test.ts b/__tests__/reporters/base.test.ts
index da871d1..dce4eed 100644
--- a/__tests__/reporters/base.test.ts
+++ b/__tests__/reporters/base.test.ts
@@ -102,6 +102,7 @@ describe('base reporter', () => {
     };
     runner.emit('journey:end', {
       journey: j1,
+      timestamp,
       status: 'failed',
       error,
       start: 0,
diff --git a/__tests__/reporters/json.test.ts b/__tests__/reporters/json.test.ts
index 49fd3ca..adade9d 100644
--- a/__tests__/reporters/json.test.ts
+++ b/__tests__/reporters/json.test.ts
@@ -163,6 +163,7 @@ describe('json reporter', () => {
     });
     runner.emit('journey:end', {
       journey: j1,
+      timestamp,
       status: 'succeeded',
       start: 0,
       end: 11,
@@ -225,6 +226,7 @@ describe('json reporter', () => {
 
     runner.emit('journey:end', {
       journey: j1,
+      timestamp,
       start: 0,
       end: 1,
       status: 'failed',
@@ -242,6 +244,7 @@ describe('json reporter', () => {
     const journeyOpts = { name: 'name', id: 'id', tags: ['tag1', 'tag2'] };
     runner.emit('journey:end', {
       journey: journey(journeyOpts, () => {}),
+      timestamp,
       start: 0,
       end: 1,
       status: 'skipped',
@@ -301,6 +304,7 @@ describe('json reporter', () => {
     const emitEnd = (options, status = 'failed' as StatusValue) =>
       runner.emit('journey:end', {
         journey: j1,
+        timestamp,
         start: 0,
         status,
         options,
diff --git a/__tests__/reporters/junit.test.ts b/__tests__/reporters/junit.test.ts
index 375c3a4..8ce7d0a 100644
--- a/__tests__/reporters/junit.test.ts
+++ b/__tests__/reporters/junit.test.ts
@@ -81,6 +81,7 @@ describe('junit reporter', () => {
     });
     runner.emit('journey:end', {
       journey: j1,
+      timestamp,
       start: 0,
       status: 'failed',
       options: {},
@@ -115,6 +116,7 @@ describe('junit reporter', () => {
     });
     runner.emit('journey:end', {
       journey: j1,
+      timestamp,
       start: 0,
       status: 'failed',
       options: {},