CI Pipeline hangs when using nx with max-parallel higher than 1
Asked by Johannes Fahrenkrug:

Hi,

I'm using Bitbucket Cloud to run the tests of an nx project.

When \`max-parallel\` is set to a value higher than 1, the pipeline times out. It seems as if Bitbucket doesn't notice that the command has finished. When I set max-parallel to 1, it works.

Here's my yml file:

```
image: node:16

options:

  max-time: 20 # configure default 20 minute timeout

pipelines:

  pull-requests:

    '**':

     - step:

        name: 'Lint, test, and build'

        caches:

         - node

        script:

         - npm ci

         - npx nx workspace-lint

         - npx nx format:check

         - npx nx affected --target=lint --base=origin/master --parallel --max-parallel=3

         - npx nx affected --target=test --base=origin/master --parallel --max-parallel=1 --ci --code-coverage

         - npx nx affected --target=build --base=origin/master --head=HEAD --parallel --max-parallel=3
```

When changing \`max-parallel\` in the second to last line to "3", the pipeline won't finish. When running it locally on my machine, it finishes just fine. Any thoughts?


Answers:
author: Theodora Boudale
body: Hi Johannes,

One possibility is that step may need more than 20 minutes to finish.

When \`max-parallel\` is set to a value higher than 1, is there any indication in the command's output that the command has finished doing what it is doing?

Is there an option to add verbose output to the command (that may show additional output about what is happening)?

Kind regards,  
Theodora


author: Melvin Williams
body: [@Johannes Fahrenkrug](/t5/user/viewprofilepage/user-id/5052886) did you find a solution for the problem. I'm facing the same behaviour.

comments: author: Johannes Fahrenkrug
body: [@Melvin Williams](/t5/user/viewprofilepage/user-id/4952334) The only "solution" we found was to set \`--max-parallel=1\` when running our tests on the BitBucket pipeline.


author: Melvin Williams
body: Thanks for the reply. Which Nx version are you using


author: Johannes Fahrenkrug
body: 16.6.0


author: aalbericio
body: Same problem here with Nx 17.3.0.

I managed to make it work using --max-parallel=1 but the rest of configurations (2, 4, etc) lead to unexpected results: sometimes it works and sometimes it hangs (always on the first package to be tested).

Is there any news on this issue? Does it have something to do with the runners conf?

[@Johannes Fahrenkrug](/t5/user/viewprofilepage/user-id/5052886) Have you found any other "workaround"?

Thanks!


author: Theodora Boudale
body: Hi everyone,

If the command hangs because it needs additional resources (memory or CPU cores), you can look into increasing the step's size:

* <https://support.atlassian.com/bitbucket-cloud/docs/step-options/#Size>

As the doc mentions, (a) this is applicable to steps running on the Bitbucket Cloud infrastructure or with a Linux Docker self-hosted runner, and (b) bigger size steps consume more build minutes.

If this is not a resources-related issue, I suggest creating an issue at nx's issue tracker here:

* <https://github.com/nrwl/nx/issues>

Kind regards,  
Theodora


Tags: bitbucket-cloud

cloud