Repository: serverless-heaven/serverless-webpack (js)

Issue:
ForceExclude
# This is a Feature Proposal

## Description

Under the current method for bundling externals, while you can exclude dependencies by marking them as devDependencies, if a downstream dependency re-includes it, it will be bundled regardless. 

While 99% of the time this makes sense, there are still a lot of projects out there, that are not lambda exclusive, which include `aws-sdk` as a dependency. While I could ask every project maintainer to move aws-sdk to peerDependency or devDependency - I think adding a `forceExclude` would be a simple fix for a wide(ish) problem

i.e.
```
webpackIncludeModules:
    forceInclude:
      - module1
      - module2
    forceExclude:
      - aws-sdk
      - module2
```



Pull request:
Support forceExclude to exclude dependencies
<!--
1. Do not remove any section of the template. If something is not applicable leave it empty but leave it in the PR
2. Please follow the template, otherwise we'll have to ask you to update it and it will take longer until your PR is merged
-->

## What did you implement:

Closes #247 

<!--
Briefly describe the feature if no issue exists for this PR. If possible only
submit PRs for existing issues. If the PR is trivial (like doc changes or simple
code fixes) it can be submitted without a related issue, but as soon as it adds
or changes functionality, a related issue should be present.
-->

## How did you implement it:

<!--
If this is a nontrivial change please briefly describe your implementation so its easy for us to understand and review your code.
-->

The configuration now allows for a `forceExclude` array besides the `forceInclude` array. All modules specified will be forcefully excluded, even if they appear as production dependencies. If the excluded dependency includes a peer dependency, this has to be excluded manually because the peer is still required, even if the excluded dependency is drawn from somewhere else.

Sample:
```yaml
#serverless.yml
custom:
  webpackIncludeModules:
    forceInclude:
      - module1
    forceExclude:
      - module2
```

The precedence is `exclude -> include`, i.e. if a module appears in include as well as in exclude, it will be excluded (exclude wins).

## How can we verify it:

<!--
Add any applicable config, commands, screenshots or other resources
to make it easy for us to verify this works. The easier you make it for us
to review a PR, the faster we can review and merge it.

Examples:
* Step by step description, how to verify
* Screenshots - Showing the difference between your output and the master
* Other - Anything else that comes to mind to help us evaluate
-->

Just use a configuration as in the sample above, do a `serverless package` and check the zip files created in `.serverless`.

## Todos:

- [x] Write tests
- [x] Write documentation
- [x] Fix linting errors
- [x] Make sure code coverage hasn't dropped
- [x] Provide verification config / commands / resources
- [x] Enable "Allow edits from maintainers" for this PR
- [x] Update the messages below

***Is this ready for review?:*** YES
***Is it a breaking change?:*** NO


Patch:
diff --git a/README.md b/README.md
index 24a8ecb..e14673b 100644
--- a/README.md
+++ b/README.md
@@ -192,6 +192,7 @@ custom:
 ```
 > Note that only relative path is supported at the moment.
 
+#### Forced inclusion
 
 Sometimes it might happen that you use dynamic requires in your code, i.e. you
 require modules that are only known at runtime. Webpack is not able to detect
@@ -209,6 +210,26 @@ custom:
       - module2
 ```
 
+#### Forced exclusion
+
+You can forcefully exclude detected external modules, e.g. if you have a module
+in your dependencies that is already installed at your provider's environment.
+
+Just add them to the `forceExclude` array property and they will not be packaged.
+
+```yaml
+# serverless.yml
+custom:
+  webpackIncludeModules:
+    forceExclude:
+      - module1
+      - module2
+```
+
+If you specify a module in both arrays, `forceInclude` and `forceExclude`, the
+exclude wins and the module will not be packaged.
+
+#### Examples
 
 You can find an example setups in the [`examples`][link-examples] folder.
 
diff --git a/lib/packExternalModules.js b/lib/packExternalModules.js
index 5c2e282..acdfaa1 100644
--- a/lib/packExternalModules.js
+++ b/lib/packExternalModules.js
@@ -7,6 +7,48 @@ const childProcess = require('child_process');
 const fse = require('fs-extra');
 const isBuiltinModule = require('is-builtin-module');
 
+/**
+ * Add the given modules to a package json's dependencies.
+ */
+function addModulesToPackageJson(externalModules, packageJson) {
+  _.forEach(externalModules, externalModule => {
+    const splitModule = _.split(externalModule, '@');
+    // If we have a scoped module we have to re-add the @
+    if (_.startsWith(externalModule, '@')) {
+      splitModule.splice(0, 1);
+      splitModule[0] = '@' + splitModule[0];
+    }
+    const moduleVersion = _.join(_.tail(splitModule), '@');
+    packageJson.dependencies = packageJson.dependencies || {};
+    packageJson.dependencies[_.first(splitModule)] = moduleVersion;
+  });
+}
+
+/**
+ * Remove a given list of excluded modules from a module list
+ * @this - The active plugin instance
+ */
+function removeExcludedModules(modules, packageForceExcludes, log) {
+  const excludedModules = _.remove(modules, externalModule => {
+    const splitModule = _.split(externalModule, '@');
+    // If we have a scoped module we have to re-add the @
+    if (_.startsWith(externalModule, '@')) {
+      splitModule.splice(0, 1);
+      splitModule[0] = '@' + splitModule[0];
+    }
+    const moduleName = _.first(splitModule);
+    return _.includes(packageForceExcludes, moduleName);
+  });
+
+  if (log && !_.isEmpty(excludedModules)) {
+    this.serverless.cli.log(`Excluding external modules: ${_.join(excludedModules, ', ')}`);
+  }
+}
+
+/**
+ * Resolve the needed versions of production depenencies for external modules.
+ * @this - The active plugin instance
+ */
 function getProdModules(externalModules, packagePath, dependencyGraph) {
   const packageJsonPath = path.join(process.cwd(), packagePath);
   const packageJson = require(packageJsonPath);
@@ -127,7 +169,9 @@ module.exports = {
       return BbPromise.resolve(stats);
     }
 
+    // Read plugin configuration
     const packageForceIncludes = _.get(includes, 'forceInclude', []);
+    const packageForceExcludes = _.get(includes, 'forceExclude', []);
     const packagePath = includes.packagePath || './package.json';
     const packageJsonPath = path.join(process.cwd(), packagePath);
 
@@ -182,6 +226,7 @@ module.exports = {
         );
         return getProdModules.call(this, externalModules, packagePath, dependencyGraph);
       }));
+      removeExcludedModules.call(this, compositeModules, packageForceExcludes, true);
 
       if (_.isEmpty(compositeModules)) {
         // The compiled code does not reference any external modules at all
@@ -200,17 +245,7 @@ module.exports = {
         description: `Packaged externals for ${this.serverless.service.service}`,
         private: true
       };
-      _.forEach(compositeModules, compositeModule => {
-        const splitModule = _.split(compositeModule, '@');
-        // If we have a scoped module we have to re-add the @
-        if (_.startsWith(compositeModule, '@')) {
-          splitModule.splice(0, 1);
-          splitModule[0] = '@' + splitModule[0];
-        }
-        const moduleVersion = _.join(_.tail(splitModule), '@');
-        compositePackage.dependencies = compositePackage.dependencies || {};
-        compositePackage.dependencies[_.first(splitModule)] = moduleVersion;
-      });
+      addModulesToPackageJson(compositeModules, compositePackage);
       this.serverless.utils.writeFileSync(compositePackageJson, JSON.stringify(compositePackage, null, 2));
 
       // (1.a.2) Copy package-lock.json if it exists, to prevent unwanted upgrades
@@ -250,16 +285,8 @@ module.exports = {
             getExternalModules.call(this, compileStats),
             _.map(packageForceIncludes, whitelistedPackage => ({ external: whitelistedPackage }))
           ), packagePath, dependencyGraph);
-        _.forEach(prodModules, prodModule => {
-          const splitModule = _.split(prodModule, '@');
-          // If we have a scoped module we have to re-add the @
-          if (_.startsWith(prodModule, '@')) {
-            splitModule.splice(0, 1);
-            splitModule[0] = '@' + splitModule[0];
-          }
-          const moduleVersion = _.join(_.tail(splitModule), '@');
-          modulePackage.dependencies[_.first(splitModule)] = moduleVersion;
-        });
+        removeExcludedModules.call(this, prodModules, packageForceExcludes);
+        addModulesToPackageJson(prodModules, modulePackage);
         this.serverless.utils.writeFileSync(modulePackageJson, JSON.stringify(modulePackage, null, 2));
 
         // Copy modules


Test patch:
diff --git a/tests/packExternalModules.test.js b/tests/packExternalModules.test.js
index cfa9307..08dcc76 100644
--- a/tests/packExternalModules.test.js
+++ b/tests/packExternalModules.test.js
@@ -458,6 +458,59 @@ describe('packExternalModules', () => {
       ]));
     });
 
+    it('should exclude external modules when forced', () => {
+      const expectedCompositePackageJSON = {
+        name: 'test-service',
+        version: '1.0.0',
+        description: 'Packaged externals for test-service',
+        private: true,
+        dependencies: {
+          '@scoped/vendor': '1.0.0',
+          bluebird: '^3.4.0',
+          pg: '^4.3.5'
+        }
+      };
+      const expectedPackageJSON = {
+        dependencies: {
+          '@scoped/vendor': '1.0.0',
+          bluebird: '^3.4.0',
+          pg: '^4.3.5'
+        }
+      };
+      serverless.service.custom = {
+        webpackIncludeModules: {
+          forceInclude: ['pg'],
+          forceExclude: ['uuid']
+        }
+      };
+      module.webpackOutputPath = 'outputPath';
+      fsExtraMock.pathExists.yields(null, false);
+      fsExtraMock.copy.yields();
+      childProcessMock.exec.onFirstCall().yields(null, '{}', '');
+      childProcessMock.exec.onSecondCall().yields(null, '', '');
+      childProcessMock.exec.onThirdCall().yields();
+      return expect(module.packExternalModules(stats)).to.be.fulfilled
+      .then(() => BbPromise.all([
+        // The module package JSON and the composite one should have been stored
+        expect(writeFileSyncStub).to.have.been.calledTwice,
+        expect(writeFileSyncStub.firstCall.args[1]).to.equal(JSON.stringify(expectedCompositePackageJSON, null, 2)),
+        expect(writeFileSyncStub.secondCall.args[1]).to.equal(JSON.stringify(expectedPackageJSON, null, 2)),
+        // The modules should have been copied
+        expect(fsExtraMock.copy).to.have.been.calledOnce,
+        // npm ls and npm prune should have been called
+        expect(childProcessMock.exec).to.have.been.calledThrice,
+        expect(childProcessMock.exec.firstCall).to.have.been.calledWith(
+          'npm ls -prod -json -depth=1'
+        ),
+        expect(childProcessMock.exec.secondCall).to.have.been.calledWith(
+          'npm install'
+        ),
+        expect(childProcessMock.exec.thirdCall).to.have.been.calledWith(
+          'npm prune'
+        )
+      ]));
+    });
+
     it('should read package-lock if found', () => {
       const expectedCompositePackageJSON = {
         name: 'test-service',