Repository: filerjs/filer
Pull request #762 (2021-02-20 04:37:16)

Using filer with webpack
This pull request includes shims for filers fs and path modules that allow them to be easily used as replacements for the node.js equivalents.

The fs shim ensures that methods on the fs module wait for the filesystem to be ready before being executed so that developers don't have to worry about the intricacies of setting up filer and can use fs much as they would in node.

I've also added shims for the file system providers (default, indexeddb and memory) which essentially re-expose existing code but in a place that makes sense to use as an alias in webpack.

Finally, I've written documentation detailing exactly how to set up your webpack config aliases.

Why? At my company we switched to using filer from BrowserFS in our product. As we have an electron version of our web app (or visa versa?) we want to use the same node.js imports (e.g. import fs from 'fs') for all of our calls to filer when we're running in the web. This is easy for us to do using webpack as we have two different configs for the web and electron. This also gives us typescript on the filer methods! =P

If there's any changes you'd like made or if you have any questions I'd be more than happy to oblige! =)

Love what you're doing here, best fs replacement we've found for the web (and we've been through a few) 👍




Patch:
diff --git a/AUTHORS b/AUTHORS
index e40f820..8d0f59f 100644
--- a/AUTHORS
+++ b/AUTHORS
@@ -5,4 +5,5 @@ Barry Tulchinsky <barry.tulchinsky@gmail.com> (@btulchinsky)
 Kieran Sedgwick <kieran.sedgwick@gmail.com> (@sedge)
 Yoav Gurevich <ygurevich@ymail.com>
 Gideon Thomas <r.gideonthomas@gmail.com>
-Abdirahman Guled <aguled2@myseneca.ca>
\ No newline at end of file
+Abdirahman Guled <aguled2@myseneca.ca>
+Ben Heidemann <ben@heidemann.co.uk>
\ No newline at end of file
diff --git a/README.md b/README.md
index 61e72bc..e1e73d5 100644
--- a/README.md
+++ b/README.md
@@ -54,6 +54,148 @@ requirejs(['filer'], function(Filer) {...}
 var Filer = window.Filer;
 ```
 
+### Webpack
+
+Filer can be used as a drop-in replacement for the node.js [fs module](http://nodejs.org/api/fs.html) using webpack.
+
+In order to use filer in place of fs, insert the following into your webpack config:
+
+```javascript
+// webpack.config.js
+module.exports = {
+  resolve: {
+    alias: {
+      'fs': 'filer/shims/fs.js',
+    }
+  }
+}
+```
+
+You can then import the node.js [fs module](http://nodejs.org/api/fs.html) as normal
+and the shim will ensure that calls to fs are appropriately handled by filer.
+
+```javascript
+import fs from 'fs';
+```
+
+If any calls are made to fs or fs.promises methods before the file system has been
+initialised, the shim will automatically delay the call until the file system is ready.
+
+If you're using filer in a typescript project, the fs shim has the added
+benefit of allowing you to use the types for the node.js [fs module](http://nodejs.org/api/fs.html),
+which filer tries to match as closely as possible. Note that some methods from fs are
+not available, even though typescript will tell you that they are! See [Getting Started](#getting-started)
+for more details on filers limitations.
+
+If you wish to use an alternative file system provider in place of the default (IndexedDB), you must also
+include an alias for this in your webpack config. For example, if you wish to use an "in memory"
+file system, configure webpack as shown below.
+
+```javascript
+// webpack.config.js
+module.exports = {
+  resolve: {
+    alias: {
+      'fsProvider': 'filer/shims/providers/memory.js',
+      'fs': 'filer/shims/fs.js',
+    }
+  }
+}
+```
+
+The current options for file system providers are:
+
+* Default (IndexedDB) - filer/shims/providers/default.js
+* IndexedDB - filer/shims/providers/default.js
+* Memory - filer/shims/providers/memory.js
+
+Though it's technically optional, it is recommended to include an alias for fsProvider in your
+webpack config. This will prevent webpack from logging unnecessary warnings.
+
+If you wish to use your own file system provider with the node.js [fs module](http://nodejs.org/api/fs.html)
+shim, it will be necessary to include an alias for fsProvider which points to your providers implementation.
+This can be done as follows:
+
+```javascript
+// webpack.config.js
+const path = require('path');
+
+module.exports = {
+  resolve: {
+    alias: {
+      'fsProvider': path.resolve(__dirname, 'example/dir/provider.js'),
+      'fs': 'filer/shims/fs.js',
+    }
+  }
+}
+```
+
+The node.js [path module](http://nodejs.org/api/path.html) also has a shim available, which can
+be applied in a similar manner to the node.js [fs module](http://nodejs.org/api/fs.html) shim.
+
+```javascript
+// webpack.config.js
+module.exports = {
+  resolve: {
+    alias: {
+      'path': 'filer/shims/path.js',
+    }
+  }
+}
+```
+
+You can then import the node.js [path module](http://nodejs.org/api/path.html) as normal and the
+shim will ensure that calls to path are appropriately handled by filer.
+
+```javascript
+import path from 'path';
+```
+
+It may be necessary in certain cases to shim the node.js [Buffer object](http://nodejs.org/api/buffer.html). This can be impoerted as follows:
+
+```javascript
+import { Buffer } from 'buffer';
+```
+
+As such it can be shimmed in much the same way as path and fs:
+
+```javascript
+// webpack.config.js
+module.exports = {
+  resolve: {
+    alias: {
+      'buffer': 'filer/shims/buffer.js',
+    }
+  }
+}
+```
+
+However, the Buffer object is globally defined in a node environment and many third party libraries will not import (or require) it.
+Using the resolve alias alone will be ineffective in such cases. Instead we must expand our webpack config to use webpacks
+[provide plugin](https://webpack.js.org/plugins/provide-plugin/), which will automatically load the module without the need for an import
+or require. This can be implemented as follows:
+
+```javascript
+// webpack.config.js
+const webpack = require('webpack');
+
+module.exports = {
+  resolve: {
+    alias: {
+      'buffer': 'filer/shims/buffer.js',
+    }
+  },
+  plugins: [
+    new webpack.ProvidePlugin({
+      Buffer: 'buffer',
+    }),
+  ]
+}
+```
+
+This will, in effect, make the Buffer object shim globally available in the same way that the node.js
+[Buffer object](http://nodejs.org/api/buffer.html) is in a node environment.
+
 ### Getting Started
 
 Filer is as close to the node.js [fs module](http://nodejs.org/api/fs.html) as possible,
diff --git a/karma.conf.js b/karma.conf.js
index 231678d..aa7d677 100644
--- a/karma.conf.js
+++ b/karma.conf.js
@@ -2,10 +2,14 @@ module.exports = function(config) {
   config.set({
     singleRun: true,
     basePath: '',
-    files: ['tests/dist/index.js'],
+    files: [
+      'node_modules/regenerator-runtime/runtime.js',
+      'tests/dist/index.js'
+    ],
     frameworks: ['mocha', 'chai'],
     reporters: ['mocha', 'summary'],
     client: {
+      captureConsole: true,
       mocha: {
         ui: 'bdd',
         timeout: 5000,
diff --git a/package.json b/package.json
index 3b26b68..c2084c4 100644
--- a/package.json
+++ b/package.json
@@ -49,6 +49,7 @@
     "minimatch": "^3.0.4"
   },
   "devDependencies": {
+    "regenerator-runtime": "^0.13.7",
     "chai": "^4.3.0",
     "chai-datetime": "^1.8.0",
     "eslint": "^7.20.0",
diff --git a/shims/buffer.js b/shims/buffer.js
new file mode 100644
index 0000000..8b15727
--- /dev/null
+++ b/shims/buffer.js
@@ -0,0 +1,7 @@
+const { Buffer } = require('../src/index');
+
+module.exports = {
+  __esModule: true,
+  default: Buffer,
+  Buffer,
+};
diff --git a/shims/fs.js b/shims/fs.js
new file mode 100644
index 0000000..d60986f
--- /dev/null
+++ b/shims/fs.js
@@ -0,0 +1,56 @@
+const { FileSystem } = require('../src/index');
+
+let Provider;
+try {
+  Provider = require('fsProvider');
+}
+catch (err) {
+  Provider = require('./providers/default');
+}
+
+const provider = new Provider();
+
+let onFsReady;
+let onFsError;
+
+let fsReady = new Promise((resolve, reject) => {
+  onFsReady = resolve;
+  onFsError = reject;
+});
+
+var fsInstance = new FileSystem({ provider }, (err) => {
+  if (err) {
+    onFsError(err);
+  } else {
+    onFsReady(true);
+  }
+});
+
+const fsPromises = new Proxy(fsInstance.promises, {
+  get(target, prop) {
+    return async (...args) => {
+      await fsReady;
+      return await target[prop](...args);
+    };
+  },
+});
+
+const fs = new Proxy(fsInstance, {
+  get(target, prop) {
+    if (prop === 'promises') {
+      return fsPromises;
+    }
+
+    return (...args) => {
+      (async () => {
+        await fsReady;
+        target[prop](...args);
+      })();
+    };
+  },
+});
+
+module.exports = {
+  __esModule: true,
+  default: fs,
+};
diff --git a/shims/path.js b/shims/path.js
new file mode 100644
index 0000000..46dd292
--- /dev/null
+++ b/shims/path.js
@@ -0,0 +1,6 @@
+const { path } = require('../src/index');
+
+module.exports = {
+  __esModule: true,
+  default: path,
+};
diff --git a/shims/providers/default.js b/shims/providers/default.js
new file mode 100644
index 0000000..2ba8094
--- /dev/null
+++ b/shims/providers/default.js
@@ -0,0 +1,2 @@
+const { Default } = require('../../src/providers/index');
+module.exports = Default;
\ No newline at end of file
diff --git a/shims/providers/indexeddb.js b/shims/providers/indexeddb.js
new file mode 100644
index 0000000..6e01dc1
--- /dev/null
+++ b/shims/providers/indexeddb.js
@@ -0,0 +1,2 @@
+const IndexedDB = require('../../src/providers/indexeddb');
+module.exports = IndexedDB;
\ No newline at end of file
diff --git a/shims/providers/memory.js b/shims/providers/memory.js
new file mode 100644
index 0000000..c33cc34
--- /dev/null
+++ b/shims/providers/memory.js
@@ -0,0 +1,2 @@
+const Memory = require('../../src/providers/memory');
+module.exports = Memory;
\ No newline at end of file


Test patch:
diff --git a/tests/index.js b/tests/index.js
index beb517f..e9245e6 100644
--- a/tests/index.js
+++ b/tests/index.js
@@ -3,6 +3,11 @@
  * get them running by default.
  */
 
+// Shims
+require('./spec/shims/fs.spec');
+require('./spec/shims/path.spec');
+require('./spec/shims/buffer.spec');
+
 // Filer
 require('./spec/filer.spec');
 require('./spec/filer.buffer.spec.js');
diff --git a/tests/lib/test-utils.js b/tests/lib/test-utils.js
index 2c58bb3..fd4461a 100644
--- a/tests/lib/test-utils.js
+++ b/tests/lib/test-utils.js
@@ -52,11 +52,13 @@ function shimIndexedDB(fn) {
     global.indexedDB = require('fake-indexeddb');
   }
 
-  fn();
+  var result = fn();
 
   if(addShim) {
     delete global.indexedDB;
   }
+
+  return result;
 }
 
 function setup(callback) {
diff --git a/tests/spec/shims/buffer.spec.js b/tests/spec/shims/buffer.spec.js
new file mode 100644
index 0000000..888207a
--- /dev/null
+++ b/tests/spec/shims/buffer.spec.js
@@ -0,0 +1,23 @@
+'use strict';
+const expect = require('chai').expect;
+const bufferDefault = require('../../../shims/buffer').default;
+const bufferNamed = require('../../../shims/buffer').Buffer;
+const bufferActual = require('../../../src/index').Buffer;
+
+describe.only('path shim', () => {
+  it('default export should be defined', () => {
+    expect(bufferDefault).to.not.be.undefined;
+  });
+
+  it('named export should be defined', () => {
+    expect(bufferNamed).to.not.be.undefined;
+  });
+
+  it('default export should be re-exposing Buffer', () => {
+    expect(bufferDefault).to.equal(bufferActual);
+  });
+
+  it('named export should be re-exposing Buffer', () => {
+    expect(bufferNamed).to.equal(bufferActual);
+  });
+});
diff --git a/tests/spec/shims/fs.spec.js b/tests/spec/shims/fs.spec.js
new file mode 100644
index 0000000..d8fb5a4
--- /dev/null
+++ b/tests/spec/shims/fs.spec.js
@@ -0,0 +1,44 @@
+'use strict';
+const expect = require('chai').expect;
+const utils = require('../../lib/test-utils');
+const fs = utils.shimIndexedDB(() => require('../../../shims/fs').default);
+
+describe.only('fs shim', () => {
+  it('should be defined', () => {
+    expect(fs).to.not.be.undefined;
+  });
+
+  it('should be an object', () => {
+    expect(typeof fs).to.equal('object');
+  });
+
+  it('should return a function when accessing fs.writeFile', () => {
+    expect(typeof fs.writeFile).to.equal('function');
+  });
+
+  it('should call callback when calling fs.writeFile', (done) => {
+    fs.writeFile('/test.txt', 'test', function(err) {
+      if(err) throw err;
+
+      done();
+    });
+  });
+
+  it('should return an object when accessing fs.promises', () => {
+    expect(typeof fs.promises).to.equal('object');
+  });
+
+  it('should return a function when accessing fs.promises.writeFile', () => {
+    expect(typeof fs.promises.writeFile).to.equal('function');
+  });
+
+  it('should return a promise which resolves when calling fs.promises.writeFile', (done) => {
+    const writeFilePromise = fs.promises.writeFile('/test2.txt', '');
+    expect(writeFilePromise instanceof Promise).to.equal(true);
+    writeFilePromise.then(() => {
+      done();
+    }).catch((err) => {
+      done(err);
+    });
+  });
+});
diff --git a/tests/spec/shims/path.spec.js b/tests/spec/shims/path.spec.js
new file mode 100644
index 0000000..0432aca
--- /dev/null
+++ b/tests/spec/shims/path.spec.js
@@ -0,0 +1,14 @@
+'use strict';
+const expect = require('chai').expect;
+const path = require('../../../shims/path').default;
+const pathActual = require('../../../src/index').path;
+
+describe.only('path shim', () => {
+  it('should be defined', () => {
+    expect(path).to.not.be.undefined;
+  });
+
+  it('should be re-exposing path', () => {
+    expect(path).to.equal(pathActual);
+  });
+});