Repository: mozilla/nunjucks
Pull request #1058 (2018-01-26 01:04:53)

Add support for ES iterators
## Summary

Proposed change:

This requires the Symbol.iterator well known symbol and Array.from to coerce them into vanilla arrays.

<!--
	Please replace this with a human-friendly description of your proposed change.
	* If you have multiple changes, try to split them into separate PRs.
	* If this change closes an issue, please write "Closes #{number}".
-->

This basically supersedes #760, see there for more info.

## Checklist

I've completed the checklist below to ensure I didn't forget anything. This makes reviewing this PR as easy as possible for the maintainers. And it gets this change released as soon as possible.

* [x] Proposed change helps towards [*purpose of this project*](https://github.com/mozilla/nunjucks/blob/master/CONTRIBUTING.md#purpose).
* [x] [*Documentation*](https://github.com/mozilla/nunjucks/tree/master/docs/) is added / updated to describe proposed change.
* [x] [*Tests*](https://github.com/mozilla/nunjucks/tree/master/tests) are added / updated to cover proposed change.
* [x] [*Changelog*](https://github.com/mozilla/nunjucks/blob/master/CHANGELOG.md) has an entry for proposed change (if user-facing fix or feature).

<!-- Tick of items by replacing `[ ]` by `[x]` -->

fdintino: Just wanted to give you a heads-up that there is a fairly big refactor / cleanup in progress and under discussion at PR #1053. I intend to merge that into master sometime next week probably. Your best bet would be to try to open this as a patch onto the `develop` branch.

I'm not sure about this approach when it comes to generators. Someone using nunjucks that wanted to convert their generators to arrays could use `Array.from` themselves, whereas true support would actually use `Generator.prototype.next()` and only lazily evaluate results during iteration. That being said, I don't see an issue with using `Array.from` for `Map` and `Set`.
maxnordlund: Ok, I've updated this PR to point to develop and your totally right about the generators, so I've removed that from the docs.
codecov-io: # [Codecov](https://codecov.io/gh/mozilla/nunjucks/pull/1058?src=pr&el=h1) Report
> Merging [#1058](https://codecov.io/gh/mozilla/nunjucks/pull/1058?src=pr&el=desc) into [develop](https://codecov.io/gh/mozilla/nunjucks/commit/e6cc590849d433fb8cfd0b70d01a4def513b08c6?src=pr&el=desc) will **increase** coverage by `0.02%`.
> The diff coverage is `100%`.

[![Impacted file tree graph](https://codecov.io/gh/mozilla/nunjucks/pull/1058/graphs/tree.svg?width=650&height=150&token=T3HgOnHq1i&src=pr)](https://codecov.io/gh/mozilla/nunjucks/pull/1058?src=pr&el=tree)

```diff
@@             Coverage Diff             @@
##           develop    #1058      +/-   ##
===========================================
+ Coverage    89.41%   89.44%   +0.02%     
===========================================
  Files           22       22              
  Lines         2966     2974       +8     
===========================================
+ Hits          2652     2660       +8     
  Misses         314      314
```


| [Impacted Files](https://codecov.io/gh/mozilla/nunjucks/pull/1058?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [nunjucks/src/runtime.js](https://codecov.io/gh/mozilla/nunjucks/pull/1058/diff?src=pr&el=tree#diff-bnVuanVja3Mvc3JjL3J1bnRpbWUuanM=) | `90.18% <100%> (+0.44%)` | :arrow_up: |
| [nunjucks/src/compiler.js](https://codecov.io/gh/mozilla/nunjucks/pull/1058/diff?src=pr&el=tree#diff-bnVuanVja3Mvc3JjL2NvbXBpbGVyLmpz) | `95.5% <100%> (ø)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/mozilla/nunjucks/pull/1058?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/mozilla/nunjucks/pull/1058?src=pr&el=footer). Last update [e6cc590...421c8cc](https://codecov.io/gh/mozilla/nunjucks/pull/1058?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


Patch:
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 42096cb..54ada2d 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -10,6 +10,8 @@ master (unreleased)
 
 * Support objects created with Object.create(null). fixes [#468](https://github.com/mozilla/nunjucks/issues/468)
 
+* Support ESNext iterators, using Array.from. Merge of
+  [#760](https://github.com/mozilla/nunjucks/pull/760)
 
 3.0.1 (May 24 2017)
 -------------------
diff --git a/docs/templating.md b/docs/templating.md
index ead8182..326b53e 100644
--- a/docs/templating.md
+++ b/docs/templating.md
@@ -252,6 +252,23 @@ var food = {
 The [`dictsort`](http://jinja.pocoo.org/docs/templates/#dictsort) filter is
 available for sorting objects when iterating over them.
 
+ES iterators are supported, like the new builtin Map and Set. But also
+anything implementing the iterator protocal.
+
+```js
+var fruits = new Map([
+  ["banana", "yellow"],
+  ["apple", "red"],
+  ["peach", "pink"]
+])
+```
+
+```jinja
+{% for fruit, color in fruits %}
+  Did you know that {{ fruit }} is {{ color }}?
+{% endfor %}
+```
+
 Additionally, Nunjucks will unpack arrays into variables:
 
 ```js
diff --git a/nunjucks/src/compiler.js b/nunjucks/src/compiler.js
index e17a678..0ab1729 100644
--- a/nunjucks/src/compiler.js
+++ b/nunjucks/src/compiler.js
@@ -662,6 +662,7 @@ class Compiler extends Obj {
     this.emitLine(';');
 
     this.emit(`if(${arr}) {`);
+    this.emitLine(arr + ' = runtime.fromIterator(' + arr + ');');
 
     // If multiple names are passed, we need to bind them
     // appropriately
@@ -754,9 +755,9 @@ class Compiler extends Obj {
 
     this.emitLine('frame = frame.push();');
 
-    this.emit('var ' + arr + ' = ');
+    this.emit('var ' + arr + ' = runtime.fromIterator(');
     this._compileExpression(node.arr, frame);
-    this.emitLine(';');
+    this.emitLine(');');
 
     if (node.name instanceof nodes.Array) {
       const arrayLen = node.name.children.length;
diff --git a/nunjucks/src/runtime.js b/nunjucks/src/runtime.js
index 272a5a6..b04ef3f 100644
--- a/nunjucks/src/runtime.js
+++ b/nunjucks/src/runtime.js
@@ -1,6 +1,11 @@
 'use strict';
 
 var lib = require('./lib');
+var arrayFrom = Array.from;
+var supportsIterators = (
+  typeof Symbol === 'function' && Symbol.iterator && typeof arrayFrom === 'function'
+);
+
 
 // Frames keep track of scoping both at compile-time and run-time so
 // we know how to access variables. Block tags can introduce special
@@ -342,6 +347,16 @@ function asyncAll(arr, dimen, func, cb) {
   }
 }
 
+function fromIterator(arr) {
+  if (arr == null || lib.isArray(arr)) {
+    return arr;
+  } else if (supportsIterators && Symbol.iterator in arr) {
+    return arrayFrom(arr);
+  } else {
+    return arr;
+  }
+}
+
 module.exports = {
   Frame: Frame,
   makeMacro: makeMacro,
@@ -360,5 +375,6 @@ module.exports = {
   markSafe: markSafe,
   asyncEach: asyncEach,
   asyncAll: asyncAll,
-  inOperator: lib.inOperator
+  inOperator: lib.inOperator,
+  fromIterator: fromIterator
 };


Test patch:
diff --git a/tests/compiler.js b/tests/compiler.js
index 608e028..767db9c 100644
--- a/tests/compiler.js
+++ b/tests/compiler.js
@@ -475,6 +475,42 @@
             },
             'showing test\nshowing 1\nshowing 2\nshowing 3\n');
         });
+        /* global Set */
+        it('should work with Set builtin', function() {
+          if (typeof Set === 'undefined') {
+            this.skip();
+          } else {
+            equal('{% ' + block + ' i in set %}{{ i }}{% ' + end + ' %}',
+              { set: new Set([1, 2, 3, 4, 5]) },
+              '12345');
+
+            equal('{% ' + block + ' i in set %}{{ i }}{% else %}empty{% ' + end + ' %}',
+              { set: new Set([1, 2, 3, 4, 5]) },
+              '12345');
+
+            equal('{% ' + block + ' i in set %}{{ i }}{% else %}empty{% ' + end + ' %}',
+              { set: new Set() },
+              'empty');
+          }
+        });
+        /* global Map */
+        it('should work with Map builtin', function() {
+          if (typeof Map === 'undefined') {
+            this.skip();
+          } else {
+            equal('{% ' + block + ' k, v in map %}[{{ k }},{{ v }}]{% ' + end + ' %}',
+              { map: new Map([[1, 2], [3, 4], [5, 6]]) },
+              '[1,2][3,4][5,6]');
+
+            equal('{% ' + block + ' k, v in map %}[{{ k }},{{ v }}]{% else %}empty{% ' + end + ' %}',
+              { map: new Map([[1, 2], [3, 4], [5, 6]]) },
+              '[1,2][3,4][5,6]');
+
+            equal('{% ' + block + ' k, v in map %}[{{ k }},{{ v }}]{% else %}empty{% ' + end + ' %}',
+              { map: new Map() },
+              'empty');
+          }
+        });
       });
     }