Repository: CloudyKit/jet
Pull request #189 (2021-02-01 15:31:25)

dump() built-in function (#111)
The following is - perhaps naive? - attempt to satisfy requirement described in issue #187. Let me know what you think - Jet internals are mystery to me, at least for now.

To be honest, I am not sure why comparison against master contains also changes implemented by 4beb91506060bd8900ec0f7d42e401d21300350f and f67928da339229e4ccb9b198a6428b8f077cf1c9 - that is confusing.

Regards, 

Jan

jan-herout: Hmm, not sure why this fails, it has nothing to do with the change.

```
FAIL
FAIL	github.com/CloudyKit/jet/v6	0.072s
?   	github.com/CloudyKit/jet/v6/examples/asset_packaging	[no test files]
?   	github.com/CloudyKit/jet/v6/examples/asset_packaging/assets/templates	[no test files]
?   	github.com/CloudyKit/jet/v6/examples/todos	[no test files]
?   	github.com/CloudyKit/jet/v6/jettest	[no test files]
```
jan-herout: I have read the ticket second time, and my attempt is something different then what was asked for. Do not waste your time, I might put it here in the end. I will sleep on it.

https://gitlab.com/Herout/jetfunc


jan-herout: OK, I had to try again. Please let me know, what do you think - could this be the solution to #187? Thanks!

Regards,

Jan.
sauerbraten: The changes look to me like you addressed #111, not #187?

> To be honest, I am not sure why comparison against master contains also changes implemented by 4beb915 and f67928d - that is confusing.

I assume you didn't pull the newest master of CloudyKit/jet into your fork before starting with the new PR? The hashes are different from your original commits since in this repository, we use a "rebase, then merge" strategy that avoids merge commits. A side effect is that the commits are rewritten with the merging person added to them as commiter (in addition to you as the commit author). See here for example: https://github.com/CloudyKit/jet/commit/f67928da339229e4ccb9b198a6428b8f077cf1c9. This means your fork has a commit with the same changes as f67928d, but it has a different hash and so Git doesn't know the code is the same in your fork and this repo, so the changes are shown in the diff.

Assuming you really did mean to address #111, I added a comment there that is helpful feedback to this PR at the same time: https://github.com/CloudyKit/jet/issues/111#issuecomment-771609451

I don't expect all of that to be done in this PR, but I'd like whatever changes you are able to make to go in that direction, maybe start with just a simple dumpVariables function without any parent scope walking here and we can expand to all the other stuff I laid out in the comment later.
sauerbraten: As for the failing tests, it seems you created devdump.jet on Windows with \r\n line endings, when Jet only produces \n line endings, so the expected output doesn't match what dump prints: https://ci.appveyor.com/project/CloudyKit/jet/builds/37566590/job/jo1ne5y95yyqs0jd#L880
jan-herout: Thanks for both comments. When possible, I will look at that. And indeed, this should be related to #111 - my bad.
And your Windows comment is spot on.
jan-herout: All right. I would appreciate your feedback.  Let's assume that this is the alfa version. Also, documenation is not entirely up to-date, and newly added functions under `dump.go` probably deserve a few well placed comments.

I did not manage to commit the test case template ins uch a manner, that EOL convention would be Unix - sorry for that - however, the test case accounts for that, and runs both on Windows and Linux correctly.

Functions used in the code are "not panicky", I did not like the idea of that.

Regards,

Jan
jan-herout: Thanks for the review, will look at these changes most likely yesterday.  This is becoming a valuable experience.
jan-herout: Let me know should you find anything else.
Regards,

Jan

Patch:
diff --git a/default.go b/default.go
index 815de14..e710fdb 100644
--- a/default.go
+++ b/default.go
@@ -151,6 +151,37 @@ func init() {
 			}
 			return reflect.ValueOf(&intsRanger{from: from, to: to})
 		})),
+		"dump": reflect.ValueOf(Func(func(a Arguments) (result reflect.Value) {
+			a.RequireNumOfArguments("dump", 0, -1)
+			noOfArgs := a.NumOfArguments()
+			// no arguments were provided, dump all; do not recurse over parents
+			if noOfArgs == 0 {
+				return dumpAll(a, 0)
+			}
+			//// some arguments were provided, grab them
+			args := make([]reflect.Value, noOfArgs)
+			ids := make([]string, noOfArgs)
+			for i := 0; i < a.NumOfArguments(); i++ {
+				args[i] = a.Get(i)
+			}
+
+			// did we use a dump(int) in our template?
+			if len(args) == 1 && args[0].Kind() == reflect.Float64 {
+				return dumpAll(a, int(args[0].Float()))
+			}
+
+			// at this moment, all arguments should be strings, otherwise we have a problem
+			for _, arg := range args {
+				if arg.Kind() != reflect.String {
+					// TODO: maybe this should panic?
+					return reflect.ValueOf("dump: err: you sent more then one argument, and some of them aren't strings, do not know what to do.")
+				}
+				ids = append(ids, arg.String())
+			}
+
+			return dumpIdentified(a.runtime, ids)
+
+		})),
 	}
 }
 
diff --git a/docs/builtins.md b/docs/builtins.md
index 35e53e2..87fc61d 100644
--- a/docs/builtins.md
+++ b/docs/builtins.md
@@ -1,17 +1,19 @@
 # Built-ins
 
-- [Functions](#functions)
-  - [From Go](#from-go)
-  - [len](#len)
-  - [isset](#isset)
-  - [exec](#exec)
-  - [ints](#ints)
-- [SafeWriter](#safewriter)
-  - [safeHtml](#safehtml)
-  - [safeJs](#safejs)
-  - [raw/unsafe](#rawunsafe)
-- [Renderer](#renderer)
-    - [writeJson](#writejson)
+- [Built-ins](#built-ins)
+  - [Functions](#functions)
+    - [From Go](#from-go)
+    - [len](#len)
+    - [isset](#isset)
+    - [exec](#exec)
+    - [ints](#ints)
+    - [dump](#dump)
+  - [SafeWriter](#safewriter)
+    - [safeHtml](#safehtml)
+    - [safeJs](#safejs)
+    - [raw/unsafe](#rawunsafe)
+  - [Renderer](#renderer)
+      - [writeJson](#writejson)
 
 ## Functions
 
@@ -51,6 +53,18 @@ It panics if you pass a value of any type other than string, array, slice, map,
 
 `ints()` takes two integers as lower and upper limit and returns a Ranger producing all the integers between them, including the lower and excluding the upper limit. It panics when the arguments can't be converted to integers or when the upper limit is not strictly greater than the lower limit.
 
+### dump
+
+`dump` is meant to support template development, and can be used to print out variables, blocks, context, and globals that are available to the template.
+The function can be used in three forms:
+
+`dump()` used without parameters will printout context, variables, globals, and blocks (in this order) in current scope, without accessing any parent.
+
+`dump(levels)` - where `levels` is an **integer** - is the same as `dump()`, and additionally will recurse over context parents to the maximum of `levels`.
+For example, `dump(1)` will additionaly print out all variables accessible to the parent of current context.
+
+`dump("name1","name2", ....)` will try to find the variable or block with the given name(s) in the current runtime.
+
 ## SafeWriter
 
 Jet includes a [`SafeWriter`](https://pkg.go.dev/github.com/CloudyKit/jet/v5?tab=doc#SafeWriter) function type for writing directly to the render output stream. This can be used to circumvent Jet's default HTML escaping. Jet has a few such functions built-in.
@@ -73,4 +87,4 @@ Jet exports a [`Renderer`](https://pkg.go.dev/github.com/CloudyKit/jet/v5?tab=do
 
 #### writeJson
 
-`writeJson` renders the JSON encoding of whatever you pass in to the output, escaping only "<", ">", and "&" (just like the `json` function).
\ No newline at end of file
+`writeJson` renders the JSON encoding of whatever you pass in to the output, escaping only "<", ">", and "&" (just like the `json` function).
diff --git a/dump.go b/dump.go
new file mode 100644
index 0000000..559c0b3
--- /dev/null
+++ b/dump.go
@@ -0,0 +1,108 @@
+package jet
+
+import (
+	"bytes"
+	"fmt"
+	"io"
+	"reflect"
+)
+
+// dumpAll returns
+//  - everything in Runtime.context
+//  - everything in Runtime.variables
+//  - everything in Runtime.set.globals
+//  - everything in Runtime.blocks
+func dumpAll(a Arguments, depth int) reflect.Value {
+	var b bytes.Buffer
+	var vars VarMap
+
+	ctx := a.runtime.context
+	fmt.Fprintln(&b, "Context:")
+	fmt.Fprintf(&b, "\t%s %#v\n", ctx.Type(), ctx)
+
+	dumpScopeVars(&b, a.runtime.scope, 0)
+	dumpScopeVarsToDepth(&b, a.runtime.parent, depth)
+
+	vars = a.runtime.set.globals
+	for i, name := range vars.SortedKeys() {
+		if i == 0 {
+			fmt.Fprintln(&b, "Globals:")
+		}
+		val := vars[name]
+		fmt.Fprintf(&b, "\t%s:=%#v // %s\n", name, val, val.Type())
+	}
+
+	blockKeys := a.runtime.scope.sortedBlocks()
+	fmt.Fprintln(&b, "Blocks:")
+	for _, k := range blockKeys {
+		block := a.runtime.blocks[k]
+		dumpBlock(&b, block)
+	}
+
+	return reflect.ValueOf(b.String())
+}
+
+// dumpScopeVarsToDepth prints all variables in the scope, and all parent scopes,
+// to the limit of maxDepth.
+func dumpScopeVarsToDepth(w io.Writer, scope *scope, maxDepth int) {
+	for i := 1; i <= maxDepth; i++ {
+		if scope == nil {
+			break // do not panic if something bad happens
+		}
+		dumpScopeVars(w, scope, i)
+		scope = scope.parent
+	}
+}
+
+// dumpScopeVars prints all variables in the scope.
+func dumpScopeVars(w io.Writer, scope *scope, lvl int) {
+	if scope == nil {
+		return // do not panic if something bad happens
+	}
+	if lvl == 0 {
+		fmt.Fprint(w, "Variables in current scope:\n")
+	} else {
+		fmt.Fprintf(w, "Variables in scope %d level(s) up:\n", lvl)
+	}
+	vars := scope.variables
+	for _, k := range vars.SortedKeys() {
+		fmt.Fprintf(w, "\t%s=%#v\n", k, vars[k])
+	}
+}
+
+// dumpIdentified accepts a runtime and slice of names.
+// Then, it prints all variables and blocks in the runtime, with names equal to one of the names
+// in the slice.
+func dumpIdentified(rnt *Runtime, ids []string) reflect.Value {
+	var b bytes.Buffer
+	for _, id := range ids {
+		dumpFindVar(&b, rnt, id)
+		dumpFindBlock(&b, rnt, id)
+
+	}
+	return reflect.ValueOf(b.String())
+}
+
+// dumpFindBlock finds the block by name, prints the header of the block, and name of the template in which it was defined.
+func dumpFindBlock(w io.Writer, rnt *Runtime, name string) {
+	if block, ok := rnt.scope.blocks[name]; ok {
+		dumpBlock(w, block)
+	}
+}
+
+// dumpBlock prints header of the block, and template in which the block was first defined.
+func dumpBlock(w io.Writer, block *BlockNode) {
+	if block == nil {
+		return
+	}
+	fmt.Fprintf(w, "\tblock %s(%s), from %s\n", block.Name, block.Parameters.String(), block.TemplatePath)
+}
+
+// dumpFindBlock finds the variable by name, and prints the variable, if it is in the runtime.
+func dumpFindVar(w io.Writer, rnt *Runtime, name string) {
+	val, err := rnt.resolve(name)
+	if err != nil {
+		return
+	}
+	fmt.Fprintf(w, "\t%s:=%#v // %s\n", name, val, val.Type())
+}
diff --git a/eval.go b/eval.go
index 0bf85e4..6528c9c 100644
--- a/eval.go
+++ b/eval.go
@@ -21,6 +21,7 @@ import (
 	"io"
 	"reflect"
 	"runtime"
+	"sort"
 	"strconv"
 	"strings"
 	"sync"
@@ -98,6 +99,15 @@ type scope struct {
 	blocks    map[string]*BlockNode
 }
 
+func (s scope) sortedBlocks() []string {
+	r := make([]string, 0, len(s.blocks))
+	for k := range s.blocks {
+		r = append(r, k)
+	}
+	sort.Strings(r)
+	return r
+}
+
 // YieldBlock yields a block in the current context, will panic if the context is not available
 func (st *Runtime) YieldBlock(name string, context interface{}) {
 	block, has := st.getBlock(name)
diff --git a/exec.go b/exec.go
index e79c98b..8ea74e8 100644
--- a/exec.go
+++ b/exec.go
@@ -21,10 +21,21 @@ package jet
 import (
 	"io"
 	"reflect"
+	"sort"
 )
 
 type VarMap map[string]reflect.Value
 
+// SortedKeys returns a sorted slice of VarMap keys
+func (scope VarMap) SortedKeys() []string {
+	keys := make([]string, 0, len(scope))
+	for k := range scope {
+		keys = append(keys, k)
+	}
+	sort.Strings(keys)
+	return keys
+}
+
 func (scope VarMap) Set(name string, v interface{}) VarMap {
 	scope[name] = reflect.ValueOf(v)
 	return scope


Test patch:
diff --git a/dump_test.go b/dump_test.go
new file mode 100644
index 0000000..c219169
--- /dev/null
+++ b/dump_test.go
@@ -0,0 +1,71 @@
+// Copyright 2016 José Santos <henrique_1609@me.com>
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package jet
+
+import (
+	"bytes"
+	"reflect"
+	"strings"
+	"testing"
+)
+
+func TestDump(t *testing.T) {
+	var b bytes.Buffer                                // writer for the template
+	tmplt, err := parseSet.GetTemplate("devdump.jet") // the testing template containing dump function
+	if err != nil {
+		t.Log(err)
+		t.FailNow()
+	}
+
+	// execute template with dummy inputs
+	// MAP
+	vars := make(VarMap)
+	aMap := make(map[string]interface{})
+	aMap["aMap-10"] = 10 // only one member, because map is unsorted; test could fail for no apparent reason.
+	vars.Set("inputMap", aMap)
+	// SLICE
+	aSlice := []string{"sliceMember1", "sliceMember2"}
+	vars.Set("aSlice", aSlice)
+
+	// prepare dummy context
+	ctx := struct {
+		Name    string
+		Surname string
+	}{Name: "John", Surname: "Doe"}
+
+	// execute template
+	err = tmplt.Execute(&b, vars, ctx)
+	if err != nil {
+		t.Log(err)
+		t.FailNow()
+	}
+
+	// normalize EOL convention and
+	// split outcome to two parts; this is necessary, because the original code
+	// was developed on windows (SORRY !!!!)
+	aux := strings.ReplaceAll(b.String(), "\r\n", "\n")
+	rslt := strings.Split(aux, "===\n")
+	if len(rslt) != 2 {
+		t.Log("expected to get two parts, did you include separator in the template?")
+		t.FailNow()
+	}
+	//t.Log(rslt[0])
+	// compare what we got with what we wanted
+	got := strings.Split(rslt[0], "\n")
+	want := strings.Split(rslt[1], "\n")
+	if !reflect.DeepEqual(got, want) {
+		t.Errorf("\ngot :%q\nwant:%q\nAS TEXT\ngot\n%swant\n%s", got, want, rslt[0], rslt[1])
+	}
+}
diff --git a/testData/devdump.jet b/testData/devdump.jet
new file mode 100644
index 0000000..a177e7c
--- /dev/null
+++ b/testData/devdump.jet
@@ -0,0 +1,65 @@
+1{{- x1:="a string" }}
+2{{- x2 := 1 }}
+3{{- b1 := true }}
+4{{- b2 := false }}
+5{{- s1 := slice("foo", "bar", "baz", "duq")}}
+6{{- m := map("foo", 123) }}
+7{{- mainMenu := "a variable, not a block!" }}
+8{{ block mainMenu(type="text", label="main") }}inside a block{{ end }}
+------------------------------------- dump without parameters
+{{ dump() }}
+------------------------------------- dump with depth of 2
+{{ dump(2) }}
+------------------------------------- dump with erroneous use
+{{ dump(1,"m") }}
+------------------------------------- dump named
+{{ dump("mainMenu", "m") }}
+done
+===
+1
+2
+3
+4
+5
+6
+7
+8inside a block
+------------------------------------- dump without parameters
+Context:
+	struct { Name string; Surname string } struct { Name string; Surname string }{Name:"John", Surname:"Doe"}
+Variables in current scope:
+	b1=true
+	b2=false
+	m=map[string]interface {}{"foo":123}
+	mainMenu="a variable, not a block!"
+	s1=[]interface {}{"foo", "bar", "baz", "duq"}
+	x1="a string"
+	x2=1
+Blocks:
+	block mainMenu(type="text",label="main"), from /devdump.jet
+
+------------------------------------- dump with depth of 2
+Context:
+	struct { Name string; Surname string } struct { Name string; Surname string }{Name:"John", Surname:"Doe"}
+Variables in current scope:
+	b1=true
+	b2=false
+	m=map[string]interface {}{"foo":123}
+	mainMenu="a variable, not a block!"
+	s1=[]interface {}{"foo", "bar", "baz", "duq"}
+	x1="a string"
+	x2=1
+Variables in scope 1 level(s) up:
+	aSlice=[]string{"sliceMember1", "sliceMember2"}
+	inputMap=map[string]interface {}{"aMap-10":10}
+Blocks:
+	block mainMenu(type="text",label="main"), from /devdump.jet
+
+------------------------------------- dump with erroneous use
+dump: err: you sent more then one argument, and some of them aren't strings, do not know what to do.
+------------------------------------- dump named
+	mainMenu:="a variable, not a block!" // string
+	block mainMenu(type="text",label="main"), from /devdump.jet
+	m:=map[string]interface {}{"foo":123} // map[string]interface {}
+
+done