Repository: CloudyKit/jet (go)

Issue:
Performance: Steep decline between v2 and v5
I've been doing several proof of concepts and the performance of v5's loading of templates is significantly less than of the v2 branch:
```
BenchmarkJets
BenchmarkJets/Jetv2
BenchmarkJets/Jetv2-8         	   48260	     24372 ns/op	    5040 B/op	      87 allocs/op
BenchmarkJets/Jetv5
BenchmarkJets/Jetv5-8         	    3828	    267862 ns/op	   36687 B/op	     456 allocs/op
```

## The setup
[Versions](https://github.com/CloudyKit/jet/releases):
- github.com/CloudyKit/jet v2.1.2+incompatible
- github.com/CloudyKit/jet/v5 v5.1.1

The implementation and templates are identical, with the exception of some differences required for the different Jet versions:
```sh
$ find . -type f -name "*.jet" -print -exec diff -Naruw ../jetv2/{} ../jetv5/{} \;
```
```diff
./views/leaderboard.jet
--- ../jetv2/./views/leaderboard.jet	2020-10-20 10:18:04.000000000 +0200
+++ ../jetv5/./views/leaderboard.jet	2020-10-20 12:55:05.000000000 +0200
@@ -1,8 +1,8 @@
-{{import "partials/leaderboard_row.jet"}}
+{{import "/partials/leaderboard_row.jet"}}

 <table>
     <!-- Control flow is a must. This includes iterating over arrays, if/elseif/else statements and ternaries. -->
-    {{range competitor := .Leaderboard}}
+    {{range _, competitor := .Leaderboard}}
     <!-- comments do not appear in the rendered HTML -->
         {* `$competitor` is of type `User` *}
         {{yield leaderboard_row(user=competitor)}}
./views/index.jet
--- ../jetv2/./views/index.jet	2020-10-20 10:18:04.000000000 +0200
+++ ../jetv5/./views/index.jet	2020-10-20 12:45:10.000000000 +0200
@@ -1,6 +1,6 @@
-{{extends "layouts/main.jet"}}
+{{extends "/layouts/main.jet"}}

-{{import "partials/ui.jet"}}
+{{import "/partials/ui.jet"}}

 {{block body()}}
     <script type="text/javascript">
./views/layouts/main.jet
./views/layouts/header.jet
--- ../jetv2/./views/layouts/header.jet	2020-10-26 09:09:56.000000000 +0100
+++ ../jetv5/./views/layouts/header.jet	2020-10-20 12:45:03.000000000 +0200
@@ -1,4 +1,4 @@
-{{import "partials/ui.jet"}}
+{{import "/partials/ui.jet"}}

 <div id="header">
     <div id="menu">
./views/layouts/footer.jet
./views/partials/ui.jet
./views/partials/leaderboard_row.jet
```
The files:
```
.
├── jet.go
└── views
    ├── index.jet
    ├── layouts
    │   ├── footer.jet
    │   ├── header.jet
    │   └── main.jet
    ├── leaderboard.jet
    └── partials
        ├── leaderboard_row.jet
        └── ui.jet
```

## The benchmark
```go
import (
	jetversion2 "github.com/CloudyKit/jet"
	jetversion5 "github.com/CloudyKit/jet/v5"
	// ..
)

func BenchmarkJets(b *testing.B) {
	if wd, err := os.Getwd(); err == nil && filepath.Base(wd) == "template" {
		_ = os.Chdir("..")
	}

	tr := reflect.ValueOf(template.TranslateFunc("en"))
	data := template.ExampleContext()

	b.Run("Jetv2", func(b *testing.B) {

		views := jetversion2.NewHTMLSet("./template/jetv2/views")
		views.SetDevelopmentMode(false)

		for i := 0; i < b.N; i++ {
			t, err := views.GetTemplate("index.jet")
			if err != nil {
				b.Errorf("Error in benchmark %s", err)
			}

			err = t.Execute(ioutil.Discard, jetversion2.VarMap{"T": tr}, data)
			if err != nil {
				b.Errorf("Error in benchmark %s", err)
			}
		}
	})

	b.Run("Jetv5", func(b *testing.B) {

		views := jetversion5.NewHTMLSet("./template/jetv5/views")
		views.SetDevelopmentMode(false)
		views.SetExtensions([]string{""}) // v5 specific optimisation

		for i := 0; i < b.N; i++ {
			t, err := views.GetTemplate("index.jet")
			if err != nil {
				b.Errorf("Error in benchmark %s", err)
			}

			err = t.Execute(ioutil.Discard, jetversion5.VarMap{"T": tr}, data)
			if err != nil {
				b.Errorf("Error in benchmark %s", err)
			}
		}
	})
}
```

## Considerations 
I noticed quite a bit of time spent in io (`Loader.Exists`). I wrote a quick-n-dirty Loader that pre-fetches all *.jet files and adds them to a map for faster subsequent lookups. This reduces the execution time to 17 ms (down from 267ms). But I think it only masks the problem, instead of properly solving it.

I noticed many invocations to `Loader.Exists` and `Loader.Load()` for already observed files. My *guess* so far is that some state should be preserved in `func (st *Runtime) executeList` when `st.executeInclude()` is invoked. Or that the way file paths are canonicalised should be more in line with how templates can be used in version 5. e.g.:
- `{{ include "./index.jet" }}`
- `{{ include "/index.jet" }}`
- `{{ include "index.jet" }}`

might all point to the same file.

![Screenshot 2020-10-26 at 16 38 55](https://user-images.githubusercontent.com/39014/97194084-06bf1f80-17aa-11eb-88d9-baca94b026b2.png)
![Screenshot 2020-10-26 at 16 40 21](https://user-images.githubusercontent.com/39014/97194095-09ba1000-17aa-11eb-853b-53150012c63d.png)


## Further findings
After I've plugged a different Loader I managed to get it down to the following. However I think this isn't the approach to take. I think the lazy-loading approach is probably a fine route to take, but right now it appears to not resolving efficiently.
```
BenchmarkJets
BenchmarkJets/Jetv2
BenchmarkJets/Jetv2-8         	   49384	     23685 ns/op	    5041 B/op	      87 allocs/op
BenchmarkJets/Jetv5
BenchmarkJets/Jetv5-8         	   69686	     17440 ns/op	    5091 B/op	      72 allocs/op
```


Pull request:
cache more aggressively; remove Set.LoadTemplate()
~~Keeping track of wether or not to cache templates that are imported/extended during Parse() is too much work, so Parse() is gone for now.~~ LoadTemplate() is removed in favor of a new in-memory Loader where you can add templates on-the-fly (which is also used for tests without template files). Using it together with a file loader via a multi loader restores the previous functionality.

I believe this fixes #180.

Patch:
diff --git a/eval.go b/eval.go
index a163ad3..b13f0e5 100644
--- a/eval.go
+++ b/eval.go
@@ -606,7 +606,7 @@ func (st *Runtime) executeInclude(node *IncludeNode) (returnValue reflect.Value)
 	}
 
 	st.set.tmx.RLock()
-	t, err := st.set.getSiblingTemplate(templatePath, node.TemplatePath)
+	t, err := st.set.getSiblingTemplate(templatePath, node.TemplatePath, true)
 	st.set.tmx.RUnlock()
 	if err != nil {
 		node.error(err)
diff --git a/loader.go b/loader.go
index cfb2aa4..aee4321 100644
--- a/loader.go
+++ b/loader.go
@@ -15,17 +15,21 @@
 package jet
 
 import (
+	"bytes"
+	"fmt"
 	"io"
+	"io/ioutil"
 	"os"
+	"path"
 	"path/filepath"
 )
 
 // Loader is a minimal interface required for loading templates.
 type Loader interface {
 	// Exists checks for template existence.
-	Exists(path string) (string, bool)
+	Exists(templatePath string) (string, bool)
 	// Open opens the underlying reader with template content.
-	Open(path string) (io.ReadCloser, error)
+	Open(templatePath string) (io.ReadCloser, error)
 }
 
 // OSFileSystemLoader implements Loader interface using OS file system (os.File).
@@ -39,22 +43,58 @@ var _ Loader = (*OSFileSystemLoader)(nil)
 // NewOSFileSystemLoader returns an initialized OSFileSystemLoader.
 func NewOSFileSystemLoader(dirPath string) *OSFileSystemLoader {
 	return &OSFileSystemLoader{
-		dir: dirPath,
+		dir: filepath.FromSlash(dirPath),
 	}
 }
 
 // Open opens a file from OS file system.
-func (l *OSFileSystemLoader) Open(path string) (io.ReadCloser, error) {
-	return os.Open(filepath.Join(l.dir, path))
+func (l *OSFileSystemLoader) Open(templatePath string) (io.ReadCloser, error) {
+	return os.Open(filepath.Join(l.dir, filepath.FromSlash(templatePath)))
 }
 
 // Exists checks if the template name exists by walking the list of template paths
 // returns true if the template file was found
-func (l *OSFileSystemLoader) Exists(path string) (string, bool) {
-	path = filepath.Join(l.dir, path)
-	stat, err := os.Stat(path)
+func (l *OSFileSystemLoader) Exists(templatePath string) (string, bool) {
+	templatePath = filepath.Join(l.dir, filepath.FromSlash(templatePath))
+	stat, err := os.Stat(templatePath)
 	if err == nil && !stat.IsDir() {
-		return path, true
+		return templatePath, true
 	}
 	return "", false
 }
+
+type InMemLoader struct {
+	files map[string][]byte
+}
+
+// compile time check that we implement Loader
+var _ Loader = (*InMemLoader)(nil)
+
+func NewInMemLoader() *InMemLoader {
+	return &InMemLoader{
+		files: map[string][]byte{},
+	}
+}
+
+func (l *InMemLoader) Open(templatePath string) (io.ReadCloser, error) {
+	f, ok := l.files[templatePath]
+	if !ok {
+		return nil, fmt.Errorf("%s does not exist", templatePath)
+	}
+
+	return ioutil.NopCloser(bytes.NewReader(f)), nil
+}
+
+func (l *InMemLoader) Exists(templatePath string) (string, bool) {
+	_, ok := l.files[templatePath]
+	if !ok {
+		return "", false
+	}
+	return templatePath, true
+}
+
+func (l *InMemLoader) Set(templatePath, contents string) {
+	templatePath = filepath.ToSlash(templatePath)
+	templatePath = path.Join("/", templatePath)
+	l.files[templatePath] = []byte(contents)
+}
diff --git a/node.go b/node.go
index d1c90f1..819dd37 100644
--- a/node.go
+++ b/node.go
@@ -17,6 +17,7 @@ package jet
 import (
 	"bytes"
 	"fmt"
+	"path/filepath"
 )
 
 var textFormat = "%s" //Changed to "%q" in tests for better error messages.
@@ -61,7 +62,7 @@ func (node *NodeBase) error(err error) {
 }
 
 func (node *NodeBase) errorf(format string, v ...interface{}) {
-	panic(fmt.Errorf("Jet Runtime Error (%q:%d): %s", node.TemplatePath, node.Line, fmt.Sprintf(format, v...)))
+	panic(fmt.Errorf("Jet Runtime Error (%q:%d): %s", filepath.ToSlash(node.TemplatePath), node.Line, fmt.Sprintf(format, v...)))
 }
 
 // Type returns itself and provides an easy default implementation
diff --git a/parse.go b/parse.go
index 572b315..cec5d1b 100644
--- a/parse.go
+++ b/parse.go
@@ -174,7 +174,7 @@ func (t *Template) recover(errp *error) {
 	return
 }
 
-func (s *Set) parse(name, text string) (t *Template, err error) {
+func (s *Set) parse(name, text string, cacheAfterParsing bool) (t *Template, err error) {
 	t = &Template{
 		Name:         name,
 		ParseName:    name,
@@ -188,7 +188,7 @@ func (s *Set) parse(name, text string) (t *Template, err error) {
 	lexer.setDelimiters(s.leftDelim, s.rightDelim)
 	lexer.run()
 	t.startParse(lexer)
-	t.parseTemplate()
+	t.parseTemplate(cacheAfterParsing)
 	t.stopParse()
 
 	if t.extends != nil {
@@ -215,7 +215,7 @@ func (t *Template) expectString(context string) string {
 
 // parse is the top-level parser for a template, essentially the same
 // It runs to EOF.
-func (t *Template) parseTemplate() (next Node) {
+func (t *Template) parseTemplate(cacheAfterParsing bool) (next Node) {
 	t.Root = t.newList(t.peek().pos)
 	// {{ extends|import stringLiteral }}
 	for t.peek().typ != itemEOF {
@@ -234,12 +234,12 @@ func (t *Template) parseTemplate() (next Node) {
 						t.errorf("Unexpected extends clause: the 'extends' clause should come before all import clauses")
 					}
 					var err error
-					t.extends, err = t.set.getSiblingTemplate(s, t.Name)
+					t.extends, err = t.set.getSiblingTemplate(s, t.Name, cacheAfterParsing)
 					if err != nil {
 						t.error(err)
 					}
 				} else {
-					tt, err := t.set.getSiblingTemplate(s, t.Name)
+					tt, err := t.set.getSiblingTemplate(s, t.Name, cacheAfterParsing)
 					if err != nil {
 						t.error(err)
 					}
diff --git a/template.go b/template.go
index 404f2b3..62d4e0d 100644
--- a/template.go
+++ b/template.go
@@ -19,6 +19,7 @@
 package jet
 
 import (
+	"errors"
 	"fmt"
 	"io"
 	"io/ioutil"
@@ -112,61 +113,34 @@ func (s *Set) Delims(left, right string) {
 	s.rightDelim = right
 }
 
-func (s *Set) getTemplateFromCache(templatePath string) (t *Template, ok bool) {
-	// check path with all possible extensions in cache
-	for _, extension := range s.extensions {
-		canonicalPath := templatePath + extension
-		if t, found := s.templates[canonicalPath]; found {
-			return t, true
-		}
-	}
-	return nil, false
-}
-
-func (s *Set) getTemplateFromLoader(templatePath string) (t *Template, err error) {
-	// check path with all possible extensions in loader
-	for _, extension := range s.extensions {
-		canonicalPath := templatePath + extension
-		if _, found := s.loader.Exists(canonicalPath); found {
-			return s.loadFromFile(canonicalPath)
-		}
-	}
-	return nil, fmt.Errorf("template %s could not be found", templatePath)
-}
-
-// GetTemplate tries to find (and load, if not yet loaded) the template at the specified path.
+// GetTemplate tries to find (and parse, if not yet parsed) the template at the specified path.
 //
-// for example, GetTemplate("catalog/products.list") with extensions set to []string{"", ".html.jet",".jet"}
+// For example, GetTemplate("catalog/products.list") with extensions set to []string{"", ".html.jet",".jet"}
 // will try to look for:
 //     1. catalog/products.list
 //     2. catalog/products.list.html.jet
 //     3. catalog/products.list.jet
 // in the set's templates cache, and if it can't find the template it will try to load the same paths via
-// the loader, and, if parsed successfully, cache the template.
+// the loader, and, if parsed successfully, cache the template (unless running in development mode).
 func (s *Set) GetTemplate(templatePath string) (t *Template, err error) {
-	if !s.developmentMode {
-		s.tmx.RLock()
-		t, found := s.getTemplateFromCache(templatePath)
-		if found {
-			s.tmx.RUnlock()
-			return t, nil
-		}
-		s.tmx.RUnlock()
-	}
+	s.tmx.Lock()
+	defer s.tmx.Unlock()
+	return s.getSiblingTemplate(templatePath, "/", true)
+}
 
-	t, err = s.getTemplateFromLoader(templatePath)
-	if err == nil && !s.developmentMode {
-		// load template into cache
-		s.tmx.Lock()
-		s.templates[t.Name] = t
-		s.tmx.Unlock()
+func (s *Set) getSiblingTemplate(templatePath, siblingPath string, cacheAfterParsing bool) (t *Template, err error) {
+	templatePath = filepath.ToSlash(templatePath)
+	siblingPath = filepath.ToSlash(siblingPath)
+	if !path.IsAbs(templatePath) {
+		siblingDir := path.Dir(siblingPath)
+		templatePath = path.Join(siblingDir, templatePath)
 	}
-	return t, err
+	return s.getTemplate(templatePath, cacheAfterParsing)
 }
 
 // same as GetTemplate, but assumes the reader already called s.tmx.RLock(), and
 // doesn't cache a template when found through the loader
-func (s *Set) getTemplate(templatePath string) (t *Template, err error) {
+func (s *Set) getTemplate(templatePath string, cacheAfterParsing bool) (t *Template, err error) {
 	if !s.developmentMode {
 		t, found := s.getTemplateFromCache(templatePath)
 		if found {
@@ -174,27 +148,36 @@ func (s *Set) getTemplate(templatePath string) (t *Template, err error) {
 		}
 	}
 
-	return s.getTemplateFromLoader(templatePath)
+	t, err = s.getTemplateFromLoader(templatePath, cacheAfterParsing)
+	if err == nil && cacheAfterParsing && !s.developmentMode {
+		s.templates[templatePath] = t
+	}
+	return t, err
 }
 
-func (s *Set) getSiblingTemplate(templatePath, siblingPath string) (t *Template, err error) {
-	if !path.IsAbs(filepath.ToSlash(templatePath)) {
-		siblingDir := filepath.Dir(siblingPath)
-		templatePath = filepath.Join(siblingDir, templatePath)
+func (s *Set) getTemplateFromCache(templatePath string) (t *Template, ok bool) {
+	// check path with all possible extensions in cache
+	for _, extension := range s.extensions {
+		canonicalPath := templatePath + extension
+		if t, found := s.templates[canonicalPath]; found {
+			return t, true
+		}
 	}
-	return s.getTemplate(templatePath)
+	return nil, false
 }
 
-// Parse parses the template without adding it to the set's templates cache.
-func (s *Set) Parse(templatePath, content string) (*Template, error) {
-	s.tmx.RLock()
-	t, err := s.parse(templatePath, content)
-	s.tmx.RUnlock()
-
-	return t, err
+func (s *Set) getTemplateFromLoader(templatePath string, cacheAfterParsing bool) (t *Template, err error) {
+	// check path with all possible extensions in loader
+	for _, extension := range s.extensions {
+		canonicalPath := templatePath + extension
+		if _, found := s.loader.Exists(canonicalPath); found {
+			return s.loadFromFile(canonicalPath, cacheAfterParsing)
+		}
+	}
+	return nil, fmt.Errorf("template %s could not be found", templatePath)
 }
 
-func (s *Set) loadFromFile(templatePath string) (template *Template, err error) {
+func (s *Set) loadFromFile(templatePath string, cacheAfterParsing bool) (template *Template, err error) {
 	f, err := s.loader.Open(templatePath)
 	if err != nil {
 		return nil, err
@@ -204,33 +187,21 @@ func (s *Set) loadFromFile(templatePath string) (template *Template, err error)
 	if err != nil {
 		return nil, err
 	}
-	return s.parse(templatePath, string(content))
+	return s.parse(templatePath, string(content), cacheAfterParsing)
 }
 
-func (s *Set) LoadTemplate(templatePath, content string) (*Template, error) {
-	if s.developmentMode {
-		s.tmx.RLock()
-		defer s.tmx.RUnlock()
-		return s.parse(templatePath, content)
+func (s *Set) Parse(templatePath, contents string) (template *Template, err error) {
+	templatePath = filepath.ToSlash(templatePath)
+	switch path.Base(templatePath) {
+	case ".", "/":
+		return nil, errors.New("template path has no base name")
 	}
+	// make sure it's absolute and clean it
+	templatePath = path.Join("/", templatePath)
 
-	// fast path (t from cache)
 	s.tmx.RLock()
-	if t, found := s.templates[templatePath]; found {
-		s.tmx.RUnlock()
-		return t, nil
-	}
-	s.tmx.RUnlock()
-
-	// not found, parse and cache
-	s.tmx.Lock()
-	defer s.tmx.Unlock()
-
-	t, err := s.parse(templatePath, content)
-	if err == nil {
-		s.templates[templatePath] = t
-	}
-	return t, err
+	defer s.tmx.RUnlock()
+	return s.parse(templatePath, contents, false)
 }
 
 // SetExtensions sets extensions.


Test patch:
diff --git a/eval_test.go b/eval_test.go
index 857bc53..ec3e3b2 100644
--- a/eval_test.go
+++ b/eval_test.go
@@ -19,16 +19,15 @@ import (
 	"fmt"
 	"io"
 	"reflect"
-	"runtime"
 	"strconv"
 	"strings"
-	"sync"
 	"testing"
 	"text/template"
 )
 
 var (
-	JetTestingSet = NewSet(nil, "")
+	JetTestingLoader = NewInMemLoader()
+	JetTestingSet    = NewSetLoader(nil, JetTestingLoader)
 
 	ww    io.Writer = (*devNull)(nil)
 	users           = []*User{
@@ -84,32 +83,24 @@ func init() {
 	}
 
 	JetTestingSet.AddGlobal("dummy", dummy)
-	JetTestingSet.LoadTemplate("actionNode_dummy", `hello {{dummy("WORLD")}}`)
-	JetTestingSet.LoadTemplate("noAllocFn", `hello {{ "José" }} {{1}} {{ "José" }}`)
-	JetTestingSet.LoadTemplate("rangeOverUsers", `{{range .}}{{.Name}}-{{.Email}}{{end}}`)
-	JetTestingSet.LoadTemplate("rangeOverUsers_Set", `{{range index,user:= . }}{{index}}{{user.Name}}-{{user.Email}}{{end}}`)
-
-	JetTestingSet.LoadTemplate("BenchNewBlock", "{{ block col(md=12,offset=0) }}\n<div class=\"col-md-{{md}} col-md-offset-{{offset}}\">{{ yield content }}</div>\n\t\t{{ end }}\n\t\t{{ block row(md=12) }}\n<div class=\"row {{md}}\">{{ yield content }}</div>\n\t\t{{ content }}\n<div class=\"col-md-1\"></div>\n<div class=\"col-md-1\"></div>\n<div class=\"col-md-1\"></div>\n\t\t{{ end }}\n\t\t{{ block header() }}\n<div class=\"header\">\n\t{{ yield row() content}}\n\t\t{{ yield col(md=6) content }}\n{{ yield content }}\n\t\t{{end}}\n\t{{end}}\n</div>\n\t\t{{content}}\n<h1>Hey</h1>\n\t\t{{ end }}")
+	JetTestingLoader.Set("actionNode_dummy", `hello {{dummy("WORLD")}}`)
+	JetTestingLoader.Set("noAllocFn", `hello {{ "José" }} {{1}} {{ "José" }}`)
+	JetTestingLoader.Set("rangeOverUsers", `{{range .}}{{.Name}}-{{.Email}}{{end}}`)
+	JetTestingLoader.Set("rangeOverUsers_Set", `{{range index,user:= . }}{{index}}{{user.Name}}-{{user.Email}}{{end}}`)
+	JetTestingLoader.Set("BenchNewBlock", "{{ block col(md=12,offset=0) }}\n<div class=\"col-md-{{md}} col-md-offset-{{offset}}\">{{ yield content }}</div>\n\t\t{{ end }}\n\t\t{{ block row(md=12) }}\n<div class=\"row {{md}}\">{{ yield content }}</div>\n\t\t{{ content }}\n<div class=\"col-md-1\"></div>\n<div class=\"col-md-1\"></div>\n<div class=\"col-md-1\"></div>\n\t\t{{ end }}\n\t\t{{ block header() }}\n<div class=\"header\">\n\t{{ yield row() content}}\n\t\t{{ yield col(md=6) content }}\n{{ yield content }}\n\t\t{{end}}\n\t{{end}}\n</div>\n\t\t{{content}}\n<h1>Hey</h1>\n\t\t{{ end }}")
 }
 
 func RunJetTest(t *testing.T, variables VarMap, context interface{}, testName, testContent, testExpected string) {
-	RunJetTestWithSet(t, JetTestingSet, variables, context, testName, testContent, testExpected)
-}
-
-func RunJetTestWithSet(t *testing.T, set *Set, variables VarMap, context interface{}, testName, testContent, testExpected string) {
-	var (
-		tt  *Template
-		err error
-	)
-
 	if testContent != "" {
-		tt, err = set.LoadTemplate(testName, testContent)
-	} else {
-		tt, err = set.GetTemplate(testName)
+		JetTestingLoader.Set(testName, testContent)
 	}
+	RunJetTestWithSet(t, JetTestingSet, variables, context, testName, testExpected)
+}
 
+func RunJetTestWithSet(t *testing.T, set *Set, variables VarMap, context interface{}, testName, testExpected string) {
+	tt, err := set.GetTemplate(testName)
 	if err != nil {
-		t.Errorf("Parsing error: %s %s %s", err.Error(), testName, testContent)
+		t.Errorf("Error parsing templates for test %s: %v", testName, err)
 		return
 	}
 	RunJetTestWithTemplate(t, tt, variables, context, testExpected)
@@ -280,7 +271,7 @@ func TestEvalBlockYieldIncludeNode(t *testing.T) {
 	RunJetTest(t, data, nil, "Block_Import", `{{import "Block_simple"}}{{yield hello() "Buddy"}}`, `Hello Buddy`)
 	RunJetTest(t, data, nil, "Block_Import", `{{import "Block_simple"}}{{yield hello() "Buddy"}}`, `Hello Buddy`)
 
-	JetTestingSet.LoadTemplate("Block_ImportInclude1", `{{yield hello() "Buddy"}}`)
+	JetTestingLoader.Set("Block_ImportInclude1", `{{yield hello() "Buddy"}}`)
 	RunJetTest(t, data, nil, "Block_ImportInclude", `{{ import "Block_simple"}}{{include "Block_ImportInclude1"}}`, `Hello Buddy`)
 	RunJetTest(t, data, nil,
 		"Block_Content",
@@ -288,12 +279,12 @@ func TestEvalBlockYieldIncludeNode(t *testing.T) {
 		"\n<div class=\"col-md-12 col-md-offset-0\"></div>\n\t\t\n\t\t\n<div class=\"row 12\">\n<div class=\"col-md-1\"></div>\n<div class=\"col-md-1\"></div>\n<div class=\"col-md-1\"></div>\n\t\t</div>\n\t\t\n\t\t\n<div class=\"header\">\n\t\n<div class=\"row 12\">\n\t\t\n<div class=\"col-md-6 col-md-offset-0\">\n\n<h1>Hey</h1>\n\t\t\n\t\t</div>\n\t\t\n\t</div>\n\t\t\n</div>\n\t\t",
 	)
 
-	JetTestingSet.LoadTemplate("BlockContentLib", "{{block col(columns)}}\n    <div class=\"col {{columns}}\">{{yield content}}</div>\n{{end}}\n{{block row(cols=\"\")}}\n    <div class=\"row\">\n        {{if len(cols) > 0}}\n            {{yield col(columns=cols) content}}{{yield content}}{{end}}\n        {{else}}\n            {{yield content}}\n        {{end}}\n    </div>\n{{end}}")
+	JetTestingLoader.Set("BlockContentLib", "{{block col(columns)}}\n    <div class=\"col {{columns}}\">{{yield content}}</div>\n{{end}}\n{{block row(cols=\"\")}}\n    <div class=\"row\">\n        {{if len(cols) > 0}}\n            {{yield col(columns=cols) content}}{{yield content}}{{end}}\n        {{else}}\n            {{yield content}}\n        {{end}}\n    </div>\n{{end}}")
 	RunJetTest(t, nil, nil, "BlockContentParam",
 		`{{import "BlockContentLib"}}{{yield row(cols="12") content}}{{cols}}{{end}}`,
 		"\n    <div class=\"row\">\n        \n            \n    <div class=\"col 12\">12</div>\n\n        \n    </div>\n")
 
-	JetTestingSet.LoadTemplate("BlockContentLib2", `
+	JetTestingLoader.Set("BlockContentLib2", `
 		{{block col(
 			columns,
 		)}}
@@ -584,7 +575,7 @@ func TestEvalStructFieldPointerExpressions(t *testing.T) {
 	RunJetTest(t, data, nil, "PointerFields_7", `{{ structWithPointerFields2.StructField }}`, "<nil>")
 
 	var set = NewSet(nil, "./testData")
-	tt, err := set.parse("PointerFields_8", `{{ structWithPointerFields2.StructField.StringField }}`)
+	tt, err := set.parse("PointerFields_8", `{{ structWithPointerFields2.StructField.StringField }}`, false)
 	if err != nil {
 		t.Error(err)
 	}
@@ -624,19 +615,22 @@ func TestEvalBuiltinExpression(t *testing.T) {
 }
 
 func TestEvalAutoescape(t *testing.T) {
-	set := NewHTMLSet("")
-	RunJetTestWithSet(t, set, nil, nil, "Autoescapee_Test1", `<h1>{{"<h1>Hello Buddy!</h1>" }}</h1>`, "<h1>&lt;h1&gt;Hello Buddy!&lt;/h1&gt;</h1>")
-	RunJetTestWithSet(t, set, nil, nil, "Autoescapee_Test2", `<h1>{{"<h1>Hello Buddy!</h1>" |unsafe }}</h1>`, "<h1><h1>Hello Buddy!</h1></h1>")
+	l := NewInMemLoader()
+	set := NewSetLoader(template.HTMLEscape, l)
+	l.Set("Autoescapee_Test1", `<h1>{{"<h1>Hello Buddy!</h1>" }}</h1>`)
+	RunJetTestWithSet(t, set, nil, nil, "Autoescapee_Test1", "<h1>&lt;h1&gt;Hello Buddy!&lt;/h1&gt;</h1>")
+	l.Set("Autoescapee_Test2", `<h1>{{"<h1>Hello Buddy!</h1>" |unsafe }}</h1>`)
+	RunJetTestWithSet(t, set, nil, nil, "Autoescapee_Test2", "<h1><h1>Hello Buddy!</h1></h1>")
 }
 
 func TestFileResolve(t *testing.T) {
 	set := NewHTMLSet("./testData/resolve")
-	RunJetTestWithSet(t, set, nil, nil, "simple", "", "simple")
-	RunJetTestWithSet(t, set, nil, nil, "simple.jet", "", "simple.jet")
-	RunJetTestWithSet(t, set, nil, nil, "extension", "", "extension.jet.html")
-	RunJetTestWithSet(t, set, nil, nil, "extension.jet.html", "", "extension.jet.html")
-	RunJetTestWithSet(t, set, nil, nil, "./sub/subextend", "", "simple - simple.jet - extension.jet.html")
-	RunJetTestWithSet(t, set, nil, nil, "./sub/extend", "", "simple - simple.jet - extension.jet.html")
+	RunJetTestWithSet(t, set, nil, nil, "simple", "simple")
+	RunJetTestWithSet(t, set, nil, nil, "simple.jet", "simple.jet")
+	RunJetTestWithSet(t, set, nil, nil, "extension", "extension.jet.html")
+	RunJetTestWithSet(t, set, nil, nil, "extension.jet.html", "extension.jet.html")
+	RunJetTestWithSet(t, set, nil, nil, "./sub/subextend", "simple - simple.jet - extension.jet.html")
+	RunJetTestWithSet(t, set, nil, nil, "./sub/extend", "simple - simple.jet - extension.jet.html")
 	//for key, _ := range set.templates {
 	//	t.Log(key)
 	//}
@@ -644,10 +638,10 @@ func TestFileResolve(t *testing.T) {
 
 func TestIncludeIfNotExists(t *testing.T) {
 	set := NewHTMLSet("./testData/includeIfNotExists")
-	RunJetTestWithSet(t, set, nil, nil, "existent", "", "Hi, i exist!!")
-	RunJetTestWithSet(t, set, nil, nil, "notExistent", "", "")
-	RunJetTestWithSet(t, set, nil, nil, "ifIncludeIfExits", "", "Hi, i exist!!\n    Was included!!\n\n\n    Was not included!!\n\n")
-	RunJetTestWithSet(t, set, nil, "World", "wcontext", "", "Hi, Buddy!\nHi, World!")
+	RunJetTestWithSet(t, set, nil, nil, "existent", "Hi, i exist!!")
+	RunJetTestWithSet(t, set, nil, nil, "notExistent", "")
+	RunJetTestWithSet(t, set, nil, nil, "ifIncludeIfExits", "Hi, i exist!!\n    Was included!!\n\n\n    Was not included!!\n\n")
+	RunJetTestWithSet(t, set, nil, "World", "wcontext", "Hi, Buddy!\nHi, World!")
 
 	// Check if includeIfExists helper bubbles up runtime errors of included templates
 	tt, err := set.GetTemplate("includeBroken")
@@ -661,49 +655,26 @@ func TestIncludeIfNotExists(t *testing.T) {
 	}
 }
 
-func TestSet_Parse(t *testing.T) {
-	set := NewHTMLSet("./testData/resolve")
-
-	var c int64 = 100
-
-	group := &sync.WaitGroup{}
-	for i, l := int64(0), c; i < l; i++ {
-		(func() {
-			template, _ := set.Parse("TestTemplate", `{{extends "sub/extend"}}`)
-			RunJetTestWithTemplate(t, template, nil, nil, "simple - simple.jet - extension.jet.html")
-			if len(set.templates) > 0 {
-				t.Fail()
-			}
-			group.Add(1)
-			runtime.SetFinalizer(template, func(ob interface{}) {
-				group.Done()
-			})
-		})()
-	}
-	runtime.GC()
-	group.Wait()
-}
-
 func TestExecReturn(t *testing.T) {
 	set := NewHTMLSet("./testData/execReturn")
-	RunJetTestWithSet(t, set, nil, nil, "simple", "", "\n\n... some content that will be discarded when this template runs inside exec() ...\n\n")
-	RunJetTestWithSet(t, set, nil, nil, "test_simple", "", "foo\n")
-	RunJetTestWithSet(t, set, nil, nil, "in_if", "", "\n\n\n")
-	RunJetTestWithSet(t, set, nil, nil, "test_in_if", "", "from inside if branch\n")
+	RunJetTestWithSet(t, set, nil, nil, "simple", "\n\n... some content that will be discarded when this template runs inside exec() ...\n\n")
+	RunJetTestWithSet(t, set, nil, nil, "test_simple", "foo\n")
+	RunJetTestWithSet(t, set, nil, nil, "in_if", "\n\n\n")
+	RunJetTestWithSet(t, set, nil, nil, "test_in_if", "from inside if branch\n")
 	context := map[string]interface{}{"arr": []string{"foo", "bar"}}
-	RunJetTestWithSet(t, set, nil, context, "in_range", "", "\n0: foo\n\n\n")
-	RunJetTestWithSet(t, set, nil, context, "test_in_range", "", "from inside if branch inside range\n")
-	RunJetTestWithSet(t, set, nil, nil, "included", "", "... some content that will be discarded when this template runs inside exec() ...\n\n")
-	RunJetTestWithSet(t, set, nil, nil, "in_include", "", "bla bla\n... some content that will be discarded when this template runs inside exec() ...\n\n\nfoo\n")
-	RunJetTestWithSet(t, set, nil, nil, "test_in_include", "", "from inside included template\n")
+	RunJetTestWithSet(t, set, nil, context, "in_range", "\n0: foo\n\n\n")
+	RunJetTestWithSet(t, set, nil, context, "test_in_range", "from inside if branch inside range\n")
+	RunJetTestWithSet(t, set, nil, nil, "included", "... some content that will be discarded when this template runs inside exec() ...\n\n")
+	RunJetTestWithSet(t, set, nil, nil, "in_include", "bla bla\n... some content that will be discarded when this template runs inside exec() ...\n\n\nfoo\n")
+	RunJetTestWithSet(t, set, nil, nil, "test_in_include", "from inside included template\n")
 }
 
 func TestTryCatch(t *testing.T) {
 	set := NewHTMLSet("./testData/tryCatch")
-	RunJetTestWithSet(t, set, nil, nil, "try", "", "before try without panic ...\n\nsome content\n\nfoo\n\nafter try without panic ...\nbefore panic ...\n\nafter panic ...")
-	RunJetTestWithSet(t, set, nil, nil, "try_catch", "", "before panic ...\n\nan error occured!\n\nafter panic ...")
-	RunJetTestWithSet(t, set, nil, nil, "try_catch_err", "", "before panic ...\n\nan error occured: Jet Runtime Error (&#34;try_catch_err.jet&#34;:3): identifier &#34;undefined_identifier_that_causes_panic&#34; not available in current (map[]) or parent scope, global, or default variables\n\nafter panic ...")
-	RunJetTestWithSet(t, set, nil, nil, "try_include", "", "before broken include ...\n\nafter broken include ...")
+	RunJetTestWithSet(t, set, nil, nil, "try", "before try without panic ...\n\nsome content\n\nfoo\n\nafter try without panic ...\nbefore panic ...\n\nafter panic ...")
+	RunJetTestWithSet(t, set, nil, nil, "try_catch", "before panic ...\n\nan error occured!\n\nafter panic ...")
+	RunJetTestWithSet(t, set, nil, nil, "try_catch_err", "before panic ...\n\nan error occured: Jet Runtime Error (&#34;/try_catch_err.jet&#34;:3): identifier &#34;undefined_identifier_that_causes_panic&#34; not available in current (map[]) or parent scope, global, or default variables\n\nafter panic ...")
+	RunJetTestWithSet(t, set, nil, nil, "try_include", "before broken include ...\n\nafter broken include ...")
 }
 
 func TestBuiltinCollectionFuncs(t *testing.T) {
@@ -744,8 +715,8 @@ func TestRanger(t *testing.T) {
 
 func TestWhitespaceControl(t *testing.T) {
 	set := NewHTMLSet("./testData/whitespaceControl")
-	RunJetTestWithSet(t, set, nil, nil, "simple", "", "beforeACTIONafter")
-	RunJetTestWithSet(t, set, nil, nil, "multiple", "", "beforeACTIONafter")
+	RunJetTestWithSet(t, set, nil, nil, "simple", "beforeACTIONafter")
+	RunJetTestWithSet(t, set, nil, nil, "multiple", "beforeACTIONafter")
 }
 
 func BenchmarkSimpleAction(b *testing.B) {
diff --git a/jettest/test.go b/jettest/test.go
index 71e5691..fd44995 100644
--- a/jettest/test.go
+++ b/jettest/test.go
@@ -23,36 +23,15 @@ import (
 	"github.com/CloudyKit/jet/v5"
 )
 
-// TestingSet holds a template set for running tests
-var TestingSet = jet.NewSet(nil, "")
-
-// Run runs jet template engine test, template will be loaded and cached in the default set TestingSet
-func Run(t *testing.T, variables jet.VarMap, context interface{}, testName, testContent, testExpected string) {
-	RunWithSet(t, TestingSet, variables, context, testName, testContent, testExpected)
-}
-
-// RunWithSet like Run but accepts a jet.Set as a parameter to be used instead of the default set
-func RunWithSet(t *testing.T, set *jet.Set, variables jet.VarMap, context interface{}, testName, testContent, testExpected string) {
-	var (
-		tt  *jet.Template
-		err error
-	)
-
-	if testContent != "" {
-		tt, err = set.LoadTemplate(testName, testContent)
-	} else {
-		tt, err = set.GetTemplate(testName)
-	}
-
+func RunWithSet(t *testing.T, set *jet.Set, variables jet.VarMap, context interface{}, testName, testExpected string) {
+	tt, err := set.GetTemplate(testName)
 	if err != nil {
-		t.Errorf("Parsing error: %s %s %s", err.Error(), testName, testContent)
+		t.Errorf("Error parsing templates for test %s: %v", testName, err)
 		return
 	}
-
 	RunWithTemplate(t, tt, variables, context, testExpected)
 }
 
-// RunWithTemplate like Run but accepts a jet.Template
 func RunWithTemplate(t *testing.T, tt *jet.Template, variables jet.VarMap, context interface{}, testExpected string) {
 	if testing.RunTests(func(pat, str string) (bool, error) {
 		return true, nil
diff --git a/loaders/httpfs/httpfs_test.go b/loaders/httpfs/httpfs_test.go
index 8e14c26..de792dc 100644
--- a/loaders/httpfs/httpfs_test.go
+++ b/loaders/httpfs/httpfs_test.go
@@ -23,8 +23,8 @@ func TestNilHTTPFileSystem(t *testing.T) {
 func TestHTTPFileSystemResolve(t *testing.T) {
 	fs := http.Dir("testData/includeIfNotExists")
 	set := jet.NewHTMLSetLoader(NewLoader(fs))
-	jettest.RunWithSet(t, set, nil, nil, "existent", "", "Hi, i exist!!")
-	jettest.RunWithSet(t, set, nil, nil, "notExistent", "", "")
-	jettest.RunWithSet(t, set, nil, nil, "ifIncludeIfExits", "", "Hi, i exist!!\n    Was included!!\n\n\n    Was not included!!\n\n")
-	jettest.RunWithSet(t, set, nil, "World", "wcontext", "", "Hi, Buddy!\nHi, World!")
+	jettest.RunWithSet(t, set, nil, nil, "existent", "Hi, i exist!!")
+	jettest.RunWithSet(t, set, nil, nil, "notExistent", "")
+	jettest.RunWithSet(t, set, nil, nil, "ifIncludeIfExits", "Hi, i exist!!\n    Was included!!\n\n\n    Was not included!!\n\n")
+	jettest.RunWithSet(t, set, nil, "World", "wcontext", "Hi, Buddy!\nHi, World!")
 }
diff --git a/loaders/multi/multi_test.go b/loaders/multi/multi_test.go
index b3d3a88..9c6c0d5 100644
--- a/loaders/multi/multi_test.go
+++ b/loaders/multi/multi_test.go
@@ -26,7 +26,7 @@ func TestTwoLoaders(t *testing.T) {
 	httpFSLoader := httpfs.NewLoader(http.Dir("../../testData"))
 	l := NewLoader(osFSLoader, httpFSLoader)
 	set := jet.NewHTMLSetLoader(l)
-	jettest.RunWithSet(t, set, nil, nil, "resolve/simple.jet", "", "simple.jet")
-	jettest.RunWithSet(t, set, nil, nil, "base.jet", "", "")
-	jettest.RunWithSet(t, set, nil, nil, "simple2", "", "simple2\n")
+	jettest.RunWithSet(t, set, nil, nil, "resolve/simple.jet", "simple.jet")
+	jettest.RunWithSet(t, set, nil, nil, "base.jet", "")
+	jettest.RunWithSet(t, set, nil, nil, "simple2", "simple2\n")
 }
diff --git a/parse_test.go b/parse_test.go
index 9c5fcf8..a062cf5 100644
--- a/parse_test.go
+++ b/parse_test.go
@@ -34,7 +34,7 @@ func (t ParserTestCase) ExpectPrintName(name, input, output string) {
 	if t.set != nil {
 		set = t.set
 	}
-	template, err := set.parse(name, input)
+	template, err := set.parse(name, input, false)
 	if err != nil {
 		t.Errorf("%q %s", input, err.Error())
 		return
@@ -55,7 +55,7 @@ func (t ParserTestCase) ExpectError(name, input, errorMessage string) {
 	if t.set != nil {
 		set = t.set
 	}
-	_, err := set.parse(name, input)
+	_, err := set.parse(name, input, false)
 	if err == nil {
 		t.Errorf("expected %q but got no error", errorMessage)
 		return
diff --git a/template_test.go b/template_test.go
index 950385d..da09f10 100644
--- a/template_test.go
+++ b/template_test.go
@@ -34,3 +34,27 @@ func TestSetSetExtensions(t *testing.T) {
 		}
 	}
 }
+
+func TestParseDoesNotCache(t *testing.T) {
+	loader := NewInMemLoader()
+	set := NewHTMLSetLoader(loader)
+	_, err := set.Parse("/asd.jet", `{{ foo := "bar" }}{{foo}}`)
+	if err != nil {
+		t.Errorf("parsing template: %v", err)
+		return
+	}
+	if len(set.templates) > 0 {
+		t.Errorf("template is cached in set after Parse()")
+	}
+
+	loader.Set("/something_to_extend.jet", "some content to extend")
+
+	_, err = set.Parse("/includes_template.jet", `{{ extends "/something_to_extend.jet" }}, and more content`)
+	if err != nil {
+		t.Errorf("parsing template: %v", err)
+		return
+	}
+	if len(set.templates) > 0 {
+		t.Errorf("one or more template(s) are cached in set after Parse()")
+	}
+}
diff --git a/utils/visitor_test.go b/utils/visitor_test.go
index 68c5c5c..6f90b63 100644
--- a/utils/visitor_test.go
+++ b/utils/visitor_test.go
@@ -3,15 +3,20 @@ package utils
 import (
 	"reflect"
 	"testing"
+	"text/template"
 
 	"github.com/CloudyKit/jet/v5"
 )
 
-var Set = jet.NewHTMLSet("")
+var (
+	Loader = jet.NewInMemLoader()
+	Set    = jet.NewSetLoader(template.HTMLEscape, Loader)
+)
 
 func TestVisitor(t *testing.T) {
 	var collectedIdentifiers []string
-	var mTemplate, _ = Set.LoadTemplate("_testing", "{{ ident1 }}\n{{ ident2(ident3)}}\n{{ if ident4 }}\n    {{ident5}}\n{{else}}\n    {{ident6}}\n{{end}}\n{{ ident7|ident8|ident9+ident10|ident11[ident12]: ident13[ident14:ident15] }}")
+	Loader.Set("_testing", "{{ ident1 }}\n{{ ident2(ident3)}}\n{{ if ident4 }}\n    {{ident5}}\n{{else}}\n    {{ident6}}\n{{end}}\n{{ ident7|ident8|ident9+ident10|ident11[ident12]: ident13[ident14:ident15] }}")
+	mTemplate, _ := Set.GetTemplate("_testing")
 	Walk(mTemplate, VisitorFunc(func(context VisitorContext, node jet.Node) {
 		if node.Type() == jet.NodeIdentifier {
 			collectedIdentifiers = append(collectedIdentifiers, node.String())
@@ -24,7 +29,8 @@ func TestVisitor(t *testing.T) {
 }
 
 func TestSimpleTemplate(t *testing.T) {
-	var mTemplate, err = Set.LoadTemplate("_testing2", "<html><head><title>Thank you!</title></head>\n\n<body>\n\tHello {{userName}},\n\n\tThanks for the order!\n\n\t{{range product := products}}\n\t\t{{product.name}}\n\n\t    {{block productPrice(price=product.Price) product}}\n            {{if price > ExpensiveProduct}}\n                Expensive!!\n            {{end}}\n        {{end}}\n\n\t\t${{product.price / 100}}\n\t{{end}}\n</body>\n</html>")
+	Loader.Set("_testing2", "<html><head><title>Thank you!</title></head>\n\n<body>\n\tHello {{userName}},\n\n\tThanks for the order!\n\n\t{{range product := products}}\n\t\t{{product.name}}\n\n\t    {{block productPrice(price=product.Price) product}}\n            {{if price > ExpensiveProduct}}\n                Expensive!!\n            {{end}}\n        {{end}}\n\n\t\t${{product.price / 100}}\n\t{{end}}\n</body>\n</html>")
+	mTemplate, err := Set.GetTemplate("_testing2")
 	if err != nil {
 		t.Error(err)
 	}