Repository: PebbleTemplates/pebble
Pull request #552 (2020-11-29 17:11:32)

issue #520: provide option to customize stock extensions
This is a draft for #520 to allow for a customization of the default extensions.

- refactored out the extension loading code from the `PebbleEngine.Builder` into the `ExtensionRegistryFactory` 
- provide a `ExtensionCustomizer` which basically allows to access extensions before loading


pepperbob: So I came up with "ExtensionCustomizer" to allow filtering out or change whatever functionality is provided by an extension. It currently only targets the baked in extensions - not sure if it would make much sense on the user provided extensions anyway. 

On the one hand it looks like it leaves the configuration API completely intact and doesn't degrade things like `allowOverrideCoreOperators` to an implementation detail.

On the other hand I'm not too happy with the current version as now other implementation details, e.g. how core functionality is split over extension classes (core, i18n, escaper) kind of leaks out as you have to know the extension class to apply the customizer.

Of course the whole target class thing can be dropped and there could be just one single customizer that is wrapped around every extension before it is loaded ... while thinking about it - i think this is would would provide the cleanest API and won't break things for users if stuff is re-organized internally at some point.. I'll rework it.

Feedback is much appreciated!
ebussieres: Can you provide some documentation for this new functionality ?
pepperbob: @ebussieres Ok, I added further JavaDoc - is that okay so far?
ebussieres: I mean add some documentation for pebble website. Doc is here : https://github.com/PebbleTemplates/pebble/tree/master/docs/src/orchid/resources
pepperbob: Oh sure, I'll check this out!

Patch:
diff --git a/docs/src/orchid/resources/wiki/guide/customize-defaults.md b/docs/src/orchid/resources/wiki/guide/customize-defaults.md
new file mode 100644
index 00000000..50be2cce
--- /dev/null
+++ b/docs/src/orchid/resources/wiki/guide/customize-defaults.md
@@ -0,0 +1,37 @@
+---
+---
+
+Pebble comes with a rich set of built-in tags and filters that will help you render your templates into  websites and other documents with ease. However, imagine a more specific use-case where the templates are not entirely under your control.
+
+In these cases it might be advised to consider stripping-down Pebbles' built-in functionality that may otherwise introduce security-concers regarding the integrity and stability of your application.
+
+### Opt-Out using ExtensionCustomizer
+
+The `ExtensionCustomizer` base class can be used to gain access to the default functionality before it is loaded into Pebbles template engine. Overwrite methods to get hold on provided default-functionality and modify whatever should be available for the template engine.
+
+The following example removes the `ForTokenParser`, i.e. the ability to parse `{% for %}{{ ... }}{% endfor %}` constructs:
+
+```java
+class ExampleOptOuts extends ExtensionCustomizer {
+
+  public ExampleOptOuts(Extension ext) {
+    super(ext);
+  }
+
+  @Override
+  public List<TokenParser> getTokenParsers() {
+    List<TokenParser> tokenParsers = Optional.ofNullable(super.getTokenParsers())
+                                        .map(ArrayList::new).orElseGet(ArrayList::new);
+      
+    tokenParsers.removeIf(x -> x instanceof ForTokenParser);
+    return tokenParsers;
+  }
+
+}
+```
+
+The `ExtensionCustomizer` will be used to wrap any Pebble-extension which is provided by default. It can be registered in your setup code to create `PebbleEngine`:
+
+```java
+PebbleEngine engine = new PebbleEngine.Builder().registerCustomizer(ExampleOptOuts::new).build();
+```
diff --git a/docs/src/orchid/resources/wiki/summary.md b/docs/src/orchid/resources/wiki/summary.md
index 5209ffed..9512175c 100644
--- a/docs/src/orchid/resources/wiki/summary.md
+++ b/docs/src/orchid/resources/wiki/summary.md
@@ -4,6 +4,7 @@
 - [Spring petclinic](https://github.com/PebbleTemplates/spring-petclinic)
 - [Pebble Spring Example](https://github.com/PebbleTemplates/pebble-example-spring)
 - [Basic Usage](guide/basic-usage.md)
+- [Customize Defaults](guide/customize-defaults.md)
 - [Escaping](guide/escaping.md)
 - [Extending Pebble](guide/extending-pebble.md)
 - [High Performance Techniques](guide/high-performance.md)
diff --git a/pebble/src/main/java/com/mitchellbosecke/pebble/PebbleEngine.java b/pebble/src/main/java/com/mitchellbosecke/pebble/PebbleEngine.java
index 498d5b35..7e5d0109 100644
--- a/pebble/src/main/java/com/mitchellbosecke/pebble/PebbleEngine.java
+++ b/pebble/src/main/java/com/mitchellbosecke/pebble/PebbleEngine.java
@@ -18,14 +18,8 @@ import com.mitchellbosecke.pebble.cache.tag.NoOpTagCache;
 import com.mitchellbosecke.pebble.cache.template.ConcurrentMapTemplateCache;
 import com.mitchellbosecke.pebble.cache.template.NoOpTemplateCache;
 import com.mitchellbosecke.pebble.error.LoaderException;
-import com.mitchellbosecke.pebble.extension.Extension;
-import com.mitchellbosecke.pebble.extension.ExtensionRegistry;
-import com.mitchellbosecke.pebble.extension.NodeVisitorFactory;
-import com.mitchellbosecke.pebble.extension.core.AttributeResolverExtension;
-import com.mitchellbosecke.pebble.extension.core.CoreExtension;
-import com.mitchellbosecke.pebble.extension.escaper.EscaperExtension;
+import com.mitchellbosecke.pebble.extension.*;
 import com.mitchellbosecke.pebble.extension.escaper.EscapingStrategy;
-import com.mitchellbosecke.pebble.extension.i18n.I18nExtension;
 import com.mitchellbosecke.pebble.lexer.LexerImpl;
 import com.mitchellbosecke.pebble.lexer.Syntax;
 import com.mitchellbosecke.pebble.lexer.TokenStream;
@@ -44,10 +38,11 @@ import com.mitchellbosecke.pebble.template.PebbleTemplateImpl;
 import java.io.IOException;
 import java.io.Reader;
 import java.util.ArrayList;
-import java.util.Collections;
 import java.util.List;
 import java.util.Locale;
 import java.util.concurrent.ExecutorService;
+import java.util.function.Function;
+
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 
@@ -277,8 +272,6 @@ public class PebbleEngine {
 
     private Loader<?> loader;
 
-    private final List<Extension> userProvidedExtensions = new ArrayList<>();
-
     private Syntax syntax;
 
     private boolean strictVariables = false;
@@ -297,18 +290,16 @@ public class PebbleEngine {
 
     private PebbleCache<CacheKey, Object> tagCache;
 
-    private final EscaperExtension escaperExtension = new EscaperExtension();
-
     private boolean literalDecimalTreatedAsInteger = false;
 
     private boolean greedyMatchMethod = false;
 
-    private boolean allowOverrideCoreOperators = false;
-
     private boolean literalNumbersAsBigDecimals = false;
 
     private MethodAccessValidator methodAccessValidator = new BlacklistMethodAccessValidator();
 
+    private final ExtensionRegistryFactory factory = new ExtensionRegistryFactory();
+
     /**
      * Creates the builder.
      */
@@ -334,7 +325,7 @@ public class PebbleEngine {
      * @return This builder object
      */
     public Builder extension(Extension... extensions) {
-      Collections.addAll(this.userProvidedExtensions, extensions);
+      this.factory.extension(extensions);
       return this;
     }
 
@@ -455,7 +446,7 @@ public class PebbleEngine {
      * @return This builder object
      */
     public Builder autoEscaping(boolean autoEscaping) {
-      this.escaperExtension.setAutoEscaping(autoEscaping);
+      this.factory.autoEscaping(autoEscaping);
       return this;
     }
 
@@ -466,7 +457,7 @@ public class PebbleEngine {
      * @return This builder object
      */
     public Builder allowOverrideCoreOperators(boolean allowOverrideCoreOperators) {
-      this.allowOverrideCoreOperators = allowOverrideCoreOperators;
+      this.factory.allowOverrideCoreOperators(allowOverrideCoreOperators);
       return this;
     }
 
@@ -477,7 +468,7 @@ public class PebbleEngine {
      * @return This builder object
      */
     public Builder defaultEscapingStrategy(String strategy) {
-      this.escaperExtension.setDefaultStrategy(strategy);
+      this.factory.defaultEscapingStrategy(strategy);
       return this;
     }
 
@@ -489,7 +480,7 @@ public class PebbleEngine {
      * @return This builder object
      */
     public Builder addEscapingStrategy(String name, EscapingStrategy strategy) {
-      this.escaperExtension.addEscapingStrategy(name, strategy);
+      this.factory.addEscapingStrategy(name, strategy);
       return this;
     }
 
@@ -566,6 +557,18 @@ public class PebbleEngine {
       return this;
     }
 
+    /**
+     * Registers an implementation of {@link ExtensionCustomizer} to change runtime-behaviour of standard
+     * functionality.
+     *
+     * @param customizer The customizer which wraps any non-user-provided extension
+     * @return This build object
+     */
+    public Builder registerExtensionCustomizer(Function<Extension, ExtensionCustomizer> customizer) {
+      this.factory.registerExtensionCustomizer(customizer);
+      return this;
+    }
+
     /**
      * Creates the PebbleEngine instance.
      *
@@ -573,7 +576,7 @@ public class PebbleEngine {
      */
     public PebbleEngine build() {
 
-      ExtensionRegistry extensionRegistry = this.buildExtensionRegistry();
+      ExtensionRegistry extensionRegistry = this.factory.buildExtensionRegistry();
 
       // default loader
       if (this.loader == null) {
@@ -617,26 +620,6 @@ public class PebbleEngine {
           this.tagCache, this.templateCache,
           this.executorService, extensionRegistry, parserOptions, evaluationOptions);
     }
-
-    private ExtensionRegistry buildExtensionRegistry() {
-      ExtensionRegistry extensionRegistry = new ExtensionRegistry();
-
-      extensionRegistry.addExtension(new CoreExtension());
-      extensionRegistry.addExtension(this.escaperExtension);
-      extensionRegistry.addExtension(new I18nExtension());
-
-      for (Extension userProvidedExtension : this.userProvidedExtensions) {
-        if (this.allowOverrideCoreOperators) {
-          extensionRegistry.addOperatorOverridingExtension(userProvidedExtension);
-        } else {
-          extensionRegistry.addExtension(userProvidedExtension);
-        }
-      }
-
-      extensionRegistry.addExtension(new AttributeResolverExtension());
-
-      return extensionRegistry;
-    }
   }
 
   public EvaluationOptions getEvaluationOptions() {
diff --git a/pebble/src/main/java/com/mitchellbosecke/pebble/extension/ExtensionCustomizer.java b/pebble/src/main/java/com/mitchellbosecke/pebble/extension/ExtensionCustomizer.java
new file mode 100644
index 00000000..ca9a2f98
--- /dev/null
+++ b/pebble/src/main/java/com/mitchellbosecke/pebble/extension/ExtensionCustomizer.java
@@ -0,0 +1,77 @@
+package com.mitchellbosecke.pebble.extension;
+
+import com.mitchellbosecke.pebble.attributes.AttributeResolver;
+import com.mitchellbosecke.pebble.operator.BinaryOperator;
+import com.mitchellbosecke.pebble.operator.UnaryOperator;
+import com.mitchellbosecke.pebble.tokenParser.TokenParser;
+
+import java.util.List;
+import java.util.Map;
+
+/**
+ *  Base class that allows implementing a customizer to modify Pebbles build-in extensions.
+ *  It is meant to provide a way to remove or replace functions, filters, tags, etc. to change
+ *  the standard behaviour. Use-cases can be down-stripping available functionality for security
+ *  reasons.
+ *
+ *  Implementations of this class are meant to overwrite methods and access registered functionality
+ *  before it is loaded into the PebbleEngine by calling super.
+ *
+ *  The ExentsionCustomizer can be registred via {@link com.mitchellbosecke.pebble.PebbleEngine.Builder#registerExtensionCustomizer}
+ *  and is applied for every non-user-provided extension.
+ *
+ */
+public abstract class ExtensionCustomizer implements Extension {
+
+  private final Extension delegate;
+
+  public ExtensionCustomizer(Extension delegate) {
+    this.delegate = delegate;
+  }
+
+  @Override
+  public Map<String, Filter> getFilters() {
+    return delegate.getFilters();
+  }
+
+  @Override
+  public Map<String, Test> getTests() {
+    return delegate.getTests();
+  }
+
+  @Override
+  public Map<String, Function> getFunctions() {
+    return delegate.getFunctions();
+  }
+
+  @Override
+  public List<TokenParser> getTokenParsers() {
+    return delegate.getTokenParsers();
+  }
+
+  @Override
+  public List<BinaryOperator> getBinaryOperators() {
+    return delegate.getBinaryOperators();
+  }
+
+  @Override
+  public List<UnaryOperator> getUnaryOperators() {
+    return delegate.getUnaryOperators();
+  }
+
+  @Override
+  public Map<String, Object> getGlobalVariables() {
+    return delegate.getGlobalVariables();
+  }
+
+  @Override
+  public List<NodeVisitorFactory> getNodeVisitors() {
+    return delegate.getNodeVisitors();
+  }
+
+  @Override
+  public List<AttributeResolver> getAttributeResolver() {
+    return delegate.getAttributeResolver();
+  }
+
+}
diff --git a/pebble/src/main/java/com/mitchellbosecke/pebble/extension/ExtensionRegistryFactory.java b/pebble/src/main/java/com/mitchellbosecke/pebble/extension/ExtensionRegistryFactory.java
new file mode 100644
index 00000000..c2c67426
--- /dev/null
+++ b/pebble/src/main/java/com/mitchellbosecke/pebble/extension/ExtensionRegistryFactory.java
@@ -0,0 +1,72 @@
+package com.mitchellbosecke.pebble.extension;
+
+import com.mitchellbosecke.pebble.extension.core.AttributeResolverExtension;
+import com.mitchellbosecke.pebble.extension.core.CoreExtension;
+import com.mitchellbosecke.pebble.extension.escaper.EscaperExtension;
+import com.mitchellbosecke.pebble.extension.escaper.EscapingStrategy;
+import com.mitchellbosecke.pebble.extension.i18n.I18nExtension;
+
+import java.util.*;
+import java.util.function.Function;
+import java.util.stream.Stream;
+
+/**
+ * Provides configuration methods and builds the {@link ExtensionRegistry}. Used only internally by
+ * the {@link com.mitchellbosecke.pebble.PebbleEngine.Builder}.
+ *
+ */
+public class ExtensionRegistryFactory {
+
+  private final List<Extension> userProvidedExtensions = new ArrayList<>();
+
+  private final EscaperExtension escaperExtension = new EscaperExtension();
+
+  private boolean allowOverrideCoreOperators = false;
+
+  private Function<Extension, Extension> customizer = Function.identity();
+
+  public ExtensionRegistry buildExtensionRegistry() {
+    ExtensionRegistry extensionRegistry = new ExtensionRegistry();
+
+    Stream.of(new CoreExtension(), this.escaperExtension, new I18nExtension())
+            .map(customizer::apply)
+            .forEach(extensionRegistry::addExtension);
+
+    for (Extension userProvidedExtension : this.userProvidedExtensions) {
+      if (this.allowOverrideCoreOperators) {
+        extensionRegistry.addOperatorOverridingExtension(userProvidedExtension);
+      } else {
+        extensionRegistry.addExtension(userProvidedExtension);
+      }
+    }
+
+    extensionRegistry.addExtension(customizer.apply(new AttributeResolverExtension()));
+
+    return extensionRegistry;
+  }
+
+  public void autoEscaping(boolean autoEscaping) {
+    this.escaperExtension.setAutoEscaping(autoEscaping);
+  }
+
+  public void addEscapingStrategy(String name, EscapingStrategy strategy) {
+    this.escaperExtension.addEscapingStrategy(name, strategy);
+  }
+
+  public void extension(Extension... extensions) {
+    Collections.addAll(this.userProvidedExtensions, extensions);
+  }
+
+  public void allowOverrideCoreOperators(boolean allowOverrideCoreOperators) {
+    this.allowOverrideCoreOperators = allowOverrideCoreOperators;
+  }
+
+  public void defaultEscapingStrategy(String strategy) {
+    this.escaperExtension.setDefaultStrategy(strategy);
+  }
+
+  public void registerExtensionCustomizer(Function<Extension, ExtensionCustomizer> customizer) {
+    this.customizer = customizer::apply;
+  }
+
+}
\ No newline at end of file


Test patch:
diff --git a/pebble/src/test/java/com/mitchellbosecke/pebble/extension/ExtensionCustomizerTest.java b/pebble/src/test/java/com/mitchellbosecke/pebble/extension/ExtensionCustomizerTest.java
new file mode 100644
index 00000000..27e33973
--- /dev/null
+++ b/pebble/src/test/java/com/mitchellbosecke/pebble/extension/ExtensionCustomizerTest.java
@@ -0,0 +1,56 @@
+package com.mitchellbosecke.pebble.extension;
+
+import com.mitchellbosecke.pebble.PebbleEngine;
+import com.mitchellbosecke.pebble.error.PebbleException;
+import com.mitchellbosecke.pebble.template.PebbleTemplate;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+
+import java.io.IOException;
+import java.io.StringWriter;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.Optional;
+
+import static org.junit.jupiter.api.Assertions.assertThrows;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+
+class ExtensionCustomizerTest {
+
+  PebbleEngine pebble;
+
+  @BeforeEach
+  void setUp() {
+    pebble = new PebbleEngine.Builder()
+            .registerExtensionCustomizer(RemoveUpperCustomizer::new)
+            .build();
+  }
+
+  @Test
+  void upperFilterCannotBeUsed() throws IOException {
+    Map<String, Object> obj = new HashMap<>();
+    obj.put("test", "abc");
+    PebbleTemplate template = pebble.getLiteralTemplate("{{ test | upper }}");
+
+    PebbleException exception = assertThrows(PebbleException.class, () -> template.evaluate(new StringWriter(), obj));
+    assertTrue(exception.getMessage().contains("upper"),
+            () -> "Expect upper-Filter to not exist, actual Problem: " + exception.getMessage());
+  }
+
+  private static class RemoveUpperCustomizer extends ExtensionCustomizer {
+
+    public RemoveUpperCustomizer(Extension core) {
+      super(core);
+    }
+
+    @Override
+    public Map<String, Filter> getFilters() {
+      Map<String, Filter> filters = Optional.ofNullable(super.getFilters()).map(HashMap::new)
+              .orElseGet(HashMap::new);
+      filters.remove("upper");
+      return filters;
+    }
+
+  }
+
+}