Repository: getnikola/nikola

Issue:
Escapes closing php statements
config `PAGES = ( ("test/*.php", "stories", "story.tmpl"), )`.

test.php: `Hello, I am <?php true; ?>, Johnny.`.

Results: `<template>Hello, I am <?php true; ?&gt;, Johnny.</template>`.

Expected: `<template>Hello, I am <?php true; ?>, Johnny.</template>`.

Escapes closing php statements
config `PAGES = ( ("test/*.php", "stories", "story.tmpl"), )`.

test.php: `Hello, I am <?php true; ?>, Johnny.`.

Results: `<template>Hello, I am <?php true; ?&gt;, Johnny.</template>`.

Expected: `<template>Hello, I am <?php true; ?>, Johnny.</template>`.



Discussion:
We do it with `post.text` through lxml. Fixing it there doesn’t help as Mako will do the exact same thing again at a later stage. As far as I can figure out, this can only be done as a (hardcoded?) filter applied to .php output:

```
if "?&gt;" in data:
    data = data.replace("?&gt;", r"?>")
```

Not ideal as this would blow up:

```
<code>&lt;?xml example ?&gt;</code>
```

Alternative approach is to generate the template with a `NIKOLA_PHP_PLACEHOLDER\n` that is later replaced with the php data after Mako is done.

Any better suggestions, @Kwpolska, @ralsina?

Found a work-around that lxml, Mako, PHP, and HTML can all live with. (Forgive me Universe.)

```
<?php
    true;
/* <!-- */ ?> <!-- -->
```

Obviously, we cannot expect end-users to use this solution.

There are more problems with the current approch. I am leaning towards generating the template from the metadata with a placeholder and destination, and inserting the PHP into the placeholder at the destination afterwards.

`$var->duck;` turns into `$var-&gt;duck;`

`include_once( "../config.php" );` vanishes (eaten by Mako).

The placeholder solution sounds the sanest, but should only be applied to PHP code.  Another solution might be trusting user input and disabling the HTML escaping features, but that might bite us in other places.

A good solution would be:
1. in Post.text(), detect PHP files and return a placeholder that does not contain any special characters; possibly a unique one for every PHP file
2. force a filter to be applied to PHP files that performs the replacement

We should also make sure that nobody tries to use PHP posts with teasers, as we’d have to re-implement the parser somewhere else.

We do it with `post.text` through lxml. Fixing it there doesn’t help as Mako will do the exact same thing again at a later stage. As far as I can figure out, this can only be done as a (hardcoded?) filter applied to .php output:

```
if "?&gt;" in data:
    data = data.replace("?&gt;", r"?>")
```

Not ideal as this would blow up:

```
<code>&lt;?xml example ?&gt;</code>
```

Alternative approach is to generate the template with a `NIKOLA_PHP_PLACEHOLDER\n` that is later replaced with the php data after Mako is done.

Any better suggestions, @Kwpolska, @ralsina?

Found a work-around that lxml, Mako, PHP, and HTML can all live with. (Forgive me Universe.)

```
<?php
    true;
/* <!-- */ ?> <!-- -->
```

Obviously, we cannot expect end-users to use this solution.

There are more problems with the current approch. I am leaning towards generating the template from the metadata with a placeholder and destination, and inserting the PHP into the placeholder at the destination afterwards.

`$var->duck;` turns into `$var-&gt;duck;`

`include_once( "../config.php" );` vanishes (eaten by Mako).

The placeholder solution sounds the sanest, but should only be applied to PHP code.  Another solution might be trusting user input and disabling the HTML escaping features, but that might bite us in other places.

A good solution would be:
1. in Post.text(), detect PHP files and return a placeholder that does not contain any special characters; possibly a unique one for every PHP file
2. force a filter to be applied to PHP files that performs the replacement

We should also make sure that nobody tries to use PHP posts with teasers, as we’d have to re-implement the parser somewhere else.


Resolving patch:
diff --git a/nikola/filters.py b/nikola/filters.py
--- a/nikola/filters.py
+++ b/nikola/filters.py
@@ -166,3 +166,18 @@ def typogrify(data):
     # data = typo.caps(data)
     data = typo.initial_quotes(data)
     return data
+
+
+@apply_to_text_file
+def php_template_injection(data):
+    import re
+    template = re.search('<\!-- __NIKOLA_PHP_TEMPLATE_INJECTION source\:(.*) csum\:(.*)__ -->', data)
+    if template:
+        source = template.group(1)
+        with codecs.open(source, "r", "utf8") as in_file:
+            phpdata = in_file.read()
+        _META_SEPARATOR = '(' + os.linesep * 2 + '|' + ('\n' * 2) + '|' + ("\r\n" * 2) + ')'
+        phpdata = re.split(_META_SEPARATOR, phpdata, maxsplit=1)[-1]
+        phpdata = re.sub(template.group(0), phpdata, data)
+
+    return phpdata
diff --git a/nikola/plugins/compile/php.py b/nikola/plugins/compile/php.py
--- a/nikola/plugins/compile/php.py
+++ b/nikola/plugins/compile/php.py
@@ -30,13 +30,10 @@
 
 import os
 import codecs
-import re
 
 from nikola.plugin_categories import PageCompiler
 from nikola.utils import makedirs, write_metadata
-
-
-_META_SEPARATOR = '(' + os.linesep * 2 + '|' + ('\n' * 2) + '|' + ("\r\n" * 2) + ')'
+from hashlib import md5
 
 
 class CompilePhp(PageCompiler):
@@ -47,11 +44,9 @@ class CompilePhp(PageCompiler):
     def compile_html(self, source, dest, is_two_file=True):
         makedirs(os.path.dirname(dest))
         with codecs.open(dest, "w+", "utf8") as out_file:
-            with codecs.open(source, "r", "utf8") as in_file:
-                data = in_file.read()
-            if not is_two_file:
-                data = re.split(_META_SEPARATOR, data, maxsplit=1)[-1]
-            out_file.write(data)
+            with open(source, "rb") as in_file:
+                hash = md5(in_file.read()).hexdigest()
+                out_file.write('<!-- __NIKOLA_PHP_TEMPLATE_INJECTION source:{0} csum:{1}__ -->'.format(source, hash))
         return True
 
     def create_post(self, path, **kw):
diff --git a/nikola/utils.py b/nikola/utils.py
--- a/nikola/utils.py
+++ b/nikola/utils.py
@@ -172,6 +172,7 @@ def req_missing(names, purpose, python=True, optional=False):
 from doit import tools
 from unidecode import unidecode
 from pkg_resources import resource_filename
+from nikola import filters as task_filters
 
 import PyRSS2Gen as rss
 
@@ -794,6 +795,12 @@ def apply_filters(task, filters, skip_ext=None):
     and the filter itself to the uptodate of the task.
     """
 
+    if '.php' in filters.keys():
+        if task_filters.php_template_injection not in filters['.php']:
+            filters['.php'].append(task_filters.php_template_injection)
+    else:
+        filters['.php'] = [task_filters.php_template_injection]
+
     def filter_matches(ext):
         for key, value in list(filters.items()):
             if isinstance(key, (tuple, list)):


Test patch: