Repository: getnikola/nikola

Issue:
import_wordpress problem with php code in blogpost
```
$ nikola import_wordpress /Users/davidak/Downloads/davidakweblog.wordpress.2016-02-23.xml
[2016-02-23T17:06:37Z] NOTICE: Nikola: The folder new_site already exists - assuming that this is a already existing Nikola site.
Traceback (most recent call last):
  File "/usr/local/lib/python3.5/site-packages/doit/doit_cmd.py", line 168, in run
    return command.parse_execute(args)
  File "/usr/local/lib/python3.5/site-packages/doit/cmd_base.py", line 122, in parse_execute
    return self.execute(params, args)
  File "/usr/local/lib/python3.5/site-packages/nikola/plugin_categories.py", line 124, in execute
    return self._execute(options, args)
  File "/usr/local/lib/python3.5/site-packages/nikola/plugins/command/import_wordpress.py", line 332, in _execute
    self.import_posts(channel)
  File "/usr/local/lib/python3.5/site-packages/nikola/plugins/command/import_wordpress.py", line 1015, in import_posts
    self.process_item_if_post_or_page(item)
  File "/usr/local/lib/python3.5/site-packages/nikola/plugins/command/import_wordpress.py", line 996, in process_item_if_post_or_page
    out_folder_slug = self.import_postpage_item(item, wordpress_namespace, 'posts', attachments)
  File "/usr/local/lib/python3.5/site-packages/nikola/plugins/command/import_wordpress.py", line 938, in import_postpage_item
    rewrite_html)
  File "/usr/local/lib/python3.5/site-packages/nikola/plugins/basic_import.py", line 130, in write_content
    doc = html.document_fromstring(content)
  File "/usr/local/lib/python3.5/site-packages/lxml/html/__init__.py", line 755, in document_fromstring
    "Document is empty")
lxml.etree.ParserError: Document is empty
```

wordpress export of the post:

```
<?xml version="1.0" encoding="UTF-8" ?>
<!-- This is a WordPress eXtended RSS file generated by WordPress as an export of your site. -->
<!-- It contains information about your site's posts, pages, comments, categories, and other content. -->
<!-- You may use this file to transfer that content from one site to another. -->
<!-- This file is not intended to serve as a complete backup of your site. -->

<!-- To import this information into a WordPress site follow these steps: -->
<!-- 1. Log in to that site as an administrator. -->
<!-- 2. Go to Tools: Import in the WordPress admin panel. -->
<!-- 3. Install the "WordPress" importer from the list. -->
<!-- 4. Activate & Run Importer. -->
<!-- 5. Upload this file using the form provided on that page. -->
<!-- 6. You will first be asked to map the authors in this export file to users -->
<!--    on the site. For each author, you may choose to map to an -->
<!--    existing user on the site or to create a new user. -->
<!-- 7. WordPress will then import each of the posts, pages, comments, categories, etc. -->
<!--    contained in this file into your site. -->

<!-- generator="WordPress/4.4.2" created="2016-02-23 17:01" -->
<rss version="2.0"
    xmlns:excerpt="http://wordpress.org/export/1.2/excerpt/"
    xmlns:content="http://purl.org/rss/1.0/modules/content/"
    xmlns:wfw="http://wellformedweb.org/CommentAPI/"
    xmlns:dc="http://purl.org/dc/elements/1.1/"
    xmlns:wp="http://wordpress.org/export/1.2/"
>

<channel>
    <title>davidak Weblog</title>
    <link>http://davidak.de/blog</link>
    <description></description>
    <pubDate>Tue, 23 Feb 2016 17:01:21 +0000</pubDate>
    <language>de-DE</language>
    <wp:wxr_version>1.2</wp:wxr_version>
    <wp:base_site_url>http://davidak.de/blog</wp:base_site_url>
    <wp:base_blog_url>http://davidak.de/blog</wp:base_blog_url>

    <wp:author><wp:author_id>3</wp:author_id><wp:author_login><![CDATA[davidak]]></wp:author_login><wp:author_email><![CDATA[post@davidak.de]]></wp:author_email><wp:author_display_name><![CDATA[davidak]]></wp:author_display_name><wp:author_first_name><![CDATA[David]]></wp:author_first_name><wp:author_last_name><![CDATA[Kleuker]]></wp:author_last_name></wp:author>

    <wp:category><wp:term_id>12</wp:term_id><wp:category_nicename><![CDATA[podcasts]]></wp:category_nicename><wp:category_parent><![CDATA[]]></wp:category_parent><wp:cat_name><![CDATA[Podcasts]]></wp:cat_name></wp:category>

    <generator>https://wordpress.org/?v=4.4.2</generator>

    <item>
        <title>Umfrage: Podcasts?</title>
        <link>http://davidak.de/blog/170-umfrage-podcasts/</link>
        <pubDate>Wed, 27 Jun 2007 13:59:10 +0000</pubDate>
        <dc:creator><![CDATA[davidak]]></dc:creator>
        <guid isPermaLink="false">http://davidak.de/blog/?p=170</guid>
        <description></description>
        <content:encoded><![CDATA[<?php get_poll(6);?>
]]></content:encoded>
        <excerpt:encoded><![CDATA[]]></excerpt:encoded>
        <wp:post_id>170</wp:post_id>
        <wp:post_date><![CDATA[2007-06-27 15:59:10]]></wp:post_date>
        <wp:post_date_gmt><![CDATA[2007-06-27 13:59:10]]></wp:post_date_gmt>
        <wp:comment_status><![CDATA[open]]></wp:comment_status>
        <wp:ping_status><![CDATA[open]]></wp:ping_status>
        <wp:post_name><![CDATA[umfrage-podcasts]]></wp:post_name>
        <wp:status><![CDATA[publish]]></wp:status>
        <wp:post_parent>0</wp:post_parent>
        <wp:menu_order>0</wp:menu_order>
        <wp:post_type><![CDATA[post]]></wp:post_type>
        <wp:post_password><![CDATA[]]></wp:post_password>
        <wp:is_sticky>0</wp:is_sticky>
        <category domain="category" nicename="internet"><![CDATA[Internet]]></category>
        <category domain="post_tag" nicename="podcasting"><![CDATA[podcasting]]></category>
        <category domain="category" nicename="podcasts"><![CDATA[Podcasts]]></category>
        <category domain="post_tag" nicename="podcasts"><![CDATA[Podcasts]]></category>
        <category domain="post_tag" nicename="umfrage"><![CDATA[umfrage]]></category>
        <wp:postmeta>
            <wp:meta_key><![CDATA[runphp]]></wp:meta_key>
            <wp:meta_value><![CDATA[1]]></wp:meta_value>
        </wp:postmeta>
    </item>

</channel>
</rss>

```



Discussion:
This is (partially) because you are using a plugin called "runPHP" in WordPress which our importer doesn't know anything about. It assumes you have a standard WordPress post.

To fix this, we need to add support for this plugin to the WordPress importer. It has to check the post metadata to check if `runphp` is set to `1` at this place: https://github.com/getnikola/nikola/blob/master/nikola/plugins/command/import_wordpress.py#L884, and then add some code here: https://github.com/getnikola/nikola/blob/master/nikola/plugins/command/import_wordpress.py#L657 to output `.php` files instead. (Also, the blog's config needs to be adjusted.)

Since I don't really know what the runPHP plugin does, and since I don't like working with PHP in general ;), I leave this to anyone interested to implement this.

the wordpress plugin executes php code ;)

the idea of Nikola is to create HTML, not PHP. so i would just assume it imports it as text.

metadata the importer don't knows should get ignored.

And it totally ignores it. And your wordpress post has nothing in it then.

The whole content of the post is 

```
<![CDATA[<?php get_poll(6);?>
]]>
```

If you have any suggestions about what to do with that...

I suppose we could make the importer not stop on that error, and import that post as a no-text post.

yes, i would assume an empty post or the php code as text.

To know that's supposed to be text we would have to do a special handler for the plugin. I'll fix it so it creates empty posts.

I guess it should be exported as php code, since after all we have a php compile plugin.

@ralsina: there's already stuff which handles empty posts. The problem is that this post is not empty, but since Nikola tries to convert it to MarkDown, it feeds it through a XML/HTML parser (lxml), which dies because it thinks the post is "empty".

i don't think that it's worth the effort to support this wordpress plugin. maybe when the first person report this as a bug. i don't care, the code only works inside wordpress with another plugin.

this plugin seams to be popular now:
https://wordpress.org/plugins/insert-php/

rather supporting this.

but i don't think that you want PHP in a static site. why should you switch from wordpress? after all, it's very intuitive.

Implementing this takes probably only a few lines, so why not?

Also, I think the PHP is executed/rendered when the static site is built, so no PHP will end up on the server. (At least, if I understood the idea behind Nikola's php compiler plugin correctly. I've never looked at it...)

Yes, and I intend to catch that and not crash, that is all.

El lun., mar. 7, 2016 3:15, Felix Fontein notifications@github.com
escribió:

> I guess it should be exported as php code, since after all we have a php
> compile plugin.
> 
> @ralsina https://github.com/ralsina: there's already stuff which
> handles empty posts. The problem is that this post is not empty, but since
> Nikola tries to convert it to MarkDown, it feeds it through a XML/HTML
> parser (lxml), which dies because it thinks the post is "empty".
> 
> —
> Reply to this email directly or view it on GitHub
> https://github.com/getnikola/nikola/issues/2263#issuecomment-193119498.

@felixfontein the php plugin just uploads PHP inside a template. It's meant for users to be able to use simple forms, that sort of thing.


Resolving patch:
diff --git a/nikola/plugins/basic_import.py b/nikola/plugins/basic_import.py
--- a/nikola/plugins/basic_import.py
+++ b/nikola/plugins/basic_import.py
@@ -127,9 +127,12 @@ def transform_content(cls, content):
     def write_content(cls, filename, content, rewrite_html=True):
         """Write content to file."""
         if rewrite_html:
-            doc = html.document_fromstring(content)
-            doc.rewrite_links(replacer)
-            content = html.tostring(doc, encoding='utf8')
+            try:
+                doc = html.document_fromstring(content)
+                doc.rewrite_links(replacer)
+                content = html.tostring(doc, encoding='utf8')
+            except etree.ParserError:
+                content = content.encode('utf-8')
         else:
             content = content.encode('utf-8')
 


Test patch: