CVE-2021-39286: Improper Neutralization of Input During Web Page Generation ('Cross-site Scripting') (CWE-79)
value: Webrecorder pywb before 2.6.0 allows XSS because it does not ensure that Jinja2 templates are autoescaped.
CWE description: The product does not neutralize or incorrectly neutralizes user-controllable input before it is placed in output that is used as a web page that is served to other users.
CVSS v3 base score: 6.1
Published: 2021-08-18T18:15Z
Repository: https://github.com/webrecorder/pywb
Language: HTML
Files changed: CHANGES.rst, config.yaml, docs/manual/access-control.rst, docs/manual/localization.rst, pywb/manager/locmanager.py, pywb/rewrite/templateview.py, pywb/templates/base.html, pywb/templates/error.html, pywb/templates/frame_insert.html, pywb/templates/head_insert.html, pywb/templates/header.html, pywb/templates/index.html, pywb/version.py, setup.py, tests/test_auto_colls.py
Commit date: 2021-06-14 17:09:00 -0700
Commit message: Localization / doc fixes (#650)

* localization / doc fixes:
- add missing header.html
- docs: support 'i18n' extra, mention in docs
- use 'default_locale' for html lang tag
- access control docs: fix documentation for adding user with acl command

* localization: add compile_catalog after extract as well to simplify updates for identity (en) locale

* ui: 
- include locale in home page collection listing
- keep locale on error page home link

* autoescape:
- ensure jinja2 templates are autoescaped to prevent xss issues (thanks @sebastian-nagel for suggested fix)
- ensure banner inserts are not double-escaped
- update tests for template autoescaping

* update CHANGES.rst

* bump version to 2.6.0b1


Diff: diff --git a/CHANGES.rst b/CHANGES.rst
index 112c6cf..352bfeb 100644
--- a/CHANGES.rst
+++ b/CHANGES.rst
@@ -1,5 +1,19 @@
+pywb 2.6.0b1 changelist
+~~~~~~~~~~~~~~~~~~~~~~~
+
+Additional documentation / localization fixes `#650 <https://github.com/webrecorder/pywb/pull/650>`_
+
+* Ensure home page and error page keeps locale, language switching is working.
+
+* Add autoescaping to Jinja2 to avoid XSS issues (suggested by @sebastian-nagel)
+
+* Add support for 'pywb[i18n]' extra to install localization dependencies
+
+Documentation typo fixes (by @ldko, `#649 <https://github.com/webrecorder/pywb/pull/649>`_)
+
+
 pywb 2.6.0b0 changelist
 ~~~~~~~~~~~~~~~~~~~~~~~
 
 Documentation Updates:
 
diff --git a/config.yaml b/config.yaml
index 5522c68..01827eb 100644
--- a/config.yaml
+++ b/config.yaml
@@ -15,8 +15,5 @@ use_js_obj_proxy: true
 enable_memento: true
 
 # Replay content in an iframe
 framed_replay: true
 
-locales:
-  - en
-  - es
diff --git a/docs/manual/access-control.rst b/docs/manual/access-control.rst
index 6dce1c7..8b1c576 100644
--- a/docs/manual/access-control.rst
+++ b/docs/manual/access-control.rst
@@ -185,11 +185,11 @@ The URL supplied can be a URL or a SURT prefix. If a SURT is supplied, it is use
   wb-manager acl add <collection> com, allow
 
 
 A specific user for user-based rules can also be specified, for example to add ``allow_ignore_embargo`` for user ``staff`` only, run::
 
-  wb-manager acl add <collection> http://httpbin.org/anything/something allow_ignore_embargo staff
+  wb-manager acl add <collection> http://httpbin.org/anything/something allow_ignore_embargo -u staff
 
 
 By default, access control rules apply to a prefix of a given URL or SURT.
 
 To have the rule apply only to the exact match, use::
diff --git a/docs/manual/localization.rst b/docs/manual/localization.rst
index adfd501..8055a3d 100644
--- a/docs/manual/localization.rst
+++ b/docs/manual/localization.rst
@@ -7,14 +7,19 @@ pywb supports configuring different language locales and loading different langu
 
 pywb can extract all text from templates and generate CSV files for translation and convert them back into a binary format used for localization/internationalization.
 
 (pywb uses the `Babel library <http://babel.pocoo.org/en/latest/>`_  which extends the `standard Python i18n system <https://docs.python.org/3/library/gettext.html>`_)
 
+To ensure all localization related dependencies are installed, first run::
+
+  pip install pywb[i18n]
+
 Locales to use are configured in the ``config.yaml``.
 
 The command-line ``wb-manager`` utility provides a way to manage locales for translation, including generating extracted text, and to update translated text.
 
+
 Adding a Locale and Extracting Text
 ===================================
 
 To add a new locale for translation and automatically extract all text that needs to be translated, run::
 
diff --git a/pywb/manager/locmanager.py b/pywb/manager/locmanager.py
index c031ac1..fcbe034 100644
--- a/pywb/manager/locmanager.py
+++ b/pywb/manager/locmanager.py
@@ -39,10 +39,12 @@ class LocManager:
                 base = os.path.join(TRANSLATIONS, loc, 'LC_MESSAGES')
                 po = os.path.join(base, 'messages.po')
                 csv = os.path.join(base, 'messages.csv')
                 po2csv([po, csv])
 
+        self.compile_catalog()
+
     def update_loc(self, locale, no_csv):
         for loc in locale:
             if not no_csv:
                 loc_dir = os.path.join(TRANSLATIONS, loc)
                 base = os.path.join(TRANSLATIONS, loc, 'LC_MESSAGES')
diff --git a/pywb/rewrite/templateview.py b/pywb/rewrite/templateview.py
index 122e18d..4a38815 100644
--- a/pywb/rewrite/templateview.py
+++ b/pywb/rewrite/templateview.py
@@ -3,11 +3,11 @@ from warcio.timeutils import timestamp_now
 
 from pywb.utils.loaders import load
 
 from six.moves.urllib.parse import urlsplit, quote
 
-from jinja2 import Environment, TemplateNotFound, contextfunction
+from jinja2 import Environment, TemplateNotFound, contextfunction, select_autoescape
 from jinja2 import FileSystemLoader, PackageLoader, ChoiceLoader
 
 from babel.support import Translations
 
 from webassets.ext.jinja2 import AssetsExtension
@@ -75,14 +75,16 @@ class JinjaEnv(object):
         if assets_path:
             extensions.append(AssetsExtension)
 
         if overlay:
             jinja_env = overlay.jinja_env.overlay(loader=loader,
+                                                  autoescape=select_autoescape(),
                                                   trim_blocks=True,
                                                   extensions=extensions)
         else:
             jinja_env = RelEnvironment(loader=loader,
+                                       autoescape=select_autoescape(),
                                        trim_blocks=True,
                                        extensions=extensions)
 
         jinja_env.filters.update(self.filters)
 
diff --git a/pywb/templates/base.html b/pywb/templates/base.html
index 0bb02c2..2160a8b 100644
--- a/pywb/templates/base.html
+++ b/pywb/templates/base.html
@@ -1,7 +1,7 @@
 <!DOCTYPE html>
-<html lang="{{ env.pywb_lang | default('en') }}">
+<html lang="{{ env.pywb_lang | default(default_locale) }}">
     <head>
         <meta http-equiv="content-type" content="text/html; charset=UTF-8;charset=utf-8"/>
         <meta name="viewport" content="width=device-width, initial-scale=1">
 
         <title>{% block title %}{% endblock %}</title>
diff --git a/pywb/templates/error.html b/pywb/templates/error.html
index 9b2f229..63e2c68 100644
--- a/pywb/templates/error.html
+++ b/pywb/templates/error.html
@@ -11,11 +11,11 @@
             <p class="lead">{% trans %}Access Blocked to {{ err_msg }}{% endtrans %}</p>
 
         {% elif err_status == 404 and err_details == 'coll_not_found' %}
             <p>{% trans %}Collection not found: <b>{{ err_msg }}{% endtrans %}</b></p>
 
-            <p><a href="/">{{ _('See list of valid collections') }}</a></p>
+            <p><a href="/{{ env.pywb_lang | default('') }}">{{ _('See list of valid collections') }}</a></p>
 
         {% elif err_status == 404 and err_details == 'static_file_not_found' %}
             <p>{% trans %}Static file not found: <b>{{ err_msg }}{% endtrans %}</b></p>
 
         {% else %}
diff --git a/pywb/templates/frame_insert.html b/pywb/templates/frame_insert.html
index 544cb76..af67eed 100644
--- a/pywb/templates/frame_insert.html
+++ b/pywb/templates/frame_insert.html
@@ -12,12 +12,16 @@ html, body
 }
 
 </style>
 <script src='{{ static_prefix }}/wb_frame.js'> </script>
 
+{% autoescape false %}
+
 {{ banner_html }}
 
+{% endautoescape %}
+
 </head>
 <body style="margin: 0px; padding: 0px;">
 
 <div id="wb_iframe_div">
 <iframe id="replay_iframe" frameborder="0" seamless="seamless" scrolling="yes" class="wb_iframe" allow="autoplay; fullscreen"></iframe>
diff --git a/pywb/templates/head_insert.html b/pywb/templates/head_insert.html
index 924c46b..f49d73a 100644
--- a/pywb/templates/head_insert.html
+++ b/pywb/templates/head_insert.html
@@ -1,5 +1,7 @@
+{% autoescape false %}
+
 <!-- WB Insert -->
 <script>
 {% set urlsplit = cdx.url | urlsplit %}
   wbinfo = {};
   wbinfo.top_url = "{{ top_url }}";
@@ -59,7 +61,9 @@
 
 {% endif %}
 
 {{ banner_html }}
 
+{% endautoescape %}
+
 <!-- End WB Insert -->
 
diff --git a/pywb/templates/header.html b/pywb/templates/header.html
new file mode 100644
index 0000000..c45a9e0
--- /dev/null
+++ b/pywb/templates/header.html
@@ -0,0 +1,13 @@
+<header>
+   {% if locales|length > 1 %}
+    <div class="language-select">
+        {{ _('Language:') }}
+        <ul role="listbox" aria-activedescendant="{{ env.pywb_lang | default(default_locale) }}" aria-labelledby="{{ _('Language select') }}">
+        {% for locale in locales %}
+          <li role="option" id="{{ locale }}"><a href="{{ switch_locale(locale) }}">{{ locale }}</a></li>
+        {% endfor %}
+        </ul>
+    </div>
+    {% endif %}
+</header>
+
diff --git a/pywb/templates/index.html b/pywb/templates/index.html
index 338232a..1e1aa92 100644
--- a/pywb/templates/index.html
+++ b/pywb/templates/index.html
@@ -7,11 +7,11 @@
     </div>
     <div class="row">
         <ul>
             {% for route in routes %}
                 <li>
-                    <a href="{{ env['pywb.app_prefix'] + '/' + route }}">{{ '/' + route }}</a>
+                    <a href="{{ env['pywb.app_prefix'] + ('/' + env.pywb_lang if env.pywb_lang else '') + '/' + route }}">{{ '/' + route }}</a>
                     {% if all_metadata and all_metadata[route] %}
                         ({{ all_metadata[route].title }})
                     {% endif %}
                 </li>
             {% endfor %}
diff --git a/pywb/version.py b/pywb/version.py
index 844b041..21f487d 100644
--- a/pywb/version.py
+++ b/pywb/version.py
@@ -1,4 +1,4 @@
-__version__ = '2.6.0b0'
+__version__ = '2.6.0b1'
 
 if __name__ == '__main__':
     print(__version__)
diff --git a/setup.py b/setup.py
index 2d347e8..bf2f721 100755
--- a/setup.py
+++ b/setup.py
@@ -105,10 +105,13 @@ setup(
         ('sample_archive/warcs', glob.glob('sample_archive/warcs/*')),
         ('sample_archive/text_content',
          glob.glob('sample_archive/text_content/*')),
     ],
     install_requires=load_requirements('requirements.txt'),
+    extras_require={
+        "i18n":  ["translate_toolkit"],
+    },
     tests_require=[
         'pytest',
         'WebTest',
         'pytest-cov',
         'mock',
diff --git a/tests/test_auto_colls.py b/tests/test_auto_colls.py
index 2df4813..0c706aa 100644
--- a/tests/test_auto_colls.py
+++ b/tests/test_auto_colls.py
@@ -311,19 +311,20 @@ class TestManagedColls(CollsDirMixin, BaseConfigTest):
         resp = self.testapp.get('/test/')
         resp.charset = 'utf-8'
         assert resp.status_int == 200
         assert resp.content_type == 'text/html'
         assert 'overriden search page: ' in resp.text
-        print(resp.text)
-        assert '"some":"value"' in resp.text, resp.text
+        #assert '"some":"value"' in resp.text, resp.text
+        assert '{&#34;some&#34;:&#34;value&#34;}' in resp.text, resp.text
 
     def test_replay_banner_metadata(self, fmod):
         """ Test adding metadata in replay banner (both framed and non-frame)
         """
         resp = self.get('/test/20140103030321{0}/http://example.com/?example=1', fmod)
         assert '<div>Custom Banner Here!</div>' in resp.text
-        assert '"some":"value"' in resp.text
+        #assert '"some":"value"' in resp.text
+        assert '{&#34;some&#34;:&#34;value&#34;}' in resp.text, resp.text
 
     def test_more_custom_templates_replay(self, fmod):
         resp = self.get('/test/20140103030321{0}/http://example.com/?example=1', fmod)
         assert resp.status_int == 200
 

Vulnerable code: locales:
  - en
  - es
  wb-manager acl add <collection> http://httpbin.org/anything/something allow_ignore_embargo staff
from jinja2 import Environment, TemplateNotFound, contextfunction
<html lang="{{ env.pywb_lang | default('en') }}">
            <p><a href="/">{{ _('See list of valid collections') }}</a></p>
                    <a href="{{ env['pywb.app_prefix'] + '/' + route }}">{{ '/' + route }}</a>
__version__ = '2.6.0b0'
        print(resp.text)
        assert '"some":"value"' in resp.text, resp.text
        assert '"some":"value"' in resp.text
Fixed code: pywb 2.6.0b1 changelist
~~~~~~~~~~~~~~~~~~~~~~~

Additional documentation / localization fixes `#650 <https://github.com/webrecorder/pywb/pull/650>`_

* Ensure home page and error page keeps locale, language switching is working.

* Add autoescaping to Jinja2 to avoid XSS issues (suggested by @sebastian-nagel)

* Add support for 'pywb[i18n]' extra to install localization dependencies

Documentation typo fixes (by @ldko, `#649 <https://github.com/webrecorder/pywb/pull/649>`_)


  wb-manager acl add <collection> http://httpbin.org/anything/something allow_ignore_embargo -u staff
To ensure all localization related dependencies are installed, first run::

  pip install pywb[i18n]


        self.compile_catalog()

from jinja2 import Environment, TemplateNotFound, contextfunction, select_autoescape
                                                  autoescape=select_autoescape(),
                                       autoescape=select_autoescape(),
<html lang="{{ env.pywb_lang | default(default_locale) }}">
            <p><a href="/{{ env.pywb_lang | default('') }}">{{ _('See list of valid collections') }}</a></p>
{% autoescape false %}

{% endautoescape %}

{% autoescape false %}

{% endautoescape %}

<header>
   {% if locales|length > 1 %}
    <div class="language-select">
        {{ _('Language:') }}
        <ul role="listbox" aria-activedescendant="{{ env.pywb_lang | default(default_locale) }}" aria-labelledby="{{ _('Language select') }}">
        {% for locale in locales %}
          <li role="option" id="{{ locale }}"><a href="{{ switch_locale(locale) }}">{{ locale }}</a></li>
        {% endfor %}
        </ul>
    </div>
    {% endif %}
</header>

                    <a href="{{ env['pywb.app_prefix'] + ('/' + env.pywb_lang if env.pywb_lang else '') + '/' + route }}">{{ '/' + route }}</a>
__version__ = '2.6.0b1'
    extras_require={
        "i18n":  ["translate_toolkit"],
    },
        #assert '"some":"value"' in resp.text, resp.text
        assert '{&#34;some&#34;:&#34;value&#34;}' in resp.text, resp.text
        #assert '"some":"value"' in resp.text
        assert '{&#34;some&#34;:&#34;value&#34;}' in resp.text, resp.text