Repository: fossasia/open-event-server

Issue:
Error during create event when location is "Wrocław"
``` log
UnicodeEncodeError: 'ascii' codec can't encode character u'\u0142' in position 4: ordinal not in range(128)
ERROR:open_event:Exception on /events/1/ [GET]
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1817, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1477, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/local/lib/python2.7/dist-packages/flask_restplus/api.py", line 536, in error_router
    return original_handler(e)
  File "/usr/local/lib/python2.7/dist-packages/flask_cors/extension.py", line 188, in wrapped_function
    return cors_after_request(app.make_response(f(*args, **kwargs)))
  File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1381, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1475, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1461, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/usr/local/lib/python2.7/dist-packages/flask_admin/base.py", line 68, in inner
    return self._run_view(f, *args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/flask_admin/base.py", line 367, in _run_view
    return fn(self, *args, **kwargs)
  File "/vagrant/open_event/views/admin/models_views/events.py", line 146, in details_view
    checklist=checklist)
  File "/usr/local/lib/python2.7/dist-packages/flask_admin/base.py", line 307, in render
    return render_template(template, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/flask/templating.py", line 128, in render_template
    context, ctx.app)
  File "/usr/local/lib/python2.7/dist-packages/flask/templating.py", line 110, in _render
    rv = template.render(context)
  File "/usr/local/lib/python2.7/dist-packages/jinja2/environment.py", line 989, in render
    return self.environment.handle_exception(exc_info, True)
  File "/usr/local/lib/python2.7/dist-packages/jinja2/environment.py", line 754, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "/vagrant/open_event/templates/gentelella/admin/event/details/details.html", line 1, in top-level template code
    {% extends 'gentelella/admin/event/event_base.html' %}
  File "/vagrant/open_event/templates/gentelella/admin/event/event_base.html", line 25, in top-level template code
    <li{% if id == active_page %} class="active"{% endif %}><a href="{{ href|e }}">{{ caption|e }}</a></li>
  File "/vagrant/open_event/templates/gentelella/admin/base.html", line 46, in top-level template code
    {% block body %}
  File "/vagrant/open_event/templates/gentelella/admin/event/event_base.html", line 80, in block "body"
    {% block content %}
  File "/vagrant/open_event/templates/gentelella/admin/event/details/details.html", line 28, in block "content"
    {% include 'gentelella/admin/event/details/widgets/general_info.html' %}
  File "/vagrant/open_event/templates/gentelella/admin/event/details/widgets/general_info.html", line 5, in top-level template code
    {% set versions_count = count_versions(event) %}
  File "/vagrant/open_event/templates/gentelella/admin/event/details/base_panel.html", line 15, in top-level template code
    {% block panel_content %}
  File "/vagrant/open_event/templates/gentelella/admin/event/details/widgets/general_info.html", line 102, in block "panel_content"
    {% for diff in side_by_side_diff(changeset[name]) %}
  File "/vagrant/open_event/__init__.py", line 174, in side_by_side_diff
    for side_by_side_diff_entry in side_by_side_diff(changeset_entry[0], changeset_entry[1]):
  File "/vagrant/open_event/helpers/versioning.py", line 58, in side_by_side_diff
    new_text = strip_tags(strip_line_breaks(str(new_text)))
UnicodeEncodeError: 'ascii' codec can't encode character u'\u0142' in position 4: ordinal not in range(128)
10.0.2.2 - - [26/Jun/2016 10:27:17] "GET /events/1/ HTTP/1.1" 500 -
```



Discussion:
@rafalkowalski 
Even import OTS json is failing because the text has some non-ascii characters. This used to work before. I wonder what recent change has caused this. 
(I did a successful OTS import on 22nd june)


Resolving patch:
diff --git a/open_event/helpers/versioning.py b/open_event/helpers/versioning.py
--- a/open_event/helpers/versioning.py
+++ b/open_event/helpers/versioning.py
@@ -6,11 +6,11 @@
 
 
 def remove_line_breaks(target_string):
-    return str(target_string).replace('\r', '')
+    return target_string.replace('\r', '')
 
 
 def strip_line_breaks(target_string):
-    return str(target_string).replace('\n', '').replace('\r', '')
+    return target_string.replace('\n', '').replace('\r', '')
 
 
 def clean_up_string(target_string):
@@ -54,8 +54,8 @@ def side_by_side_diff(old_text, new_text):
     if not new_text:
         new_text = ''
 
-    old_text = strip_tags(strip_line_breaks(str(old_text)))
-    new_text = strip_tags(strip_line_breaks(str(new_text)))
+    old_text = strip_tags(strip_line_breaks(unicode(old_text).encode('utf-8', errors='ignore')))
+    new_text = strip_tags(strip_line_breaks(unicode(new_text).encode('utf-8', errors='ignore')))
 
     def yield_open_entry(open_entry):
         """ Yield all open changes. """


Test patch: