Repository: CTFd/CTFd

Issue:
Hints can't be unlocked when challenge name has special charactes (SQLite)
I've been running a Czech CTF for a couple days now. Users have been complaining about the inability to buy hints. I investigated and found out that the presence of characters like "ž" or "é" in the challenge name does not play nice with SQLite when buying hints. I'm attaching the error message. The challenge name was _Cože? Jaké heslo?_  (means _What password?_). Removing the Czech characters fixed the problem, but it forces me to not use correct Czech.

Instance details:
 - Debian 9 (stretch)
 - CTFd-1.0.3
 - SQLite (libsqlite3-0:amd64 - 3.16.2-5)


```
ProgrammingError: (sqlite3.ProgrammingError) You must not use 8-bit bytestrings unless you use a text_factory that can interpret 8-bit bytestrings (like text_factory = str). It is highly recommended that you instead just switch your application to Unicode strings. [SQL: u'INSERT INTO awards (teamid, name, description, date, value, category, icon) VALUES (?, ?, ?, ?, ?, ?, ?)'] [parameters: (24, 'Hint for Co\xc5\xbee? Jak\xc3\xa9 heslo?', None, '2017-09-07 09:46:29.728823', -25, None, None)]
```


Discussion:
Hmm.... unicode has been fairly well tested in master. Are you using the specific 1.0.3 release or using master? 

A new version will be coming out soon. 
Actually I can reproduce this on master right now. Thanks for reporting this. 
I just played around with this a bit. I am not sure what the solution is in Python 2 besides switching the text_factory. 

But! With Python 3 you don't run into this issue. If you can, you should try to switch to Python 3. 

EDIT:

I just tried this on Python 2 and modified all the columns to use `db.Unicode` and it seems like it didn't fix it. 
Oh, right! Thanks for your help :-)
I should also say that while testing in Python 3, I needed to clear the
cookies that were saved in the browser from Python 2.
On Fri, Sep 8, 2017 at 3:07 AM Martin Černáč <notifications@github.com>
wrote:

> Oh, right! Thanks for your help :-)
>
> —
> You are receiving this because you commented.
>
>
> Reply to this email directly, view it on GitHub
> <https://github.com/CTFd/CTFd/issues/377#issuecomment-328019180>, or mute
> the thread
> <https://github.com/notifications/unsubscribe-auth/AAKJvd9i42DnWHU0hmOiGp0RnAnhWV-tks5sgOfCgaJpZM4PPlKw>
> .
>

Yes, that completely solved the issue I initially reported.

However - upon switching to Python 3, there are some issues with e-mails (using my own mailserver). I'm attaching what I did to migrate to Python 3 and what happened, when I tried to reset my password via e-mail.

I'm still using the same configuration, I also executed:
```
apt install python3-dev python3-pip
pip uninstall gunicorn
pip3 install gunicorn
pip3 install -r CTFd-1.0.3/requirements.txt
```

The error:
```
[2017-09-08 10:08:03,547] ERROR in app: Exception on /reset_password [POST]
Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 1982, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 1614, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 1517, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/local/lib/python3.5/dist-packages/flask/_compat.py", line 33, in reraise
    raise value
  File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/CTFd-1.0.3/CTFd/auth.py", line 97, in reset_password
    """.format(url_for('auth.reset_password', _external=True), urllib.quote_plus(token.encode('base64')))
AttributeError: module 'urllib' has no attribute 'quote_plus'
```

So I can't really use Python 3 because it seems to break more stuff
Yeah master has a variety of Python 3 fixes that you would need to use. The 1.0.3 release is behind and there will be a new release this weekend. 
Okay @octaroot here is the solution:

In `challenges.py`
Look for this line:
```python
award = Awards(teamid=session['id'], name='Hint for {}'.format(chal.name), value=(-hint.cost))
```

and change it to:

```python
award = Awards(teamid=session['id'], name=unicode('Hint for {}'.format(chal.name)), value=(-hint.cost))
```

There will be a better fix for this in master. 

Resolving patch:
diff --git a/CTFd/challenges.py b/CTFd/challenges.py
--- a/CTFd/challenges.py
+++ b/CTFd/challenges.py
@@ -11,6 +11,7 @@
 from CTFd.plugins.challenges import get_chal_class
 
 from CTFd import utils
+from CTFd.utils import text_type
 
 challenges = Blueprint('challenges', __name__)
 
@@ -40,7 +41,7 @@ def hints_view(hintid):
             if team.score() < hint.cost:
                 return jsonify({'errors': 'Not enough points'})
             unlock = Unlocks(model='hints', teamid=session['id'], itemid=hint.id)
-            award = Awards(teamid=session['id'], name='Hint for {}'.format(chal.name), value=(-hint.cost))
+            award = Awards(teamid=session['id'], name=text_type('Hint for {}'.format(chal.name)), value=(-hint.cost))
             db.session.add(unlock)
             db.session.add(award)
             db.session.commit()
diff --git a/CTFd/utils.py b/CTFd/utils.py
--- a/CTFd/utils.py
+++ b/CTFd/utils.py
@@ -31,6 +31,13 @@
 
 from CTFd.models import db, WrongKeys, Pages, Config, Tracking, Teams, Files, ip2long, long2ip
 
+if six.PY2:
+    text_type = unicode
+    binary_type = str
+else:
+    text_type = str
+    binary_type = bytes
+
 cache = Cache()
 migrate = Migrate()
 markdown = mistune.Markdown()


Test patch:
diff --git a/tests/helpers.py b/tests/helpers.py
--- a/tests/helpers.py
+++ b/tests/helpers.py
@@ -3,6 +3,14 @@
 from sqlalchemy_utils import database_exists, create_database, drop_database
 from sqlalchemy.engine.url import make_url
 import datetime
+import six
+
+if six.PY2:
+    text_type = unicode
+    binary_type = str
+else:
+    text_type = str
+    binary_type = bytes
 
 
 def create_ctfd(ctf_name="CTFd", name="admin", email="admin@ctfd.io", password="password", setup=True):
diff --git a/tests/user/test_challenges.py b/tests/user/test_challenges.py
--- a/tests/user/test_challenges.py
+++ b/tests/user/test_challenges.py
@@ -81,6 +81,7 @@ def test_chals_solves():
             ''')
             received = json.loads(output)
             assert saved == received
+    destroy_ctfd(app)
 
 
 def test_submitting_correct_flag():
@@ -185,3 +186,45 @@ def test_submitting_flags_with_large_ips():
             assert resp.get('status') == 1 and resp.get('message') == "Correct"
             assert Solves.query.filter_by(ip=ip_address).first()
     destroy_ctfd(app)
+
+
+def test_unlocking_hints_with_no_cost():
+    '''Test that hints with no cost can be unlocked'''
+    app = create_ctfd()
+    with app.app_context():
+        register_user(app)
+        chal = gen_challenge(app.db)
+        chal_id = chal.id
+        hint = gen_hint(app.db, chal_id)
+
+        client = login_as_user(app)
+        with client.session_transaction() as sess:
+            data = {
+                "nonce": sess.get('nonce')
+            }
+        r = client.post('/hints/1', data=data)
+        output = r.get_data(as_text=True)
+        output = json.loads(output)
+        assert output.get('hint') == 'This is a hint'
+    destroy_ctfd(app)
+
+
+def test_unlocking_hint_for_unicode_challenge():
+    '''Test that hints for challenges with unicode names can be unlocked'''
+    app = create_ctfd()
+    with app.app_context():
+        register_user(app)
+        chal = gen_challenge(app.db, name=text_type('🐺'))
+        chal_id = chal.id
+        hint = gen_hint(app.db, chal_id)
+
+        client = login_as_user(app)
+        with client.session_transaction() as sess:
+            data = {
+                "nonce": sess.get('nonce')
+            }
+        r = client.post('/hints/1', data=data)
+        output = r.get_data(as_text=True)
+        output = json.loads(output)
+        assert output.get('hint') == 'This is a hint'
+    destroy_ctfd(app)