internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: CTFd/CTFd

Issue:
Not possible to download files as anonymous user
**Environment**:

  - CTFd Version/Commit: 2.0.0 from master
  - Operating System: Debian Stretch, Ubuntu 18.04, ...
  - Web Browser and Version: Firefox 63

**What happened?**

* In admin, set visibility of challenges to public
* Logout
* Open challenge view (`/challenges`)
* Click on a challenge with a file
* Try to download the file (`/files/c378d661d2c9e103c4409cd4c92d801f/alice_bob.py`
* => Error 403

**What did you expect to happen?**

* ...
* Click on a challenge with a file
* Try to download the file
* File downloads

**How to reproduce your issue**

_see above_

**Any associated stack traces or error logs**

_none_


Discussion:
I think the reason is this `if` (or rather the corresponding `else`): 
https://github.com/CTFd/CTFd/blob/master/CTFd/views.py#L191

We fixed this in our fork by removing this `else` and adding a `@check_challenge_visibility` decorator to the function instead.
I just realized that there is a test for this: 
https://github.com/CTFd/CTFd/blob/master/tests/users/test_views.py#L178

So it appears this is intended behavior.  But I am not sure why. If i enable a public challenge list, I also want user's to be able to download files.

I propose to declare this a bug in the "specification" then.

An alternative is to create a separate config switch for that. 
Tests for the way I understood that functionality: 

```python
            set_config('challenge_visibility', 'public')
            # Unauthed user should return 200
            with app.test_client() as client:
                r = client.get(url)

                assert r.status_code == 200
                assert r.get_data(as_text=True) == 'testing file load'

            set_config('challenge_visibility', 'private')
            # Unauthed user should return 403
            with app.test_client() as client:
                r = client.get(url)

                assert r.status_code == 403 or r.status_code == 302
                assert r.get_data(as_text=True) != 'testing file load'
```
I think this is an oversight. Thanks for pointing out the issue!
By the way I don't think wrapping the files handler in check_challenge_visibility is the best solution because that handler also deals with file downloads for regular non-challenge files. 
Good point. 

So it probably makes more sense to include `get_config('challenge_visibility')` somewhere in the check (for challenge files). 

Like changing line 190 to something like this, and no further changes:

`if f.type == 'challenge' and get_config('challenge_visibility') == 'private':`
The function you probably want is:

```
from CTFd.utils.config.visibility import challenges_visible
```

I'll have a fix ready soon :) 
@stefan2904 this should fix your issue. Would be great if you could confirm. Thanks for reporting!

Resolving patch:
diff --git a/CTFd/__init__.py b/CTFd/__init__.py
--- a/CTFd/__init__.py
+++ b/CTFd/__init__.py
@@ -21,7 +21,7 @@
     reload(sys)
     sys.setdefaultencoding("utf-8")
 
-__version__ = '2.0.0'
+__version__ = '2.0.1'
 
 
 class CTFdFlask(Flask):


Test patch:
Repository: CTFd/CTFd

Issue:
Can we condense the url_for calls for themes further? 
url_for for themes files currently doesn't know what theme is being used and so we pass it in. We should probably default it inside the view. But we would need to default the parameters to None and then set it in the function itself. 


Discussion:

Resolving patch:
diff --git a/CTFd/utils/initialization/__init__.py b/CTFd/utils/initialization/__init__.py
--- a/CTFd/utils/initialization/__init__.py
+++ b/CTFd/utils/initialization/__init__.py
@@ -71,6 +71,11 @@ def inject_user():
             return dict(session)
         return dict()
 
+    @app.url_defaults
+    def inject_theme(endpoint, values):
+        if 'theme' not in values and app.url_map.is_endpoint_expecting(endpoint, 'theme'):
+            values['theme'] = ctf_theme()
+
     @app.before_request
     def needs_setup():
         if request.path == url_for('views.setup') or request.path.startswith('/themes'):


Test patch:
diff --git a/tests/api/v1/user/test_hints.py b/tests/api/v1/user/test_hints.py
--- a/tests/api/v1/user/test_hints.py
+++ b/tests/api/v1/user/test_hints.py
@@ -107,12 +107,12 @@ def test_api_hint_admin_access():
         admin = login_as_user(app, "admin")
         register_user(app)
         client = login_as_user(app)
-        r = client.patch('/api/v1/hints/1')
-        assert r.status_code == 302
-        r = client.delete('/api/v1/hints/1')
-        assert r.status_code == 302
+        r = client.patch('/api/v1/hints/1', json="")
+        assert r.status_code == 403
+        r = client.delete('/api/v1/hints/1', json="")
+        assert r.status_code == 403
         r_admin = admin.patch('/api/v1/hints/1', json={"cost": 2})
         assert r_admin.status_code == 200
-        r_admin = admin.delete('/api/v1/hints/1')
+        r_admin = admin.delete('/api/v1/hints/1', json="")
         assert r_admin.status_code == 200
     destroy_ctfd(app)
Repository: CTFd/CTFd

Issue:
get_config return default
get_config(key) should probably be get_config(key, default=None). This helps in some ideas where you want to do different behavior if get_config returns None. 


Discussion:
This was abandoned before b/c the memoization was causing issues but it needs to be re-evaluated. 

Resolving patch:
diff --git a/CTFd/__init__.py b/CTFd/__init__.py
--- a/CTFd/__init__.py
+++ b/CTFd/__init__.py
@@ -22,7 +22,7 @@
     reload(sys)
     sys.setdefaultencoding("utf-8")
 
-__version__ = '2.0.3'
+__version__ = '2.0.4'
 
 
 class CTFdRequest(Request):


Test patch:
Repository: CTFd/CTFd

Issue:
Imports may need to go through the application factory process again
If the CTF has plugins that create tables, those tables may not be re-created when importing/resetting. This might be leading to issues where tables don't exist in the database but are referred to by plugins. 
Imports may need to go through the application factory process again
If the CTF has plugins that create tables, those tables may not be re-created when importing/resetting. This might be leading to issues where tables don't exist in the database but are referred to by plugins. 


Discussion:

Resolving patch:
diff --git a/CTFd/utils/exports/__init__.py b/CTFd/utils/exports/__init__.py
--- a/CTFd/utils/exports/__init__.py
+++ b/CTFd/utils/exports/__init__.py
@@ -6,6 +6,7 @@
 from datafreeze.format import SERIALIZERS
 from flask import current_app as app
 from datafreeze.format.fjson import JSONSerializer, JSONEncoder
+from sqlalchemy.exc import OperationalError
 import dataset
 import datafreeze
 import datetime
@@ -159,6 +160,16 @@ def import_ctf(backup, erase=True):
 
     alembic_version = json.loads(backup.open('db/alembic_version.json').read())["results"][0]["version_num"]
     upgrade(revision=alembic_version)
+
+    # Create tables created by plugins
+    try:
+        app.db.create_all()
+    except OperationalError as e:
+        if not postgres:
+            raise e
+        else:
+            print("Allowing error during app.db.create_all() due to Postgres")
+
     members.remove('db/alembic_version.json')
 
     for member in members:


Test patch:
Repository: CTFd/CTFd

Issue:
Sphinx Documentation
CTFds documentation is lacking and we need a system for it. 


Discussion:
Hello there. I was thinking we might want to get a "Documentation" or "Needs Doc" tag for issues. I'm totally up for adding docs, but it would be easier to tag them. 
Sure that makes sense. I created the `documentation` tag. 
I'm going to create the docs folder and start looking at getting the basic documentation flow started.

Resolving patch:
diff --git a/docs/conf.py b/docs/conf.py
--- a/docs/conf.py
+++ b/docs/conf.py
@@ -20,13 +20,13 @@
 # -- Project information -----------------------------------------------------
 
 project = u'CTFd'
-copyright = u'2019, Kevin Chung'
+copyright = u'2019, CTFd LLC'
 author = u'Kevin Chung'
 
 # The short X.Y version
 version = u''
 # The full version, including alpha/beta/rc tags
-release = u'2.1.0'
+release = u'2.1.2'
 
 
 # -- General configuration ---------------------------------------------------
@@ -78,7 +78,7 @@
 # The theme to use for HTML and HTML Help pages.  See the documentation for
 # a list of builtin themes.
 #
-html_theme = 'alabaster'
+html_theme = 'sphinx_rtd_theme'
 
 # Theme options are theme-specific and customize the look and feel of a theme
 # further.  For a list of options available for each theme, see the
@@ -154,7 +154,7 @@
 #  dir menu entry, description, category)
 texinfo_documents = [
     (master_doc, 'CTFd', u'CTFd Documentation',
-     author, 'CTFd', 'One line description of project.',
+     author, 'CTFd', 'A Capture The Flag framework focusing on ease of use and customizability.',
      'Miscellaneous'),
 ]
 


Test patch:
Repository: CTFd/CTFd

Issue:
Exporting always downloads the same zip
**Environment**:

  - CTFd Version/Commit: 2.1.2
  - Operating System: Standard docker-compose launch on Linux
  - Web Browser and Version: Firefox 67.0.3

**What happened?**

When exporting the challenges from `/admin/config#backup`, the same zip downloads as the first export. It only contains the one challenge that was there when the backup was made, although there are around 30 challenges.

**What did you expect to happen?**

The export should download a zip file with all of the current challenges.

**How to reproduce your issue**

Export multiple times with different numbers of challenges.




Discussion:
I ran into this too. I worked around it by disabling caching in my browser using developer tools. Ideally the cache header sent to the browser would have a very short TTL.

Resolving patch:
diff --git a/CTFd/admin/__init__.py b/CTFd/admin/__init__.py
--- a/CTFd/admin/__init__.py
+++ b/CTFd/admin/__init__.py
@@ -110,7 +110,9 @@ def export_ctf():
     ctf_name = ctf_config.ctf_name()
     day = datetime.datetime.now().strftime("%Y-%m-%d")
     full_name = u"{}.{}.zip".format(ctf_name, day)
-    return send_file(backup, as_attachment=True, attachment_filename=full_name)
+    return send_file(
+        backup, cache_timeout=-1, as_attachment=True, attachment_filename=full_name
+    )
 
 
 @admin.route("/admin/export/csv")


Test patch:
Repository: CTFd/CTFd

Issue:
get_place should return None instead of 0
The place, get_place attributes/functions should return None instead of 0 if a user/team has no place. 


Discussion:

Resolving patch:
diff --git a/CTFd/models/__init__.py b/CTFd/models/__init__.py
--- a/CTFd/models/__init__.py
+++ b/CTFd/models/__init__.py
@@ -358,7 +358,7 @@ def get_place(self, admin=False, numeric=False):
                 k = i % 10
                 return "%d%s" % (i, "tsnrhtdd"[(i / 10 % 10 != 1) * (k < 4) * k :: 4])
         except ValueError:
-            return 0
+            return None
 
 
 class Admins(Users):
@@ -486,7 +486,7 @@ def get_place(self, admin=False):
             k = i % 10
             return "%d%s" % (i, "tsnrhtdd"[(i / 10 % 10 != 1) * (k < 4) * k :: 4])
         except ValueError:
-            return 0
+            return None
 
 
 class Submissions(db.Model):
diff --git a/populate.py b/populate.py
--- a/populate.py
+++ b/populate.py
@@ -184,6 +184,12 @@
     '.jav', '.pl', '.bak', '.gho', '.old', '.ori', '.tmp', '.dmg',
     '.iso', '.toa', '.vcd', '.gam', '.nes', '.rom', '.sav', '.msi',
 ]
+companies = [
+    'Corp',
+    'Inc.',
+    'Squad',
+    'Team',
+]
 
 
 def gen_sentence():
@@ -206,6 +212,10 @@ def gen_category():
     return random.choice(categories)
 
 
+def gen_affiliation():
+    return (random.choice(hipsters) + " " + random.choice(companies)).title()
+
+
 def gen_value():
     return random.choice(range(100, 500, 50))
 
@@ -223,6 +233,10 @@ def random_date(start, end):
         seconds=random.randint(0, int((end - start).total_seconds())))
 
 
+def random_chance():
+    return random.random() > 0.5
+
+
 if __name__ == '__main__':
     with app.app_context():
         db = app.db
@@ -274,6 +288,10 @@ def random_date(start, end):
                     name=name,
                     password="password"
                 )
+                if random_chance():
+                    team.affiliation = gen_affiliation()
+                if random_chance():
+                    team.oauth_id = random.randint(1, 1000)
                 db.session.add(team)
                 count += 1
 
@@ -295,6 +313,10 @@ def random_date(start, end):
                         password='password'
                     )
                     user.verified = True
+                    if random_chance():
+                        user.affiliation = gen_affiliation()
+                    if random_chance():
+                        user.oauth_id = random.randint(1, 1000)
                     if mode == 'teams':
                         user.team_id = random.randint(1, TEAM_AMOUNT)
                     db.session.add(user)


Test patch:
Repository: CTFd/CTFd

Issue:
import will crash ctfd
<!--
If this is a bug report please fill out the template below.

If this is a feature request please describe the behavior that you'd like to see.
-->

**Environment**:

  - CTFd Version/Commit: 2.1.3
  - Operating System: ubuntu 18.04
  - Web Browser and Version: Opera 60.0.3255.170

**What happened?**
trying to import db (zip file)
**What did you expect to happen?**
it would import db (zip file)
**How to reproduce your issue**

**Any associated stack traces or error logs**
Failed to disable foreign key checks. Continuing.
Error: No support for ALTER of constraints in SQLite dialect

I believe it's Alembic fault  


Discussion:
Can you provide an export that reproduces this behavior or some kind of stack trace? Imports are tested for this kind of scenario so it's not really clear what you're seeing. 
I had the same import crash. If you use mark-2.1.3 docker image, you'll get the "Failed to disable foreign key checks..." message and the DB is broken. If you use 2.1.2 or latest docker image as below, it doesn't show the messabe but the DB is broken.

Steps:
1. docker run -p 8000:8000 -it ctfd/ctfd (or ctfd/ctfd:mark-2.1.3)
2. http://localhost:8000 -> create user "admin" with a password
3. login as admin
4. go to Admin Panel -> back up -> export (zip file downloaded)
5. Admin Panel -> import -> give the downloaded zip file -> [import]

Result:
DB is broken. The following http requests all return HTTP 50x.
I confirmed docker image 'ctfd/ctfd:2.0.6' doesn't have the problem. It looks the import feature was broken after 2.0.6.
Are you using SQLite for your database backend? 
I've reproduced this. Fix should be available shortly. 

Resolving patch:
diff --git a/migrations/versions/b5551cd26764_add_captain_column_to_teams.py b/migrations/versions/b5551cd26764_add_captain_column_to_teams.py
--- a/migrations/versions/b5551cd26764_add_captain_column_to_teams.py
+++ b/migrations/versions/b5551cd26764_add_captain_column_to_teams.py
@@ -30,7 +30,11 @@
 def upgrade():
     # ### commands auto generated by Alembic - please adjust! ###
     op.add_column('teams', sa.Column('captain_id', sa.Integer(), nullable=True))
-    op.create_foreign_key('team_captain_id', 'teams', 'users', ['captain_id'], ['id'])
+
+    bind = op.get_bind()
+    url = str(bind.engine.url)
+    if url.startswith('sqlite') is False:
+        op.create_foreign_key('team_captain_id', 'teams', 'users', ['captain_id'], ['id'])
 
     connection = op.get_bind()
     for team in connection.execute(teams_table.select()):


Test patch:
Repository: CTFd/CTFd

Issue:
Theme reset from CLI
There needs to be a way to reset the theme and other kinds of unrecoverable errors from the CLI. 

Context: https://community.majorleaguecyber.org/t/ctfd-templatenotfound/51


Discussion:

Resolving patch:
diff --git a/manage.py b/manage.py
--- a/manage.py
+++ b/manage.py
@@ -3,6 +3,7 @@
 from flask_script import Manager
 from flask_migrate import Migrate, MigrateCommand
 from CTFd import create_app
+from CTFd.utils import get_config as get_config_util, set_config as set_config_util
 from CTFd.models import *
 
 app = create_app()
@@ -10,5 +11,18 @@
 manager = Manager(app)
 manager.add_command('db', MigrateCommand)
 
-if __name__ == '__main__':
+
+@manager.command
+def get_config(key):
+    with app.app_context():
+        print(get_config_util(key))
+
+
+@manager.command
+def set_config(key, value):
+    with app.app_context():
+        print(set_config_util(key, value).value)
+
+
+if __name__ == "__main__":
     manager.run()


Test patch:
Repository: CTFd/CTFd

Issue:
Re-enable jinja cache
```
    @app.before_request
    def disable_jinja_cache():
        app.jinja_env.cache = {}
```

Explore how we can re-enable the Jinja cache by perhaps storing it on Redis. This may require very custom code. 


Discussion:
Related PR: https://github.com/CTFd/CTFd/pull/630
4f081f082636a6f41db02d675e89b4e4138bb6d7 A better solution than emptying the cache on every request. 
Potentially useful gist: https://gist.github.com/rtt/5029885
Looking at implementing the BytecodeCache based off of Flask-Caching runs into the issue that it's not easy to invalidate cached templates because the SimpleCache doesn't seem to have a way to search through keys. 
PR #1119 related but not finished. 
This issue is difficult to close and it could cause a lot of other issues. It technically should be beneficial and would save the disk read but we'd be increasing complexity. 

Resolving patch:
diff --git a/CTFd/__init__.py b/CTFd/__init__.py
--- a/CTFd/__init__.py
+++ b/CTFd/__init__.py
@@ -1,6 +1,7 @@
 import datetime
 import os
 import sys
+import weakref
 from distutils.version import StrictVersion
 
 from flask import Flask, Request
@@ -71,11 +72,33 @@ class SandboxedBaseEnvironment(SandboxedEnvironment):
     def __init__(self, app, **options):
         if "loader" not in options:
             options["loader"] = app.create_global_jinja_loader()
-        # Disable cache entirely so that themes can be switched (#662)
-        # If the cache is enabled, switching themes will cause odd rendering errors
-        SandboxedEnvironment.__init__(self, cache_size=0, **options)
+        SandboxedEnvironment.__init__(self, **options)
         self.app = app
 
+    def _load_template(self, name, globals):
+        if self.loader is None:
+            raise TypeError("no loader for this environment specified")
+
+        # Add theme to the LRUCache cache key
+        cache_name = name
+        if name.startswith("admin/") is False:
+            theme = str(utils.get_config("ctf_theme"))
+            cache_name = theme + "/" + name
+
+        # Rest of this code is copied from Jinja
+        # https://github.com/pallets/jinja/blob/master/src/jinja2/environment.py#L802-L815
+        cache_key = (weakref.ref(self.loader), cache_name)
+        if self.cache is not None:
+            template = self.cache.get(cache_key)
+            if template is not None and (
+                not self.auto_reload or template.is_up_to_date
+            ):
+                return template
+        template = self.loader.load(self, name, globals)
+        if self.cache is not None:
+            self.cache[cache_key] = template
+        return template
+
 
 class ThemeLoader(FileSystemLoader):
     """Custom FileSystemLoader that switches themes based on the configuration value"""


Test patch:
Repository: CTFd/CTFd

Issue:
log rotation does not works
In the file https://github.com/CTFd/CTFd/blob/master/CTFd/utils/initialization/__init__.py#L107
the log handlers `logging.handlers.RotatingFileHandler` are not created with the right parameters (see [RotatingFileHandler](https://docs.python.org/3/library/logging.handlers.html#logging.handlers.RotatingFileHandler)) because _if either of maxBytes or backupCount is zero, rollover never occurs_ (`backupCount` parameter is missing)


Discussion:
Related to #844. 

Thanks for pointing this out I suspected there was an issue with rotation. Do you have any suggestions with regards to what `backupCount` should be? 

Resolving patch:
diff --git a/CTFd/utils/initialization/__init__.py b/CTFd/utils/initialization/__init__.py
--- a/CTFd/utils/initialization/__init__.py
+++ b/CTFd/utils/initialization/__init__.py
@@ -102,11 +102,13 @@ def init_logs(app):
                 open(log, "a").close()
 
         submission_log = logging.handlers.RotatingFileHandler(
-            logs["submissions"], maxBytes=10000
+            logs["submissions"], maxBytes=10485760, backupCount=5
+        )
+        login_log = logging.handlers.RotatingFileHandler(
+            logs["logins"], maxBytes=10485760, backupCount=5
         )
-        login_log = logging.handlers.RotatingFileHandler(logs["logins"], maxBytes=10000)
         registration_log = logging.handlers.RotatingFileHandler(
-            logs["registrations"], maxBytes=10000
+            logs["registrations"], maxBytes=10485760, backupCount=5
         )
 
         logger_submissions.addHandler(submission_log)


Test patch:
Repository: CTFd/CTFd

Issue:
unlocks api does not check already unlocked hints 
There is not check in the unlocks api for already unlocked hints in the file [unlocks.py](https://github.com/CTFd/CTFd/blob/master/CTFd/api/v1/unlocks.py)

It is possible to unlock multiple times the same hint by just calling the api.


Discussion:
Hmm how did you identify this issue? This wouldn't normally show up during usage because the hint endpoints are always hit before the unlock endpoints. 

Resolving patch:
diff --git a/CTFd/api/v1/unlocks.py b/CTFd/api/v1/unlocks.py
--- a/CTFd/api/v1/unlocks.py
+++ b/CTFd/api/v1/unlocks.py
@@ -59,6 +59,16 @@ def post(self):
         if response.errors:
             return {"success": False, "errors": response.errors}, 400
 
+        existing = Unlocks.query.filter_by(**req).first()
+        if existing:
+            return (
+                {
+                    "success": False,
+                    "errors": {"target": "You've already unlocked this this target"},
+                },
+                400,
+            )
+
         db.session.add(response.data)
 
         award_schema = AwardSchema()


Test patch:
diff --git a/tests/api/v1/user/test_hints.py b/tests/api/v1/user/test_hints.py
--- a/tests/api/v1/user/test_hints.py
+++ b/tests/api/v1/user/test_hints.py
@@ -95,6 +95,27 @@ def test_api_hint_unlocked():
     destroy_ctfd(app)
 
 
+def test_api_hint_double_unlock():
+    """Can a target hint be unlocked twice"""
+    app = create_ctfd()
+    with app.app_context():
+        chal = gen_challenge(app.db)
+        gen_hint(app.db, chal.id, content="This is a hint", cost=1, type="standard")
+        register_user(app)
+        # Give user points with an award
+        gen_award(app.db, 2)
+        client = login_as_user(app)
+        r = client.get("/api/v1/hints/1")
+        assert r.status_code == 200
+        r = client.post("/api/v1/unlocks", json={"target": 1, "type": "hints"})
+        assert r.status_code == 200
+        r = client.get("/api/v1/hints/1")
+        assert r.status_code == 200
+        r = client.post("/api/v1/unlocks", json={"target": 1, "type": "hints"})
+        assert r.status_code == 400
+    destroy_ctfd(app)
+
+
 def test_api_hints_admin_access():
     """Can the users access /api/v1/hints if not admin"""
     app = create_ctfd()