Repository: torchbox/wagtail-webstories (language: python)

Issue:
url_hash field in ExternalStory model is too short
For example, for this webstory: 

https://www-pcgamesn-com.cdn.ampproject.org/c/s/www.pcgamesn.com/amp-stories/best-battle-royale-games.html

The value returned by hexdigest is "8e3616b35b2f4d30f7dae987e23468ded00858c1", which has more than 32 characters so it produces an

(1406, "Data too long for column 'url_hash' at row 1")

error when publishing in wagtail.

I believe the author of the code had MD5 in mind, which does return a 32 character length hexdigest but it is instead using sha1, so I suggest changing to MD5 in the get_for_url method in the ExternalStory model, or increasing max character length for url_hash to 40.


Discussion / hints:

Fix patch:
diff --git a/wagtail_webstories/migrations/0002_extend_url_hash.py b/wagtail_webstories/migrations/0002_extend_url_hash.py
new file mode 100644
index 0000000..35819ec
--- /dev/null
+++ b/wagtail_webstories/migrations/0002_extend_url_hash.py
@@ -0,0 +1,18 @@
+# Generated by Django 2.2.17 on 2021-05-27 10:23
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('wagtail_webstories', '0001_initial'),
+    ]
+
+    operations = [
+        migrations.AlterField(
+            model_name='externalstory',
+            name='url_hash',
+            field=models.CharField(db_index=True, editable=False, max_length=40, unique=True),
+        ),
+    ]
diff --git a/wagtail_webstories/models.py b/wagtail_webstories/models.py
index cd15e92..a6ed266 100644
--- a/wagtail_webstories/models.py
+++ b/wagtail_webstories/models.py
@@ -439,7 +439,7 @@ def get_story_page_models():
 class ExternalStory(models.Model):
     url = models.TextField()
     # a SHA-1 hash of the URL
-    url_hash = models.CharField(max_length=32, editable=False, unique=True, db_index=True)
+    url_hash = models.CharField(max_length=40, editable=False, unique=True, db_index=True)
     title = models.TextField(blank=True, editable=False)
     publisher = models.TextField(blank=False, editable=False)
     publisher_logo_src = models.TextField('Publisher logo URL', blank=True, editable=False)


Test patch:
diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml
index a52bd7c..bf936a9 100644
--- a/.github/workflows/test.yml
+++ b/.github/workflows/test.yml
@@ -7,11 +7,12 @@ on:
   pull_request:
 
 # Current configuration:
-# - django 2.2, python 3.6, wagtail 2.5
-# - django 2.2, python 3.7, wagtail 2.7
-# - django 3.0, python 3.8, wagtail 2.10
-# - django 3.1, python 3.8, wagtail 2.11
-# - django 3.1, python 3.9, wagtail main (allow failures)
+# - django 2.2, python 3.6, wagtail 2.5, sqlite
+# - django 2.2, python 3.7, wagtail 2.7, postgres
+# - django 3.0, python 3.8, wagtail 2.11, postgres
+# - django 3.1, python 3.8, wagtail 2.12, sqlite
+# - django 3.2, python 3.9, wagtail 2.13, postgres
+# - django 3.2, python 3.9, wagtail main, sqlite (allow failures)
 jobs:
   test:
     runs-on: ubuntu-latest
@@ -22,24 +23,41 @@ jobs:
           - python: 3.6
             django: "Django>=2.2,<2.3"
             wagtail: "wagtail>=2.5,<2.6"
+            database: "sqlite3"
             experimental: false
           - python: 3.7
             django: "Django>=2.2,<2.3"
             wagtail: "wagtail>=2.7,<2.8"
+            database: "postgresql"
             experimental: false
           - python: 3.8
             django: "Django>=3.0,<3.1"
-            wagtail: "wagtail>=2.10,<2.11"
+            wagtail: "wagtail>=2.11,<2.12"
+            database: "postgresql"
             experimental: false
           - python: 3.8
             django: "Django>=3.1,<3.2"
-            wagtail: "wagtail>=2.11,<2.12"
+            wagtail: "wagtail>=2.12,<2.13"
+            database: "sqlite3"
+            experimental: false
+          - python: 3.8
+            django: "Django>=3.2,<3.3"
+            wagtail: "wagtail>=2.13,<2.14"
+            database: "postgresql"
             experimental: false
           - python: 3.9
-            django: "Django>=3.1,<3.2"
+            django: "Django>=3.2,<3.3"
             wagtail: "git+https://github.com/wagtail/wagtail.git@main#egg=wagtail"
+            database: "sqlite3"
             experimental: true
 
+    services:
+      postgres:
+        image: postgres:10.8
+        ports:
+          - 5432:5432
+        options: --health-cmd pg_isready --health-interval 10s --health-timeout 5s --health-retries 5
+
     steps:
       - uses: actions/checkout@v2
       - name: Set up Python ${{ matrix.python }}
@@ -49,8 +67,14 @@ jobs:
       - name: Install dependencies
         run: |
           python -m pip install --upgrade pip
+          pip install "psycopg2>=2.6"
           pip install "${{ matrix.django }}"
           pip install "${{ matrix.wagtail }}"
           pip install -e .[testing]
       - name: Test
         run: ./runtests.py
+        env:
+          DATABASE_ENGINE: django.db.backends.${{ matrix.database }}
+          DATABASE_HOST: localhost
+          DATABASE_USER: postgres
+          DATABASE_PASS: postgres
diff --git a/tests/settings.py b/tests/settings.py
index 5bdd80a..0596cc1 100644
--- a/tests/settings.py
+++ b/tests/settings.py
@@ -79,10 +79,24 @@
 # Database
 # https://docs.djangoproject.com/en/{{ docs_version }}/ref/settings/#databases
 
+db_engine = os.environ.get('DATABASE_ENGINE', 'django.db.backends.sqlite3'),
+if db_engine == 'django.db.backends.sqlite3':
+    default_db_name = os.path.join(BASE_DIR, 'db.sqlite3')
+else:
+    default_db_name = 'wagtail_webstories'
+
+
 DATABASES = {
     'default': {
-        'ENGINE': 'django.db.backends.sqlite3',
-        'NAME': os.path.join(BASE_DIR, 'db.sqlite3'),
+        'ENGINE': db_engine,
+        'NAME': os.environ.get('DATABASE_NAME', default_db_name),
+        'USER': os.environ.get('DATABASE_USER', None),
+        'PASSWORD': os.environ.get('DATABASE_PASS', None),
+        'HOST': os.environ.get('DATABASE_HOST', None),
+
+        'TEST': {
+            'NAME': os.environ.get('DATABASE_NAME', None),
+        }
     }
 }