Repository: sissbruecker/linkding

Issue:
web_archive_snapshot_url not showing up in REST API
I enable Internet Archive integration in my Settings and have access to the archived URLs by hovering the timestamp in the Dashboard, but am not sure why `web_archive_snapshot_url` is empty for all my links. The Django admin also doesn't show the snapshot URLs.

Did I overlook anything? Thanks.

(I'm running 1.36.0 on railway)


Discussion:
Since [1.31.1](https://github.com/sissbruecker/linkding/releases/tag/v1.31.1) the UI will dynamically generate a "fallback" link to the web archive if no specific snapshot URL is stored in the bookmark. But that is not the case for the API currently. I guess it should.

For some background: In general, new snapshots on the IA are only created when you add a single bookmark, either via the UI or the API. It doesn't create snapshots for bookmarks that you have imported though. I removed that functionality as that created a large number of background tasks which then consistently failed due to rate limiting with the web archive API. It also didn't make sense to generate, and then link to, a snapshot from a website in it's current form if the bookmark was added two years ago. So instead it just generates a dynamic link to a snapshot from the time the bookmark was created, and then the IA will usually show you the nearest one. Worst case no snapshot exists but that should be rare.

I guess your issue is with bookmark that were imported then? Otherwise something would be wrong with background tasks, then you could check `background_tasks.log` for errors.
Makes sense. Indeed I imported loads of links into Linkding for a test drive. 



Patch:
diff --git a/bookmarks/api/serializers.py b/bookmarks/api/serializers.py
index 153341e8..124a55b1 100644
--- a/bookmarks/api/serializers.py
+++ b/bookmarks/api/serializers.py
@@ -10,6 +10,7 @@
     enhance_with_website_metadata,
 )
 from bookmarks.services.tags import get_or_create_tag
+from bookmarks.services.wayback import generate_fallback_webarchive_url
 
 
 class TagListField(serializers.ListField):
@@ -59,9 +60,10 @@ class Meta:
 
     # Custom tag_names field to allow passing a list of tag names to create/update
     tag_names = TagListField(required=False)
-    # Custom fields to return URLs for favicon and preview image
+    # Custom fields to generate URLs for favicon, preview image, and web archive snapshot
     favicon_url = serializers.SerializerMethodField()
     preview_image_url = serializers.SerializerMethodField()
+    web_archive_snapshot_url = serializers.SerializerMethodField()
     # Add dummy website title and description fields for backwards compatibility but keep them empty
     website_title = serializers.SerializerMethodField()
     website_description = serializers.SerializerMethodField()
@@ -82,6 +84,12 @@ def get_preview_image_url(self, obj: Bookmark):
         preview_image_url = request.build_absolute_uri(preview_image_file_path)
         return preview_image_url
 
+    def get_web_archive_snapshot_url(self, obj: Bookmark):
+        if obj.web_archive_snapshot_url:
+            return obj.web_archive_snapshot_url
+
+        return generate_fallback_webarchive_url(obj.url, obj.date_added)
+
     def get_website_title(self, obj: Bookmark):
         return None
 


Test patch:
diff --git a/bookmarks/tests/test_bookmarks_api.py b/bookmarks/tests/test_bookmarks_api.py
index 21667732..c53967fb 100644
--- a/bookmarks/tests/test_bookmarks_api.py
+++ b/bookmarks/tests/test_bookmarks_api.py
@@ -1,15 +1,18 @@
+import datetime
 import urllib.parse
 from collections import OrderedDict
 from unittest.mock import patch
 
 from django.contrib.auth.models import User
 from django.urls import reverse
+from django.utils import timezone
 from rest_framework import status
 from rest_framework.authtoken.models import Token
 from rest_framework.response import Response
 
 from bookmarks.models import Bookmark, BookmarkSearch, UserProfile
 from bookmarks.services import website_loader
+from bookmarks.services.wayback import generate_fallback_webarchive_url
 from bookmarks.services.website_loader import WebsiteMetadata
 from bookmarks.tests.helpers import LinkdingApiTestCase, BookmarkFactoryMixin
 
@@ -33,7 +36,10 @@ def assertBookmarkListEqual(self, data_list, bookmarks):
             expectation["title"] = bookmark.title
             expectation["description"] = bookmark.description
             expectation["notes"] = bookmark.notes
-            expectation["web_archive_snapshot_url"] = bookmark.web_archive_snapshot_url
+            expectation["web_archive_snapshot_url"] = (
+                bookmark.web_archive_snapshot_url
+                or generate_fallback_webarchive_url(bookmark.url, bookmark.date_added)
+            )
             expectation["favicon_url"] = (
                 f"http://testserver/static/{bookmark.favicon_file}"
                 if bookmark.favicon_file
@@ -590,6 +596,23 @@ def test_get_bookmark_with_more_details(self):
         response = self.get(url, expected_status_code=status.HTTP_200_OK)
         self.assertBookmarkListEqual([response.data], [bookmark])
 
+    def test_get_bookmark_returns_fallback_webarchive_url(self):
+        self.authenticate()
+        bookmark = self.setup_bookmark(
+            web_archive_snapshot_url="",
+            url="https://example.com/",
+            added=timezone.datetime(
+                2023, 8, 11, 21, 45, 11, tzinfo=datetime.timezone.utc
+            ),
+        )
+
+        url = reverse("bookmarks:bookmark-detail", args=[bookmark.id])
+        response = self.get(url, expected_status_code=status.HTTP_200_OK)
+        self.assertEqual(
+            response.data["web_archive_snapshot_url"],
+            "https://web.archive.org/web/20230811214511/https://example.com/",
+        )
+
     def test_update_bookmark(self):
         self.authenticate()
         bookmark = self.setup_bookmark()