internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: digitalfabrik/integreat-cms

Issue:
Autosave changes unversioned `Page` model
### Describe the Bug
<!-- A clear and concise description of what the bug is. -->
When an autosave is triggered, the complete `Page` and `PageTranslation` model instances are saved.
Since just the `PageTranslation` is versioned, this might lead to unexpected changes on public pages.

### Steps to Reproduce

1. Go to a page form
2. Edit something in the content textarea to trigger the tinymce autosave
3. Change some field in the page form, e.g. embed live content
4. Wait for the autosave to get triggered
5. Return without saving

### Expected Behavior
<!-- A clear and concise description of what you expected to happen. -->
All page fields should be unchanged, since they are not versioned and there is no backup of the "public" state. If the page field is exited without saving, the page model instance should be exactly the same as before.

### Actual Behavior
<!-- A clear and concise description of what actually happened. -->
All fields are automatically saved and the page instance is saved as well.

### Additional Information
<!-- Add any other context (e.g. logs, screenshots, etc.) about the problem here. -->
Probably, this can be solved by limiting the `FormData` in [src/frontend/js/forms/autosave.ts](https://github.com/Integreat/integreat-cms/blob/8219586acbb98c681635210d0c45817285a0c13e/src/frontend/js/forms/autosave.ts#L7) so only the fields `slug`, `title` and `text`/`description` are saved.
_Edit:_ Probably not, since this would cause the validation to fail or remove specific values... so maybe we need a completely new view for the autosave?


Discussion:

Resolving patch:
diff --git a/src/cms/views/pages/page_view.py b/src/cms/views/pages/page_view.py
--- a/src/cms/views/pages/page_view.py
+++ b/src/cms/views/pages/page_view.py
@@ -244,8 +244,13 @@ def post(self, request, *args, **kwargs):
             messages.info(request, _("No changes detected, autosave skipped"))
 
         else:
-            # Save forms
-            page_translation_form.instance.page = page_form.save()
+            # Only save page form if page does not yet exist or if translation is no auto save
+            if (
+                not page_instance
+                or page_translation_form.instance.status != status.AUTO_SAVE
+            ):
+                page_translation_form.instance.page = page_form.save()
+            # Save page translation form
             page_translation_form.save()
             # Add the success message and redirect to the edit page
             if not page_instance:


Test patch:
Repository: digitalfabrik/integreat-cms

Issue:
Filtering for POIs in event list not working
### Describe the Bug
<!-- A clear and concise description of what the bug is. -->
When filters for POIs in the event list are applied, all events are returned, independent from their location

### Steps to Reproduce

1. Go to event list
2. Filter for event location

### Expected Behavior
<!-- A clear and concise description of what you expected to happen. -->
Only the events with the selected location should be shown

### Actual Behavior
<!-- A clear and concise description of what actually happened. -->
All events are shown

### Additional Information
<!-- Add any other context (e.g. logs, screenshots, etc.) about the problem here. -->




Discussion:

Resolving patch:
diff --git a/integreat_cms/cms/views/events/event_form_view.py b/integreat_cms/cms/views/events/event_form_view.py
--- a/integreat_cms/cms/views/events/event_form_view.py
+++ b/integreat_cms/cms/views/events/event_form_view.py
@@ -65,7 +65,10 @@ def get(self, request, *args, **kwargs):
         recurrence_rule_instance = RecurrenceRule.objects.filter(
             event=event_instance
         ).first()
-        poi_instance = region.pois.filter(events=event_instance).first()
+        if event_instance:
+            poi_instance = event_instance.location
+        else:
+            poi_instance = None
 
         # Make form disabled if event is archived or user doesn't have the permission to edit the event
         if event_instance and event_instance.archived:


Test patch:
Repository: digitalfabrik/integreat-cms

Issue:
Page tree broken after searching and going back
### Describe the Bug
When using the back button of the browser after searching, the page tree is expanded. After that, the collapsing function is broken.

Can we tell the browser to reload the full page after using the back button? However, it would be the best solution to actually keep the status of the tree after using the back button.

~~This could be related to #1131~~


Discussion:
This is probably browser-specific cache, see e.g. https://stackoverflow.com/questions/49547/how-do-we-control-web-page-caching-across-all-browsers
e.g. for firefox, here is a bit of background information about their backward/forward-navigation caching techniques: https://developer.mozilla.org/en-US/docs/Mozilla/Firefox/Releases/1.5/Using_Firefox_1.5_caching

> [Firefox 1.5](https://developer.mozilla.org/en-US/docs/Mozilla/Firefox/Releases/1.5) uses in-memory caching for entire Web pages, including their JavaScript states, for a single browser session.
I think we should use the meta tag approach and add the following tags to the page tree DOM tree.

```
<meta http-equiv="Cache-Control" content="no-cache, no-store, must-revalidate">
<meta http-equiv="Pragma" content="no-cache">
<meta http-equiv="Expires" content="0">
```
Just tested this locally and unfortunately, the meta tags do not have an effect on the in-memory cache of the DOM tree.
Instead, we need the HTTP headers, I think this SO answer explains it very well: https://stackoverflow.com/a/5493543

Resolving patch:
diff --git a/integreat_cms/cms/views/pages/page_tree_view.py b/integreat_cms/cms/views/pages/page_tree_view.py
--- a/integreat_cms/cms/views/pages/page_tree_view.py
+++ b/integreat_cms/cms/views/pages/page_tree_view.py
@@ -107,7 +107,7 @@ def get(self, request, *args, **kwargs):
             filter_form = PageFilterForm()
             filter_form.changed_data.clear()
 
-        return render(
+        response = render(
             request,
             self.template_name,
             {
@@ -118,6 +118,9 @@ def get(self, request, *args, **kwargs):
                 "filter_form": filter_form,
             },
         )
+        # Disable browser cache of page tree to prevent subpages from being expanded after using "back"-button
+        response["Cache-Control"] = "no-store, must-revalidate"
+        return response
 
     def post(self, request, *args, **kwargs):
         r"""


Test patch:
Repository: digitalfabrik/integreat-cms

Issue:
PDF Export URL pattern
### Describe the Bug
The web app calls `/REGION/LANG/wp-json/ig-mpdf/v1/pdf` to export a PDF which returns a 404. Our API currently uses `REGION/LANG/pdf`.

The normal mapping does not work, as we

### Steps to Reproduce

```shell
curl 'https://malte-test.tuerantuer.org/joerdenstorf/de/wp-json/ig-mpdf/v1/pdf'
```

### Expected Behavior
Map old URL pattern to new endpoint.


### Actual Behavior
404


Discussion:

Resolving patch:
diff --git a/integreat_cms/api/urls.py b/integreat_cms/api/urls.py
--- a/integreat_cms/api/urls.py
+++ b/integreat_cms/api/urls.py
@@ -136,6 +136,11 @@
                     "<slug:language_slug>/wp-json/extensions/v3/",
                     include(content_api_urlpatterns),
                 ),
+                path(
+                    "<slug:language_slug>/wp-json/ig-mpdf/v1/pdf/",
+                    pdf_export,
+                    name="pdf_export",
+                ),
             ]
         ),
     ),


Test patch:
Repository: digitalfabrik/integreat-cms

Issue:
Language Tree isn't handled correctly by cache in Page Tree
### Describe the Bug
So far, our Table rows aren't consistent after we delete a language tree node and recreate it. We should probably empty the cacheops cache after we delete a language tree node.



Discussion:

Resolving patch:
diff --git a/integreat_cms/cms/forms/language_tree/language_tree_node_form.py b/integreat_cms/cms/forms/language_tree/language_tree_node_form.py
--- a/integreat_cms/cms/forms/language_tree/language_tree_node_form.py
+++ b/integreat_cms/cms/forms/language_tree/language_tree_node_form.py
@@ -4,6 +4,8 @@
 from django.utils.text import capfirst
 from django.utils.translation import ugettext_lazy as _
 
+from cacheops import invalidate_obj
+
 from ..custom_model_form import CustomModelForm
 from ..custom_tree_node_form import CustomTreeNodeForm
 from ...models import Language, LanguageTreeNode
@@ -119,3 +121,25 @@ def clean(self):
             "LanguageTreeNodeForm validated [2] with cleaned data %r", cleaned_data
         )
         return cleaned_data
+
+    def save(self, commit=True):
+        """
+        This method extends the default ``save()``-method of the base :class:`~django.forms.ModelForm` to flush
+        the cache after commiting.
+
+        :param commit: Whether or not the changes should be written to the database
+        :type commit: bool
+
+        :return: The saved page translation object
+        :rtype: ~integreat_cms.cms.models.pages.page_translation.PageTranslation
+        """
+        # Save CustomModelForm and flush Cache
+        result = super().save(commit=commit)
+
+        for page in self.instance.region.pages.all():
+            invalidate_obj(page)
+        for poi in self.instance.region.pois.all():
+            invalidate_obj(poi)
+        for event in self.instance.region.events.all():
+            invalidate_obj(event)
+        return result


Test patch:
Repository: digitalfabrik/integreat-cms

Issue:
Disable Matomo tracking setting
### Motivation
We want our test system to work normally, but we do not want to track API requests in the statistics. We should have a setting that only disables tracking API requests.


### Proposed Solution
Add a Setting to disable the tracking function (decorator).


### Alternatives
Use a different Matomo.


Discussion:

Resolving patch:
diff --git a/integreat_cms/api/decorators.py b/integreat_cms/api/decorators.py
--- a/integreat_cms/api/decorators.py
+++ b/integreat_cms/api/decorators.py
@@ -160,7 +160,11 @@ def wrap(request, *args, **kwargs):
         :return: The response of the given function or an 404 :class:`~django.http.JsonResponse`
         :rtype: ~django.http.JsonResponse
         """
-        if not request.region.matomo_id or not request.region.matomo_token:
+        if (
+            not settings.MATOMO_TRACKING
+            or not request.region.matomo_id
+            or not request.region.matomo_token
+        ):
             return func(request, *args, **kwargs)
         data = {
             "idsite": request.region.matomo_id,
diff --git a/integreat_cms/core/settings.py b/integreat_cms/core/settings.py
--- a/integreat_cms/core/settings.py
+++ b/integreat_cms/core/settings.py
@@ -31,6 +31,11 @@
     "INTEGREAT_CMS_MATOMO_URL", "https://statistics.integreat-app.de"
 )
 
+#: Enable tracking of API requests in Matomo
+MATOMO_TRACKING = bool(
+    strtobool(os.environ.get("INTEGREAT_CMS_MATOMO_TRACKING", "False"))
+)
+
 #: The slug for the legal notice (see e.g. :class:`~integreat_cms.cms.models.pages.imprint_page_translation.ImprintPageTranslation`)
 IMPRINT_SLUG = "imprint"
 


Test patch:
Repository: digitalfabrik/integreat-cms

Issue:
Push notifications can be sent multiple times
### Describe the Bug
<!-- A clear and concise description of what the bug is. -->
It should not be possible to send push notifications multiple times to prevent the accidental re-sending of news.
Instead, only the "save" button should be enabled which can be used to update the messages that are delivered via the API endpoint.

### Steps to Reproduce

1. Go to News
2. Send new push notification
3. Send again

### Expected Behavior
<!-- A clear and concise description of what you expected to happen. -->
The news should only be sent the first time and only saved on all subsequent submissions

### Actual Behavior
<!-- A clear and concise description of what actually happened. -->
The news is sent twice



Discussion:

Resolving patch:
diff --git a/integreat_cms/cms/views/push_notifications/push_notification_form_view.py b/integreat_cms/cms/views/push_notifications/push_notification_form_view.py
--- a/integreat_cms/cms/views/push_notifications/push_notification_form_view.py
+++ b/integreat_cms/cms/views/push_notifications/push_notification_form_view.py
@@ -192,7 +192,7 @@ def post(self, request, *args, **kwargs):
                     ),
                 )
 
-            if "submit_send" in request.POST:
+            if "submit_send" in request.POST and not pn_form.instance.sent_date:
                 if not request.user.has_perm("cms.send_push_notification"):
                     logger.warning(
                         "%r does not have the permission to send %r",


Test patch:
Repository: digitalfabrik/integreat-cms

Issue:
Cache does not get invalidated correctly when LanguageTreeNode is deleted
### Describe the Bug
<!-- A clear and concise description of what the bug is. -->
If you delete a language the page tree still shows the translation status of the corresponding language. This is probably a caching issue.

Cache does not get invalidated correctly when LanguageTreeNode is deleted
### Describe the Bug
<!-- A clear and concise description of what the bug is. -->
If you delete a language the page tree still shows the translation status of the corresponding language. This is probably a caching issue.



Discussion:

Resolving patch:
diff --git a/integreat_cms/cms/views/language_tree/language_tree_actions.py b/integreat_cms/cms/views/language_tree/language_tree_actions.py
--- a/integreat_cms/cms/views/language_tree/language_tree_actions.py
+++ b/integreat_cms/cms/views/language_tree/language_tree_actions.py
@@ -11,6 +11,7 @@
 from django.db import transaction
 
 from treebeard.exceptions import InvalidPosition, InvalidMoveToDescendant
+from cacheops import invalidate_obj
 
 from ...constants import position
 from ...decorators import permission_required
@@ -128,6 +129,16 @@ def delete_language_tree_node(request, region_slug, language_tree_node_id):
 
     logger.debug("%r deleted by %r", language_node, request.user)
     language_node.delete()
+
+    for page in region.pages.all():
+        invalidate_obj(page)
+    for event in region.events.all():
+        invalidate_obj(event)
+    for poi in region.pois.all():
+        invalidate_obj(poi)
+    for push_notification in region.push_notifications.all():
+        invalidate_obj(push_notification)
+
     messages.success(
         request,
         _(


Test patch:
Repository: digitalfabrik/integreat-cms

Issue:
API: Wordpress slug compatibility
### Motivation
Django does not allow special characters in slugs, while WordPress does allow them. We're stripping them during the migration. It would be nice if links all over the internet and within our content to pages with special characters in their slugs do not break.


### Proposed Solution
If a slug transmitted via the API does not match a translation, run the slugify method on it and then try again to match. This is the same operation we perform on slags after migrating. Alternatively, the slugify method could be run on all slugs if the compute time is not too long?


### Alternatives
Old links break or need to be fixed manually over time.


### Additional Context
https://issues.tuerantuer.org/browse/INFRA-161




Discussion:

Resolving patch:
diff --git a/integreat_cms/api/v3/pages.py b/integreat_cms/api/v3/pages.py
--- a/integreat_cms/api/v3/pages.py
+++ b/integreat_cms/api/v3/pages.py
@@ -9,6 +9,7 @@
 from django.shortcuts import get_object_or_404
 from django.views.decorators.csrf import csrf_exempt
 from django.utils.html import strip_tags
+from django.utils.text import slugify
 
 from ...cms.models import Page
 from ...cms.forms import PageTranslationForm
@@ -142,7 +143,7 @@ def get_single_page(request, language_slug):
         # Strip leading and trailing slashes to avoid ambiguous urls
         url = request.GET.get("url").strip("/")
         # The last path component of the url is the page translation slug
-        page_translation_slug = url.split("/")[-1]
+        page_translation_slug = slugify(url.split("/")[-1], allow_unicode=True)
         # Get page by filtering for translation slug and translation language slug
         filtered_pages = region.pages.filter(
             translations__slug=page_translation_slug,


Test patch:
Repository: digitalfabrik/integreat-cms

Issue:
Separate XLIFF export options
### Motivation
<!-- A clear and concise description of what the motivation for the new feature is, and what problem it is solving. -->
Apparently, the two options to export XLIFF files (either all pages or only public pages) are too similar and lead to accidental misclicks.

### Proposed Solution
<!-- A clear and concise description of the feature you would like to add, and how it solves the motivating problem. -->
- Move the public option to the top (since this is the most used)
- Reword the "all pages" option to make it more obvious that this includes drafts
- Maybe add an exclamation mark or similar to the "all pages" option

### Alternatives
<!-- A clear and concise description of any alternative solutions or features you've considered, and why you're proposed solution is better. -->


### Additional Context
<!-- Add any other information or screenshots about the feature request here. -->




Discussion:
@svenseeberg An additional idea: We could also include the selected option in the success message here:
https://github.com/digitalfabrik/integreat-cms/blob/2db58e4e4318f2504b571f738704132c075e872a/integreat_cms/cms/views/pages/page_bulk_actions.py#L103
e.g. "XLIFF file for published translations ..." vs  "XLIFF file for published and unpublished translations" depending on the value of `only_public`?

Resolving patch:
diff --git a/integreat_cms/cms/views/pages/page_bulk_actions.py b/integreat_cms/cms/views/pages/page_bulk_actions.py
--- a/integreat_cms/cms/views/pages/page_bulk_actions.py
+++ b/integreat_cms/cms/views/pages/page_bulk_actions.py
@@ -100,9 +100,13 @@ def post(self, request, *args, **kwargs):
             messages.success(
                 request,
                 __(
-                    _("XLIFF file for translation to {} successfully created.").format(
-                        target_language
-                    ),
+                    _(
+                        "XLIFF file with published pages only for translation to {} successfully created."
+                    ).format(target_language)
+                    if self.only_public
+                    else _(
+                        "XLIFF file with unpublished and published pages for translation to {} successfully created."
+                    ).format(target_language),
                     _(
                         "If the download does not start automatically, please click {}here{}."
                     ).format(


Test patch:
Repository: digitalfabrik/integreat-cms

Issue:
Statistics numbers don't add up
### Describe the Bug
When downloading the statistics numbers from the cms, the numbers in all columns don't add up to the total shown in the total column.

### Steps to Reproduce

1. Go to 'Kreis Bergstraße'
2. Click on 'Analyse.'
3. Select a timeframe of a few months
4. Set 'Auswertung' to 'monatlich'
5. Download as csv
6. Add up the rows minus the total column
8. See error

### Expected Behavior
Total column and added up individual cells show the same value


### Actual Behavior
They show a different value


### Additional Information
![image](https://user-images.githubusercontent.com/64901227/184635082-41e52c8c-6cdf-449a-b6f6-9df20fbe87c3.png)



Discussion:
~~I think the problem here is that the numbers per language are filtered by the `/pages/` subpath, while the total number is not filtered by that. That means that the total amount probably also includes events etc.~~ We only collect children and pages endpoint calls in our statistics. That means the sum of all numbers should add up.
Okay I think I found the issue, which is probably hard to solve and always existed: The problem comes from the method of Matomo to count unique visitors. In some cases the requests from multiple users are combined into one unique visitor, for example when by chance the random IP is the same. This is why the sum of the languages + offline downloads is always larger than the total amount of visitors. One easy way to avoid the issue is by summing up the languages in Django and ignoring the total number from Matomo. Alternatively we may find other end points that provide the actual number of requests. `VisitsSummary.getActions` may provide better numbers (https://developer.matomo.org/api-reference/reporting-api#api-method-list).

Resolving patch:
diff --git a/integreat_cms/cms/utils/matomo_api_manager.py b/integreat_cms/cms/utils/matomo_api_manager.py
--- a/integreat_cms/cms/utils/matomo_api_manager.py
+++ b/integreat_cms/cms/utils/matomo_api_manager.py
@@ -190,7 +190,7 @@ def get_total_visits(self, start_date, end_date, period=matomo_periods.DAY):
         query_params = {
             "date": f"{start_date},{end_date}",
             "idSite": self.matomo_id,
-            "method": "VisitsSummary.getVisits",
+            "method": "VisitsSummary.getActions",
             "period": period,
         }
 
@@ -298,7 +298,7 @@ def get_visits_per_language(self, start_date, end_date, period):
             "filter_limit": "-1",
             "format_metrics": "1",
             "idSite": self.matomo_id,
-            "method": "VisitsSummary.getVisits",
+            "method": "VisitsSummary.getActions",
             "period": period,
         }
         logger.debug(


Test patch: