internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: ietf-tools/datatracker

Issue:
Group edit form always uses default used_roles
### Describe the issue

When editing a group, the default set of `used_roles` from the `GroupFeatures` for the group type is always used. A group's custom `used_roles` should take precedence.

This is probably caused by incorrect logic here: https://github.com/ietf-tools/datatracker/blob/eda494a2990f2775f0dfc34c096fc0fac16ed1ab/ietf/group/forms.py#L85

### Code of Conduct

- [X] I agree to follow the [IETF's Code of Conduct](https://github.com/ietf-tools/.github/blob/main/CODE_OF_CONDUCT.md)


Discussion:

Resolving patch:
diff --git a/ietf/group/forms.py b/ietf/group/forms.py
--- a/ietf/group/forms.py
+++ b/ietf/group/forms.py
@@ -79,10 +79,10 @@ def __init__(self, *args, **kwargs):
             self.used_roles = self.group.used_roles or group_features.default_used_roles
         else:
             group_features = GroupFeatures.objects.filter(type_id=self.group_type).first()
+            self.used_roles = group_features.default_used_roles
 
         log.assertion('group_features is not None')
         if group_features is not None:
-            self.used_roles = group_features.default_used_roles
             parent_types = group_features.parent_types.all()
             need_parent = group_features.need_parent
             default_parent = group_features.default_parent


Test patch:
diff --git a/ietf/group/tests_info.py b/ietf/group/tests_info.py
--- a/ietf/group/tests_info.py
+++ b/ietf/group/tests_info.py
@@ -16,6 +16,7 @@
 import debug                            # pyflakes:ignore
 
 from django.conf import settings
+from django.test import RequestFactory
 from django.urls import reverse as urlreverse
 from django.urls import NoReverseMatch
 from django.utils import timezone
@@ -1086,6 +1087,16 @@ def test_edit_roles(self):
                     RoleFactory(name_id=rslug, group=group, person=PersonFactory())
                 self.do_edit_roles_test(group)
 
+    def test_used_roles_overrides_default(self):
+        """A group's used_roles should override the GroupFeatures default"""
+        request_factory = RequestFactory()
+        group = GroupFactory(type_id='wg')
+        self.assertEqual(GroupForm(request_factory.get('/some/url'), group=group).used_roles, group.features.default_used_roles)
+        group.used_roles = ['ad', 'chair']
+        group.save()
+        self.assertNotEqual(group.used_roles, group.features.default_used_roles)
+        self.assertEqual(GroupForm(request_factory.get('/some/url'), group=group).used_roles, group.used_roles)
+
     def test_need_parent(self):
         """GroupForm should enforce non-null parent when required"""
         group = GroupFactory()
Repository: ietf-tools/datatracker

Issue:
v1 api crashes on some content when serializing to xml
See the very old ticket at https://github.com/django-tastypie/django-tastypie/issues/1107.

submission.first_two_pages can contain formfeeds. These break tastypie's xml serialization. Json serialization succeeds.

The quick fix is to stop exposing first_two_pages through the API.




Discussion:

Resolving patch:
diff --git a/ietf/submit/resources.py b/ietf/submit/resources.py
--- a/ietf/submit/resources.py
+++ b/ietf/submit/resources.py
@@ -68,6 +68,7 @@ class Meta:
             "group": ALL_WITH_RELATIONS,
             "draft": ALL_WITH_RELATIONS,
         }
+        excludes = ('first_two_pages',)
 api.submit.register(SubmissionResource())
 
 from ietf.person.resources import PersonResource


Test patch:
Repository: ietf-tools/datatracker

Issue:
Chat logs from past meetings not accessible
### Describe the issue

On the agenda page there used to be links to Jabber logs, which have been replaced by links to Zulip _narrows_. Zulip narrows don't have any message archived from past meetings, though. 

See for instance https://datatracker.ietf.org/meeting/111/agenda/



### Code of Conduct

- [X] I agree to follow the [IETF's Code of Conduct](https://github.com/ietf-tools/.github/blob/main/CODE_OF_CONDUCT.md)


Discussion:
I edited the title as the same happens for interims details as well, e.g., https://datatracker.ietf.org/meeting/interim-2022-cbor-11/session/cbor

Resolving patch:
diff --git a/ietf/meeting/models.py b/ietf/meeting/models.py
--- a/ietf/meeting/models.py
+++ b/ietf/meeting/models.py
@@ -1266,10 +1266,13 @@ def chat_room_url(self):
         return settings.CHAT_URL_PATTERN.format(chat_room_name=self.chat_room_name())
 
     def chat_archive_url(self):
-        # Zulip has no separate archive
-        if hasattr(settings,'CHAT_ARCHIVE_URL_PATTERN'):
+        # datatracker 8.8.0 released on 2022 July 15; before that, fall back to old log URL
+        if self.meeting.date <= datetime.date(2022, 7, 15):
+            return f'https://www.ietf.org/jabber/logs/{ self.chat_room_name() }?C=M;O=D'
+        elif hasattr(settings,'CHAT_ARCHIVE_URL_PATTERN'):
             return settings.CHAT_ARCHIVE_URL_PATTERN.format(chat_room_name=self.chat_room_name())
         else:
+            # Zulip has no separate archive
             return self.chat_room_url()
 
     def notes_id(self):


Test patch:
diff --git a/ietf/meeting/tests_models.py b/ietf/meeting/tests_models.py
--- a/ietf/meeting/tests_models.py
+++ b/ietf/meeting/tests_models.py
@@ -1,7 +1,9 @@
 # Copyright The IETF Trust 2021, All Rights Reserved
 # -*- coding: utf-8 -*-
 """Tests of models in the Meeting application"""
-from ietf.meeting.factories import MeetingFactory
+import datetime
+
+from ietf.meeting.factories import MeetingFactory, SessionFactory
 from ietf.stats.factories import MeetingRegistrationFactory
 from ietf.utils.test_utils import TestCase
 
@@ -49,3 +51,10 @@ def test_get_attendance(self):
         self.assertIsNotNone(attendance)
         self.assertEqual(attendance.online, 0)
         self.assertEqual(attendance.onsite, 5)
+
+
+class SessionTests(TestCase):
+    def test_chat_archive_url_with_jabber(self):
+        # datatracker 8.8.0 rolled out on 2022-07-15. Before that, chat logs were jabber logs hosted at www.ietf.org.
+        session_with_jabber = SessionFactory(group__acronym='fakeacronym', meeting__date=datetime.date(2022,7,14))
+        self.assertEqual(session_with_jabber.chat_archive_url(), 'https://www.ietf.org/jabber/logs/fakeacronym?C=M;O=D')
Repository: ietf-tools/datatracker

Issue:
The nomcom settings page does not let the chair set the "accepting volunteers" option
Right now, the secretariat has to set this option on the chair's behalf (using the admin). It should appear on the form at `/nomcom/yyyy/private/edit-nomcom/` as well.


Discussion:
I suspect the first_call_for_volunteers date should also be set if the chair is the first to turn is_accepting_volunteers on.

Resolving patch:
diff --git a/ietf/nomcom/forms.py b/ietf/nomcom/forms.py
--- a/ietf/nomcom/forms.py
+++ b/ietf/nomcom/forms.py
@@ -120,7 +120,7 @@ def __init__(self, *args, **kwargs):
     class Meta:
         model = NomCom
         fields = ('public_key', 'initial_text', 'show_nominee_pictures', 'show_accepted_nominees',
-                  'send_questionnaire', 'reminder_interval')
+                  'send_questionnaire', 'is_accepting_volunteers', 'first_call_for_volunteers', 'reminder_interval')
         widgets = {'public_key':FileInput, }
 
     def clean_public_key(self):
diff --git a/ietf/nomcom/migrations/0013_update_accepting_volunteers_helptext.py b/ietf/nomcom/migrations/0013_update_accepting_volunteers_helptext.py
new file mode 100644
--- /dev/null
+++ b/ietf/nomcom/migrations/0013_update_accepting_volunteers_helptext.py
@@ -0,0 +1,18 @@
+# Generated by Django 2.2.28 on 2022-07-23 13:14
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('nomcom', '0012_populate_volunteers'),
+    ]
+
+    operations = [
+        migrations.AlterField(
+            model_name='nomcom',
+            name='is_accepting_volunteers',
+            field=models.BooleanField(default=False, help_text='Is this nomcom currently accepting volunteers?', verbose_name='Accepting volunteers'),
+        ),
+    ]
diff --git a/ietf/nomcom/models.py b/ietf/nomcom/models.py
--- a/ietf/nomcom/models.py
+++ b/ietf/nomcom/models.py
@@ -60,7 +60,7 @@ class NomCom(models.Model):
     show_accepted_nominees = models.BooleanField(verbose_name='Show accepted nominees', default=True, 
                                                  help_text='Show accepted nominees on the public nomination page')
     is_accepting_volunteers = models.BooleanField(verbose_name="Accepting volunteers", default=False,
-                                                  help_text='Is this nomcom is currently accepting volunteers?')
+                                                  help_text='Is this nomcom currently accepting volunteers?')
     first_call_for_volunteers = models.DateField(verbose_name='Date of the first call for volunteers', blank=True, null=True)
 
     class Meta:


Test patch:
Repository: ietf-tools/datatracker

Issue:
Don't show needed ballot counts on documents that have already been approved
Having a page that says
```
 Summary: Needs one more YES or NO OBJECTION position to pass.
```
on a document that has already been approved is confusing and incorrect.


Discussion:

Resolving patch:
diff --git a/ietf/doc/views_doc.py b/ietf/doc/views_doc.py
--- a/ietf/doc/views_doc.py
+++ b/ietf/doc/views_doc.py
@@ -1216,6 +1216,8 @@ def document_ballot_content(request, doc, ballot_id, editable=True):
         else:
             position_groups.append(g)
 
+    iesg = doc.get_state("draft-iesg")
+    iesg_state = iesg.slug if iesg else None
     if (ballot.ballot_type.slug == "irsg-approve"):
         summary = irsg_needed_ballot_positions(doc, [p for p in positions if not p.is_old_pos])
     else:
@@ -1241,6 +1243,7 @@ def document_ballot_content(request, doc, ballot_id, editable=True):
                                    deferred=deferred,
                                    summary=summary,
                                    all_ballots=all_ballots,
+                                   iesg_state=iesg_state,
                                    ),
                               request=request)
 


Test patch:
Repository: ietf-tools/datatracker

Issue:
In the new agenda, links in the material box don't open in a new window
### Describe the issue
<img width="1410" alt="Screenshot 2022-07-25 at 16 40 37" src="https://user-images.githubusercontent.com/8420764/180804830-1b5d0af8-0c81-4544-96f5-5218ff97c658.png">



### Code of Conduct

- [X] I agree to follow the [IETF's Code of Conduct](https://github.com/ietf-tools/.github/blob/main/CODE_OF_CONDUCT.md)


Discussion:
I clicked on a link in the dispatch agenda to the slides.
Actually the box only show the agenda and not the rest of the material.
And one more issue about the new agenda (might be a separate issue, please decide to split up yourself): I have a setting to always show the left-side menu but it does not show up with the new agenda.
left menu for that page is a known issue
This might be a bit problematic to solve as the contents are rendered inside an iframe, but we could add `<base target="_blank" />` to the page loaded inside the iframe to force all links to open in a new window (or `_parent` to re-use the same window).

Resolving patch:
diff --git a/ietf/meeting/views.py b/ietf/meeting/views.py
--- a/ietf/meeting/views.py
+++ b/ietf/meeting/views.py
@@ -258,7 +258,7 @@ def materials_document(request, document, num=None, ext=None):
                     content_type = content_type.replace('plain', 'markdown', 1)
                     break;
                 elif atype[0] == 'text/html':
-                    bytes = "<html>\n<head></head>\n<body>\n%s\n</body>\n</html>\n" % markdown.markdown(bytes.decode())
+                    bytes = "<html>\n<head><base target=\"_blank\" /></head>\n<body>\n%s\n</body>\n</html>\n" % markdown.markdown(bytes.decode())
                     content_type = content_type.replace('plain', 'html', 1)
                     break;
                 elif atype[0] == 'text/plain':


Test patch:
Repository: ietf-tools/datatracker

Issue:
cvs export of group documents is dumping b'' strings
From Tim Wicinski:
>FYI
>
>So I went and dumped the WG documents via the CSV link:
>
>https://datatracker.ietf.org/group/dnsop/documents/csv/
>
>Then I imported them into the googles sheets.
>The CSV file has all strings as "b'<string>'" which
>makes the import needing a bit of cleanup. 
>
>tim

cc @moonshiner



Discussion:

Resolving patch:
diff --git a/ietf/community/views.py b/ietf/community/views.py
--- a/ietf/community/views.py
+++ b/ietf/community/views.py
@@ -208,7 +208,7 @@ def export_to_csv(request, username=None, acronym=None, group_type=None):
         row.append(str(doc.ad) if doc.ad else "")
         e = doc.latest_event()
         row.append(e.time.strftime("%Y-%m-%d") if e else "")
-        writer.writerow([v.encode("utf-8") for v in row])
+        writer.writerow(row)
 
     return response
 


Test patch:
Repository: ietf-tools/datatracker

Issue:
Can't create new status change
### Describe the issue

Submitting https://datatracker.ietf.org/doc/start-rfc-status-change/ doesn't work, because "Shepherding AD" is a required field. Which it should not be, since there is a state "Needs Shepherd" which allows this to be empty.

### Code of Conduct

- [X] I agree to follow the [IETF's Code of Conduct](https://github.com/ietf-tools/.github/blob/main/CODE_OF_CONDUCT.md)


Discussion:

Resolving patch:
diff --git a/ietf/doc/views_status_change.py b/ietf/doc/views_status_change.py
--- a/ietf/doc/views_status_change.py
+++ b/ietf/doc/views_status_change.py
@@ -465,7 +465,7 @@ class StartStatusChangeForm(forms.Form):
     document_name = forms.CharField(max_length=255, label="Document name", help_text="A descriptive name such as status-change-md2-to-historic is better than status-change-rfc1319.", required=True)
     title = forms.CharField(max_length=255, label="Title", required=True)
     ad = forms.ModelChoiceField(Person.objects.filter(role__name="ad", role__group__state="active",role__group__type='area').order_by('name'), 
-                                label="Shepherding AD", empty_label="(None)", required=True)
+                                label="Shepherding AD", empty_label="(None)", required=False)
     create_in_state = forms.ModelChoiceField(State.objects.filter(type="statchg", slug__in=("needshep", "adrev")), empty_label=None, required=False)
     notify = forms.CharField(max_length=255, label="Notice emails", help_text="Separate email addresses with commas.", required=False)
     telechat_date = forms.TypedChoiceField(coerce=lambda x: datetime.datetime.strptime(x, '%Y-%m-%d').date(), empty_value=None, required=False, widget=forms.Select(attrs={'onchange':'make_bold()'}))
@@ -725,4 +725,4 @@ def last_call(request, name):
                                     last_call_form  = form,
                                    ),
                            )
-               
\ No newline at end of file
+               


Test patch:
Repository: ietf-tools/datatracker

Issue:
Submit page has a link to idnits at author-tools that always produces an error.
Following the current link on `https://datatracker.ietf.org/submit/` to 
```
https://author-tools.ietf.org/api/idnits
```
always results in
```
{"error":"URL is missing"}
```
A different user experience is needed. Perhaps pointing to the main author-tools page and add prose to the datatracker page saying to use the `Validate` functionality there.
Or build a URL at author-tools that allows a friendlier guided experience into validating a document.


Discussion:

Resolving patch:
diff --git a/ietf/settings.py b/ietf/settings.py
--- a/ietf/settings.py
+++ b/ietf/settings.py
@@ -542,6 +542,7 @@ def skip_unreadable_post(record):
 IDTRACKER_BASE_URL = "https://datatracker.ietf.org"
 RFCDIFF_BASE_URL = "https://www.ietf.org/rfcdiff"
 IDNITS_BASE_URL = "https://author-tools.ietf.org/api/idnits"
+IDNITS_SERVICE_URL = "https://author-tools.ietf.org/idnits"
 
 # Content security policy configuration (django-csp)
 CSP_DEFAULT_SRC = ("'self'", "'unsafe-inline'", f"data: {IDTRACKER_BASE_URL} https://www.ietf.org/ https://analytics.ietf.org/ https://fonts.googleapis.com/")


Test patch:
Repository: ietf-tools/datatracker

Issue:
AD field lists also non-ADs
### Describe the issue

On https://datatracker.ietf.org/group/chartering/create/wg/

### Code of Conduct

- [X] I agree to follow the [IETF's Code of Conduct](https://github.com/ietf-tools/.github/blob/main/CODE_OF_CONDUCT.md)


Discussion:
This needs similar treatment as #4276 (see also #3711)

Resolving patch:
diff --git a/ietf/group/forms.py b/ietf/group/forms.py
--- a/ietf/group/forms.py
+++ b/ietf/group/forms.py
@@ -13,6 +13,7 @@
 from django.utils.html import mark_safe # type:ignore
 from django.db.models import F
 from django.core.exceptions import ValidationError, ObjectDoesNotExist
+from django.db.models import Q
 
 # IETF imports
 from ietf.group.models import Group, GroupHistory, GroupStateName, GroupFeatures 
@@ -118,6 +119,10 @@ def __init__(self, *args, **kwargs):
             if fieldname == 'delegate_roles':
                 field_args['max_entries'] = MAX_GROUP_DELEGATES
                 field_args['help_text'] = mark_safe("Chairs can delegate the authority to update the state of group documents - at most %s persons at a given time." % MAX_GROUP_DELEGATES)
+            if role_slug == "ad":
+                field_args['extra_prefetch'] = Email.objects.filter(Q(role__name__in=('pre-ad', 'ad'), role__group__type='area', role__group__state='active')).distinct()
+                field_args['disable_ajax'] = True  # only use the prefetched options
+                field_args['min_search_length'] = 0  # do not require typing to display options
             self.fields[fieldname] = SearchableEmailsField(**field_args)
             self.fields[fieldname].initial = Email.objects.filter(person__role__name_id=role_slug,person__role__group=self.group,person__role__email__pk=F('pk')).distinct()
 
@@ -391,4 +396,4 @@ class ReviewSecretarySettingsForm(forms.ModelForm):
     class Meta:
         model = ReviewSecretarySettings
         fields = ['remind_days_before_deadline', 'max_items_to_show_in_reviewer_list',
-                  'days_to_show_in_reviewer_list']
\ No newline at end of file
+                  'days_to_show_in_reviewer_list']
diff --git a/ietf/utils/fields.py b/ietf/utils/fields.py
--- a/ietf/utils/fields.py
+++ b/ietf/utils/fields.py
@@ -284,7 +284,7 @@ def prepare_value(self, value):
         for d in pre:
             if isinstance(value, list):
                 d["selected"] = any([v.pk == d["id"] for v in value])
-            else:
+            elif value:
                 d["selected"] = value.exists() and value.filter(pk__in=[d["id"]]).exists()
         self.widget.attrs["data-pre"] = json.dumps(list(pre))
 


Test patch:
Repository: ietf-tools/datatracker

Issue:
"Duplicate timeslots" option for new rooms does not respect timezones
### Describe the issue

Reported by @flynnliz,

> On the page [...]/secr/meetings/115/liz-flynn-1/rooms/ , I entered a bunch of rooms. At the bottom of the page, checking the box "Duplicate timeslots from previous meeting for new rooms?" results in creating the previous time slots but in the new time zone — for example, 114’s 10:00 timeslot in Philadelphia becomes 14:00 in 115’s London. (This essentially makes the feature useless, as now every single time slot is at the wrong time. I usually do everything from scratch every meeting anyway so this being broken isn’t a big deal.)

### Code of Conduct

- [X] I agree to follow the [IETF's Code of Conduct](https://github.com/ietf-tools/.github/blob/main/CODE_OF_CONDUCT.md)


Discussion:

Resolving patch:
diff --git a/ietf/secr/meetings/views.py b/ietf/secr/meetings/views.py
--- a/ietf/secr/meetings/views.py
+++ b/ietf/secr/meetings/views.py
@@ -64,15 +64,23 @@ def build_timeslots(meeting,room=None):
         else:
             source_meeting = get_last_meeting(meeting)
 
-        delta = meeting.date - source_meeting.date
         timeslots = []
-        time_seen = set()
+        time_seen = set()  # time of source_meeting timeslot
         for t in source_meeting.timeslot_set.filter(type='regular'):
             if not t.time in time_seen:
                 time_seen.add(t.time)
                 timeslots.append(t)
         for t in timeslots:
-            new_time = t.time + delta
+            # Create new timeslot at the same wall clock time on the same day relative to meeting start
+            day_offset = t.local_start_time().date() - source_meeting.date
+            new_date = meeting.date + day_offset
+            new_time = make_aware(
+                datetime.datetime.combine(
+                    new_date,
+                    t.local_start_time().time(),
+                ),
+                meeting.tz(),
+            )
             for room in rooms:
                 TimeSlot.objects.create(type_id='regular',
                                         meeting=meeting,


Test patch:
Repository: ietf-tools/datatracker

Issue:
post_submission() creates bibxml that likely has incorrect timezone handling
### Describe the issue

The `doc/bibxml.xml` template rendered by `post_submission()` includes
```
      <date month="{{doc.date|date:"F"}}" day="{{doc.date.day}}" year="{{doc.date.year}}" />
```
which will give inconsistent day/year for a few hours at start of the day.

### Code of Conduct

- [X] I agree to follow the [IETF's Code of Conduct](https://github.com/ietf-tools/.github/blob/main/CODE_OF_CONDUCT.md)


Discussion:
The code I quoted there is ok - `doc.date` is a `datetime.date` so it's ok to refer to its day and year fields directly in a template. The code that calls it is now going to be handing it dates computed from the UTC time when the `NewRevisionDocEvent` was created instead of the PST8PDT time. @rjsparks, do we need to worry about that level of precision for this date?
It should match whatever shows for a date for the NewRevisionDocEvent on the history tab for the document.

Resolving patch:
diff --git a/ietf/doc/utils.py b/ietf/doc/utils.py
--- a/ietf/doc/utils.py
+++ b/ietf/doc/utils.py
@@ -13,6 +13,7 @@
 
 from collections import defaultdict, namedtuple
 from urllib.parse import quote
+from zoneinfo import ZoneInfo
 
 from django.conf import settings
 from django.contrib import messages
@@ -1351,14 +1352,15 @@ def bibxml_for_draft(doc, rev=None):
     latest_revision_event = doc.latest_event(NewRevisionDocEvent, type="new_revision")
     latest_revision_rev = latest_revision_event.rev if latest_revision_event else None
     best_events = NewRevisionDocEvent.objects.filter(doc__name=doc.name, rev=(rev or latest_revision_rev))
+    tzinfo = ZoneInfo(settings.TIME_ZONE)
     if best_events.exists():
         # There was a period where it was possible to get more than one NewRevisionDocEvent for a revision.
         # A future data cleanup would allow this to be simplified
         best_event = best_events.order_by('time').first()
         log.assertion('doc.rev == best_event.rev')
-        doc.date = best_event.time.date()
+        doc.date = best_event.time.astimezone(tzinfo).date()
     else:
-        doc.date = doc.time.date()      # Even if this may be incoreect, what would be better?
+        doc.date = doc.time.astimezone(tzinfo).date()      # Even if this may be incorrect, what would be better?
 
     return render_to_string('doc/bibxml.xml', {'name':doc.name, 'doc': doc, 'doc_bibtype':'I-D'})
 


Test patch: