internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: ietf-tools/datatracker

Issue:
Many tests failing with YangModelExtractor error
### Describe the issue

Starting with [test run 790](https://github.com/ietf-tools/datatracker/actions/runs/3172961553), a few dozen tests are failing with the error
```
AttributeError: 'YangModuleExtractor' object has no attribute 'extract_yang_model'
```
The last passing test run was a couple hours before the release of [xym 0.6.0](https://pypi.org/project/xym/0.6.0/#history), so it seems likely this is related.

### Code of Conduct

- [X] I agree to follow the [IETF's Code of Conduct](https://github.com/ietf-tools/.github/blob/main/CODE_OF_CONDUCT.md)


Discussion:
Looks like the `extract_yang_model()` method was replaced with `extract_yang_model_text()` and `extact_yang_model_xml()`

Resolving patch:
diff --git a/ietf/submit/checkers.py b/ietf/submit/checkers.py
--- a/ietf/submit/checkers.py
+++ b/ietf/submit/checkers.py
@@ -147,7 +147,7 @@ def check_file_txt(self, path):
                 saved_stderr = sys.stderr
                 sys.stdout = io.StringIO()
                 sys.stderr = io.StringIO()
-                extractor.extract_yang_model(file.readlines())
+                extractor.extract_yang_model_text(file.read())
                 model_list = extractor.get_extracted_models(False, True)
                 out = sys.stdout.getvalue()
                 err = sys.stderr.getvalue()


Test patch:
Repository: ietf-tools/datatracker

Issue:
Reset to Template Text button on shepherd writeup form generates wrong template type
At doc/draft-ietf-stir-identity-header-errors-handling/edit/shepherdwriteup/, before a writeup was generated, the form presents the template appropriate for a wg document. Pressing the Reset to Template Text button replaces it with the template for an individual submission document.



Discussion:
cc @bacampbe

Resolving patch:
diff --git a/ietf/doc/views_draft.py b/ietf/doc/views_draft.py
--- a/ietf/doc/views_draft.py
+++ b/ietf/doc/views_draft.py
@@ -989,14 +989,16 @@ def edit_shepherd_writeup(request, name):
                 return redirect("ietf.doc.views_doc.document_main", name=doc.name)
 
         elif "reset_text" in request.POST:
+            if not doc.group.type.slug or doc.group.type.slug != "wg":
+                generate_type = "individ"
+            else:
+                generate_type = "group"           
             init = {
                 "content": render_to_string(
                     "doc/shepherd_writeup.txt",
                     dict(
                         doc=doc,
-                        type="individ"
-                        if not doc.group.type.slug or doc.group.type.slug != "ietf"
-                        else "group",
+                        type=generate_type,
                         stream=doc.stream.slug,
                         group=doc.group.type.slug,
                     ),
@@ -1020,13 +1022,15 @@ def edit_shepherd_writeup(request, name):
         if previous_writeup:
             init["content"] = previous_writeup.text
         else:
+            if not doc.group.type.slug or doc.group.type.slug != "wg":
+                generate_type = "individ"
+            else:
+                generate_type = "group"
             init["content"] = render_to_string(
                 "doc/shepherd_writeup.txt",
                 dict(
                     doc=doc,
-                    type="individ"
-                    if not doc.group.type.slug or doc.group.type.slug != "wg"
-                    else "group",
+                    type=generate_type,
                     stream=doc.stream.slug,
                     group=doc.group.type.slug,
                 ),


Test patch:
diff --git a/ietf/doc/tests_draft.py b/ietf/doc/tests_draft.py
--- a/ietf/doc/tests_draft.py
+++ b/ietf/doc/tests_draft.py
@@ -1960,6 +1960,10 @@ def test_shepherd_writeup_generation(self):
         login_testing_unauthorized(self, "secretary", url)
         r = self.client.get(url)
         self.assertContains(r, "for Individual Documents", status_code=200)
+        r = self.client.post(url,dict(reset_text=''))
+        self.assertContains(r, "for Individual Documents", status_code=200)
         url = urlreverse('ietf.doc.views_draft.edit_shepherd_writeup', kwargs=dict(name=wg_draft.name))
         r = self.client.get(url)
         self.assertContains(r, "for Group Documents", status_code=200)
+        r = self.client.post(url,dict(reset_text=''))
+        self.assertContains(r, "for Group Documents", status_code=200)
Repository: ietf-tools/datatracker

Issue:
When an AD puts a document in AD is watching, email emitted says the IESG is starting processing
(Reported by Ben Campbell)

That confuses chairs. The message should be more specific in this case.

cc @mskucherawy


Discussion:
cc @bacampbe

Resolving patch:
diff --git a/ietf/doc/views_draft.py b/ietf/doc/views_draft.py
--- a/ietf/doc/views_draft.py
+++ b/ietf/doc/views_draft.py
@@ -592,7 +592,7 @@ def to_iesg(request,name):
                 e.by = by
                 e.doc = doc
                 e.rev = doc.rev
-                e.desc = "IESG process started in state <b>%s</b>" % target_state['iesg'].name
+                e.desc = "Document is now in IESG state <b>%s</b>" % target_state['iesg'].name
                 e.save()
                 events.append(e)
 
@@ -713,7 +713,7 @@ def edit_info(request, name):
                 e.by = by
                 e.doc = doc
                 e.rev = doc.rev
-                e.desc = "IESG process started in state <b>%s</b>" % doc.get_state("draft-iesg").name
+                e.desc = "Document is now in IESG state <b>%s</b>" % doc.get_state("draft-iesg").name
                 e.save()
                 events.append(e)
 


Test patch:
Repository: ietf-tools/datatracker

Issue:
iCalendar outputs do not handle UTC times properly
### Describe the issue

Per [RFC 5545](https://www.rfc-editor.org/rfc/rfc5545.html#section-3.3.5), a DATE-TIME in an iCalendar file must represent UTC times using a Z suffix (e.g., `2022-08-08T23:01:00Z`). For local times with a time zone reference, the `TZID` attribute is to be used (e.g., `TZID=America/Los_Angeles:2022-08-08T16:01:00`).

In at least `agenda.ics`, meeting time zones are specified using the `TZID` attribute regardless of its value. This should instead use the -Z suffix when the meeting times are in UTC.

### Code of Conduct

- [X] I agree to follow the [IETF's Code of Conduct](https://github.com/ietf-tools/.github/blob/main/CODE_OF_CONDUCT.md)


Discussion:

Resolving patch:
diff --git a/ietf/doc/templatetags/ietf_filters.py b/ietf/doc/templatetags/ietf_filters.py
--- a/ietf/doc/templatetags/ietf_filters.py
+++ b/ietf/doc/templatetags/ietf_filters.py
@@ -502,6 +502,36 @@ def ics_esc(text):
     text = re.sub(r"([\n,;\\])", r"\\\1", text)
     return text
 
+
+@register.simple_tag
+def ics_date_time(dt, tzname):
+    """Render a datetime as an iCalendar date-time
+
+    dt a datetime, localized to the timezone to be displayed
+    tzname is the name for this timezone
+
+    Caller must arrange for a VTIMEZONE for the tzname to be included in the iCalendar file.
+    Output includes a ':'. Use like:
+      DTSTART{% ics_date_time timestamp 'America/Los_Angeles' %}
+    to get
+      DTSTART;TZID=America/Los_Angeles:20221021T111200
+
+    >>> ics_date_time(datetime.datetime(2022,1,2,3,4,5), 'utc')
+    ':20220102T030405Z'
+
+    >>> ics_date_time(datetime.datetime(2022,1,2,3,4,5), 'UTC')
+    ':20220102T030405Z'
+
+    >>> ics_date_time(datetime.datetime(2022,1,2,3,4,5), 'America/Los_Angeles')
+    ';TZID=America/Los_Angeles:20220102T030405'
+    """
+    timestamp = dt.strftime('%Y%m%dT%H%M%S')
+    if tzname.lower() == 'utc':
+        return f':{timestamp}Z'
+    else:
+        return f';TZID={ics_esc(tzname)}:{timestamp}'
+
+
 @register.filter
 def consensus(doc):
     """Returns document consensus Yes/No/Unknown."""


Test patch:
Repository: ietf-tools/datatracker

Issue:
htmlizer entry point does wrong thing with cased inputs
From @agmalis

> I note that the URL
> 
> https://datatracker.ietf.org/doc/html/rfc7209
> 
> correctly displays RFC 7209.
> 
> However, the URL
> 
> https://datatracker.ietf.org/doc/html/RFC7209
> 
> actually displays draft-ietf-l2vpn-evpn-req-07.txt .
> 
> Cheers,
> Andy


Discussion:

Resolving patch:
diff --git a/ietf/doc/utils.py b/ietf/doc/utils.py
--- a/ietf/doc/utils.py
+++ b/ietf/doc/utils.py
@@ -1293,6 +1293,8 @@ def fuzzy_find_documents(name, rev=None):
     document.
     """
     # Handle special case name formats
+    if re.match(r"^\s*rfc", name, flags=re.IGNORECASE):
+        name = re.sub(r"\s+", "", name.lower())
     if name.startswith('rfc0'):
         name = "rfc" + name[3:].lstrip('0')
     if name.startswith('review-') and re.search(r'-\d\d\d\d-\d\d$', name):
@@ -1303,8 +1305,6 @@ def fuzzy_find_documents(name, rev=None):
         rev = rev[-2:]
     if re.match("^[0-9]+$", name):
         name = f'rfc{name}'
-    if re.match("^[Rr][Ff][Cc] [0-9]+$",name):
-        name = f'rfc{name[4:]}'
 
     # see if we can find a document using this name
     docs = Document.objects.filter(docalias__name=name, type_id='draft')


Test patch:
Repository: ietf-tools/datatracker

Issue:
Eliminate use of django.utils.datetime_safe
### Describe the issue

In at least `ietf/meeting/models.py`, the `django.utils.datetime_safe` library is used. The [datetime_safe module is reportedly going to go away](https://stackoverflow.com/questions/21991666/what-is-django-utils-datetime-safe-and-should-i-use-it-instead-of-standard-datet) and as far as I have found is not documented outside of the source code.

It's unclear to me why this was used, but I think it should be straightforward to drop and replace with the standard datetime library.

Minor, but must be addressed before or when migrating to Django 4.

### Code of Conduct

- [X] I agree to follow the [IETF's Code of Conduct](https://github.com/ietf-tools/.github/blob/main/CODE_OF_CONDUCT.md)


Discussion:
I'm on it and can confirm this is only in `ietf/meeting/models.py`:

```
justin@ubuntu:~/git/IETF/datatracker$ grep -R "datetime_safe" .
./ietf/meeting/models.py:from django.utils import datetime_safe
./ietf/meeting/models.py:    d = datetime_safe.new_date(o)
```

Resolving patch:
diff --git a/ietf/meeting/models.py b/ietf/meeting/models.py
--- a/ietf/meeting/models.py
+++ b/ietf/meeting/models.py
@@ -60,16 +60,6 @@
 timezones.sort()
 
 
-# this is used in models to format dates, as the built-in json serializer
-# can not deal with them, and the django provided serializer is inaccessible.
-from django.utils import datetime_safe
-DATE_FORMAT = "%Y-%m-%d"
-TIME_FORMAT = "%H:%M:%S"
-
-def fmt_date(o):
-    d = datetime_safe.new_date(o)
-    return d.strftime(DATE_FORMAT)
-
 class Meeting(models.Model):
     # number is either the number for IETF meetings, or some other
     # identifier for interim meetings/IESG retreats/liaison summits/...


Test patch:
Repository: ietf-tools/datatracker

Issue:
Links to non-existing person profiles are being generated
### What happened?

For example, on http://127.0.0.1:8000/ipr/2670/history/, a link to http://127.0.0.1:8000/person/laura.hill@parc.com is being generated, which 404s.

### What browser(s) are you seeing the problem on?

_No response_

### Code of Conduct

- [X] I agree to follow the IETF's Code of Conduct


Discussion:
In this particular case, a `Person` record exists with that email address but there's no `Email` record.
Profile view checks `Email` records if the profile URL has an email address, that's the reason the profile page is 404.
I think the correct behaviour is to check both `Email` records and `Person` records in the profile view.

Resolving patch:
diff --git a/ietf/person/views.py b/ietf/person/views.py
--- a/ietf/person/views.py
+++ b/ietf/person/views.py
@@ -68,11 +68,13 @@ def ajax_select2_search(request, model_name):
     return HttpResponse(select2_id_name_json(objs), content_type='application/json')
 
 def profile(request, email_or_name):
+    aliases = Alias.objects.filter(name=email_or_name)
+    persons = list(set([ a.person for a in aliases ]))
+
     if '@' in email_or_name:
-        persons = [ get_object_or_404(Email, address=email_or_name).person, ]
-    else:
-        aliases = Alias.objects.filter(name=email_or_name)
-        persons = list(set([ a.person for a in aliases ]))
+        emails = Email.objects.filter(address=email_or_name)
+        persons += list(set([ e.person for e in emails ]))
+
     persons = [ p for p in persons if p and p.id ]
     if not persons:
         raise Http404


Test patch:
diff --git a/ietf/person/tests.py b/ietf/person/tests.py
--- a/ietf/person/tests.py
+++ b/ietf/person/tests.py
@@ -104,6 +104,23 @@ def test_person_profile(self):
         r = self.client.get(photo_url)
         self.assertEqual(r.status_code, 200)
 
+    def test_person_profile_without_email(self):
+        person = PersonFactory(name="foobar@example.com")
+        # delete Email record
+        person.email().delete()
+        url = urlreverse("ietf.person.views.profile", kwargs={ "email_or_name": person.plain_name()})
+        r = self.client.get(url)
+        self.assertContains(r, person.name, status_code=200)
+
+    def test_person_profile_404(self):
+        urls = [
+                urlreverse("ietf.person.views.profile", kwargs={ "email_or_name": "nonexistent@example.com"}),
+                urlreverse("ietf.person.views.profile", kwargs={ "email_or_name": "Nonexistent Person"}),]
+
+        for url in urls:
+            r = self.client.get(url)
+            self.assertEqual(r.status_code, 404)
+
     def test_person_photo(self):
         person = PersonFactory(with_bio=True)
         
@@ -403,8 +420,3 @@ def test_dots(self):
         self.assertEqual(get_dots(ncmember),['nomcom'])
         ncchair = RoleFactory(group__acronym='nomcom2020',group__type_id='nomcom',name_id='chair').person
         self.assertEqual(get_dots(ncchair),['nomcom'])
-
-
-
-
-       
\ No newline at end of file
Repository: ietf-tools/datatracker

Issue:
New outgoing liaison form fails at the From contact 
On submission, validation of valid from contacts fail - the field is cleared, and unless you are in very special role, the field is also disabled, so there is no recourse.


Discussion:
The `disabled` flag is set on the input - this has the effect both of disabling it (desired) and omitting its value from the POST (less desired).
(It was changed recently from `readonly` which does not seem to work with the reworked searchable fields - changing it back to readonly fixes the form but leaves the field editable.)
Is there any need to show this field if it's not editable? I suspect the cleanest fix is to get rid of it and fill in the correct value automatically.
See if this works? https://stackoverflow.com/questions/41807096/select2-make-it-readonly-not-disabled-from-js/55001516#55001516

Resolving patch:
diff --git a/ietf/liaisons/forms.py b/ietf/liaisons/forms.py
--- a/ietf/liaisons/forms.py
+++ b/ietf/liaisons/forms.py
@@ -510,7 +510,7 @@ def set_from_fields(self):
         else:
             email = self.person.email_address()
         self.fields['from_contact'].initial = email
-        self.fields['from_contact'].widget.attrs['disabled'] = True
+        self.fields['from_contact'].widget.attrs['readonly'] = True
 
     def set_to_fields(self):
         '''Set to_groups and to_contacts options and initial value based on user


Test patch:
Repository: ietf-tools/datatracker

Issue:
ietf links filter overmatches
From EKR:
![image](https://user-images.githubusercontent.com/10996692/200128592-07f10f62-6dd3-4130-ae33-0302afd6af4d.png)


The agenda parser seems to think that anything matching "RFC<ws>####" is an RFC, even if there is a line break, as in the thing above.


Discussion:
We need to handle one line break, because we wordwrap some content and that can create one, but we shouldn't allow more than one. I'll make that change.

Resolving patch:
diff --git a/ietf/doc/templatetags/ietf_filters.py b/ietf/doc/templatetags/ietf_filters.py
--- a/ietf/doc/templatetags/ietf_filters.py
+++ b/ietf/doc/templatetags/ietf_filters.py
@@ -249,7 +249,7 @@ def urlize_ietf_docs(string, autoescape=None):
         flags=re.IGNORECASE | re.ASCII,
     )
     string = re.sub(
-        r"\b(?<![/\-:=#\"\'])((RFC|BCP|STD|FYI)\s*0*(\d+))\b",
+        r"\b(?<![/\-:=#\"\'])((RFC|BCP|STD|FYI) *\n? *0*(\d+))\b",
         link_other_doc_match,
         string,
         flags=re.IGNORECASE | re.ASCII,


Test patch:
Repository: ietf-tools/datatracker

Issue:
(nomcom) http URLs used in 'IETF Nomination Information' emails
### Describe the issue

In the 'IETF Nomination Information' email that is sent to a nomcom nominee, http URLs are used. For example:

```
You have been nominated for the position of Test Area Director.

Please follow one of the links below to accept or decline this nomination.

Accept: http://datatracker.ietf.org/nomcom/7437/process-nomination-status/1230/accepted/20221105/.../'

Decline: http://datatracker.ietf.org/nomcom/7437/process-nomination-status/1230/declined/20221105/.../'
```
https URLs should be used instead.

### Code of Conduct

- [X] I agree to follow the [IETF's Code of Conduct](https://github.com/ietf-tools/.github/blob/main/CODE_OF_CONDUCT.md)
htmlizer entry point does wrong thing with cased inputs
From @agmalis

> I note that the URL
> 
> https://datatracker.ietf.org/doc/html/rfc7209
> 
> correctly displays RFC 7209.
> 
> However, the URL
> 
> https://datatracker.ietf.org/doc/html/RFC7209
> 
> actually displays draft-ietf-l2vpn-evpn-req-07.txt .
> 
> Cheers,
> Andy


Discussion:

Resolving patch:
diff --git a/ietf/doc/utils.py b/ietf/doc/utils.py
--- a/ietf/doc/utils.py
+++ b/ietf/doc/utils.py
@@ -1308,6 +1308,8 @@ def fuzzy_find_documents(name, rev=None):
     document.
     """
     # Handle special case name formats
+    if re.match(r"^\s*rfc", name, flags=re.IGNORECASE):
+        name = re.sub(r"\s+", "", name.lower())
     if name.startswith('rfc0'):
         name = "rfc" + name[3:].lstrip('0')
     if name.startswith('review-') and re.search(r'-\d\d\d\d-\d\d$', name):
@@ -1318,8 +1320,6 @@ def fuzzy_find_documents(name, rev=None):
         rev = rev[-2:]
     if re.match("^[0-9]+$", name):
         name = f'rfc{name}'
-    if re.match("^[Rr][Ff][Cc] [0-9]+$",name):
-        name = f'rfc{name[4:]}'
 
     # see if we can find a document using this name
     docs = Document.objects.filter(docalias__name=name, type_id='draft')
diff --git a/ietf/nomcom/utils.py b/ietf/nomcom/utils.py
--- a/ietf/nomcom/utils.py
+++ b/ietf/nomcom/utils.py
@@ -425,7 +425,7 @@ def make_nomineeposition_for_newperson(nomcom, candidate_name, candidate_email,
 
     return make_nomineeposition(nomcom, email.person, position, author)
 
-def getheader(header_text, default="ascii"):
+def getheader(header_text, default="utf-8"):
     """Decode the specified header"""
 
     tuples = decode_header(header_text)
@@ -433,7 +433,7 @@ def getheader(header_text, default="ascii"):
     return "".join(header_sections)
 
 
-def get_charset(message, default="ascii"):
+def get_charset(message, default="utf-8"):
     """Get the message charset"""
 
     if message.get_content_charset():


Test patch:
Repository: ietf-tools/datatracker

Issue:
Rework select->readonly usage
Rework what had been done at #4708 with more conformant html to avoid the need for #4786.


Discussion:

Resolving patch:
diff --git a/ietf/liaisons/forms.py b/ietf/liaisons/forms.py
--- a/ietf/liaisons/forms.py
+++ b/ietf/liaisons/forms.py
@@ -471,7 +471,6 @@ def set_to_fields(self):
 
 
 class OutgoingLiaisonForm(LiaisonModelForm):
-    from_contact = SearchableEmailField(only_users=True)
     approved = forms.BooleanField(label="Obtained prior approval", required=False)
 
     class Meta:
@@ -501,6 +500,7 @@ def set_from_fields(self):
             self.fields['from_groups'].initial = [flat_choices[0][0]]
         
         if has_role(self.user, "Secretariat"):
+            self.fields['from_contact'] = SearchableEmailField(only_users=True)  # secretariat can edit this field!
             return
 
         if self.person.role_set.filter(name='liaiman',group__state='active'):
@@ -509,8 +509,10 @@ def set_from_fields(self):
             email = self.person.role_set.filter(name__in=('ad','chair'),group__state='active').first().email.address
         else:
             email = self.person.email_address()
+
+        # Non-secretariat user cannot change the from_contact field. Fill in its value.
+        self.fields['from_contact'].disabled = True
         self.fields['from_contact'].initial = email
-        self.fields['from_contact'].widget.attrs['readonly'] = True
 
     def set_to_fields(self):
         '''Set to_groups and to_contacts options and initial value based on user


Test patch:
diff --git a/ietf/utils/test_runner.py b/ietf/utils/test_runner.py
--- a/ietf/utils/test_runner.py
+++ b/ietf/utils/test_runner.py
@@ -196,7 +196,6 @@ def vnu_filter_message(msg, filter_db_issues, filter_test_issues):
 
     return re.search(
         r"""document\ is\ not\ mappable\ to\ XML\ 1|
-             Attribute\ 'readonly'\ not\ allowed\ on\ element\ 'select'|
             ^Attribute\ 'required'\ not\ allowed\ on\ element\ 'div'|
             ^The\ 'type'\ attribute\ is\ unnecessary\ for\ JavaScript|
             is\ not\ in\ Unicode\ Normalization\ Form\ C""",
Repository: ietf-tools/datatracker

Issue:
bibxml3 returns incorrect results for some older draft revisions where the draft has been published as an RFC
Because of a questionable decision to change the document name of DocHistory objects, right now
```
view-source:https://datatracker.ietf.org/doc/bibxml3/draft-ietf-aaa-diameter-17.xml
```
returns
```
   <seriesInfo name="Internet-Draft" value="rfc3588-17" />
   <format type="TXT" target="https://www.ietf.org/archive/id/rfc3588-17.txt" />
```

The bibxml3 (aka bibxml-ids) code should pay attention and make sure it's returning the _draft_ name in these places.

The problem space will change here when we change RFCs to be their own document type.


Discussion:

Resolving patch:
diff --git a/ietf/doc/utils.py b/ietf/doc/utils.py
--- a/ietf/doc/utils.py
+++ b/ietf/doc/utils.py
@@ -1362,5 +1362,9 @@ def bibxml_for_draft(doc, rev=None):
     else:
         doc.date = doc.time.astimezone(tzinfo).date()      # Even if this may be incorrect, what would be better?
 
-    return render_to_string('doc/bibxml.xml', {'name':doc.name, 'doc': doc, 'doc_bibtype':'I-D'})
+    name = doc.name if isinstance(doc, Document) else doc.doc.name
+    if name.startswith('rfc'): # bibxml3 does not speak of RFCs
+        raise Http404()
+        
+    return render_to_string('doc/bibxml.xml', {'name':name, 'doc':doc, 'doc_bibtype':'I-D'})
 


Test patch: