internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: hydroshare/hydroshare

Issue:
PR Builds are failing to create hydroshare container
**Description of the bug**
The PR build Jenkins job is failing to create the hydroshare container with the following error:

`ERROR: for hydroshare  Cannot create container for service hydroshare: b'invalid IP address in add-host: ""'`

Steps to reproduce the bug:
1. Build a PR
2. Look in the console log for the job.

**Expected behavior**
A clean build.

**Additional information**
~~Apparently sometime yesterday a blank IP stopped resolving to localhost (and it was unreasonable to expect it to do so) at RENCI. The fix is to explicitly specify 127.0.0.1 in the extra-hosts directive of scripts/templates/docker-compose-local-irods.template for local irods hosts.~~ The build fails because the local irods containers fail to stay up long enough to query their IPs.



Discussion:

Resolving patch:
diff --git a/hs_file_types/models/reftimeseries.py b/hs_file_types/models/reftimeseries.py
--- a/hs_file_types/models/reftimeseries.py
+++ b/hs_file_types/models/reftimeseries.py
@@ -1,20 +1,19 @@
 import json
 import logging
-from dateutil import parser
-from urllib.request import Request, urlopen
+import ssl
 from urllib.error import URLError
-import jsonschema
+from urllib.request import Request, urlopen
 
-from django.utils import timezone
-from django.db import models, transaction
+import jsonschema
+from dateutil import parser
 from django.core.exceptions import ValidationError
+from django.db import models, transaction
 from django.template import Template, Context
-
+from django.utils import timezone
 from dominate.tags import div, form, button, h4, p, textarea, legend, table, tbody, tr, \
     th, td, a
 
 from hs_core.signals import post_add_reftimeseries_aggregation
-
 from .base import AbstractFileMetaData, AbstractLogicalFile, FileTypeContext
 
 
@@ -939,11 +938,14 @@ def _validate_json_data(json_data):
             if not url:
                 raise ValidationError("methodLink has a value of empty string")
             if url.lower() != 'unknown':
+                ctx = ssl.create_default_context()
+                ctx.check_hostname = False
+                ctx.verify_mode = ssl.CERT_NONE
                 url = Request(url)
                 if url not in urls:
                     urls.append(url)
                     try:
-                        urlopen(url)
+                        urlopen(url, context=ctx)
                     except URLError:
                         raise Exception(err_msg.format("Invalid method link found"))
 


Test patch:
Repository: hydroshare/hydroshare

Issue:
Allow deletion of Web App Icon
An error is currently shown when you delete the value of the Icon URL field in a Web App.  Add support for saving an empty value to the Icon URL field.


Discussion:

Resolving patch:
diff --git a/hs_tools_resource/models.py b/hs_tools_resource/models.py
--- a/hs_tools_resource/models.py
+++ b/hs_tools_resource/models.py
@@ -487,7 +487,7 @@ def _validate_tool_icon(cls, url):
     def create(cls, **kwargs):
         if 'value' in kwargs and "data_url" not in kwargs:
             url = kwargs["value"]
-            data_url = cls._validate_tool_icon(url)
+            data_url = cls._validate_tool_icon(url) if url else ""
 
             metadata_obj = kwargs['content_object']
             new_meta_instance = ToolIcon.objects.create(content_object=metadata_obj)
@@ -510,7 +510,7 @@ def update(cls, element_id, **kwargs):
         meta_instance = ToolIcon.objects.get(id=element_id)
         if 'value' in kwargs:
             url = kwargs["value"]
-            data_url = cls._validate_tool_icon(url)
+            data_url = cls._validate_tool_icon(url) if url else ""
             meta_instance.value = url
             meta_instance.data_url = data_url
             meta_instance.save()


Test patch:
Repository: hydroshare/hydroshare

Issue:
Image formats handling for icon url (web app resources)
The "feature suggstion" part of this:
It seems like .ico "Icon file" should be accepted by the Icon URL input for Web App resources.  SVG files seem like they should be possible as well.  Further, both of these file types correctly render a preview when the user inputs the url.  We should probably not show the preview if we are not accepting the file?  This could make the user think the file format restrictions are arbitrary (maybe to some extent they are).  

The "bug" part of this:
For these ico and svg files, the right error message is not always displayed.  For "https://duckduckgo.com/favicon.ico" I get a "Error! Metadata failed to update." as opposed to the expected "Error! Failed to update metadata element 'toolicon'. Supported icon image types are png, gif and jpeg."  So if the user tried https://duckduckgo.com/favicon.ico they would have no insight into the issue or how to successfully implement the icon.

[002.zip](https://github.com/hydroshare/hydroshare/files/2421102/002.zip)




Discussion:
related to #2895 
@horsburgh I believe this falls under the Resource Model Category; adding this label but please feel free to reassign if there is a better fit. 

Resolving patch:
diff --git a/hs_tools_resource/models.py b/hs_tools_resource/models.py
--- a/hs_tools_resource/models.py
+++ b/hs_tools_resource/models.py
@@ -1,6 +1,7 @@
 import requests
 import base64
 import imghdr
+from hs_core.hydroshare.utils import get_file_mime_type
 
 from django.db import models, transaction
 from django.contrib.contenttypes.fields import GenericRelation
@@ -472,12 +473,17 @@ def _validate_tool_icon(cls, url):
         if response.status_code != 200:
             raise HttpResponse("Failed to read data from given url. HTTP_code {0}".
                                format(response.status_code))
-        image_size_mb = float(response.headers["content-length"])
+        if 'Transfer-Encoding' in response.headers and response.headers["Transfer-Encoding"] == "chunked":
+            image_size_mb = len(response.content)
+        else:
+            image_size_mb = float(response.headers["content-length"])
         if image_size_mb > 1000000:  # 1mb
             raise ValidationError("Icon image size should be less than 1MB.")
         image_type = imghdr.what(None, h=response.content)
-        if image_type not in ["png", "gif", "jpeg"]:
-            raise ValidationError("Supported icon image types are png, gif and jpeg")
+        if not image_type:
+            image_type = get_file_mime_type(url).rsplit('/', 1)[1]
+        if image_type not in ["png", "gif", "jpeg", "svg+xml", "vnd.microsoft.icon", "svg", "ico"]:
+            raise ValidationError("Supported icon image types are png, gif, jpeg, ico, and svg")
         base64_string = base64.b64encode(response.content)
         data_url = "data:image/{image_type};base64,{base64_string}". \
             format(image_type=image_type, base64_string=base64_string.decode())


Test patch:
Repository: hydroshare/hydroshare

Issue:
Web App Connector cannot add Home page url
**Description of the bug**
When creating or editing a web app connector, you cannot change the App Home Page URL field
Steps to reproduce the bug:
1. Create a new Web App Connector (name does not matter)
2. Type a valid URL into the metadata field "App Home Page URL"
3. Click on the Save Changes button 
4. See error

**Expected behavior**
Saves metadata and Save successful message appears on top of screen

**Additional information**
First noticed Apr. 20th, may have to do with the iRods upgrade that occured on Apr. 19th.



Discussion:

Resolving patch:
diff --git a/hs_tools_resource/forms.py b/hs_tools_resource/forms.py
--- a/hs_tools_resource/forms.py
+++ b/hs_tools_resource/forms.py
@@ -115,7 +115,6 @@ class Meta:
 
 class UrlValidationForm(forms.Form):
     value = forms.URLField(max_length=1024, required=False)
-    data_url = forms.CharField(required=False)
 
 
 class AppResourceLevelUrlValidationForm(forms.Form):
@@ -527,8 +526,8 @@ class Meta:
 
 
 class ToolIconValidationForm(forms.Form):
-    value = forms.CharField(max_length=1024, required=False)
-    data_url = forms.URLField(required=False)
+    value = forms.URLField(max_length=1024, required=False)
+    data_url = forms.CharField(required=False)
 
 
 class MetadataField(Field):
diff --git a/hs_tools_resource/receivers.py b/hs_tools_resource/receivers.py
--- a/hs_tools_resource/receivers.py
+++ b/hs_tools_resource/receivers.py
@@ -11,7 +11,7 @@
                                     SupportedFileExtensionsValidationForm, \
                                     AppAggregationLevelUrlValidationForm, \
                                     AppResourceLevelUrlValidationForm, \
-                                    AppFileLevelUrlValidationForm
+                                    AppFileLevelUrlValidationForm, ToolIconValidationForm
 
 from .default_icon import default_icon_data_url
 
@@ -56,7 +56,7 @@ def validate_form(request, element_name):
     elif element_name == 'supportedaggtypes':
         element_form = SupportedAggTypesValidationForm(data=request.POST)
     elif element_name == 'toolicon':
-        element_form = UrlValidationForm(data=request.POST)
+        element_form = ToolIconValidationForm(data=request.POST)
     elif element_name == 'supportedsharingstatus':
         element_form = SupportedSharingStatusValidationForm(data=request.POST)
     elif element_name == 'supportedfileextensions':


Test patch:
Repository: hydroshare/hydroshare

Issue:
unwanted email sending from beta.hydroshare
**Description of the bug**
The [Celery task for sending quota emails](https://github.com/hydroshare/hydroshare/blob/1.57.1/hs_core/tasks.py#L225) sent 8 emails out to users last week from _beta.hydroshare_ This should only be possible if beta is in DEBUG mode. I'm not sure why it would've been...

However, this incident makes it clear that our email sending logic on beta might need revision.
Should we add more safeguards to ensure that unintended emails don't go out?
Change the django email backend configs?
Use a mailinator?



Discussion:
I vote that we 

- create a custom email backend just for Beta
- add that backend as the default in `/projects/hydroshare/hsdata/zdt/beta/deploy` (and add it to local_settings.template under source control)
- modify `/projects/hydroshare/hsdata/zdt/beta/deploy/hs_email` so that it **sed**s this new backend appropriately


Resolving patch:
diff --git a/hs_core/tasks.py b/hs_core/tasks.py
--- a/hs_core/tasks.py
+++ b/hs_core/tasks.py
@@ -17,6 +17,7 @@
 from celery.task import periodic_task
 from django.conf import settings
 from django.core.mail import send_mail
+from django.contrib.sites.models import Site
 from django.core.exceptions import ObjectDoesNotExist, ValidationError
 from rest_framework import status
 
@@ -217,7 +218,7 @@ def send_over_quota_emails():
 
                 msg_str += '\n\nHydroShare Support'
                 subject = 'Quota warning'
-                if settings.DEBUG:
+                if settings.DEBUG or "www.hydroshare.org" not in Site.objects.get_current().domain:
                     logger.info("quota warning email not sent out on debug server but logged instead: "
                                 "{}".format(msg_str))
                 else:


Test patch:
Repository: hydroshare/hydroshare

Issue:
Resource is broken (with error page) after zipping, bagging, and unzipping operations
**Description of the bug**
After testing of zipping, bagging, and unzipping operations, a resource became unusual.  These operations weren't automated or otherwise executed in a non-human way.

Steps to reproduce the bug:
1.  Upload a zip
3. Create a zip in the content browser
4. Unzip both files
5. Create a BagIt
6. Unzip the BagIt
7. Unzip the BagIt again, with overwrites

<img width="1440" alt="Bildschirmfoto 2022-08-29 um 11 57 27 PM" src="https://user-images.githubusercontent.com/43424907/187346185-76ff23e0-939e-4264-a1e1-5e1ce48e99b2.png">
https://beta.hydroshare.org/resource/e007cf2132364c5990ba3246001a403d/

**Expected behavior**
Resource remains in-tact.

**Additional information**
Chrome 104.0.5112.101


Discussion:
@ndebuhr Can you explain what Step 2 "Create a zip in the content browser" means? How can I create a zip in the content browser? Do you mean using "Download zipped" from the content browser to download a zip file, then adding the downloaded zip file to the same resource?

Resolving patch:
diff --git a/hs_core/models.py b/hs_core/models.py
--- a/hs_core/models.py
+++ b/hs_core/models.py
@@ -2571,7 +2571,13 @@ def get_readme_file_content(self):
         characters in the file will be escaped when we return the file content.
         """
         readme_file = self.readme_file
-        if readme_file is not None:
+        # check the file exists on irods
+
+        if readme_file is None:
+            return readme_file
+
+        # check the file exists on irods
+        if readme_file.exists:
             readme_file_content = readme_file.read().decode('utf-8', 'ignore')
             if readme_file.extension.lower() == '.md':
                 markdown_file_content = markdown(readme_file_content)
@@ -2579,7 +2585,13 @@ def get_readme_file_content(self):
                         'file_name': readme_file.file_name, 'file_type': 'md'}
             else:
                 return {'content': readme_file_content, 'file_name': readme_file.file_name}
-        return readme_file
+        else:
+            file_name = readme_file.file_name
+            readme_file.delete()
+            logger = logging.getLogger(__name__)
+            log_msg = f"readme file ({file_name}) is missing on iRODS. Deleting the file from Django."
+            logger.warning(log_msg)
+            return None
 
     @property
     def logical_files(self):


Test patch:
Repository: hydroshare/hydroshare

Issue:
Unzipping bagit in place results in empty directory
**Description of the bug**
Unzipping bagit in place results in empty directory

Steps to reproduce the bug:
1. Create new resource
2. Add a file
3. Download as zipped bagit
4. Upload the zip that you just downloaded back into the resource
5. Unzip in place withing the File browser
6. A directory is created that is empty

https://user-images.githubusercontent.com/17934193/187804318-1f22755e-61a9-4731-8789-504a197d656b.mov



**Expected behavior**
Unzipped directory should contain the original files in the zip archive

**Additional information**
rc-1.58, 20c0f8e3a01708dfb8f754e93d2774ba8dbbd86e, Firefox
Seems that this issue is intermittent? Occurred consistently for a while, then I checked out a different branch, then checked out 20c0f8e3a01708dfb8f754e93d2774ba8dbbd86e again and had trouble reproducing. Rebuilt local stack and then was able to see the bug again. 



Discussion:

Resolving patch:
diff --git a/hs_core/views/utils.py b/hs_core/views/utils.py
--- a/hs_core/views/utils.py
+++ b/hs_core/views/utils.py
@@ -1112,7 +1112,7 @@ def unzip_file(user, res_id, zip_with_rel_path, bool_remove_original,
         added_resource_files = []
         for file in res_files:
             destination_file = _get_destination_filename(file.name, unzipped_foldername)
-            destination_file = destination_file.replace(res_id + "/", "")
+            destination_file = destination_file.replace(res_id + "/", "", 1)
             destination_file = resource.get_irods_path(destination_file)
             res_file = link_irods_file_to_django(resource, destination_file)
             added_resource_files.append(res_file)


Test patch:
Repository: hydroshare/hydroshare

Issue:
Change Mezzanine Form to Disallow Username Changes
**Describe the feature you'd like and what it will do**
The internal mezzanine form for the admin account should be altered to no longer allow username changes if possible, since doing so breaks the resource. 


**Why is this feature important?**
This will be policy moving forward, that usernames cannot be changed. 
I will work with Scott to see if this change is feasible. 



Discussion:
@sblack-usu We said I'd make an issue for this and speak about it


Resolving patch:
diff --git a/hs_core/admin.py b/hs_core/admin.py
--- a/hs_core/admin.py
+++ b/hs_core/admin.py
@@ -14,6 +14,7 @@ def __init__(self, *args, **kwargs):
         self.fields['email'] = forms.EmailField(label=_("E-mail"), max_length=75)
 
 UserAdmin.add_form = UserCreationFormExtended
+UserAdmin.readonly_fields = ('username',)
 UserAdmin.add_fieldsets = (
     (None, {
         'classes': ('wide',),


Test patch:
Repository: hydroshare/hydroshare

Issue:
unwanted email sending from beta during nightly celery task
**Description of the bug**
Related to [#4716](ttps://github.com/hydroshare/hydroshare/issues/4716)
Beta is sending emails to help@cuahsi.org/zendesk like the following:

> d8f59a93f41d4074b61728eb46024713 does not have published date in its metadata.
231886df055e40f2a8186c16753f77f0 does not have published date in its metadata.
cf40bed6ddfe4a94977365105a4ffcec does not have published date in its metadata.

**Expected behavior**
We need to disable these emails on beta




Discussion:

Resolving patch:
diff --git a/hs_core/tasks.py b/hs_core/tasks.py
--- a/hs_core/tasks.py
+++ b/hs_core/tasks.py
@@ -16,7 +16,6 @@
 from celery.schedules import crontab
 from django.conf import settings
 from django.core.mail import send_mail
-from django.contrib.sites.models import Site
 from django.core.exceptions import ObjectDoesNotExist, ValidationError
 from rest_framework import status
 
@@ -185,7 +184,7 @@ def manage_task_nightly():
             msg_lst.append("{res_id} does not have published date in its metadata.".format(
                 res_id=res.short_id))
 
-    if msg_lst:
+    if msg_lst and not settings.DISABLE_TASK_EMAILS:
         email_msg = '\n'.join(msg_lst)
         subject = 'Notification of pending DOI deposition/activation of published resources'
         # send email for people monitoring and follow-up as needed
@@ -237,7 +236,7 @@ def send_over_quota_emails():
 
                 msg_str += '\n\nHydroShare Support'
                 subject = 'Quota warning'
-                if settings.DEBUG or "www.hydroshare.org" not in Site.objects.get_current().domain:
+                if settings.DEBUG or settings.DISABLE_TASK_EMAILS:
                     logger.info("quota warning email not sent out on debug server but logged instead: "
                                 "{}".format(msg_str))
                 else:
diff --git a/hydroshare/settings.py b/hydroshare/settings.py
--- a/hydroshare/settings.py
+++ b/hydroshare/settings.py
@@ -132,6 +132,10 @@
 # production. Best set to ``True`` in local_settings.py
 DEBUG = False
 
+# A boolean that determines whether tasks related emails should be sent.
+# Best set to ``True`` in local_settings.py
+DISABLE_TASK_EMAILS = False
+
 # Whether a user's session cookie expires when the Web browser is closed.
 SESSION_EXPIRE_AT_BROWSER_CLOSE = True
 


Test patch:
Repository: hydroshare/hydroshare

Issue:
hsapi/userDetails/ endpoint does not accept usernames and emails in OpenAPI interface
**Description of the bug**
A clear and concise description of what the bug is.

Steps to reproduce the bug:
1. Go to hydroshare.org/hsapi
2. Click on /userDetails/{user_identiier}/
3. See that user_identifier requires an integer

**Expected behavior**
The userDetails endpoint should accept usernames and emails in addition to the identifiers.  The endpoint actually accepts usernames and emails if you hit the endpoint outside of the OpenAPI interface (hsclient uses it).

**Additional information**
[Relevant lines](https://github.com/hydroshare/hydroshare/blob/develop/hs_rest_api/urls.py#L206-L207)
<img width="1422" alt="image" src="https://user-images.githubusercontent.com/35820390/195150588-8491baf1-3fb7-448d-9b72-172eddffcb20.png">



Discussion:

Resolving patch:
diff --git a/hs_core/views/__init__.py b/hs_core/views/__init__.py
--- a/hs_core/views/__init__.py
+++ b/hs_core/views/__init__.py
@@ -1760,7 +1760,7 @@ def get_metadata_terms_page(request, *args, **kwargs):
     return render(request, 'pages/metadata_terms.html')
 
 
-uid = openapi.Parameter('user_identifier', openapi.IN_PATH, description="id of the user for which data is needed", type=openapi.TYPE_INTEGER)
+uid = openapi.Parameter('user_identifier', openapi.IN_PATH, description="email, username, or id of the user for which data is needed", type=openapi.TYPE_STRING)
 @swagger_auto_schema(method='get', operation_description="Get user data",
                      responses={200: "Returns JsonResponse containing user data"}, manual_parameters=[uid])
 @api_view(['GET'])


Test patch:
Repository: hydroshare/hydroshare

Issue:
"Complete profile" broken link on Home page
**Description of the bug**
"Complete profile" broken link on Home page

Steps to reproduce the bug:

https://user-images.githubusercontent.com/17934193/203313781-fa04aae1-e18a-430e-90f1-9d0f20dd51b9.mov



Discussion:

Resolving patch:
diff --git a/theme/views.py b/theme/views.py
--- a/theme/views.py
+++ b/theme/views.py
@@ -62,7 +62,7 @@ def get_context_data(self, **kwargs):
             except:
                 u = User.objects.get(username=self.request.GET['user'])
 
-        elif not self.request.user.is_anonymous():
+        elif not self.request.user.is_anonymous:
             # if the user is logged in and no user is specified, show logged in user
             u = User.objects.get(pk=int(self.request.user.id))
 


Test patch:
Repository: hydroshare/hydroshare

Issue:
DOI status pending while DOI is active (pt 3?)
**Description of the bug**
DOI status says pending on resource landing page while DOI is active: https://doi.org/10.4211/hs.632e550deaea46b080bdae986fd19156

Steps to reproduce the bug:
1. Go to https://doi.org/10.4211/hs.632e550deaea46b080bdae986fd19156

**Expected behavior**
DOI should not stay status pending

**Additional information**
I couldn't find the previous ticket for this issue, apologies. If anyone can please link it


Discussion:
https://github.com/hydroshare/hydroshare/issues/4605
Other pending resources
https://www.hydroshare.org/resource/632e550deaea46b080bdae986fd19156/
https://www.hydroshare.org/resource/efb41192ddbd4785a2b0ef00bf5e7c62/
https://www.hydroshare.org/resource/a70e86faebe448128ecb0e208626ba4d/
https://www.hydroshare.org/resource/ebd3c73f946344d8adf0431f20576d1c/
https://www.hydroshare.org/resource/aefa69aefa98407ca2c0f0dd55c57cd4/
https://www.hydroshare.org/resource/674721ff170e4b67aa92a2136a0aa5a4/
https://www.hydroshare.org/resource/0b71b6ffeea04179acd9f3d5b13473b0/
```
# grep 632e550deaea46b080bdae986fd19156 django.log
[05/Dec/2022 15:34:18] WARNING [django.request:230] Forbidden: /hsapi/resource/632e550deaea46b080bdae986fd19156/file_list/
[05/Dec/2022 15:34:18] WARNING [django.request:230] Forbidden: /hsapi/resource/632e550deaea46b080bdae986fd19156/file_list/
[05/Dec/2022 15:51:58] WARNING [django.request:230] Forbidden: /hsapi/resource/632e550deaea46b080bdae986fd19156/file_list/
[05/Dec/2022 15:53:32] ERROR [django.request:230] Forbidden (Permission denied): /hsapi/_internal/632e550deaea46b080bdae986fd19156/publish/
```
perhaps due to:
https://github.com/celery/celery/issues/3589#issuecomment-290979095

Testing a fix

Resolving patch:
diff --git a/hs_core/tasks.py b/hs_core/tasks.py
--- a/hs_core/tasks.py
+++ b/hs_core/tasks.py
@@ -71,7 +71,7 @@ def __init__(self, error_message):
         super(FileOverrideException, self).__init__(self, error_message)
 
 
-@celery_app.on_after_configure.connect
+@celery_app.on_after_finalize.connect
 def setup_periodic_tasks(sender, **kwargs):
     if (hasattr(settings, 'DISABLE_PERIODIC_TASKS') and settings.DISABLE_PERIODIC_TASKS):
         logger.debug("Periodic tasks are disabled in SETTINGS")


Test patch: