internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: hydroshare/hydroshare

Issue:
Keycloak Integration
**Describe the feature you'd like and what it will do**
To support hydroshare login through other identity providers (ORCID) and to lay the groundwork for single sign on for related applications, HydroShare will use the OIDC protocol for authentication through keycloak.

**Why is this feature important?**
Support for single sign on and login through other identity providers.

**Additional context**
A pull request with the code changes necessary to integrate keycloak may be found here.  https://github.com/hydroshare/hydroshare/pull/5152.  There are a couple of UI changes that have been identified and will be updated.  There is also a change for API authentication to allow us to continue to support Basic Authetnication via the API.



Discussion:
Diagram describing the changes for API authentication
![HydroShare Keycloak Flow Diagram-API Authentication(1)](https://github.com/hydroshare/hydroshare/assets/35820390/926cad73-0fe0-4cfe-a78b-25084a23b4f8)

Diagram showing the UI in the login flows:
![HydroShare Keycloak Flow Diagram-Screenshots(2)](https://github.com/hydroshare/hydroshare/assets/35820390/dd042679-9853-4c39-a8d3-6c6321dfba26)


Resolving patch:
diff --git a/hs_core/middleware.py b/hs_core/middleware.py
new file mode 100644
--- /dev/null
+++ b/hs_core/middleware.py
@@ -0,0 +1,25 @@
+class SunsetMiddleware:
+
+    def __init__(self, get_response):
+        self.get_response = get_response
+
+    def __call__(self, request):
+        from django.utils.http import http_date
+        from datetime import datetime
+
+        response = self.get_response(request)
+        auth = request.headers.get('Authorization')
+        if not auth or 'Basic' not in auth:
+            return response
+
+        deprecation_date = datetime(2024, 4, 1)
+        http_deprecation = http_date(deprecation_date.timestamp())
+        help_link = "https://help.hydroshare.org/about-hydroshare/cuahsi-single-sign-on/"
+        if datetime.now() > deprecation_date:
+            response['Deprecation'] = http_deprecation
+            response['Link'] = f'''<{help_link}>; rel="deprecation"; type="text/html"'''
+        else:
+            # https://datatracker.ietf.org/doc/html/rfc8594
+            response['Sunset'] = http_deprecation
+            response['Link'] = f'''<{help_link}>; rel="sunset"; type="text/html"'''
+        return response
diff --git a/hydroshare/settings.py b/hydroshare/settings.py
--- a/hydroshare/settings.py
+++ b/hydroshare/settings.py
@@ -439,6 +439,7 @@ def _(s): return s # noqa
     "mezzanine.core.middleware.FetchFromCacheMiddleware",
     "hs_core.robots.RobotFilter",
     "hs_tracking.middleware.Tracking",
+    "hs_core.middleware.SunsetMiddleware",
 )
 
 # security settings


Test patch:
Repository: hydroshare/hydroshare

Issue:
Unable to publish collection
**Description of the bug**
[publish_resource](https://github.com/hydroshare/hydroshare/blob/2.10/hs_core/hydroshare/resource.py#L1083) fails with
```
>>> publish_resource(user, "c037167e497546a1bc1508dfb32a9cff")
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/hydroshare/hs_core/hydroshare/resource.py", line 1139, in publish_resource
    resource.metadata.create_element('Publisher', **md_args)
  File "/hydroshare/hs_core/models.py", line 4785, in create_element
    element = model_type.model_class().create(**kwargs)
  File "/hydroshare/hs_core/models.py", line 1354, in create
    raise ValidationError("Invalid publisher name")
```
It fails at these lines:
https://github.com/hydroshare/hydroshare/blob/2.10/hs_core/models.py#L1353-L1354
when attempting to publish a **collection** that doesn't have any files
```
>>> res.files.all()
<QuerySet []>
```

**Additional information**
HS v2.10



Discussion:

Resolving patch:
diff --git a/hs_core/models.py b/hs_core/models.py
--- a/hs_core/models.py
+++ b/hs_core/models.py
@@ -1327,6 +1327,8 @@ def create(cls, **kwargs):
         metadata_obj = kwargs['content_object']
         # get matching resource
         resource = BaseResource.objects.filter(object_id=metadata_obj.id).first()
+        if not resource:
+            raise ValidationError("Resource not found")
         if not resource.raccess.published:
             raise ValidationError("Publisher element can't be created for a resource that "
                                   "is not yet published.")
@@ -1348,13 +1350,14 @@ def create(cls, **kwargs):
             kwargs['url'] = 'https://www.cuahsi.org'
         else:
             # make sure we are not setting CUAHSI as publisher for a resource
-            # that has no content files
-            if 'name' in kwargs:
-                if kwargs['name'].lower() == publisher_CUAHSI.lower():
-                    raise ValidationError("Invalid publisher name")
-            if 'url' in kwargs:
-                if kwargs['url'].lower() == 'https://www.cuahsi.org':
-                    raise ValidationError("Invalid publisher URL")
+            # that has no content files, unless it is a Collection
+            if resource.resource_type.lower() != "collectionresource":
+                if 'name' in kwargs:
+                    if kwargs['name'].lower() == publisher_CUAHSI.lower():
+                        raise ValidationError("Invalid publisher name")
+                if 'url' in kwargs:
+                    if kwargs['url'].lower() == 'https://www.cuahsi.org':
+                        raise ValidationError("Invalid publisher URL")
 
         return super(Publisher, cls).create(**kwargs)
 


Test patch:
Repository: hydroshare/hydroshare

Issue:
nightly_zips_cleanup task fails with deep recursion
**Description of the bug**
```
<MaybeEncodingError: Error sending result: ''(1, <ExceptionInfo: Retry(Retry(...), SessionException(SessionException(...), "Error processing IRODS request: 3. stderr follows:\\n\\nb\'remote addresses: 152.54.2.121 ERROR: rmUtil: rm error for /hydroshareZone/home/wwwHydroProxy/zips/2023-11-18, status = -819000 status = -819000 CAT_SUCCESS_BUT_WITH_NO_INFO\\\\n\'"), 0)>, None)''. Reason: '"RecursionError('maximum recursion depth exceeded while calling a Python object')"'.> 
```

This recursion is caused by these lines in the task:
https://github.com/hydroshare/hydroshare/blob/2.10.3/hs_core/tasks.py#L153-L156
The remedy is to change the query to something like:
```
BaseResource.objects.order_by().values_list('resource_federation_path').distinct()
```
so that it returns one single result for each federation path.

The other part of the issue is a recursion error for deeply nested collections in Irods.
We use [irm -rf](https://docs.irods.org/4.2.6/icommands/user/#irm) and the -r fails if there are too many zips from a given day


Steps to reproduce the bug:
Occurs nightly, when there are lots of zip files from a given day

**Expected behavior**
Task should succeed without irm recursion error

**Additional information**
HS v2.10.3



Discussion:

Resolving patch:
diff --git a/hs_core/tasks.py b/hs_core/tasks.py
--- a/hs_core/tasks.py
+++ b/hs_core/tasks.py
@@ -150,8 +150,7 @@ def nightly_zips_cleanup():
     istorage = IrodsStorage()
     if istorage.exists(zips_daily_date):
         istorage.delete(zips_daily_date)
-    federated_prefixes = BaseResource.objects.all().values_list('resource_federation_path')\
-        .distinct()
+    federated_prefixes = BaseResource.objects.order_by().values_list('resource_federation_path').distinct()
 
     for p in federated_prefixes:
         prefix = p[0]  # strip tuple


Test patch:
Repository: hydroshare/hydroshare

Issue:
Failed Celery Tasks should email dev team, not support user
Related to https://github.com/hydroshare/hydroshare/issues/5265 we have some tasks that are failing (in that case due to iRods recursion)
Rather than emailing HELP on task failure, here we send email to DEFAULT_FROM_EMAIL
Also small change to make sure that tasks get marked FAILED.


Discussion:

Resolving patch:
diff --git a/hs_core/tasks.py b/hs_core/tasks.py
--- a/hs_core/tasks.py
+++ b/hs_core/tasks.py
@@ -81,19 +81,20 @@ def __init__(self, error_message):
 
 class HydroshareRequest(Request):
     '''A Celery custom request to log failures.
-    https://docs.celeryq.dev/en/v4.4.7/userguide/tasks.html?#requests-and-custom-requests
+    https://docs.celeryq.dev/en/v5.2.7/userguide/tasks.html#requests-and-custom-requests
     '''
     def on_failure(self, exc_info, send_failed_event=True, return_ok=False):
         super(HydroshareRequest, self).on_failure(
             exc_info,
-            send_failed_event=send_failed_event,
-            return_ok=return_ok
+            # always mark failed
+            send_failed_event=True,
+            return_ok=False
         )
-        warning_message = f'Failure detected for task {self.task.name}'
+        warning_message = f"Failure detected for task {self.task.name}. Exception: {exc_info}"
         logger.warning(warning_message)
         if not settings.DISABLE_TASK_EMAILS:
             subject = 'Notification of failing Celery task'
-            send_mail(subject, warning_message, settings.DEFAULT_FROM_EMAIL, [settings.DEFAULT_SUPPORT_EMAIL])
+            send_mail(subject, warning_message, settings.DEFAULT_FROM_EMAIL, [settings.DEFAULT_FROM_EMAIL])
 
 
 class HydroshareTask(Task):


Test patch:
Repository: hydroshare/hydroshare

Issue:
Publication notifications are inadvertently sent during metadata update (re-deposit)
**Description of the bug**
Publication email (example below) should only be sent upon initial publication of a resource.
However, they are being sent during metadata update on a previously published resource.

> Dear Resource Owner,
The following resource that you submitted for publication: http://www.hydroshare.org/resource/31f30d14c88748d986842d278d125a5c/ has been reviewed and determined to meet HydroShare's minimum metadata standards and community guidelines.
> The publication request was processed by [Crossref.org](https://www.crossref.org/). The Digital Object Identifier (DOI) for your resource is: https://doi.org/10.4211/hs.31f30d14c88748d986842d278d125a5c
> 
> Thank you,
> 
> The HydroShare Team

Relevant code block:
https://github.com/hydroshare/hydroshare/blob/2.13.3/hs_core/tasks.py#L379-L392

**Additional information**
HS v 2.13.3



Discussion:

Resolving patch:
diff --git a/hs_core/tasks.py b/hs_core/tasks.py
--- a/hs_core/tasks.py
+++ b/hs_core/tasks.py
@@ -340,6 +340,8 @@ def manage_task_hourly():
     pending_resources = BaseResource.objects.filter(raccess__published=True,
                                                     doi__endswith=CrossRefSubmissionStatus.PENDING.value)
     for res in pending_resources:
+        # save the doi status
+        is_metadata_update = CrossRefSubmissionStatus.UPDATE_PENDING.value in res.doi
         meta_published_date = res.metadata.dates.all().filter(type='published').first()
         if meta_published_date:
             pub_date = meta_published_date
@@ -389,7 +391,10 @@ def manage_task_hourly():
                 msg_lst.append(msg)
                 logger.debug(response.content)
             else:
-                notify_owners_of_publication_success(res)
+                if is_metadata_update:
+                    logger.info("Crossref deposit successfully updated for resource {}".format(res.short_id))
+                else:
+                    notify_owners_of_publication_success(res)
         else:
             msg_lst.append("{res_id} does not have published date in its metadata.".format(
                 res_id=res.short_id))


Test patch:
Repository: bokeh/bokeh

Issue:
Update six requirement to 1.5.2
It is outdated (1.3.0), giving import errors when you try to run the server:

```
(bokeh_dev)damian@damian-S400CA:/media/datos/Desarrollos/bokeh$ python bokeh-server -d
Traceback (most recent call last):
  File "bokeh-server", line 5, in <module>
    from bokeh.server import start
  File "/media/datos/Desarrollos/bokeh/bokeh/__init__.py", line 6, in <module>
    from . import sampledata
  File "/media/datos/Desarrollos/bokeh/bokeh/sampledata/__init__.py", line 5, in <module>
    from six.moves.urllib.request import urlopen
ImportError: No module named urllib.request
```

Update six requirement to 1.5.2
It is outdated (1.3.0), giving import errors when you try to run the server:

```
(bokeh_dev)damian@damian-S400CA:/media/datos/Desarrollos/bokeh$ python bokeh-server -d
Traceback (most recent call last):
  File "bokeh-server", line 5, in <module>
    from bokeh.server import start
  File "/media/datos/Desarrollos/bokeh/bokeh/__init__.py", line 6, in <module>
    from . import sampledata
  File "/media/datos/Desarrollos/bokeh/bokeh/sampledata/__init__.py", line 5, in <module>
    from six.moves.urllib.request import urlopen
ImportError: No module named urllib.request
```



Discussion:

Resolving patch:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -77,22 +77,32 @@ def package_path(path, package_data_dirs):
         'Jinja2==2.7',
         'MarkupSafe==0.18',
         'Werkzeug==0.9.1',
+        'argparse==1.2.1',
         'greenlet==0.4.1',
         'itsdangerous==0.21',
-        'numpy',
-        'pandas',
-        'python-dateutil',
-        'pytz',
-        'requests',
-        'six',
+        'numpy>=1.7.1',
+        'pandas>=0.11.0',
+        'python-dateutil==2.1',
+        'pytz==2013b',
+        'requests==1.2.3',
+        'six==1.5.2',
+        'wsgiref==0.1.2',
+        'pygments==1.6',
+        'pystache==0.5.3',
+        'markdown==2.3.1',
+        'PyYAML==3.10',
+        # tests
+        'mock==1.0.1',
+        'websocket==0.2.1',
+        'colorama==0.2.7'
     ]
 if sys.version_info[0] != 3:
     REQUIRES.extend([
-        'gevent',
-        'gevent-websocket',
+        'gevent==0.13.8',
+        'gevent-websocket==0.3.6',
     ])
 if sys.platform != "win32":
-    REQUIRES.append('redis')
+    REQUIRES.append('redis==2.7.6')
 
 setup(
     name = 'bokeh',


Test patch:
Repository: bokeh/bokeh

Issue:
output server fails when data contains nans
change `output_file` to `output_server` in the code below to reproduce.

```
import datetime
import pandas
import pandas.io.data as web

output_file("boll.html")

hold()

start = datetime.date(2012, 1, 1)
end = datetime.date.today()
data = web.DataReader('AAPL', 'google', start, end)
close = data['Close'].values  # Type: Numpy array
dates = data.index.values  # Type: Numpy array

# Plotting raw stock data.
x = data.index  # Type: Pandas DatetimeIndex
y = close  # Type: Numpy array
line(x, y, color='blue', x_axis_type='datetime')  # DatetimeIndex vs. Numpy array

# Define simple moving average:
sma50 = pandas.rolling_mean(close,50,min_periods=50)  # Type: Numpy array
print sma50

# Define upper and lower Bollinger band lines:

stdev = pandas.rolling_std(close,50,min_periods=50)  # Type: Numpy array

upperband = sma50 + (2 * stdev)   # Type: Numpy array
lowerband = sma50 - (2 * stdev)   # Type: Numpy array

# Bollinger shading glyph:
reverse_upper = upperband[::-1]   # Type: Numpy array
bandprice = np.append(lowerband, reverse_upper)   # Type: Numpy array
banddates = np.append(dates, dates[::-1])   # Type: Numpy array
patch(pandas.to_datetime(banddates), bandprice, color='#A6CEE3', fill_alpha=0.3, x_axis_type='datetime')   # DatetimeIndex vs. Numpy array
show()
```



Discussion:

Resolving patch:
diff --git a/bokeh/protocol.py b/bokeh/protocol.py
--- a/bokeh/protocol.py
+++ b/bokeh/protocol.py
@@ -28,11 +28,11 @@
 class NumpyJSONEncoder(json.JSONEncoder):
     def default(self, obj):
         if isinstance(obj, pd.Series):
-            return obj.tolist()
+            return self.transform_list(obj.tolist())
         elif isinstance(obj, np.ndarray):
             if obj.dtype.kind == 'M':
-                return obj.astype('datetime64[ms]').astype('int64').tolist()
-            return obj.tolist()
+                obj = obj.astype('datetime64[ms]').astype('int64')
+            return self.transform_list(obj.tolist())
         elif isinstance(obj, np.number):
             if isinstance(obj, np.integer):
                 return int(obj)
@@ -43,6 +43,18 @@ def default(self, obj):
         else:
             return super(NumpyJSONEncoder, self).default(obj)
 
+    def transform_list(self, l):
+        for k, v in enumerate(l):
+            if isinstance(v, list):
+                v = self.transform_list(v)
+            elif np.isnan(v):
+                l[k] = "NaN"
+            elif np.isposinf(v):
+                l[k] = "Infinity"
+            elif np.isneginf(v):
+                l[k] = "-Infinity"
+        return l
+
 def serialize_json(obj, encoder=NumpyJSONEncoder, **kwargs):
     return json.dumps(obj, cls=encoder, **kwargs)
 deserialize_json = json.loads


Test patch:
Repository: bokeh/bokeh

Issue:
datetime x_axis shows the wrong label at year resolution
Special handling for years is mentioned specifically in https://github.com/ContinuumIO/bokeh/pull/291. But that's closed, so I hope I'm right in making a new ticket.

To reproduce:
1. Go to stocks in the gallery (http://bokeh.pydata.org/docs/gallery/stocks.html) 
2. Zoom in on the end of the data in the first graph (Closing Prices). You can see data ends in March 2013.
   ![download](https://cloud.githubusercontent.com/assets/1521009/2583728/0e274bba-b9d5-11e3-9f3c-438788b4951a.png)
3. Zoom out so that the tick spacing is one per year.  Now it appears that the data ends in March 2012.
   ![download 1](https://cloud.githubusercontent.com/assets/1521009/2583746/5398fef0-b9d5-11e3-8664-dadd6cff5905.png)

My guess is the date spacing is being set using the end of the periods (so format year from December 31st instead of from January 1st).



Discussion:
I can reproduce this on master. 


Resolving patch:
diff --git a/bokeh/objects.py b/bokeh/objects.py
--- a/bokeh/objects.py
+++ b/bokeh/objects.py
@@ -150,6 +150,9 @@ class DaysTicker(Ticker):
 class MonthsTicker(Ticker):
     months = List(Int)
 
+class YearsTicker(Ticker):
+    pass
+
 class BasicTicker(Ticker):
     pass
 


Test patch:
Repository: bokeh/bokeh

Issue:
Second round of improvements to properties
This adds validation to remaining properties (e.g. `Dict`, `Either`, `Color`).



Discussion:
I have read it and looks OK...
@mattpap, this is labeled as WIP, do you want to add more things here?

@damianavila, now it's ready ;)

I think this is OK, just a minor comment about a py3 incompatibility...

So, how we're standing with this? There are no big changes here, just more validation, which we could take advantage of.

LGTM, as I said before...

OK, thanks.


Resolving patch:
diff --git a/bokeh/objects.py b/bokeh/objects.py
--- a/bokeh/objects.py
+++ b/bokeh/objects.py
@@ -87,13 +87,13 @@ class ServerDataSource(DataSource):
     owner_username = String()
     # allow us to add some data that isn't on the remote source
     # and join it to the remote data
-    data = Dict() 
+    data = Dict(String, Any)
     # allow us to specify slicing of data on updates
     index_slice = List(Any)
     data_slice = List(Any)
     # allow us to transpose data on updates (for image data)
     transpose = Bool(False)
-    
+
 class PandasDataSource(DataSource):
     """ Represents serverside data.  This gets stored into the plot server's
     database, but it does not have any client side representation.  Instead,


Test patch:
Repository: bokeh/bokeh

Issue:
Add simple tool names for horiz/vert-constrained pan and zoom
Instead of the toolname as "pan", it could be "panx", "pany", "zoomx", "zoomy".  Or "yzoom" and "xzoom".  I'm open to either approach.



Discussion:

Resolving patch:
diff --git a/bokeh/plotting_helpers.py b/bokeh/plotting_helpers.py
--- a/bokeh/plotting_helpers.py
+++ b/bokeh/plotting_helpers.py
@@ -59,7 +59,7 @@ def _glyph_doc(args, props, desc):
     plot : :py:class:`Plot <bokeh.objects.Plot>`
     """ % (desc, params, props)
 
-def _match_data_params(argnames, glyphclass, datasource, serversource, 
+def _match_data_params(argnames, glyphclass, datasource, serversource,
                        args, kwargs):
     """ Processes the arguments and kwargs passed in to __call__ to line
     them up with the argnames of the underlying Glyph
@@ -291,8 +291,16 @@ def _new_xy_plot(x_range=None, y_range=None, plot_width=None, plot_height=None,
             continue
         if tool == "pan":
             tool_obj = PanTool(plot=p, dimensions=["width", "height"])
+        elif tool == "xpan":
+            tool_obj = PanTool(plot=p, dimensions=["width"])
+        elif tool == "ypan":
+            tool_obj = PanTool(plot=p, dimensions=["height"])
         elif tool == "wheel_zoom":
             tool_obj = WheelZoomTool(plot=p, dimensions=["width", "height"])
+        elif tool == "xwheel_zoom":
+            tool_obj = WheelZoomTool(plot=p, dimensions=["width"])
+        elif tool == "ywheel_zoom":
+            tool_obj = WheelZoomTool(plot=p, dimensions=["height"])
         elif tool == "save":
             tool_obj = PreviewSaveTool(plot=p)
         elif tool == "resize":
@@ -322,7 +330,7 @@ def _new_xy_plot(x_range=None, y_range=None, plot_width=None, plot_height=None,
         elif tool == "object_explorer":
             tool_obj = ObjectExplorerTool()
         else:
-            known_tools = "pan, wheel_zoom, box_zoom, save, resize, crosshair, select, previewsave, reset, hover, or embed"
+            known_tools = "pan, xpan, ypan, wheel_zoom, xwheel_zoom, ywheel_zoom, box_zoom, save, resize, crosshair, select, previewsave, reset, hover, or embed"
             raise ValueError("invalid tool: %s (expected one of %s)" % (tool, known_tools))
 
         tool_objs.append(tool_obj)


Test patch:
Repository: bokeh/bokeh

Issue:
Trouble with encoding on Windows
From SO: http://stackoverflow.com/questions/24075840/python-trouble-with-encoding-on-windows-bokeh-plotting-library

Proposed solution, add "encoding" parameter defaulting to UTF-8.



Discussion:
Proposed solutions seems good.

As requested on an original SO answer (http://stackoverflow.com/a/24090474/1414455), my architecture is 64-bit Windows 7 SP1.

Python is 64-bit Python 3.3.0. Latest version of Bokeh installed from Github yesterday.

Can you also provide the locale you are using? Thanks

A call to locale.getdefaultlocale() returns ('en_US', 'cp1252').

Pretty sure my vm was set to that too. I definitely don't understand this. Were there any special characters in your script?

> On Jun 7, 2014, at 16:49, fgnu notifications@github.com wrote:
> 
> A call to locale.getdefaultlocale() returns ('en_US', 'cp1252').
> 
> —
> Reply to this email directly or view it on GitHub.

@fgnu any update?

absent more info, closing. Please re-open if more information comes to light. 

I just had this issue. I am on 64-bit Windows 7 SP1, with 64 BIt Python 3.4. If I change the lines in plotting.py to     

``` python
with open(filename, "wb") as f:
        f.write(html.encode('utf-8'))
```

It works.

I will submit a pull request.


Resolving patch:
diff --git a/bokeh/plotting.py b/bokeh/plotting.py
--- a/bokeh/plotting.py
+++ b/bokeh/plotting.py
@@ -7,6 +7,7 @@
 import os
 import uuid
 import warnings
+import io
 
 from . import browserlib
 from . import _glyph_functions as gf
@@ -19,6 +20,7 @@
 )
 from .resources import Resources
 from .session import Cloud, DEFAULT_SERVER_URL, Session
+from .utils import decode_utf8
 
 logger = logging.getLogger(__name__)
 
@@ -280,8 +282,8 @@ def save(filename=None, resources=None):
         return
 
     html = file_html(curdoc(), resources, _default_file['title'])
-    with open(filename, "w") as f:
-        f.write(html)
+    with io.open(filename, "w", encoding="utf-8") as f:
+        f.write(decode_utf8(html))
 
 def push(session=None, document=None):
     """ Updates the server with the data for the current document.


Test patch: