internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: flask-admin/flask-admin

Issue:
MongoImageInput widget broken in latest release (1.5.3)
Trying to use flask-admin with a mongengine `ImageField` throws:

 ```
TypeError: 'MongoImageInput' object is not callable
```

Looks like the `MongoImageInput` widget got broken [here ](https://github.com/flask-admin/flask-admin/commit/402e9a7a23d3fed04fa97f9b40c93639dfadaa24#diff-1ba51c8939e5bd0f32302da670e4f6d9R51) with a missing tab so it no longer defines `__call__()`.


Discussion:

Resolving patch:
diff --git a/flask_admin/contrib/mongoengine/view.py b/flask_admin/contrib/mongoengine/view.py
--- a/flask_admin/contrib/mongoengine/view.py
+++ b/flask_admin/contrib/mongoengine/view.py
@@ -140,7 +140,8 @@ class MyModelView(BaseModelView):
 
     allowed_search_types = (mongoengine.StringField,
                             mongoengine.URLField,
-                            mongoengine.EmailField)
+                            mongoengine.EmailField,
+                            mongoengine.ReferenceField)
     """
         List of allowed search field types.
     """
@@ -466,7 +467,12 @@ def _search(self, query, search_term):
         criteria = None
 
         for field in self._search_fields:
-            flt = {'%s__%s' % (field.name, op): term}
+            if type(field) == mongoengine.ReferenceField:
+                import re
+                regex = re.compile('.*%s.*' % term)
+            else:
+                regex = term
+            flt = {'%s__%s' % (field.name, op): regex}
             q = mongoengine.Q(**flt)
 
             if criteria is None:
diff --git a/flask_admin/contrib/mongoengine/widgets.py b/flask_admin/contrib/mongoengine/widgets.py
--- a/flask_admin/contrib/mongoengine/widgets.py
+++ b/flask_admin/contrib/mongoengine/widgets.py
@@ -47,10 +47,8 @@ class MongoImageInput(object):
                 ' <input type="checkbox" name="%(marker)s">Delete</input>'
                 '</div>')
 
-
-def __call__(self, field, **kwargs):
+    def __call__(self, field, **kwargs):
         kwargs.setdefault('id', field.id)
-
         placeholder = ''
         if field.data and isinstance(field.data, ImageGridFsProxy):
             args = helpers.make_thumb_args(field.data)


Test patch:
Repository: flask-admin/flask-admin

Issue:
Small typo in flask_admin.base
Should be provided rather than povided.


Discussion:

Resolving patch:
diff --git a/flask_admin/base.py b/flask_admin/base.py
--- a/flask_admin/base.py
+++ b/flask_admin/base.py
@@ -257,7 +257,7 @@ def create_blueprint(self, admin):
                 self.static_folder = 'static'
                 self.static_url_path = '/static/admin'
 
-        # If name is not povided, use capitalized endpoint name
+        # If name is not provided, use capitalized endpoint name
         if self.name is None:
             self.name = self._prettify_class_name(self.__class__.__name__)
 


Test patch:
Repository: flask-admin/flask-admin

Issue:
`Save and continue` issue in edit view
I have a flask_admin application behind a reverse proxy which rewrites the prefix of the url.
The issue is that when working in the edit view, the `Save and continue` button uses `request.url` which is missing the prefix.
So we are redirected to a wrong url.

It works ok in the create_view but fails in the edit_view.

I think we should change [this](https://github.com/flask-admin/flask-admin/blob/master/flask_admin/model/base.py#L2147):
```python
return redirect(request.url)
```

To same as create_view:

```python
return redirect(self.get_url('.edit_view', id=self.get_pk_value(model)))
```

I'll create a simple PR for it.


Discussion:

Resolving patch:
diff --git a/flask_admin/model/base.py b/flask_admin/model/base.py
--- a/flask_admin/model/base.py
+++ b/flask_admin/model/base.py
@@ -2144,7 +2144,7 @@ def edit_view(self):
                 if '_add_another' in request.form:
                     return redirect(self.get_url('.create_view', url=return_url))
                 elif '_continue_editing' in request.form:
-                    return redirect(request.url)
+                    return redirect(self.get_url('.edit_view', id=self.get_pk_value(model)))
                 else:
                     # save button
                     return redirect(self.get_save_return_url(model, is_created=False))


Test patch:
Repository: flask-admin/flask-admin

Issue:
WTForms 3.0.0 breaks DateTimeField format compatibility
When using flask-admin v1.5.8 and WTForms 3.0.0 the form doesn't validate on submit when a `DateTimeField` is used with a custom format. The error message `Not a valid datetime value.` appears below the field.

To reproduce it just configure a `DateTimeField` with a different format from the default value. E.g.:

```
    form_args = {
        'my_datetime_field': {'format': '%d/%m/%Y %H:%M:%S'},
    }
```

This happens because the flask-admin's `DateTimeField` (which inherits from WTForms' `DateTimeField`) doesn't pass the `format` parameter to its parent constructor. I'll make a pull request with a fix that solves this issue and works for WTForms 2.3 and 3.0.


Discussion:

Resolving patch:
diff --git a/flask_admin/form/fields.py b/flask_admin/form/fields.py
--- a/flask_admin/form/fields.py
+++ b/flask_admin/form/fields.py
@@ -36,9 +36,8 @@ def __init__(self, label=None, validators=None, format=None, **kwargs):
             :param kwargs:
                 Any additional parameters
         """
-        super(DateTimeField, self).__init__(label, validators, **kwargs)
-
-        self.format = format or '%Y-%m-%d %H:%M:%S'
+        super(DateTimeField, self).__init__(
+            label, validators, format or '%Y-%m-%d %H:%M:%S', **kwargs)
 
 
 class TimeField(fields.Field):


Test patch:
Repository: flask-admin/flask-admin

Issue:
MenuLink's `target` parameter does nothing/is not rendered when template_mode="bootstrap4"
It appears to be due to a missing jinga conditional check in the `menu_links` macro.


Discussion:

Resolving patch:
diff --git a/flask_admin/model/typefmt.py b/flask_admin/model/typefmt.py
--- a/flask_admin/model/typefmt.py
+++ b/flask_admin/model/typefmt.py
@@ -36,8 +36,8 @@ def bool_formatter(view, value):
             Value to check
     """
     glyph = 'ok-circle' if value else 'minus-sign'
-    fa = 'fas fa-check-circle' if value else 'far fa-circle'
-    return Markup('<span class="%s glyphicon glyphicon-%s icon-%s"></span>' % (fa, glyph, glyph))
+    fa = 'fa-check-circle' if value else 'fa-minus-circle'
+    return Markup('<span class="fa %s glyphicon glyphicon-%s icon-%s"></span>' % (fa, glyph, glyph))
 
 
 def list_formatter(view, values):


Test patch:
Repository: flask-admin/flask-admin

Issue:
sqla.validators.ItemsRequired doesn't sheck for "_should_delete"
I'm using this for an inline model where items can be added and deleted but at least one item is required.

```python
    form_args = {
        'names': {
            'validators': [sqla.validators.ItemsRequired()]
        }
    }

    inline_models = (
        (
            ChannelName,
            dict(
                form_label='Name',
                form_columns=('id', 'name', 'default')
            )
        ),
    )
```

This works on creating items but it's still possible to delete all items.

The validator checks for `if len(field.data) < self.min` only but it should only count items not selected for deletion.


Discussion:

Resolving patch:
diff --git a/flask_admin/contrib/sqla/validators.py b/flask_admin/contrib/sqla/validators.py
--- a/flask_admin/contrib/sqla/validators.py
+++ b/flask_admin/contrib/sqla/validators.py
@@ -6,6 +6,8 @@
 except ImportError:
     from wtforms.validators import Required as InputRequired
 
+from flask_admin._compat import filter_list
+
 
 class Unique(object):
     """Checks field value unicity against specified table field.
@@ -55,7 +57,8 @@ def __init__(self, min=1, message=None):
         self.min = min
 
     def __call__(self, form, field):
-        if len(field.data) < self.min:
+        items = filter_list(lambda e: not field.should_delete(e), field.entries)
+        if len(items) < self.min:
             if self.message is None:
                 message = field.ngettext(
                     u"At least %(num)d item is required",


Test patch:
diff --git a/flask_admin/tests/sqla/test_inlineform.py b/flask_admin/tests/sqla/test_inlineform.py
--- a/flask_admin/tests/sqla/test_inlineform.py
+++ b/flask_admin/tests/sqla/test_inlineform.py
@@ -155,6 +155,17 @@ class UserModelView(ModelView):
     assert User.query.count() == 1
     assert UserEmail.query.count() == 1
 
+    # Attempted delete, prevented by ItemsRequired
+    data = {
+        'name': 'hasEmail',
+        'del-emails-0': 'on',
+        'emails-0-email': 'foo@bar.com',
+    }
+    rv = client.post('/admin/user/edit/?id=1', data=data)
+    assert rv.status_code == 200
+    assert User.query.count() == 1
+    assert UserEmail.query.count() == 1
+
 
 def test_inline_form_ajax_fk():
     app, db, admin = setup()
Repository: flask-admin/flask-admin

Issue:
_request_ctx_stack DeprecationWarning for Flask 2.2+
```
  /usr/local/lib/python3.10/site-packages/flask_admin/form/widgets.py:2: DeprecationWarning: '_request_ctx_stack' is deprecated and will be remoevd in Flask 2.3.
    from flask.globals import _request_ctx_stack
```

I am unsure what the correct solution is, but it looks like you can get the jinja environment right from app (based on this https://flask.palletsprojects.com/en/2.2.x/templating/#registering-filters). So maybe something like this

```diff
# form/widgets.py
< from flask.globals import _request_ctx_stack
> from flask import current_app

<        ctx = _request_ctx_stack.top
<        jinja_env = ctx.app.jinja_env
>        jinja_env = current_app.jinja_env

```


Discussion:

Resolving patch:
diff --git a/flask_admin/form/widgets.py b/flask_admin/form/widgets.py
--- a/flask_admin/form/widgets.py
+++ b/flask_admin/form/widgets.py
@@ -1,5 +1,5 @@
 from wtforms import widgets
-from flask.globals import _request_ctx_stack
+from flask import current_app
 from flask_admin.babel import gettext, ngettext
 from flask_admin import helpers as h
 
@@ -89,9 +89,6 @@ def __init__(self, template):
         self.template = template
 
     def __call__(self, field, **kwargs):
-        ctx = _request_ctx_stack.top
-        jinja_env = ctx.app.jinja_env
-
         kwargs.update({
             'field': field,
             '_gettext': gettext,
@@ -99,5 +96,5 @@ def __call__(self, field, **kwargs):
             'h': h,
         })
 
-        template = jinja_env.get_template(self.template)
+        template = current_app.jinja_env.get_template(self.template)
         return template.render(kwargs)


Test patch:
Repository: flask-admin/flask-admin

Issue:
Fix search placeholder
Fixes #1723

The search box placeholder now shows the appropriate translation, but also expands to accommodate more placeholder text, and truncates with '...' when the placeholder text becomes too large.


Discussion:
This is ready to go. Anyone mind reviewing?

Resolving patch:
diff --git a/flask_admin/contrib/sqla/view.py b/flask_admin/contrib/sqla/view.py
--- a/flask_admin/contrib/sqla/view.py
+++ b/flask_admin/contrib/sqla/view.py
@@ -624,10 +624,10 @@ class MyModelView(BaseModelView):
         for searchable in self.column_searchable_list:
             if isinstance(searchable, InstrumentedAttribute):
                 placeholders.append(
-                    self.column_labels.get(searchable.key, searchable.key))
+                    str(self.column_labels.get(searchable.key, searchable.key)))
             else:
                 placeholders.append(
-                    self.column_labels.get(searchable, searchable))
+                    str(self.column_labels.get(searchable, searchable)))
 
         return u', '.join(placeholders)
 
diff --git a/flask_admin/form/rules.py b/flask_admin/form/rules.py
--- a/flask_admin/form/rules.py
+++ b/flask_admin/form/rules.py
@@ -106,7 +106,7 @@ def __call__(self, form, form_opts=None, field_args={}):
         result = []
 
         for r in self.rules:
-            result.append(r(form, form_opts, field_args))
+            result.append(str(r(form, form_opts, field_args)))
 
         return Markup(self.separator.join(result))
 


Test patch:
Repository: microsoft/PubSec-Info-Assistant

Issue:
Error: Diagnostic settings does not support retention for new diagnostic settings
After a couple of attempts to deploy the PubSec suite (deploy, delete, repeat switching from australia east to eastus), I began to encounter this error 'Diagnostic settings does not support retention for new diagnostic settings.' and the deployment would fail. With each attempt I had deleted all of the services created by the previous attempt, and changed the WORKSPACE="" to be unique. 

I happened to come across this article:
https://learn.microsoft.com/en-us/azure/azure-monitor/essentials/migrate-to-azure-storage-lifecycle-policy

After changing lines 138, 146 and 156 in the main.bicep file, setting the days value to 0 (instead of the default value of 30) the deployment completed successfully. 

based on the information in the article, we'll need to update these setting after September when the deprecation comes into effect.



Discussion:
@RandomDrew , thanks for the heads up. I will add this to our backlog for the 0.4-Delta release 

Resolving patch:
diff --git a/app/backend/app.py b/app/backend/app.py
--- a/app/backend/app.py
+++ b/app/backend/app.py
@@ -192,3 +192,10 @@ def get_all_upload_status():
         logging.exception("Exception in /getalluploadstatus")
         return jsonify({"error": str(e)}), 500
     return jsonify(results)
+
+
+# Return AZURE_OPENAI_CHATGPT_DEPLOYMENT
+@app.route("/getInfoData")
+def get_info_data():
+    response = jsonify({"AZURE_OPENAI_CHATGPT_DEPLOYMENT": f"{AZURE_OPENAI_CHATGPT_DEPLOYMENT}"})
+    return response
\ No newline at end of file


Test patch:
Repository: microsoft/PubSec-Info-Assistant

Issue:
Web app doesn't start when there is new AOAI resource used
**Describe the bug**
With the new Delta version the web app doesn't start when there is new AOAI resource used. 

**To Reproduce**
Steps to reproduce the behavior:
1. Deploy a brand new Delta version with new AOAI resource (USE_EXISTING_AOAI=false)
2. Web app expecting model deployment with the name of "chat" but the default name is "gpt-35-turbo-16k"

**Expected behavior**
Web app up and running :)

**Delta version details**
 - GitHub branch: main
 - Latest commit: 0cc8d71


Discussion:
Please keep in mind that if you just simply change the name of the model deployment name to "chat", in this case web app will be fine, but enrichmentweb app won't work. 

Resolving patch:
diff --git a/app/backend/app.py b/app/backend/app.py
--- a/app/backend/app.py
+++ b/app/backend/app.py
@@ -41,7 +41,7 @@
 AZURE_OPENAI_SERVICE = os.environ.get("AZURE_OPENAI_SERVICE") or "myopenai"
 AZURE_OPENAI_RESOURCE_GROUP = os.environ.get("AZURE_OPENAI_RESOURCE_GROUP") or ""
 AZURE_OPENAI_CHATGPT_DEPLOYMENT = (
-    os.environ.get("AZURE_OPENAI_CHATGPT_DEPLOYMENT") or "chat"
+    os.environ.get("AZURE_OPENAI_CHATGPT_DEPLOYMENT") or "gpt-35-turbo-16k"
 )
 AZURE_OPENAI_CHATGPT_MODEL_NAME = ( os.environ.get("AZURE_OPENAI_CHATGPT_MODEL_NAME") or "")
 AZURE_OPENAI_CHATGPT_MODEL_VERSION = ( os.environ.get("AZURE_OPENAI_CHATGPT_MODEL_VERSION") or "")


Test patch:
Repository: microsoft/PubSec-Info-Assistant

Issue:
Text Enrichment function not quoting blob paths correctly
We have some files with percentage (%) symbols in them, which appear to cause an issue when getting to the Text Enrichment stage of the Function App due to the way the `get_blob_and_sas` function works. Example file name: `Unemployment rate back up to 3.7% in October _ Australian Bureau of Statistics.pdf`

I would suggest replacing the code that manually substitutes spaces (below) with a proper URL quoting function like `blob_path = urllib.parse.quote(blob_path)`

https://github.com/microsoft/PubSec-Info-Assistant/blob/7fa4561652211b023965d4522b2bfd7168af4060/functions/shared_code/utilities_helper.py#L52





Discussion:

Resolving patch:
diff --git a/functions/shared_code/utilities_helper.py b/functions/shared_code/utilities_helper.py
--- a/functions/shared_code/utilities_helper.py
+++ b/functions/shared_code/utilities_helper.py
@@ -3,6 +3,7 @@
 
 import os
 import logging
+import urllib.parse
 from datetime import datetime, timedelta
 from azure.storage.blob import generate_blob_sas, BlobSasPermissions
 
@@ -48,7 +49,7 @@ def  get_blob_and_sas(self, blob_path):
             permission=BlobSasPermissions(read=True),
             expiry=datetime.utcnow() + timedelta(hours=1)
         )
+        blob_path = urllib.parse.quote(blob_path)
         source_blob_path = f'{self.azure_blob_storage_endpoint}{blob_path}?{sas_token}'
-        source_blob_path = source_blob_path.replace(" ", "%20")
         logging.info("Path and SAS token for file in azure storage are now generated \n")
         return source_blob_path
\ No newline at end of file


Test patch:
Repository: microsoft/PubSec-Info-Assistant

Issue:
Tags can only contain ASCII, fails silently
**Describe the bug**
Adding tags containing none-ascii characters when uploading files results in an error.
In the console: 
`deserializationPolicy.js:164  Uncaught (in promise) RestError: The metadata specified is invalid. It has characters that are not permitted.`

The reason is that Azure blob storage metadata can only contain ASCII.

**To Reproduce**
1. Go to *Manage content*, *Upload files*
2. Add a tag: `rødgrød`
3. Drop a file in the square.

The file is written to the log-database ("uploaded"), but it never actually reaches the blob storage.

**Expected behavior**
* At least display an error in the UI.
* The tag should be encoded in some way and decoded when in use.

**Alpha version details**
 - GitHub branch: main



Discussion:

Resolving patch:
diff --git a/app/enrichment/app.py b/app/enrichment/app.py
--- a/app/enrichment/app.py
+++ b/app/enrichment/app.py
@@ -10,6 +10,7 @@
 import base64
 import requests
 import random
+from urllib.parse import unquote
 from azure.storage.blob import BlobServiceClient
 from azure.storage.queue import QueueClient, TextBase64EncodePolicy
 from azure.search.documents import SearchClient
@@ -266,14 +267,15 @@ def get_tags_and_upload_to_cosmos(blob_service_client, blob_path):
     tags = blob_properties.metadata.get("tags")
     if tags is not None:
         if isinstance(tags, str):
-            tags_list = [tags]
+            tags_list = [unquote(tags)]
         else:
-            tags_list = tags.split(",")
+            tags_list = [unquote(tag) for tag in tags.split(",")]
     else:
         tags_list = []
     # Write the tags to cosmos db
     tagsHelper.upsert_document(blob_path, tags_list)
     return tags_list
+
         
 @app.on_event("startup") 
 @repeat_every(seconds=5, logger=log, raise_exceptions=True)


Test patch:
Repository: microsoft/PubSec-Info-Assistant

Issue:
Fail to process documents due to missing language pickle file
**Describe the bug**
Upon installation, documents uploaded fail to process with error message and 200 response code in logs:
![Screenshot 2023-11-22 093007](https://github.com/microsoft/PubSec-Info-Assistant/assets/93607917/207fe184-4e99-45af-98b6-2fa6f56885fb)

Upon investigation, it appears that the download of the English language file has failed, though the ZIP file is present:

![Screenshot 2023-11-22 112703](https://github.com/microsoft/PubSec-Info-Assistant/assets/93607917/0ea018f7-b5b9-4fed-9ac5-133e0e087ceb)

Per other deployments, this ZIP file should have been extracted. During runtime, the service attempts to download the file again, but then sees the ZIP file, and stops. Likely the root cause is the file not being extracted. 

Note, the built-in gzip tool is unable to decompress as it does not recognize a ".zip" extension. Unknown if that is related to the file not being decompressed?


**Version details**
 - GitHub branch: main (Delta)
 - Latest commit: 
    - @lon-tierney ➜ /workspaces/PubSec-Info-Assistant (main) $ git log -n 1 main
commit d46657d84e63bccbd236817b9c18bceab46665aa (HEAD -> main, origin/main, origin/HEAD)
Merge: fbcd78f 679f7f5
Author: ryonsteele <ryonsteele@gmail.com>
Date:   Tue Nov 21 13:24:29 2023 -0500

    Merge pull request #351 from microsoft/ryonsteele/6258-deploymentname-documentation
    
    Update default bicep params to use default flavor model for deployment name

**Additional context**
Add any other context about the problem here.



Discussion:

Resolving patch:
diff --git a/functions/shared_code/utilities.py b/functions/shared_code/utilities.py
--- a/functions/shared_code/utilities.py
+++ b/functions/shared_code/utilities.py
@@ -6,13 +6,29 @@
 import html
 from datetime import datetime
 from enum import Enum
+import zipfile
+import os
 from azure.storage.blob import BlobServiceClient
 from shared_code.utilities_helper import UtilitiesHelper
 from nltk.tokenize import sent_tokenize
 import tiktoken
 import nltk
+# Try to download using nltk.download
 nltk.download('punkt')
 
+punkt_dir = os.path.join(nltk.data.path[0], 'tokenizers/punkt')
+
+# Check if the 'punkt' directory exists
+if not os.path.exists(punkt_dir):
+    punkt_zip_path = os.path.join(nltk.data.path[0], 'tokenizers/punkt.zip')
+
+    # If the 'punkt.zip' file exists, unzip it
+    if os.path.exists(punkt_zip_path):
+        with zipfile.ZipFile(punkt_zip_path, 'r') as zip_ref:
+            zip_ref.extractall(os.path.join(nltk.data.path[0], 'tokenizers/'))
+    else:
+        raise Exception("Failed to download 'punkt' package")
+
 class ParagraphRoles(Enum):
     """ Enum to define the priority of paragraph roles """
     PAGE_HEADER      = 1


Test patch: