internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: ckan/ckan

Issue:
Wrong auth function used when editing organizations
**CKAN version**
2.9.4

**Describe the bug**
Organization edit view uses group_update auth function instead of organization_update

**Steps to reproduce**
Steps to reproduce the behavior:
- Override `group_update` auth function to always fail (allows only sysadmins)
- Attempt to edit an organization as a regular user

**Expected behavior**
Editing succeeds

**Additional details**
This is because `ckan.views.group.EditGroupView._prepare` calls `check_access` instead of the group type aware `_check_access`.

PR incoming



Discussion:

Resolving patch:
diff --git a/ckan/views/group.py b/ckan/views/group.py
--- a/ckan/views/group.py
+++ b/ckan/views/group.py
@@ -1064,7 +1064,7 @@ def _prepare(self, id, is_organization, data=None):  # noqa
 
         try:
             _action(u'group_show')(context, data_dict)
-            check_access(u'group_update', context)
+            _check_access(u'group_update', context, {u'id': id})
         except NotAuthorized:
             base.abort(403, _(u'Unauthorized to create a group'))
         except NotFound:


Test patch:
Repository: ckan/ckan

Issue:
The title of a dataset is translated in some cases
**CKAN version**
2.8.4

**Describe the bug**
The title of a dataset is translated in some cases if you run ckan localized.

**Steps to reproduce**
Steps to reproduce the behavior:

- run ckan localized, choose the german language (de)
- create a new dataset with the title "private"
- review the created dataset in the detail page: It shows up as "Privat" in title and Breadcrumb

Same happens for example if you name the created dataset "Draft" - it will lead to the title "Entwurf".

**Expected behavior**
The dataset keeps it's given name (stays untranslated).

**Additional details**
![grafik](https://user-images.githubusercontent.com/67383107/149520488-5dcda39b-992f-442d-ad52-3c439fee3a34.png)
![grafik](https://user-images.githubusercontent.com/67383107/149520547-f5f76241-ca9d-42d3-a88e-28305c87d612.png)



Discussion:

Resolving patch:
diff --git a/ckan/lib/helpers.py b/ckan/lib/helpers.py
--- a/ckan/lib/helpers.py
+++ b/ckan/lib/helpers.py
@@ -2748,8 +2748,7 @@ def get_translated(data_dict, field):
     try:
         return data_dict[field + u'_translated'][language]
     except KeyError:
-        val = data_dict.get(field, '')
-        return _(val) if val and isinstance(val, str) else val
+        return data_dict.get(field, '')
 
 
 @core_helper


Test patch:
Repository: ckan/ckan

Issue:
organization_facets not working, instead replaced by group_facets but no description given about the change
**CKAN version**
2.9

**Describe the bug**
[organization_facets](https://docs.ckan.org/en/2.9/extensions/plugin-interfaces.html#ckan.plugins.interfaces.IFacets.organization_facets) that were working in the 2.8 version of ckan are not working any more and there is no description in the documentation about changing of any thing. Instead group_facets function is being used to render facets on organization page, which was supposed to work on group page.

**Steps to reproduce**
I dig deep down into the framework code and got to know that upexpected and unnecessary changes have been made which are not even documented. I researching thoroughly I got to know that organization_facet is nowehere registering or updating our facets instead [group_facets](https://docs.ckan.org/en/2.9/extensions/plugin-interfaces.html#ckan.plugins.interfaces.IFacets.group_facets), which is supposed to work on groups page is being used to render facets on organization page

**Expected behavior**
The organization_facets function supposed to work as it used to work in ckan 2.8 version i.e. render the facets on organization page but now it is not updating the facet_dict that needs to be rendered.

**Additional details**
links have been attached and the necessary detail is provided regarding the issue


Discussion:
@amercader any updates on this issue?

@amercader @hussnainwithss @mominimtiaz786 
The issue exists because the [function](https://github.com/ckan/ckan/blob/2.8/ckan/controllers/organization.py#L27) for organizations was not migrated from pylons to flask correctly.

@TomeCirun was working on this and he would provide a PR later this week

Resolving patch:
diff --git a/ckan/views/group.py b/ckan/views/group.py
--- a/ckan/views/group.py
+++ b/ckan/views/group.py
@@ -377,7 +377,11 @@ def pager_url(q=None, page=None):  # noqa
 
 def _update_facet_titles(facets, group_type):
     for plugin in plugins.PluginImplementations(plugins.IFacets):
-        facets = plugin.group_facets(facets, group_type, None)
+        facets = (
+            plugin.group_facets(facets, group_type, None)
+            if group_type == "group"
+            else plugin.organization_facets(facets, group_type, None)
+        )
     return facets
 
 


Test patch:
Repository: ckan/ckan

Issue:
Obsolete requirements
**CKAN version**

2.9.5

**Describe the bug**

Obsolete requirements:
* pytz is pinned to 2016, https://github.com/ckan/ckan/blob/2.9/requirements.in#L23
* Zope.interface is pinned to a version from 2016, prior to python 3.6 support. https://github.com/ckan/ckan/blob/2.9/requirements.in#L38 (ref: https://pypi.org/project/zope.interface/)

Specifically, (at least on python 3) the zope.interface pin is the one that's preventing us from using modern setuptools and requiring a pin.  (at least, until one installs ckanext-dcat, which is limited to setuptools<58.0 due to rdflib-jsonld)

**Proposed fix**

Unpin pytz. Update zope.interface to <5.0




Discussion:
:+1: from me, these seem like sensible requirements to upgrade even for a point release.

Resolving patch:
diff --git a/ckan/lib/search/index.py b/ckan/lib/search/index.py
--- a/ckan/lib/search/index.py
+++ b/ckan/lib/search/index.py
@@ -52,6 +52,7 @@ def clear_index() -> None:
     query = "+site_id:\"%s\"" % (config.get_value('ckan.site_id'))
     try:
         conn.delete(q=query)
+        conn.commit()
     except socket.error as e:
         err = 'Could not connect to SOLR %r: %r' % (conn.url, e)
         log.error(err)


Test patch:
Repository: ckan/ckan

Issue:
error [500] - when providing template-bs3 in the config
**CKAN version**
Master


**Describe the bug**
![image](https://user-images.githubusercontent.com/72216462/156154500-4ea303a0-a28c-445f-b88c-c3cc7c5b9203.png)



**Steps to reproduce**
Go to `ckan.ini` file and change the default option for `ckan.base_templates_folder` to `templates-bs3` and when you run `CKAN` you will get `500`



The problem occurs because `CKAN Master` could not find `flask.pop_message()` it was removed in #6239.
I will make `PR` by the end of the day and  will replace it with the built-in `h.get_flashed_messages`.



Discussion:

Resolving patch:
diff --git a/ckan/lib/helpers.py b/ckan/lib/helpers.py
--- a/ckan/lib/helpers.py
+++ b/ckan/lib/helpers.py
@@ -2695,6 +2695,9 @@ def resource_formats() -> dict[str, list[str]]:
     global _RESOURCE_FORMATS
     if not _RESOURCE_FORMATS:
         format_file_path = config.get_value('ckan.resource_formats')
+        if not format_file_path:
+            format_file_path = resource_formats_default_file()
+
         with open(format_file_path, encoding='utf-8') as format_file:
             try:
                 file_resource_formats = json.loads(format_file.read())


Test patch:
Repository: ckan/ckan

Issue:
Searching organization with trailing space failed
**CKAN version**
2.9

**Describe the bug**
When doing an organization search, the search terms are not trimmed. Searching with `GSA ` will not find organization **GSA**.

**Steps to reproduce**
https://demo.ckan.org/organization/?q=organization+
It gives no results.

**Expected behavior**
Show results including **Sample Organization**.

**Additional details**
Search terms need to be trimmed.



Discussion:
@FuhuXia I am interested to work in this issue.

Resolving patch:
diff --git a/ckan/logic/action/get.py b/ckan/logic/action/get.py
--- a/ckan/logic/action/get.py
+++ b/ckan/logic/action/get.py
@@ -339,7 +339,7 @@ def _group_or_org_list(
         if errors:
             raise ValidationError(errors)
     sort = data_dict.get('sort') or config.get_value('ckan.default_group_sort')
-    q = data_dict.get('q')
+    q = data_dict.get('q', '').strip()
 
     all_fields = asbool(data_dict.get('all_fields', None))
 


Test patch:
Repository: ckan/ckan

Issue:
remove extra comma
Because typo of additional comma we see: 
`jinja2.exceptions.TemplateSyntaxError: invalid syntax for function call expression`

- [ ] includes tests covering changes
- [ ] includes updated documentation
- [ ] includes user-visible changes
- [ ] includes API changes
- [ ] includes bugfix for possible backport

Please [X] all the boxes above that apply



Discussion:

Resolving patch:
diff --git a/ckan/views/api.py b/ckan/views/api.py
--- a/ckan/views/api.py
+++ b/ckan/views/api.py
@@ -478,7 +478,7 @@ def snippet(snippet_path, ver=API_REST_DEFAULT_VERSION):
 def i18n_js_translations(lang, ver=API_REST_DEFAULT_VERSION):
 
     if lang not in get_locales_from_config():
-        return _finish_bad_request('Unknown locale: {}'.format(lang))
+        return _finish_bad_request(u'Unknown locale: {}'.format(lang))
 
     ckan_path = os.path.join(os.path.dirname(__file__), u'..')
     source = os.path.abspath(os.path.join(ckan_path, u'public',


Test patch:
diff --git a/ckan/tests/controllers/test_api.py b/ckan/tests/controllers/test_api.py
--- a/ckan/tests/controllers/test_api.py
+++ b/ckan/tests/controllers/test_api.py
@@ -274,6 +274,7 @@ def test_jsonp_does_not_work_on_post_requests(self, app):
             [dataset1["name"], dataset2["name"]]
         )
 
+
 def test_i18n_only_known_locales_are_accepted(app):
 
     url = url_for("api.i18n_js_translations", ver=2, lang="fr")
Repository: ckan/ckan

Issue:
Documented form setting create_on_ui_requires_resources not implemented
**CKAN 2.9**

**Describe the bug**
The form setting `ckan.dataset.create_on_ui_requires_resources` [is documented](https://ckan.readthedocs.io/en/2.9/maintaining/configuration.html#ckan-dataset-create-on-ui-requires-resources), but does not appear to be implemented.

**Steps to reproduce**
Set `ckan.dataset.create_on_ui_requires_resources` to `False` in `ckan.ini`, restart CKAN, observe that resources are still required to be added when adding a new dataset.

**Expected behavior**
Setting `ckan.dataset.create_on_ui_requires_resources` to `False` should make the adding of resources optional.

**Additional details**
As of this writing, it appears not to be implemented at all. Running `grep -Ri create_on_ui_requires_resources` on the master branch of the ckan repository only returns a hit in `ckan/config/config_declaration.yaml`, which describes the expected behaviour but does not implement it.


Discussion:

Resolving patch:
diff --git a/ckan/views/dataset.py b/ckan/views/dataset.py
--- a/ckan/views/dataset.py
+++ b/ckan/views/dataset.py
@@ -566,13 +566,27 @@ def post(self, package_type: str) -> Union[Response, str]:
             data_dict[u'type'] = package_type
             pkg_dict = get_action(u'package_create')(context, data_dict)
 
+            create_on_ui_requires_resources = config.get_value(
+                'ckan.dataset.create_on_ui_requires_resources'
+            )
             if ckan_phase:
-                # redirect to add dataset resources
-                url = h.url_for(
-                    u'{}_resource.new'.format(package_type),
-                    id=pkg_dict[u'name']
+                if create_on_ui_requires_resources:
+                    # redirect to add dataset resources if
+                    # create_on_ui_requires_resources is set to true
+                    url = h.url_for(
+                        u'{}_resource.new'.format(package_type),
+                        id=pkg_dict[u'name']
+                    )
+                    return h.redirect_to(url)
+
+                get_action(u'package_update')(
+                    cast(Context, dict(context, allow_state_change=True)),
+                    dict(pkg_dict, state=u'active')
+                )
+                return h.redirect_to(
+                    u'{}.read'.format(package_type),
+                    id=pkg_dict["id"]
                 )
-                return h.redirect_to(url)
 
             return _form_save_redirect(
                 pkg_dict[u'name'], u'new', package_type=package_type


Test patch:
Repository: ckan/ckan

Issue:
Updating a non-existing resource causes an internal sever error
**CKAN version**
master

**Describe the bug**
Trying to update a resource that does not exist causes an internal server error instead of a not found error

**Steps to reproduce**
- using `httpie`: `http POST https://demo.ckan.org/api/action/resource_update id=doesnotexist`
- Causes an internal server error

**Expected behavior**
- "Not found" error

**Additional details**
Caused by [this line](https://github.com/ckan/ckan/blob/master/ckan/logic/action/update.py#L76) being placed before [these lines](https://github.com/ckan/ckan/blob/master/ckan/logic/action/update.py#L78-L80).



Discussion:
Hey, @bzar, I just tested this via curl, I sent an id that doesn't exist, and it gives me the proper error message: 404 not found.
https://github.com/ckan/ckan/blob/7586d78682c30f205027522214f33ee2bf413055/ckan/logic/action/update.py#L73-L76
if you look at the code you can see in line 73 we have another check

```python 
if resource is None
```

and here are my logs:
![image](https://user-images.githubusercontent.com/72216462/174585256-fd39dc89-87cd-4591-9611-9242e68aa163.png)

![image](https://user-images.githubusercontent.com/72216462/174585391-8d0d13f4-c4ca-4ed4-81b3-71868305818d.png)

It's actually 2.9 and earlier issue, master has a check for it. 2.9 does not https://github.com/ckan/ckan/blob/ca537ad25e13320b6fbf91f41031eadc30500f12/ckan/logic/action/update.py#L73-L79

Resolving patch:
diff --git a/ckan/logic/action/update.py b/ckan/logic/action/update.py
--- a/ckan/logic/action/update.py
+++ b/ckan/logic/action/update.py
@@ -71,13 +71,13 @@ def resource_update(context, data_dict):
         data_dict['url'] = ''
 
     resource = model.Resource.get(id)
-    context["resource"] = resource
-    old_resource_format = resource.format
-
     if not resource:
         log.debug('Could not find resource %s', id)
         raise NotFound(_('Resource was not found.'))
 
+    context["resource"] = resource
+    old_resource_format = resource.format
+
     _check_access('resource_update', context, data_dict)
     del context["resource"]
 


Test patch:
Repository: ckan/ckan

Issue:
Robots.txt can no longer be easily customised
**CKAN version**

2.9

**Describe the bug**

`robots.txt` was moved back to the `public` directory as part of #4801. However, this reverts the implementation of https://github.com/ckan/ideas-and-roadmap/issues/178 and makes it harder to customise the file (it can still be overridden with a different version, but not using Jinja syntax).



Discussion:
In case it's relevant to anyone, serving `robots.txt` from the templates directory can be re-enabled with a blueprint rule similar to:

    blueprint.add_url_rule(u'/robots.txt', 'robots', view_func=lambda: render('robots.txt'))

(Also, if the `robots.txt` file contains `{% ckan_extends %}`, that directive will need to be removed.)
yes very strange

routing removed here: https://github.com/ckan/ckan/commit/8ee21484d3ceb6c0f0d9eb8d239a3a97baf2e5ac
jinja template parts removed here: https://github.com/ckan/ckan/commit/4a05d5cd09ada5abde1dd74ae545679eef61aad8
@amercader was there a reason for serving robots.txt from the public directory instead of a view?
I can't remember about the reasoning behind this. I guess that reducing complexity by just having a single robots.txt file in the public folder that users can override from their extension.

But it seems like the old implementation gave more flexibility so happy to revert it. This should be easy to put back following the commits that @duttonw linked to.

Resolving patch:
diff --git a/ckan/views/home.py b/ckan/views/home.py
--- a/ckan/views/home.py
+++ b/ckan/views/home.py
@@ -5,7 +5,7 @@
 from urllib.parse import urlencode
 from typing import Any, Optional, cast, List, Tuple
 
-from flask import Blueprint, abort, redirect, request
+from flask import Blueprint, make_response, abort, redirect, request
 
 import ckan.model as model
 import ckan.logic as logic
@@ -14,7 +14,7 @@
 import ckan.lib.helpers as h
 
 from ckan.common import g, config, current_user, _
-from ckan.types import Context
+from ckan.types import Context, Response
 
 
 CACHE_PARAMETERS = [u'__cache', u'__no_cache__']
@@ -95,6 +95,13 @@ def about() -> str:
     return base.render(u'home/about.html', extra_vars={})
 
 
+def robots_txt() -> Response:
+    '''display robots.txt'''
+    resp = make_response(base.render('home/robots.txt'))
+    resp.headers['Content-Type'] = "text/plain; charset=utf-8"
+    return resp
+
+
 def redirect_locale(target_locale: str, path: Optional[str] = None) -> Any:
 
     target = f'/{target_locale}/{path}' if path else f'/{target_locale}'
@@ -107,7 +114,8 @@ def redirect_locale(target_locale: str, path: Optional[str] = None) -> Any:
 
 util_rules: List[Tuple[str, Any]] = [
     (u'/', index),
-    (u'/about', about)
+    (u'/about', about),
+    (u'/robots.txt', robots_txt)
 ]
 for rule, view_func in util_rules:
     home.add_url_rule(rule, view_func=view_func)


Test patch:
Repository: ckan/ckan

Issue:
Incorrect disclaimer displayed in latest docs
It's showing this:

![Screenshot 2022-07-05 at 11-40-16 Overview — CKAN 2 10 0a0 documentation](https://user-images.githubusercontent.com/200230/177300667-2782f892-e86a-4331-aecd-6999dbd019fa.png)

When it should be showing this:

![Screenshot 2022-07-05 at 11-45-33 Overview — CKAN 2 10 0a0 documentation](https://user-images.githubusercontent.com/200230/177300701-74056bc0-98f0-4abb-8133-76bcc4201a28.png)



Discussion:

Resolving patch:
diff --git a/doc/conf.py b/doc/conf.py
--- a/doc/conf.py
+++ b/doc/conf.py
@@ -355,7 +355,7 @@ def write_substitutions_file(**kwargs):
 latest_release_tag_value = get_latest_release_tag()
 latest_release_version = get_latest_release_version()
 latest_minor_version = latest_release_version[:latest_release_version.find(".", 3)]
-is_master = release.endswith('a')
+is_master = "a" in release.split(".")[-1]
 is_supported = get_status_of_this_version() == 'supported'
 is_latest_version = version == latest_release_version
 


Test patch:
Repository: ckan/ckan

Issue:
datastore_search full_text and fields parameters not processed properly
**CKAN version**
master

**Describe the bug**
When the new `full_text` parameter is used along with the `fields` parameter, `datastore_search` always returns the entire record, along with the "hidden" rank column (used for relevance in fts search)

**Steps to reproduce**
Use `datastore_search` `full_text` and `fields` in the same call.

**Expected behavior**
`datastore_search` will process `fields` properly when `full_text` is specified, and the hidden rank value should not be returned.



Discussion:
@jqnatividad , I am interested to work on this issue. At present I am trying to reproduce this issue but unable to reproduce it. Can you please tell how to use **datastore_search   full_text** and  **fields** in the same call?
Hi @jqnatividad , I would like to work on this issue.

Hi @jqnatividad , I'm unable to reproduce the issue.
I also checked the API documentation for  [datastore_search](https://docs.ckan.org/en/2.9/maintaining/datastore.html#ckanext.datastore.logic.action.datastore_search), but there is no reference for `full_text` in it. 

Hi @jqnatividad , I'm able to reproduce the issue. Are we trying to remove the `rank` column from the records response when we hit the API, i.e. from `'records': [{'_id': 2, 'a': 2, 'b': 'zzz', 'rank': 0.057308756}].  Please confirm my understanding.


Resolving patch:
diff --git a/ckanext/datastore/backend/postgres.py b/ckanext/datastore/backend/postgres.py
--- a/ckanext/datastore/backend/postgres.py
+++ b/ckanext/datastore/backend/postgres.py
@@ -1901,6 +1901,8 @@ def datastore_search(
         elif fields and fts_q:
             field_ids = datastore_helpers.get_list(fields)
             all_field_ids = list(fields_types.keys())
+            if "rank" not in fields:
+                all_field_ids.remove("rank")
             field_intersect = [x for x in field_ids
                                if x not in all_field_ids]
             field_ids = all_field_ids + field_intersect


Test patch:
diff --git a/ckanext/datastore/tests/test_search.py b/ckanext/datastore/tests/test_search.py
--- a/ckanext/datastore/tests/test_search.py
+++ b/ckanext/datastore/tests/test_search.py
@@ -2089,10 +2089,8 @@ def test_fts_on_field_calculates_ranks_specific_field_and_all_fields(self):
         }
         result = helpers.call_action("datastore_search", **search_data)
         ranks_from = [r["rank from"] for r in result["records"]]
-        ranks = [r["rank"] for r in result["records"]]
         assert len(result["records"]) == 2
         assert len(set(ranks_from)) == 1
-        assert len(set(ranks)) == 2
 
     @pytest.mark.ckan_config("ckan.plugins", "datastore")
     @pytest.mark.usefixtures("clean_datastore", "with_plugins")