internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: ckan/ckan

Issue:
CKAN Data API extensions
This is a placeholder ticket for work on improving the CKAN Data API template and making it extendable for new client languages and for cases when the example data can be generated to match the specific table (e.g. w/table designer)


Discussion:

Resolving patch:
diff --git a/ckan/lib/helpers.py b/ckan/lib/helpers.py
--- a/ckan/lib/helpers.py
+++ b/ckan/lib/helpers.py
@@ -367,12 +367,6 @@ def url_for(*args: Any, **kw: Any) -> str:
     # remove __ckan_no_root and add after to not pollute url
     no_root = kw.pop('__ckan_no_root', False)
 
-    # All API URLs generated should provide the version number
-    if kw.get('controller') == 'api' or args and args[0].startswith('api.'):
-        ver = kw.get('ver')
-        if not ver:
-            raise Exception('API URLs must specify the version (eg ver=3)')
-
     _auto_flask_context = _get_auto_flask_context()
     try:
         if _auto_flask_context:
diff --git a/ckanext/datastore/plugin.py b/ckanext/datastore/plugin.py
--- a/ckanext/datastore/plugin.py
+++ b/ckanext/datastore/plugin.py
@@ -80,6 +80,7 @@ def update_config(self, config: CKANConfig):
         templates_base = config.get('ckan.base_templates_folder')
 
         p.toolkit.add_template_directory(config, templates_base)
+        p.toolkit.add_resource('assets', 'ckanext_datastore')
         self.backend = DatastoreBackend.get_active_backend()
 
     # IConfigurable


Test patch:
diff --git a/ckanext/datastore/tests/test_info.py b/ckanext/datastore/tests/test_info.py
--- a/ckanext/datastore/tests/test_info.py
+++ b/ckanext/datastore/tests/test_info.py
@@ -73,15 +73,14 @@ def test_api_info(app):
 
     page = app.get(url, status=200)
 
-    # check we built all the urls ok
-    expected_urls = (
-        "http://test.ckan.net/api/3/action/datastore_create",
-        "http://test.ckan.net/api/3/action/datastore_upsert",
-        "<code>http://test.ckan.net/api/3/action/datastore_search",
-        "http://test.ckan.net/api/3/action/datastore_search_sql",
-        "url: 'http://test.ckan.net/api/3/action/datastore_search'",
-        "http://test.ckan.net/api/3/action/datastore_search"
+    # check we built some urls, examples properly
+    expected_html = (
+        "http://test.ckan.net/api/action/datastore_search",
+        "http://test.ckan.net/api/action/datastore_search_sql",
+        '<pre class="example-curl"><code class="language-bash"',
+        f'"sql": "SELECT * FROM \\"{resource["id"]}\\" WHERE',
+        "RemoteCKAN('http://test.ckan.net/', apikey=API_TOKEN)",
     )
     content = page.get_data(as_text=True)
-    for url in expected_urls:
-        assert url in content
+    for html in expected_html:
+        assert html in content
Repository: ckan/ckan

Issue:
Customize R/W datastore resource `url_type`s
The datastore API returns an error when `datastore_create`, `datastore_upsert`, `datastore_delete` are used on resources that have a resource_type value different than `"datastore"`, unless `force=True` is passed.

This is a *good* thing for the common case of data in the datastore created by xloader/datapusher because any updated data will be lost on the next reload.

Other cases (like table designer resources) should be able to allow use without `force=True`, so I'd like to make it possible for plugins to define new resource types that allow datastore modifications.


Discussion:

Resolving patch:
diff --git a/ckanext/datastore/helpers.py b/ckanext/datastore/helpers.py
--- a/ckanext/datastore/helpers.py
+++ b/ckanext/datastore/helpers.py
@@ -239,3 +239,12 @@ def datastore_search_sql_enabled(*args: Any) -> bool:
         return tk.asbool(config)
     except (tk.ObjectNotFound, tk.NotAuthorized):
         return False
+
+
+def datastore_rw_resource_url_types() -> list[str]:
+    """
+    Return a list of resource url_type values that do not require passing
+    force=True when used with datastore_create, datastore_upsert,
+    datastore_delete
+    """
+    return ["datastore"]
diff --git a/ckanext/datastore/logic/action.py b/ckanext/datastore/logic/action.py
--- a/ckanext/datastore/logic/action.py
+++ b/ckanext/datastore/logic/action.py
@@ -700,10 +700,14 @@ def _check_read_only(context: Context, resource_id: str):
     '''
     res = p.toolkit.get_action('resource_show')(
         context, {'id': resource_id})
-    if res.get('url_type') != 'datastore':
+    if res.get('url_type') not in (
+            p.toolkit.h.datastore_rw_resource_url_types()
+        ):
         raise p.toolkit.ValidationError({
-            'read-only': ['Cannot edit read-only resource. Either pass'
-                          '"force=True" or change url_type to "datastore"']
+            'read-only': ['Cannot edit read-only resource because changes '
+                          'made may be lost. Use a resource created for '
+                          'editing e.g. with datastore_create or use '
+                          '"force=True" to ignore this warning.']
         })
 
 
diff --git a/ckanext/datastore/plugin.py b/ckanext/datastore/plugin.py
--- a/ckanext/datastore/plugin.py
+++ b/ckanext/datastore/plugin.py
@@ -258,10 +258,12 @@ def datastore_search(self, context: Context, data_dict: dict[str, Any],
     def get_helpers(self) -> dict[str, Callable[..., object]]:
         conf_dictionary = datastore_helpers.datastore_dictionary
         conf_sql_enabled = datastore_helpers.datastore_search_sql_enabled
+        rw_url_types = datastore_helpers.datastore_rw_resource_url_types
 
         return {
             'datastore_dictionary': conf_dictionary,
-            'datastore_search_sql_enabled': conf_sql_enabled
+            'datastore_search_sql_enabled': conf_sql_enabled,
+            'datastore_rw_resource_url_types': rw_url_types,
         }
 
     # IForkObserver


Test patch:
Repository: ckan/ckan

Issue:
`user_show` returns `NotFound` before checking authentication for anonymous users
## CKAN version 2.10.1

## Describe the bug

The `ckan.logic.action.get.user_show` action returns a [NotFound error](https://github.com/ckan/ckan/blob/ckan-2.10.1/ckan/logic/action/get.py#L1466) before checking any auth function. This also breaks [`ckan.auth.public_user_details`](https://docs.ckan.org/en/2.10/maintaining/configuration.html#ckan-auth-public-user-details).

### Steps to reproduce
Call `user_show` anonymously. Instead of "access denied", "not found" is returned.

### Expected behavior
For anonymous users, an "access denied" error should be returned. Also, the configuration variable [`ckan.auth.public_user_details`](https://docs.ckan.org/en/2.10/maintaining/configuration.html#ckan-auth-public-user-details) seems to have no effect at all anymore.



Discussion:

Resolving patch:
diff --git a/ckan/logic/action/get.py b/ckan/logic/action/get.py
--- a/ckan/logic/action/get.py
+++ b/ckan/logic/action/get.py
@@ -1444,13 +1444,13 @@ def user_show(context: Context, data_dict: DataDict) -> ActionResult.UserShow:
     else:
         user_obj = None
 
-    if not user_obj:
-        raise NotFound
-
-    context['user_obj'] = user_obj
+    if user_obj:
+        context['user_obj'] = user_obj
 
     _check_access('user_show', context, data_dict)
 
+    if not user_obj:
+        raise NotFound
 
     # include private and draft datasets?
     requester = context.get('user')


Test patch:
Repository: ckan/ckan

Issue:
current_user.is_anonymous doesn't care if context has an user
## CKAN version
2.10

## Describe the bug
While looking into why https://github.com/ckan/ckan/pull/7266 started failing on unmodified ckan, I noticed that deciphering anonymous users has changed. On 2.9 `restrict_anon` works as follows 
https://github.com/ckan/ckan/blob/c4e2818818e08e60bb69d64229f8dbba531f8439/ckan/logic/auth/__init__.py#L51-L55

where `authz.auth_is_anon_user` checks if the context has an user in it. On 2.10 the functionality has changed:

https://github.com/ckan/ckan/blob/d46613e346f9dc551aedb54c8c24baad919f78c1/ckan/logic/auth/__init__.py#L93-L97

which does not check for context at all and the password reset started failing on 2.10. Should the `is_anonymous` check for user in context or are we just relying on what flask login says about the user? 




Discussion:

Resolving patch:
diff --git a/ckan/logic/auth/__init__.py b/ckan/logic/auth/__init__.py
--- a/ckan/logic/auth/__init__.py
+++ b/ckan/logic/auth/__init__.py
@@ -8,8 +8,8 @@
 from typing_extensions import Literal
 
 import ckan.logic as logic
+import ckan.authz as authz
 from ckan.types import Context, AuthResult, DataDict
-from ckan.common import current_user
 
 if TYPE_CHECKING:
     import ckan.model as model_
@@ -86,7 +86,7 @@ def get_user_object(
 
 
 def restrict_anon(context: Context) -> AuthResult:
-    if current_user.is_anonymous:
+    if authz.auth_is_anon_user(context):
         return {'success': False}
     else:
         return {'success': True}


Test patch:
diff --git a/ckan/tests/logic/auth/test_get.py b/ckan/tests/logic/auth/test_get.py
--- a/ckan/tests/logic/auth/test_get.py
+++ b/ckan/tests/logic/auth/test_get.py
@@ -38,7 +38,7 @@ def test_user_list_email_parameter():
 class TestGetAuth(object):
     @pytest.mark.ckan_config(u"ckan.auth.public_user_details", False)
     @mock.patch("flask_login.utils._get_user")
-    def test_auth_user_show(self, current_user):
+    def test_restrict_anon_auth_when_user_is_anonymouus(self, current_user):
         fred = factories.User()
         fred["capacity"] = "editor"
         current_user.return_value = mock.Mock(is_anonymous=True)
@@ -46,6 +46,15 @@ def test_auth_user_show(self, current_user):
         with pytest.raises(logic.NotAuthorized):
             helpers.call_auth("user_show", context=context, id=fred["id"])
 
+    @pytest.mark.ckan_config(u"ckan.auth.public_user_details", False)
+    @mock.patch("flask_login.utils._get_user")
+    def test_restrict_anon_auth_when_user_is_in_context(self, current_user):
+        fred = factories.User()
+        fred["capacity"] = "editor"
+        current_user.return_value = mock.Mock(is_anonymous=True)
+        context = {"user": fred['id'], "model": model}
+        assert helpers.call_auth("user_show", context=context, id=fred["id"])
+
     def test_authed_user_show(self):
         fred = factories.User()
         fred["capacity"] = "editor"
Repository: ckan/ckan

Issue:
Package Search breaks when '+state' is in fq_list
## CKAN version
2.9.x, at least

## Describe the bug

This call:
`get_action('package_search')(None, {'fq_list':['+state:((anything other than active))']})`

results in a solr query like:
`...&fq=state:((anything other than active))&fq=%2Bsite_id:"default"&fq=%2Bstate:active&...`

due to https://github.com/ckan/ckan/blob/2.9/ckan/lib/search/query.py#L332C11-L332C11

```
        if 'fq' in query:
            fq.append(query['fq'])
        fq.extend(query.get('fq_list', []))

        # show only results from this CKAN instance
        fq.append('+site_id:%s' % solr_literal(config.get('ckan.site_id')))

        # filter for package status
        if not '+state:' in query.get('fq', ''):
            fq.append('+state:active')
```

because the filter for package status is only checking the passed in `fq`, and not the `fq_list`.

### Expected behavior

I'd expect that `fq_list` is a more convenient version of `fq`.

I think a fix for this would be:
```
        # filter for package status
        if not any('+state:' in _item for _item in fq):
            fq.append('+state:active')
```



Discussion:

Resolving patch:
diff --git a/ckan/lib/search/query.py b/ckan/lib/search/query.py
--- a/ckan/lib/search/query.py
+++ b/ckan/lib/search/query.py
@@ -365,7 +365,7 @@ def run(self,
         fq.append('+site_id:%s' % solr_literal(config.get('ckan.site_id')))
 
         # filter for package status
-        if not '+state:' in query.get('fq', ''):
+        if not any('+state:' in _item for _item in fq):
             fq.append('+state:active')
 
         # only return things we should be able to see


Test patch:
Repository: ckan/ckan

Issue:
Core resource_read template relies on Datastore
## CKAN version

2.10

## Describe the bug

The datastore plugin provides its own version of the `package/resource_read.html` template if enabled. However, the core template still includes datastore-related logic, checking `datastore_active` and including datastore-specific content if true. As a result, if datastore entries exist but then the datastore is disabled, the resource view page will crash.

### Steps to reproduce

- Create a CKAN instance with datastore enabled.
- Create a resource.
- Create a datastore entry for the resource.
- Disable the datastore plugin.
- Visit the resource.

### Expected behavior

If the datastore plugin is not enabled, then datastore features should simply be ignored.

### Additional details
If possible, please provide the full stack trace of the error raised,  or add screenshots to help explain your problem.

      File "/mnt/local_data/ckan_venv/src/ckan/ckan/templates/package/resource_read.html", line 56, in block 'download_resource_button'
        <a class="dropdown-item" href="{{ h.url_for('datastore.dump', resource_id=res.id, bom=True) }}"
      File "/mnt/local_data/ckan_venv/src/ckan/ckan/lib/helpers.py", line 385, in url_for
        my_url = _url_for_flask(*args, **kw)
      File "/mnt/local_data/ckan_venv/src/ckan/ckan/lib/helpers.py", line 443, in _url_for_flask
        my_url = _flask_default_url_for(*args, **kw)
      File "/usr/lib/ckan/default/lib64/python3.7/site-packages/flask/helpers.py", line 338, in url_for
        return appctx.app.handle_url_build_error(error, endpoint, values)
      File "/usr/lib/ckan/default/lib64/python3.7/site-packages/flask/helpers.py", line 326, in url_for
        endpoint, values, method=method, force_external=external
      File "/usr/lib/ckan/default/lib64/python3.7/site-packages/werkzeug/routing.py", line 2315, in build
        raise BuildError(endpoint, values, method, self)
    werkzeug.routing.BuildError: Could not build url for endpoint 'datastore.dump' with values ['bom', 'resource_id']. Did you mean 'dataset.read' instead?



Discussion:

Resolving patch:
diff --git a/ckan/lib/helpers.py b/ckan/lib/helpers.py
--- a/ckan/lib/helpers.py
+++ b/ckan/lib/helpers.py
@@ -2270,7 +2270,6 @@ def resource_view_is_filterable(resource_view: dict[str, Any]) -> bool:
 @core_helper
 def resource_view_get_fields(resource: dict[str, Any]) -> list["str"]:
     '''Returns sorted list of text and time fields of a datastore resource.'''
-
     if not resource.get('datastore_active'):
         return []
 


Test patch:
Repository: ckan/ckan

Issue:
`truncate` helper is deprecated, but does not work
## CKAN version
2.10.0

## Describe the bug
`truncate` helper is deprecated, but does not work

### Steps to reproduce
- Try to use `h.truncate` in a template
- Load the page
- `ckan.exceptions.HelperError: Helper 'truncate' has not been defined.`

### Expected behavior
- No crash

### Additional details
Swapping the order of decorators `maintain.deprecated` and `core_helper` fixes the issue.



Discussion:
Hi @wardi 
I tried using "h.truncate" in an HTML file of CKAN 2.10.0, and the issue was reproduced.
I would like to work on this issue.

Resolving patch:
diff --git a/ckan/lib/maintain.py b/ckan/lib/maintain.py
--- a/ckan/lib/maintain.py
+++ b/ckan/lib/maintain.py
@@ -3,6 +3,7 @@
 ''' This module contains code that helps in maintaining the Ckan codebase. '''
 from __future__ import annotations
 
+import functools
 import inspect
 import time
 import logging
@@ -41,6 +42,7 @@ def decorator(fn: Callable[P, RT]) -> Callable[P, RT]:
                             'It must include the word `deprecated`.'
                             % (fn.__name__, fn.__module__))
 
+        @functools.wraps(fn)
         def wrapped(*args: P.args, **kw: P.kwargs) -> RT:
             since_msg = f'since CKAN v{since}' if since else ''
             msg = (


Test patch:
Repository: ckan/ckan

Issue:
aria-labels without translations
## CKAN version
2.10

## Describe the bug
There are some aria-labels that are not covered by the translations, causing a mis-match in the UI language for people that are using screen readers.

### Steps to reproduce
Using a CKAN instance with a German localization (100% translated according to transifex), such as https://opendata.leipzig.de:
- shrink screen to "smartphone" size: Navigation menu button is labeled "Toggle navigation" - [hardcoded here](https://github.com/ckan/ckan/blob/9acb5af18fdbc6d084606be399837f680b49b634/ckan/templates/header.html#L92)
- fail to log in, error message close button is labeled "Close" - [probably a recurring issue](https://github.com/search?q=repo%3Ackan%2Fckan+aria-label%3D%22Close%22&type=code)

### Expected behavior
Navigation elements have localized labels.


## Comments
I might expand this list if more missing label translations are found. I'm also looking into transifex and whether I can get this fixed myself.


Discussion:
Thanks for this @toothstone , the strings you point to won't appear in Transifex because they should be wrapped with the translation function `_` so they are extracted.

For html templates it's easy:

```diff
diff --git a/ckan/templates/header.html b/ckan/templates/header.html
index 610e6eb85..48989501f 100644
--- a/ckan/templates/header.html
+++ b/ckan/templates/header.html
@@ -89,7 +89,7 @@
 
        </hgroup>
       <button class="navbar-toggler" type="button" data-bs-toggle="collapse" data-bs-target="#main-navigation-toggle"
-        aria-controls="main-navigation-toggle" aria-expanded="false" aria-label="Toggle navigation">
+        aria-controls="main-navigation-toggle" aria-expanded="false" aria-label="{{ _('Toggle navigation') }}">
         <span class="fa fa-bars text-white"></span>
       </button>
```
For JS modules is a bit more tricky but definitely worth doing.
There are a few attributes that need to be fixed (`grep -r 'aria-label="[^{]' ckan ckanext`) but I'll submit a PR soon. Once this is merged and backported to the `dev-v2.10` we will extract the new messages and upload them to Transifex, so if you register there you'll be able to add the German translation for it, which will be included in 2.10.2.





Resolving patch:
diff --git a/ckan/cli/generate.py b/ckan/cli/generate.py
--- a/ckan/cli/generate.py
+++ b/ckan/cli/generate.py
@@ -4,7 +4,7 @@
 import contextlib
 import os
 import json
-from packaging.version import parse as version_parse
+from packaging.version import Version
 import shutil
 from typing import Optional
 
@@ -90,8 +90,7 @@ def extension(output_dir: str):
 
     include_examples = int(click.confirm(
         "Do you want to include code examples?"))
-
-    full_ckan_version = version_parse(ckan.__version__)
+    full_ckan_version: Version = Version(ckan.__version__)
     ckan_version = f"{full_ckan_version.major}.{full_ckan_version.minor}"
 
     context = {


Test patch:
Repository: ckan/ckan

Issue:
Catch AttributeErrors in license retrieval
The following code segment may also result in an `AttributeError`:

https://github.com/ckan/ckan/blob/8ddc2e800793d683c856b4a56f3d3f0ad3f1d168/ckan/views/resource.py#L95-L96

If, for some reason, the license object returned by `model.Package.get_license_register()[license_id]` does not have an `isopen` attribute or method, trying to call `isopen()` would result in an `AttributeError`, as well if `model.Package.get_license_register()[license_id]` evaluates to `None`. Currently, only `KeyErrors` are being caught.

It is advisable to include `AttributeError` in the except handle of the try block:

```python
except (KeyError, AttributeError):
    package[u'isopen'] = False
```


Discussion:
@amercader @thorge , I am interested to work on this issue. I will submit a PR.

Resolving patch:
diff --git a/ckan/views/resource.py b/ckan/views/resource.py
--- a/ckan/views/resource.py
+++ b/ckan/views/resource.py
@@ -94,7 +94,7 @@ def read(package_type: str, id: str, resource_id: str) -> Union[Response, str]:
     try:
         package[u'isopen'] = model.Package.get_license_register()[license_id
                                                                   ].isopen()
-    except KeyError:
+    except (KeyError, AttributeError):
         package[u'isopen'] = False
 
     resource_views = get_action(u'resource_view_list')(


Test patch:
Repository: ckan/ckan

Issue:
readthedocs sphinx build failures
## CKAN version
master

## Describe the bug
infinite loop in build, looks like no tags are returned from `git log`?

### Steps to reproduce
check sphinx logs

### Expected behavior
build docs on rtd working

### Additional details
```python-traceback
Traceback (most recent call last):
  File "/home/docs/checkouts/readthedocs.org/user_builds/ckan/envs/latest/lib/python3.10/site-packages/sphinx/config.py", line 358, in eval_config_file
    exec(code, namespace)  # NoQA: S102
  File "/home/docs/checkouts/readthedocs.org/user_builds/ckan/checkouts/latest/doc/conf.py", line 388, in <module>
    current_release_tag_value = get_current_release_tag()
  File "/home/docs/checkouts/readthedocs.org/user_builds/ckan/checkouts/latest/doc/conf.py", line 211, in get_current_release_tag
    return get_latest_release_tag()
  File "/home/docs/checkouts/readthedocs.org/user_builds/ckan/checkouts/latest/doc/conf.py", line 228, in get_latest_release_tag
    return get_latest_release_version()
  File "/home/docs/checkouts/readthedocs.org/user_builds/ckan/checkouts/latest/doc/conf.py", line 237, in get_latest_release_version
    version = get_latest_release_tag()[len('ckan-'):]
  File "/home/docs/checkouts/readthedocs.org/user_builds/ckan/checkouts/latest/doc/conf.py", line 228, in get_latest_release_tag
    return get_latest_release_version()
  File "/home/docs/checkouts/readthedocs.org/user_builds/ckan/checkouts/latest/doc/conf.py", line 237, in get_latest_release_version
    version = get_latest_release_tag()[len('ckan-'):]
…
```


Discussion:

Resolving patch:
diff --git a/doc/conf.py b/doc/conf.py
--- a/doc/conf.py
+++ b/doc/conf.py
@@ -85,7 +85,9 @@
 extensions = [
     'sphinx.ext.autodoc', 'sphinx.ext.todo',
     'sphinx.ext.autosummary', 'ckan.plugins.toolkit_sphinx_extension',
+    'sphinx_rtd_theme',
 ]
+html_theme = 'sphinx_rtd_theme'
 autodoc_member_order = 'bysource'
 todo_include_todos = True
 


Test patch:
Repository: ckan/ckan

Issue:
Broken Link for Font Awesome in Documentation
## CKAN version
latest

## Describe the bug
Documentation update

https://github.com/ckan/ckan/blob/3b3ac6a2d8e990293a3d274ce75ac0013fc750d1/ckan/plugins/interfaces.py#L286

Contains a link that produces a 404 while also looking like it references a very old version of FA.  Happy to submit a PR, but am unsure if better to link to https://fontawesome.com or https://github.com/FortAwesome/Font-Awesome and if a specific version should be referenced.






Discussion:
Either is fine @pdekraker-epa let's drop the version, a PR would be great!

Resolving patch:
diff --git a/ckan/plugins/interfaces.py b/ckan/plugins/interfaces.py
--- a/ckan/plugins/interfaces.py
+++ b/ckan/plugins/interfaces.py
@@ -283,7 +283,7 @@ def info(self) -> dict[str, Any]:
         :returns: a dictionary with the view type configuration
         :rtype: dict
 
-        .. _Font Awesome: http://fortawesome.github.io/Font-Awesome/3.2.1/icons
+        .. _Font Awesome: https://fontawesome.com/search
         '''
         return {u'name': self.__class__.__name__}
 


Test patch: