internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: ckan/ckan

Issue:
routes manual reference URL in comment is broken
**CKAN version**
latest

**Describe the bug**
The url in [comment ](https://github.com/ckan/ckan/blob/0f87337fd937a15545ed761367b5d27d888e3803/ckan/config/routing.py#L6) is broken.

**Steps to reproduce**
Steps to reproduce the behavior:
Open a browser and go to "http://routes.groovie.org/docs/"
![image](https://user-images.githubusercontent.com/37072369/86097650-42283680-bab5-11ea-9210-cd5baf3dded4.png)


**Expected behavior**
A valid documentation reference.




Discussion:

Resolving patch:
diff --git a/ckan/config/routing.py b/ckan/config/routing.py
--- a/ckan/config/routing.py
+++ b/ckan/config/routing.py
@@ -3,7 +3,7 @@
 
 The more specific and detailed routes should be defined first so they
 may take precedent over the more generic routes. For more information
-refer to the routes manual at http://routes.groovie.org/docs/
+refer to the routes manual at https://routes.readthedocs.io/en/latest/
 
 """
 import re


Test patch:
Repository: ckan/ckan

Issue:
Fix line breaks in translatable strings
I've got reports of confusing strings from translators, eg:

![Screenshot_2020-07-16 CKAN Transifex Editor](https://user-images.githubusercontent.com/200230/87644994-2a0b1500-c74d-11ea-8b61-1d6deaad827e.png)

There is a line break and extra spaces in the middle of the source string (`msgid`) and it's unclear to users if they should keep it in their translations.

This is how the `msgid` looks like:

```
#: ckan/templates/snippets/changes/extension_fields.html:3                                                                                                                                                                                                                                                           
msgid ""                                                                                                                                                                                                                                                                                                             
"Changed value of field <q>{key}</q> to <q>{value}</q> in\n"                                                                                                                                                                                                                                                         
"        {pkg_link}"                                                                                                                                                                                                                                                                                                 
msgstr ""   
```

The source file contains a line break probably created by an overzealous code formatter:
```
   {{ _('Changed value of field <q>{key}</q> to <q>{value}</q> in
        {pkg_link}')
```

Most if not all the strings seemed to be part of the snippets added in the `changes` folder on https://github.com/ckan/ckan/pull/4929 so I think it's ok if for now we manually fix the strings in the snippets, extract them and update the msgids on the po files using the `ckan translation sync-msgids` command we introduced in #5339 








Discussion:

Resolving patch:
diff --git a/ckan/logic/validators.py b/ckan/logic/validators.py
--- a/ckan/logic/validators.py
+++ b/ckan/logic/validators.py
@@ -937,9 +937,8 @@ def email_is_unique(key, data, errors, context):
                 return
 
     raise Invalid(
-                _('The email address \'{email}\' \
-                    belongs to a registered user.').
-                        format(email=data[key]))
+        _('The email address \'{email}\' belongs to a registered user.').format(email=data[key]))
+
 
 def one_of(list_of_value):
     ''' Checks if the provided value is present in a list '''


Test patch:
Repository: ckan/ckan

Issue:
Incorrectly Cached License Translations
**CKAN version**

2.8.5->master.

Probably introduced in #4594.

**Describe the bug**

ckan.models.license:LicenseRegister.load_licenses translates the license text, which is then cached as a classmember in the package class. (as called from ckan.lib.helpers.license_options)

This cache is for the lifetime of uwsgi process, not the request. 

The license options should be translated at the helper or the template level, and not cached between multiple requests, unless the cache is keyed to the request language. 

**Steps to reproduce**

On a site with multiple localizations, restart ckan to clear any caches, then request `/[lang]/dataset/edit/[name]` several times to make sure all wsgi processes have loaded the licenses.

Then visit ` `/en/dataset/edit/[name]` and the licenses will be in the previous localization. 

**Expected behavior**

Each license popup would be rendered in the correct localization. 


Incorrectly Cached License Translations
**CKAN version**

2.8.5->master.

Probably introduced in #4594.

**Describe the bug**

ckan.models.license:LicenseRegister.load_licenses translates the license text, which is then cached as a classmember in the package class. (as called from ckan.lib.helpers.license_options)

This cache is for the lifetime of uwsgi process, not the request. 

The license options should be translated at the helper or the template level, and not cached between multiple requests, unless the cache is keyed to the request language. 

**Steps to reproduce**

On a site with multiple localizations, restart ckan to clear any caches, then request `/[lang]/dataset/edit/[name]` several times to make sure all wsgi processes have loaded the licenses.

Then visit ` `/en/dataset/edit/[name]` and the licenses will be in the previous localization. 

**Expected behavior**

Each license popup would be rendered in the correct localization. 




Discussion:

Resolving patch:
diff --git a/ckan/lib/helpers.py b/ckan/lib/helpers.py
--- a/ckan/lib/helpers.py
+++ b/ckan/lib/helpers.py
@@ -2840,7 +2840,7 @@ def license_options(existing_license_id=None):
         license_ids.insert(0, existing_license_id)
     return [
         (license_id,
-         register[license_id].title if license_id in register else license_id)
+         _(register[license_id].title) if license_id in register else license_id)
         for license_id in license_ids]
 
 
diff --git a/ckan/model/license.py b/ckan/model/license.py
--- a/ckan/model/license.py
+++ b/ckan/model/license.py
@@ -141,8 +141,6 @@ def load_licenses(self, license_url):
         for license in license_data:
             if isinstance(license, string_types):
                 license = license_data[license]
-            if license.get('title'):
-                license['title'] = _(license['title'])
         self._create_license_list(license_data, license_url)
 
     def _create_license_list(self, license_data, license_url=''):


Test patch:
Repository: ckan/ckan

Issue:
Typo in is_positive_integer validator
**CKAN version**
2.9

**Describe the bug**
The validation error message is "Must be a postive integer"
https://github.com/ckan/ckan/blob/16164bd50d5f97225125d2af07b4331d2f3daa40/ckan/logic/validators.py#L127

**Expected behavior**
The error should read "Must be a positive integer"



Discussion:

Resolving patch:
diff --git a/ckan/logic/validators.py b/ckan/logic/validators.py
--- a/ckan/logic/validators.py
+++ b/ckan/logic/validators.py
@@ -124,7 +124,7 @@ def natural_number_validator(value, context):
 def is_positive_integer(value, context):
     value = int_validator(value, context)
     if value < 1:
-        raise Invalid(_('Must be a postive integer'))
+        raise Invalid(_('Must be a positive integer'))
     return value
 
 def boolean_validator(value, context):


Test patch:
Repository: ckan/ckan

Issue:
Download options in Datatables_view do not work
**CKAN version**
2.9

**Describe the bug**
Using datatables_view as a default resource view, which works well.  Apart from the nicer UI and pagination, one benefit of the view is that you can download a filtered version of the resource (https://github.com/ckan/ckan/pull/4497).  However, none of the datatables_view download buttons work to download the filtered data.

**Steps to reproduce**

1. Add a CSV resource to a dataset
2. Create a datatables resource view (labeled 'Table' in the resource view picklist)
3. Go to resource view and try to use the Download button for any format type
4. A 404 error page replaces the datatables control




Discussion:
This sounds like a regression related to the URL generation for these endpoints since the move to flask. https://github.com/ckan/ckan/blob/878dbf6bf2a058de4efa0d8fce9b66e2ef5b7fa3/ckanext/datatablesview/templates/datatables/datatables_view.html#L48-L50 would be the place I would start looking.

Resolving patch:
diff --git a/ckanext/datatablesview/blueprint.py b/ckanext/datatablesview/blueprint.py
--- a/ckanext/datatablesview/blueprint.py
+++ b/ckanext/datatablesview/blueprint.py
@@ -100,7 +100,7 @@ def ajax(resource_view_id):
 
 
 def filtered_download(resource_view_id):
-    params = json.loads(request.params[u'params'])
+    params = json.loads(request.form[u'params'])
     resource_view = get_action(u'resource_view_show'
                                )(None, {
                                    u'id': resource_view_id
@@ -132,14 +132,14 @@ def filtered_download(resource_view_id):
 
     cols = [c for (c, v) in zip(cols, params[u'visible']) if v]
 
-    h.redirect_to(
+    return h.redirect_to(
         h.
         url_for(u'datastore.dump', resource_id=resource_view[u'resource_id']) +
         u'?' + urlencode({
             u'q': search_text,
             u'sort': u','.join(sort_list),
             u'filters': json.dumps(filters),
-            u'format': request.params[u'format'],
+            u'format': request.form[u'format'],
             u'fields': u','.join(cols),
         })
     )
@@ -151,5 +151,5 @@ def filtered_download(resource_view_id):
 
 datatablesview.add_url_rule(
     u'/datatables/filtered-download/<resource_view_id>',
-    view_func=filtered_download
+    view_func=filtered_download, methods=[u'POST']
 )


Test patch:
Repository: ckan/ckan

Issue:
No way to disable reloader when starting the development server
**CKAN version**
2.9.0

**Describe the bug**
When trying to start the development server without a reloader i encountered problems with the `--reloader` argument.
The reloader option requires a TEXT argument, therefore i expected that --reloader False disables the reloader.


**Steps to reproduce**
Start ckan with following command:

`ckan -c [PATH_TO_CONFIG] run --host 0.0.0.0 --reloader False`

**Expected behavior**
Server starts without reloader

**Additional details**

Currently the `reloader` option is passed as string and if it's not provided it defaults to the boolean value `True`

So we have two cases when the `run_simple` method is called:
1. `--reloader` argument is not provided --> reloader=True
2. `--reloader` argument is provided --> some string is passed as reloader argument to the `run_simple` method, which evaluates to true in the if statement distinguishing whether the reloader should be used or not.

So the `--reloader` argument does not affect anything.

_My suggestion:_ rename the argument to `disable-reloader` and turn it into a boolean flag. This enables the user to disable the reloader and the default behaviour (i.e. the dev server starts with a reloader) stays the same.



Discussion:

Resolving patch:
diff --git a/ckan/cli/server.py b/ckan/cli/server.py
--- a/ckan/cli/server.py
+++ b/ckan/cli/server.py
@@ -5,8 +5,8 @@
 import click
 from werkzeug.serving import run_simple
 
-from ckan.common import config
 import ckan.plugins.toolkit as tk
+from ckan.common import config
 
 log = logging.getLogger(__name__)
 
@@ -14,7 +14,8 @@
 @click.command(u"run", short_help=u"Start development server")
 @click.option(u"-H", u"--host", default=u"localhost", help=u"Set host")
 @click.option(u"-p", u"--port", default=5000, help=u"Set port")
-@click.option(u"-r", u"--reloader", default=True, help=u"Use reloader")
+@click.option(u"-r", u"--disable-reloader", is_flag=True,
+              help=u"Disable reloader")
 @click.option(
     u"-t", u"--threaded", is_flag=True,
     help=u"Handle each request in a separate thread"
@@ -25,8 +26,9 @@
     help=u"Maximum number of concurrent processes"
 )
 @click.pass_context
-def run(ctx, host, port, reloader, threaded, extra_files, processes):
+def run(ctx, host, port, disable_reloader, threaded, extra_files, processes):
     u"""Runs the Werkzeug development server"""
+    use_reloader = not disable_reloader
     threaded = threaded or tk.asbool(config.get(u"ckan.devserver.threaded"))
     processes = processes or tk.asint(
         config.get(u"ckan.devserver.multiprocess", 1)
@@ -48,7 +50,7 @@ def run(ctx, host, port, reloader, threaded, extra_files, processes):
         host,
         port,
         ctx.obj.app,
-        use_reloader=reloader,
+        use_reloader=use_reloader,
         use_evalex=True,
         threaded=threaded,
         processes=processes,


Test patch:
Repository: ckan/ckan

Issue:
User list sorting by number_created_packages is not stable
**CKAN version**
master / python3

**Describe the bug**
Sometimes python3 builds fail on error when sorted user list is on different order than expected list, for example here https://app.circleci.com/pipelines/github/ckan/ckan/949/workflows/440c43de-7e94-4bc7-83b9-8c73903318dc/jobs/8525

If the build is rerun, it probably passes.


Discussion:
It was agreed that there should be secondary sorting by user name, so it would be stable even with when package count is the same.

Resolving patch:
diff --git a/ckan/logic/action/get.py b/ckan/logic/action/get.py
--- a/ckan/logic/action/get.py
+++ b/ckan/logic/action/get.py
@@ -881,6 +881,9 @@ def user_list(context, data_dict):
                 else_=model.User.fullname
             )
         )
+    elif order_by_field == 'number_created_packages' or order_by_field == 'fullname' \
+            or order_by_field == 'about' or order_by_field == 'sysadmin':
+        query = query.order_by(order_by_field, model.User.name)
     else:
         query = query.order_by(order_by_field)
 


Test patch:
Repository: ckan/ckan

Issue:
tag_create Documentation needs update
### CKAN Version if known (or site URL)
2.8.3

### The Issue
The tag_create API documentation has 2 issues:
1. For the tag name space is an allowed character but is not currently listed

2. The example of using the vocabulary name ('Genre') is incorrect the actual id is what is required. Issue #2362 sought a fix the software to match the documentation but was not completed. Unless it is doing to get fixed the documentation should be updated.





Discussion:
@pdekraker-epa @Zharktas , I am working on this issue. In tag name "space" is an allowed character but it is not listed in the document. I think vocabulary_id (string) – the id of the vocabulary that the new tag should be added to, e.g. the id of vocabulary 'Genre' is correct as it clearly states that in the end " e.g. the id of vocabulary 'Genre' ", Where  'Genre' is just an example. It can be id of any other vocabulary as well which has been created.

Resolving patch:
diff --git a/ckan/logic/action/create.py b/ckan/logic/action/create.py
--- a/ckan/logic/action/create.py
+++ b/ckan/logic/action/create.py
@@ -1302,7 +1302,8 @@ def tag_create(context, data_dict):
     :py:func:`~ckan.logic.action.update.package_update` function.)
 
     :param name: the name for the new tag, a string between 2 and 100
-        characters long containing only alphanumeric characters and ``-``,
+        characters long containing only alphanumeric characters, 
+        spaces and the characters ``-``,
         ``_`` and ``.``, e.g. ``'Jazz'``
     :type name: string
     :param vocabulary_id: the id of the vocabulary that the new tag


Test patch:
Repository: ckan/ckan

Issue:
"Top Rated Datasets" needs to be removed from stats
**CKAN version**
2.9.1

**Describe the bug**
"Top Rated Datasets" is still in /stats even though ratings are being deprecated #5558

**Steps to reproduce**
With stats enabled, go to /stats page.

**Expected behavior**
"Top Rated Datasets" is removed from stats.



Discussion:

Resolving patch:
diff --git a/ckanext/stats/blueprint.py b/ckanext/stats/blueprint.py
--- a/ckanext/stats/blueprint.py
+++ b/ckanext/stats/blueprint.py
@@ -13,7 +13,6 @@
 def index():
     stats = stats_lib.Stats()
     extra_vars = {
-        u'top_rated_packages': stats.top_rated_packages(),
         u'largest_groups': stats.largest_groups(),
         u'top_tags': stats.top_tags(),
         u'top_package_creators': stats.top_package_creators(),
diff --git a/ckanext/stats/stats.py b/ckanext/stats/stats.py
--- a/ckanext/stats/stats.py
+++ b/ckanext/stats/stats.py
@@ -32,30 +32,6 @@ def datetime2date(datetime_):
 
 class Stats(object):
 
-    @classmethod
-    def top_rated_packages(cls, limit=10):
-        # NB Not using sqlalchemy as sqla 0.4 doesn't work using both group_by
-        # and apply_avg
-        package = table('package')
-        rating = table('rating')
-        sql = select(
-            [
-                package.c.id,
-                func.avg(rating.c.rating),
-                func.count(rating.c.rating)
-            ],
-            from_obj=[package.join(rating)]
-        ).where(and_(package.c.private == False, package.c.state == 'active')
-                ).group_by(package.c.id).order_by(
-                    func.avg(rating.c.rating).desc(),
-                    func.count(rating.c.rating).desc()
-                ).limit(limit)
-        res_ids = model.Session.execute(sql).fetchall()
-        res_pkgs = [(
-            model.Session.query(model.Package).get(text_type(pkg_id)), avg, num
-        ) for pkg_id, avg, num in res_ids]
-        return res_pkgs
-
     @classmethod
     def largest_groups(cls, limit=10):
         member = table('member')


Test patch:
diff --git a/ckanext/stats/tests/test_stats_lib.py b/ckanext/stats/tests/test_stats_lib.py
--- a/ckanext/stats/tests/test_stats_lib.py
+++ b/ckanext/stats/tests/test_stats_lib.py
@@ -54,10 +54,6 @@ def initial_data(self, clean_db, with_request_context):
         model.Package.by_name(u"test3").notes = "Test 3 notes"
         model.repo.commit_and_remove()
 
-    def test_top_rated_packages(self):
-        pkgs = Stats.top_rated_packages()
-        assert pkgs == []
-
     def test_largest_groups(self):
         grps = Stats.largest_groups()
         grps = [(grp.name, count) for grp, count in grps]
Repository: ckan/ckan

Issue:
French: "alphanumeric characters" translated to "lettres minuscules, des chiffres"
The English string

> Tag \"%s\" must be alphanumeric characters or symbols: -_.

...which is used in ckan/logic/validators.py:428 to validate tags, has an inaccurate translation in French:

> Le mot-clé \"%s\" ne peut contenir que des lettres minuscules, des chiffres ou les symboles : -_.

An alphanumeric character can be a digit or any letter, whether lowercase or not, so the French description is overly restrictive. An accurate translation could therefore be:
"Le mot-clé \"%s\" ne peut contenir que des lettres, des chiffres ou les symboles : -_."

This affects CKAN 2.4.1, and persists in the master branch as of a3a6b582fcd758727c98b628d13d198b4317787c.

Note that it is unclear which (special) symbols these strings refer to.


Discussion:
@Chealer that sounds good. Translations are managed on Transifex (https://www.transifex.com/okfn/ckan/). If you register for an account you can update the strings for the 2.6, 2.7 and 2.8 resources, and next time we do a patch release we will pull the latest translations with the fixes:

![Screenshot_2019-12-10 CKAN Languages](https://user-images.githubusercontent.com/200230/70528781-49a2e500-1b4f-11ea-919d-5def961b8bd4.png)

Thank you @amercader but I'm afraid an efficient solution for this string would start with a clarification of the English version.
@Chealer sorry, I don't follow. What needs clarification on the English string?
@amercader the "symbols" (special characters) which are allowed (if I understand correctly, dashes and underscores).
> Tag "%s" must be alphanumeric characters or symbols: -_.

The only [allowed](https://github.com/ckan/ckan/blob/master/ckan/logic/validators.py#L432) punctuation symbols are hyphen (`-`), underscore (`_`) and full stop (`.`). That's what it says on the message. Perhaps is the colon what is confusing?

If you can think of a better message please do send a PR and then we can create a new French translation.



Resolving patch:
diff --git a/ckan/logic/validators.py b/ckan/logic/validators.py
--- a/ckan/logic/validators.py
+++ b/ckan/logic/validators.py
@@ -454,8 +454,9 @@ def tag_name_validator(value, context):
 
     tagname_match = re.compile('[\w \-.]*$', re.UNICODE)
     if not tagname_match.match(value):
-        raise Invalid(_('Tag "%s" must be alphanumeric '
-                        'characters or symbols: -_.') % (value))
+        raise Invalid(_(''Tag "%s" can only contain alphanumeric '
+                        'characters, spaces (" "), hyphens ("-"), '
+                        'underscores ("_") or dots (".")') % (value))
     return value
 
 def tag_not_uppercase(value, context):


Test patch:
Repository: ckan/ckan

Issue:
Wrong Exception in package_collaborator_create
**CKAN version**
2.9.1

**Describe the bug**
When attempting to add a non-existent user as a dataset collaborator a NotAuthorized exception is raised, which should probably be a validation or not found exception. The message 'Not allowed to add collaborators' also does not make sense for the underlying cause. When using this functionality via the web interface (at least in my implementation) the NotAuthorized exception logs the user out, which is not desired behavior.

https://github.com/ckan/ckan/blob/254b18cd274dc23f2895d9de7b7eb3022b49a534/ckan/logic/action/create.py#L671





Discussion:

Resolving patch:
diff --git a/ckan/logic/action/create.py b/ckan/logic/action/create.py
--- a/ckan/logic/action/create.py
+++ b/ckan/logic/action/create.py
@@ -668,7 +668,7 @@ def package_collaborator_create(context, data_dict):
 
     user = model.User.get(user_id)
     if not user:
-        raise NotAuthorized(_('Not allowed to add collaborators'))
+        raise NotFound(_('User not found'))
 
     if not authz.check_config_permission('allow_dataset_collaborators'):
         raise ValidationError(_('Dataset collaborators not enabled'))
diff --git a/ckan/views/dataset.py b/ckan/views/dataset.py
--- a/ckan/views/dataset.py
+++ b/ckan/views/dataset.py
@@ -1313,10 +1313,12 @@ def post(self, package_type, id):
         except NotAuthorized:
             message = _(u'Unauthorized to edit collaborators {}').format(id)
             return base.abort(401, _(message))
-        except NotFound:
-            return base.abort(404, _(u'Resource not found'))
+        except NotFound as e:
+            h.flash_error(_('User not found'))
+            return h.redirect_to(u'dataset.new_collaborator', id=id)
         except ValidationError as e:
             h.flash_error(e.error_summary)
+            return h.redirect_to(u'dataset.new_collaborator', id=id)
         else:
             h.flash_success(_(u'User added to collaborators'))
 


Test patch:
diff --git a/ckan/tests/logic/action/test_create.py b/ckan/tests/logic/action/test_create.py
--- a/ckan/tests/logic/action/test_create.py
+++ b/ckan/tests/logic/action/test_create.py
@@ -1243,12 +1243,12 @@ def test_create_dataset_not_found(self):
                 'package_collaborator_create',
                 id=dataset['id'], user_id=user['id'], capacity=capacity)
 
-    def test_create_user_not_authorized(self):
+    def test_create_user_not_found(self):
         dataset = factories.Dataset()
         user = {'id': 'yyy'}
         capacity = 'editor'
 
-        with pytest.raises(logic.NotAuthorized):
+        with pytest.raises(logic.NotFound):
             helpers.call_action(
                 'package_collaborator_create',
                 id=dataset['id'], user_id=user['id'], capacity=capacity)