Repository: ckan/ckan

Issue:
Remove or replace formencode support
Although it's not explicitly listed as a requirement in `requirements.in` (because it's a Pylons requirement), formencode is used in two places:

* Its `OneOf` validator is added to the standard CKAN ones [here](https://github.com/ckan/ckan/blob/f43d6a572838c792193f3239827d04f9ffea9206/ckan/logic/__init__.py#L677).
* There is support for defining formencode validators in the CKAN schema ([docs](https://docs.ckan.org/en/2.8/extensions/adding-custom-fields.html?highlight=formencode#custom-validators), [code](https://github.com/ckan/ckan/blob/b9e45e2723d4abd70fa72b16ec4a0bebc795c56b/ckan/lib/navl/dictization_functions.py#L219:L233))

We need to remove the requirement, because it hasn't been updated in 3 years and its setup.py only lists support up until Python 3.5.

We use `OneOf` in core and someone might be using it in an extension so we need to implement it, but I doubt that anyone is using the second feature, and that we should drop it.

In the future we can add similar support for a better maintained validation library like [WTForms](https://wtforms.readthedocs.io/).

#### Approach

* Write our own implementation of `OneOf` in `validators.py` (and rename it to `one_of` for consistency, but keeping the old name for backwards compatibility).
* Remove all formencode bits from the code base.

#### Story points

2



Discussion:
should I write it from scratch or just plug in WTForms in place of OneOf
and which branch should I implement this on
@batitaye sorry I missed your comment. You should use the current `master` branch as a base for this.
I'd prefer if you re-implemented `OneOf` on our code base to avoid the WTForms requirement for now. You can add it to [this module](https://github.com/ckan/ckan/blob/f43d6a572838c792193f3239827d04f9ffea9206/ckan/logic/validators.py) with the rest of the core ones.
Let me know if you need any more pointers and thanks for your help!
Hi @amercader , is there anyone working on this issue?
@iamarnavgarg No AFAICT. Unless @batitaye is working on it feel free to submit a PR. As I mentioned, OneOf should be re-implemented in our own [validators module](https://github.com/ckan/ckan/blob/f43d6a572838c792193f3239827d04f9ffea9206/ckan/logic/validators.py), and references to it updated.
@amercader in formencode OneOf is a class and so CKAN expects the implementation of OneOf to be an object but in [validators module](https://github.com/ckan/ckan/blob/f43d6a572838c792193f3239827d04f9ffea9206/ckan/logic/validators.py "validator.py") every validation implementations are functions so is there a reason why the new implementation of OneOf can't be a class?
Hi @amercader , I would like to inform you that, I have started working on this issue.
Hi @amercader , As per my analysis -`one_of(value,context)` should be defined at this location `/src/ckan/ckan/logic/validators.py` . In this defined function, it will take `value` as a parameter and a context parameter that contains a list or a tuple.

Then, using an iterator it will check whether the list(available in context parameter) contains the value. if it exists in the list/tuple, it will return the same value, otherwise raise an exception if not found. 

Could you please confirm with my understanding ? 

Resolving patch:
diff --git a/ckan/logic/__init__.py b/ckan/logic/__init__.py
--- a/ckan/logic/__init__.py
+++ b/ckan/logic/__init__.py
@@ -1,4 +1,5 @@
-# encoding: utf-8
+
+#encoding: utf-8
 
 import functools
 import logging
@@ -6,13 +7,13 @@
 import sys
 from collections import defaultdict
 
-import formencode.validators
 from six import string_types, text_type
 
 import ckan.model as model
 import ckan.authz as authz
 import ckan.lib.navl.dictization_functions as df
 import ckan.plugins as p
+import ckan.logic.validators as valid_one_of
 
 from ckan.common import _, c
 
@@ -674,7 +675,7 @@ def get_validator(validator):
         _validators_cache.update(validators)
         validators = _import_module_functions('ckan.logic.validators')
         _validators_cache.update(validators)
-        _validators_cache.update({'OneOf': formencode.validators.OneOf})
+        _validators_cache.update({'OneOf': valid_one_of.one_of})
         converters = _import_module_functions('ckan.logic.converters')
         _validators_cache.update(converters)
 
diff --git a/ckan/logic/validators.py b/ckan/logic/validators.py
--- a/ckan/logic/validators.py
+++ b/ckan/logic/validators.py
@@ -1,3 +1,4 @@
+
 # encoding: utf-8
 
 import collections
@@ -858,3 +859,14 @@ def email_validator(value, context):
         if not email_pattern.match(value):
             raise Invalid(_('Email {email} is not a valid format').format(email=value))
     return value
+
+def one_of(list_of_value):
+   ''' Implementation of OneOf method
+       for Python Library Formencode
+   '''
+   def callable(value):
+      if value not in list_of_value:
+	      raise Invalid(_('Value must be one of {}'.format(list_of_value)))
+      return value
+
+


Test patch:
diff --git a/ckan/tests/logic/test_validators.py b/ckan/tests/logic/test_validators.py
--- a/ckan/tests/logic/test_validators.py
+++ b/ckan/tests/logic/test_validators.py
@@ -1,3 +1,4 @@
+
 # encoding: utf-8
 
 '''Unit tests for ckan/logic/validators.py.
@@ -667,4 +668,17 @@ def call_validator(*args, **kwargs):
             errors[key] = []
             call_validator(key, {key: url}, errors, None)
 
+class TestOneOfValidator(object):
+
+    def test_val_in_list(self):
+        cont = [1,2,3,4]
+        func = validators.one_of(cont)
+        assert_equals(func(1),1)
+
+    @raises_Invalid
+    def test_val_not_in_list(self):
+        cont = [1,2,3,4]
+        func = validators.one_of(cont)
+        raises_Invalid(func)(5)
+
 # TODO: Need to test when you are not providing owner_org and the validator queries for the dataset with package_show