Repository: oppia/oppia

Issue:
Move get_full_customization_args from exp_domain to utils
Related to #4946 -- see https://github.com/oppia/oppia/pull/4946#discussion_r189789445

The get_full_customization_args method currently in exp_domain is not relevant there anymore. Now, it is being used from stats_domain as well, and it is better to move this method to utils.py instead.
Move get_full_customization_args from exp_domain to utils
Related to #4946 -- see https://github.com/oppia/oppia/pull/4946#discussion_r189789445

The get_full_customization_args method currently in exp_domain is not relevant there anymore. Now, it is being used from stats_domain as well, and it is better to move this method to utils.py instead.


Discussion:
Hi @pranavsid98  -- this issue description is insufficient. Please write a short description explaining the context and what needs to be done. Thanks!
@pranavsid98 are you currently working on this?
@jacobdavis11 Its not a blocking issue and I thought Id get to this after my GSOC. Hope thats fine. Thanks.
@pranavsid98 any updates on this?
@pranavsid98 GSoC is over :) 
Would you be able to pick this up now?
Checked with @pranavsid98, he'll take this up and complete it by 30 Sep. Thanks! 
What's the status of this issue? @brianrodri 
Hi @brianrodri, any updates?
I don't have much bandwidth to work on this, so I'm unassigning myself and giving it a space in the project page for someone to pick up.
Hi @brianrodri, May I take this up, Thanks!
Sure, it's all yours @ankita240796! Thanks!
Thanks! @brianrodri!
Hi @pranavsid98  -- this issue description is insufficient. Please write a short description explaining the context and what needs to be done. Thanks!
@pranavsid98 are you currently working on this?
@jacobdavis11 Its not a blocking issue and I thought Id get to this after my GSOC. Hope thats fine. Thanks.
@pranavsid98 any updates on this?
@pranavsid98 GSoC is over :) 
Would you be able to pick this up now?
Checked with @pranavsid98, he'll take this up and complete it by 30 Sep. Thanks! 
What's the status of this issue? @brianrodri 
Hi @brianrodri, any updates?
I don't have much bandwidth to work on this, so I'm unassigning myself and giving it a space in the project page for someone to pick up.
Hi @brianrodri, May I take this up, Thanks!
Sure, it's all yours @ankita240796! Thanks!
Thanks! @brianrodri!

Resolving patch:
diff --git a/core/domain/customization_args_util.py b/core/domain/customization_args_util.py
new file mode 100644
--- /dev/null
+++ b/core/domain/customization_args_util.py
@@ -0,0 +1,121 @@
+# coding: utf-8
+#
+# Copyright 2018 The Oppia Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS-IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Utility methods for customization args of interactions."""
+
+import logging
+
+import schema_utils
+import utils
+
+
+def get_full_customization_args(customization_args, ca_specs):
+    """Populates the given customization_args dict with default values
+    if any of the expected customization_args are missing.
+
+    Args:
+        customization_args: dict. The customization dict. The keys are names
+            of customization_args and the values are dicts with a
+            single key, 'value', whose corresponding value is the value of
+            the customization arg.
+        ca_specs: list(dict). List of spec dictionaries. Is used to check if
+            some keys are missing in customization_args. Dicts have the
+            following structure:
+                - name: str. The customization variable name.
+                - description: str. The customization variable description.
+                - default_value: *. The default value of the customization
+                    variable.
+
+    Returns:
+        dict. The customization_args dict where missing keys are populated
+        with the default values.
+    """
+    for ca_spec in ca_specs:
+        if ca_spec.name not in customization_args:
+            customization_args[ca_spec.name] = {
+                'value': ca_spec.default_value
+            }
+    return customization_args
+
+
+def validate_customization_args_and_values(
+        item_name, item_type, customization_args,
+        ca_specs_to_validate_against):
+    """Validates the given `customization_args` dict against the specs set
+    out in 'ca_specs_to_validate_against'. 'item_name' and 'item_type' are
+    used to populate any error messages that arise during validation.
+    Note that this may modify the given customization_args dict, if it has
+    extra or missing keys. It also normalizes any HTML in the
+    customization_args dict.
+
+    Args:
+        item_name: str. This is always 'interaction'.
+        item_type: str. The item_type is the ID of the interaction.
+        customization_args: dict. The customization dict. The keys are names
+            of customization_args and the values are dicts with a
+            single key, 'value', whose corresponding value is the value of
+            the customization arg.
+        ca_specs_to_validate_against: list(dict). List of spec dictionaries.
+            Is used to check if some keys are missing in customization_args.
+            Dicts have the following structure:
+                - name: str. The customization variable name.
+                - description: str. The customization variable description.
+                - default_value: *. The default value of the customization
+                    variable.
+
+    Raises:
+        ValidationError: The given 'customization_args' is not valid.
+    """
+    ca_spec_names = [
+        ca_spec.name for ca_spec in ca_specs_to_validate_against]
+
+    if not isinstance(customization_args, dict):
+        raise utils.ValidationError(
+            'Expected customization args to be a dict, received %s'
+            % customization_args)
+
+    # Validate and clean up the customization args.
+
+    # Populate missing keys with the default values.
+    customization_args = get_full_customization_args(
+        customization_args, ca_specs_to_validate_against)
+
+    # Remove extra keys.
+    extra_args = []
+    for arg_name in customization_args.keys():
+        if not isinstance(arg_name, basestring):
+            raise utils.ValidationError(
+                'Invalid customization arg name: %s' % arg_name)
+        if arg_name not in ca_spec_names:
+            extra_args.append(arg_name)
+            logging.warning(
+                '%s %s does not support customization arg %s.'
+                % (item_name.capitalize(), item_type, arg_name))
+    for extra_arg in extra_args:
+        del customization_args[extra_arg]
+
+    # Check that each value has the correct type.
+    for ca_spec in ca_specs_to_validate_against:
+        try:
+            customization_args[ca_spec.name]['value'] = (
+                schema_utils.normalize_against_schema(
+                    customization_args[ca_spec.name]['value'],
+                    ca_spec.schema))
+        except Exception:
+            # TODO(sll): Raise an actual exception here if parameters are
+            # not involved (If they are, can we get sample values for the
+            # state context parameters?).
+            pass
diff --git a/core/domain/state_domain.py b/core/domain/state_domain.py
--- a/core/domain/state_domain.py
+++ b/core/domain/state_domain.py
@@ -20,114 +20,15 @@
 import logging
 
 from constants import constants
+from core.domain import customization_args_util
 from core.domain import html_cleaner
 from core.domain import interaction_registry
 from core.domain import param_domain
 import feconf
 import jinja_utils
-import schema_utils
 import utils
 
 
-def get_full_customization_args(customization_args, ca_specs):
-    """Populates the given customization_args dict with default values
-    if any of the expected customization_args are missing.
-
-    Args:
-        customization_args: dict. The customization dict. The keys are names of
-            customization_args and the values are dicts with a
-            single key, 'value', whose corresponding value is the value of
-            the customization arg.
-        ca_specs: list(dict). List of spec dictionaries. Is used to check if
-            some keys are missing in customization_args. Dicts have the
-            following structure:
-                - name: str. The customization variable name.
-                - description: str. The customization variable description.
-                - default_value: *. The default value of the customization
-                    variable.
-
-    Returns:
-        dict. The customization_args dict where missing keys are populated with
-        the default values.
-    """
-    for ca_spec in ca_specs:
-        if ca_spec.name not in customization_args:
-            customization_args[ca_spec.name] = {
-                'value': ca_spec.default_value
-            }
-    return customization_args
-
-
-def validate_customization_args_and_values(
-        item_name, item_type, customization_args,
-        ca_specs_to_validate_against):
-    """Validates the given `customization_args` dict against the specs set out
-    in 'ca_specs_to_validate_against'. 'item_name' and 'item_type' are used to
-    populate any error messages that arise during validation.
-    Note that this may modify the given customization_args dict, if it has
-    extra or missing keys. It also normalizes any HTML in the
-    customization_args dict.
-
-    Args:
-        item_name: str. This is always 'interaction'.
-        item_type: str. The item_type is the ID of the interaction.
-        customization_args: dict. The customization dict. The keys are names of
-            customization_args and the values are dicts with a
-            single key, 'value', whose corresponding value is the value of
-            the customization arg.
-        ca_specs_to_validate_against: list(dict). List of spec dictionaries. Is
-            used to check if some keys are missing in customization_args. Dicts
-            have the following structure:
-                - name: str. The customization variable name.
-                - description: str. The customization variable description.
-                - default_value: *. The default value of the customization
-                    variable.
-
-    Raises:
-        ValidationError: The given 'customization_args' is not valid.
-    """
-    ca_spec_names = [
-        ca_spec.name for ca_spec in ca_specs_to_validate_against]
-
-    if not isinstance(customization_args, dict):
-        raise utils.ValidationError(
-            'Expected customization args to be a dict, received %s'
-            % customization_args)
-
-    # Validate and clean up the customization args.
-
-    # Populate missing keys with the default values.
-    customization_args = get_full_customization_args(
-        customization_args, ca_specs_to_validate_against)
-
-    # Remove extra keys.
-    extra_args = []
-    for arg_name in customization_args.keys():
-        if not isinstance(arg_name, basestring):
-            raise utils.ValidationError(
-                'Invalid customization arg name: %s' % arg_name)
-        if arg_name not in ca_spec_names:
-            extra_args.append(arg_name)
-            logging.warning(
-                '%s %s does not support customization arg %s.'
-                % (item_name.capitalize(), item_type, arg_name))
-    for extra_arg in extra_args:
-        del customization_args[extra_arg]
-
-    # Check that each value has the correct type.
-    for ca_spec in ca_specs_to_validate_against:
-        try:
-            customization_args[ca_spec.name]['value'] = (
-                schema_utils.normalize_against_schema(
-                    customization_args[ca_spec.name]['value'],
-                    ca_spec.schema))
-        except Exception:
-            # TODO(sll): Raise an actual exception here if parameters are not
-            # involved (If they are, can we get sample values for the state
-            # context parameters?).
-            pass
-
-
 class AnswerGroup(object):
     """Value object for an answer group. Answer groups represent a set of rules
     dictating whether a shared feedback should be shared with the user. These
@@ -369,7 +270,7 @@ def to_dict(self):
             'id': self.id,
             'customization_args': (
                 {} if self.id is None else
-                get_full_customization_args(
+                customization_args_util.get_full_customization_args(
                     self.customization_args,
                     interaction_registry.Registry.get_interaction_by_id(
                         self.id).customization_arg_specs)),
@@ -498,7 +399,7 @@ def validate(self, exp_param_specs_dict):
         except KeyError:
             raise utils.ValidationError('Invalid interaction id: %s' % self.id)
 
-        validate_customization_args_and_values(
+        customization_args_util.validate_customization_args_and_values(
             'interaction', self.id, self.customization_args,
             interaction.customization_arg_specs)
 
diff --git a/core/domain/stats_domain.py b/core/domain/stats_domain.py
--- a/core/domain/stats_domain.py
+++ b/core/domain/stats_domain.py
@@ -20,10 +20,10 @@
 import sys
 
 from core.domain import action_registry
+from core.domain import customization_args_util
 from core.domain import exp_domain
 from core.domain import interaction_registry
 from core.domain import playthrough_issue_registry
-from core.domain import state_domain
 from core.platform import models
 import feconf
 import utils
@@ -615,7 +615,7 @@ def validate(self):
             raise utils.ValidationError('Invalid issue type: %s' % (
                 self.issue_type))
 
-        state_domain.validate_customization_args_and_values(
+        customization_args_util.validate_customization_args_and_values(
             'issue', self.issue_type, self.issue_customization_args,
             issue.customization_arg_specs)
 
@@ -662,7 +662,7 @@ def to_dict(self):
         return {
             'issue_type': self.issue_type,
             'issue_customization_args': (
-                state_domain.get_full_customization_args(
+                customization_args_util.get_full_customization_args(
                     self.issue_customization_args,
                     playthrough_issue_registry.Registry.get_issue_by_type(
                         self.issue_type).customization_arg_specs)),
@@ -760,7 +760,7 @@ def validate(self):
             raise utils.ValidationError('Invalid issue type: %s' % (
                 self.issue_type))
 
-        state_domain.validate_customization_args_and_values(
+        customization_args_util.validate_customization_args_and_values(
             'issue', self.issue_type, self.issue_customization_args,
             issue.customization_arg_specs)
 
@@ -803,7 +803,7 @@ def to_dict(self):
         return {
             'action_type': self.action_type,
             'action_customization_args': (
-                state_domain.get_full_customization_args(
+                customization_args_util.get_full_customization_args(
                     self.action_customization_args,
                     action_registry.Registry.get_action_by_type(
                         self.action_type).customization_arg_specs)),
@@ -869,7 +869,7 @@ def validate(self):
             raise utils.ValidationError(
                 'Invalid action type: %s' % self.action_type)
 
-        state_domain.validate_customization_args_and_values(
+        customization_args_util.validate_customization_args_and_values(
             'action', self.action_type, self.action_customization_args,
             action.customization_arg_specs)
 


Test patch:
diff --git a/core/domain/customization_args_util_test.py b/core/domain/customization_args_util_test.py
new file mode 100644
--- /dev/null
+++ b/core/domain/customization_args_util_test.py
@@ -0,0 +1,333 @@
+# coding: utf-8
+#
+# Copyright 2018 The Oppia Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS-IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Unit tests for core.domain.customization_args_utils."""
+
+import logging
+
+from core.domain import customization_args_util
+from core.domain import interaction_registry
+from core.tests import test_utils
+import utils
+
+
+class CustomizationArgsUtilUnitTests(test_utils.GenericTestBase):
+    """Test customization args generation and validation."""
+
+    def test_get_full_customization_args(self):
+        """Test get full customization args method."""
+        ca_continue_specs = interaction_registry.Registry.get_interaction_by_id(
+            'Continue').customization_arg_specs
+        complete_customization_args = {
+            'buttonText': {'value': 'Please Continue'}
+        }
+
+        complete_customization_args_with_extra_arg = {
+            'buttonText': {'value': 'Please Continue'},
+            'extraArg': {'value': ''}
+        }
+
+        # Check if no new key is added to customization arg dict if all specs
+        # are present.
+        self.assertEqual(
+            complete_customization_args,
+            customization_args_util.get_full_customization_args(
+                complete_customization_args, ca_continue_specs
+            )
+        )
+
+        # Check if no new key is added to customization arg dict and extra keys
+        # are not removed if all specs are present.
+        self.assertEqual(
+            complete_customization_args_with_extra_arg,
+            customization_args_util.get_full_customization_args(
+                complete_customization_args_with_extra_arg,
+                ca_continue_specs
+            )
+        )
+
+        ca_fraction_input_specs = (
+            interaction_registry.Registry.get_interaction_by_id(
+                'FractionInput').customization_arg_specs
+        )
+
+        incomplete_customization_args = {
+            'requireSimplestForm': {'value': False},
+            'allowNonzeroIntegerPart': {'value': False}
+        }
+
+        incomplete_customization_args_with_extra_arg = {
+            'requireSimplestForm': {'value': False},
+            'allowNonzeroIntegerPart': {'value': False},
+            'extraArg': {'value': ''}
+        }
+
+        expected_complete_customization_args = {
+            'requireSimplestForm': {'value': False},
+            'allowImproperFraction': {'value': True},
+            'allowNonzeroIntegerPart': {'value': False},
+            'customPlaceholder': {'value': ''}
+        }
+
+        expected_complete_customization_args_with_extra_arg = {
+            'requireSimplestForm': {'value': False},
+            'allowImproperFraction': {'value': True},
+            'allowNonzeroIntegerPart': {'value': False},
+            'customPlaceholder': {'value': ''},
+            'extraArg': {'value': ''}
+        }
+
+        # Check if missing specs are added to customization arg dict without
+        # making any other change.
+        self.assertEqual(
+            expected_complete_customization_args,
+            customization_args_util.get_full_customization_args(
+                incomplete_customization_args, ca_fraction_input_specs
+            )
+        )
+
+        # Check if missing specs are added to customization arg dict without
+        # making any other change and without removing extra args.
+        self.assertEqual(
+            expected_complete_customization_args_with_extra_arg,
+            customization_args_util.get_full_customization_args(
+                incomplete_customization_args_with_extra_arg,
+                ca_fraction_input_specs
+            )
+        )
+
+    def test_validate_customization_args_and_values(self):
+        """Test validate customization args and values method."""
+
+        observed_log_messages = []
+
+        def mock_logging_function(msg, *_):
+            observed_log_messages.append(msg)
+
+        ca_item_selection_specs = (
+            interaction_registry.Registry.get_interaction_by_id(
+                'ItemSelectionInput').customization_arg_specs
+        )
+
+        complete_customization_args = {
+            'minAllowableSelectionCount': {'value': 1},
+            'maxAllowableSelectionCount': {'value': 1},
+            'choices': ['']
+        }
+
+        complete_customization_args_with_invalid_arg_name = {
+            'minAllowableSelectionCount': {'value': 1},
+            'maxAllowableSelectionCount': {'value': 1},
+            'choices': [''],
+            23: {'value': ''}
+        }
+
+        complete_customization_args_with_extra_arg = {
+            'minAllowableSelectionCount': {'value': 1},
+            'maxAllowableSelectionCount': {'value': 1},
+            'choices': [''],
+            'extraArg': {'value': ''}
+        }
+
+        complete_customization_args_with_invalid_arg_type = {
+            'minAllowableSelectionCount': {'value': 'invalid'},
+            'maxAllowableSelectionCount': {'value': 1},
+            'choices': ['']
+        }
+
+        expected_customization_args_after_validation = {
+            'minAllowableSelectionCount': {'value': 1},
+            'maxAllowableSelectionCount': {'value': 1},
+            'choices': ['']
+        }
+
+        expected_customization_args_after_validation_with_invalid_arg_type = (
+            complete_customization_args_with_invalid_arg_type
+        )
+
+        # The next four checks are for cases where customization args dict
+        # contains all required specs.
+
+        # Check if no error is produced for valid customization args.
+        customization_args_util.validate_customization_args_and_values(
+            'interaction',
+            'ItemSelectionInput',
+            complete_customization_args,
+            ca_item_selection_specs
+        )
+        self.assertEqual(
+            expected_customization_args_after_validation,
+            complete_customization_args
+        )
+
+        # Check if error is produced when arg name is invalid.
+        with self.assertRaisesRegexp(
+            utils.ValidationError,
+            'Invalid customization arg name: 23'
+        ):
+            customization_args_util.validate_customization_args_and_values(
+                'interaction',
+                'ItemSelectionInput',
+                complete_customization_args_with_invalid_arg_name,
+                ca_item_selection_specs
+            )
+
+        # Check if error is produced when extra args are present.
+        with self.swap(logging, 'warning', mock_logging_function):
+            customization_args_util.validate_customization_args_and_values(
+                'interaction',
+                'ItemSelectionInput',
+                complete_customization_args_with_extra_arg,
+                ca_item_selection_specs
+            )
+            self.assertEqual(len(observed_log_messages), 1)
+            self.assertEqual(
+                observed_log_messages[0],
+                (
+                    'Interaction ItemSelectionInput does not support '
+                    'customization arg extraArg.'
+                )
+            )
+            self.assertEqual(
+                expected_customization_args_after_validation,
+                complete_customization_args_with_extra_arg
+            )
+
+        # Check if no error is produced when arg type is not valid.
+        customization_args_util.validate_customization_args_and_values(
+            'interaction',
+            'ItemSelectionInput',
+            complete_customization_args_with_invalid_arg_type,
+            ca_item_selection_specs
+        )
+
+        self.assertEqual(
+            expected_customization_args_after_validation_with_invalid_arg_type,
+            complete_customization_args_with_invalid_arg_type
+        )
+
+        ca_fraction_input_specs = (
+            interaction_registry.Registry.get_interaction_by_id(
+                'FractionInput').customization_arg_specs
+        )
+
+        incomplete_customization_args = {
+            'requireSimplestForm': {'value': False},
+            'allowNonzeroIntegerPart': {'value': False}
+        }
+
+        incomplete_customization_args_with_invalid_arg_name = {
+            'requireSimplestForm': {'value': False},
+            False: {'value': False},
+        }
+
+        incomplete_customization_args_with_extra_arg = {
+            'requireSimplestForm': {'value': False},
+            'allowNonzeroIntegerPart': {'value': False},
+            'extraArg': {'value': ''}
+        }
+
+        incomplete_customization_args_with_invalid_arg_type = {
+            'requireSimplestForm': {'value': False},
+            'allowNonzeroIntegerPart': {'value': False},
+            'customPlaceholder': {'value': 12}
+        }
+
+        expected_customization_args_after_validation = {
+            'requireSimplestForm': {'value': False},
+            'allowImproperFraction': {'value': True},
+            'allowNonzeroIntegerPart': {'value': False},
+            'customPlaceholder': {'value': ''}
+        }
+
+        expected_customization_args_after_validation_with_invalid_arg_type = (
+            incomplete_customization_args_with_invalid_arg_type
+        )
+
+        # The next four checks are for cases where customization args dict
+        # does not contain some of the required specs.
+
+        # Check if no error is produced for valid customization args.
+        customization_args_util.validate_customization_args_and_values(
+            'interaction',
+            'FractionInput',
+            incomplete_customization_args,
+            ca_fraction_input_specs
+        )
+        self.assertEqual(
+            expected_customization_args_after_validation,
+            incomplete_customization_args
+        )
+
+        # Check if error is produced when arg name is invalid.
+        with self.assertRaisesRegexp(
+            utils.ValidationError,
+            'Invalid customization arg name: False'
+        ):
+            customization_args_util.validate_customization_args_and_values(
+                'interaction',
+                'FractionInput',
+                incomplete_customization_args_with_invalid_arg_name,
+                ca_fraction_input_specs
+            )
+
+        # Check if error is produced when extra args are present.
+        with self.swap(logging, 'warning', mock_logging_function):
+            customization_args_util.validate_customization_args_and_values(
+                'interaction',
+                'FractionInput',
+                incomplete_customization_args_with_extra_arg,
+                ca_fraction_input_specs
+            )
+            self.assertEqual(len(observed_log_messages), 2)
+            self.assertEqual(
+                observed_log_messages[1],
+                (
+                    'Interaction FractionInput does not support customization '
+                    'arg extraArg.'
+                )
+            )
+            self.assertEqual(
+                expected_customization_args_after_validation,
+                incomplete_customization_args_with_extra_arg
+            )
+
+        # Check if no error is produced when arg type is not valid.
+        customization_args_util.validate_customization_args_and_values(
+            'interaction',
+            'FractionInput',
+            incomplete_customization_args_with_invalid_arg_type,
+            ca_fraction_input_specs
+        )
+        self.assertEqual(
+            expected_customization_args_after_validation_with_invalid_arg_type,
+            incomplete_customization_args_with_invalid_arg_type
+        )
+
+        # A general check to see if error are produced when customization args
+        # is not of type dict.
+        customization_args_with_invalid_type = 23
+        with self.assertRaisesRegexp(
+            utils.ValidationError,
+            'Expected customization args to be a dict, received %s'
+            % customization_args_with_invalid_type
+        ):
+            customization_args_util.validate_customization_args_and_values(
+                'interaction',
+                'FractionInput',
+                customization_args_with_invalid_type,
+                ca_fraction_input_specs
+            )