Repository: oppia/oppia

Issue:
Check for the number of training examples before generating a classifier.
As of now, the [code](https://github.com/oppia/oppia/blob/develop/core/domain/classifier_services.py#L78) used for generating classifier models does not check for the number of examples in the state dict before generating a classifier. We need to decide the threshold of certain things like "minimum total number of training examples", "minimum number of labels (or outcomes) with training examples" that need to be checked before generating classifiers. Consequently, the back-end code should check for these thresholds and generate models only when these criteria are met.

The two thresholds can be something like (residing in feconf) -
1. MIN_TOTAL_TRAINING_EXAMPLES - should be around 50 (we might have to set this empirically)
2. MIN_ASSIGNED_LABELS - should be around 2 (since classification for a single class is not of any use) 

This issue depends on the completion of #3172 


Discussion:
@anmolshkl . I'll take this issue up. This also depends on merging of the caching classifiers PR though.
@pranavsid98 yep, sure. 
I'm keeping this issue on hold as this requires the completion of the caching issue and there's more work left in that. Should I de-assign myself until that @anmolshkl ?
@anmolshkl .
The LDAStringClassifier already has defined a few of these thresholds.
    _DEFAULT_MIN_DOCS_TO_PREDICT = 20
    _DEFAULT_MIN_LABELS_TO_PREDICT = 3

Isn't this sort of what we are trying to accomplish?
@pranavsid98 yes it's exactly that. Instead of keeping this check within every classifier, we just would like to keep it at some higher level.
@giritheja how's this going on? 

FYI, our strategy to generate classifiers was wrong. So, we need to generate classifiers when the exploration is saved and not when we receive a request for prediction from learner's view. But, you should be implementing the condition checking code in classifier_services.py and so it should not get affected by this decision in anyway.
@giritheja @anmolshkl . Since this issue was directly related to my subsequent PRs on caching classifiers, I have finished implementing the function for checking training conditions. Sorry for taking it up when you assigned yourself.
No problem. :)

Resolving patch:
diff --git a/core/domain/exp_domain.py b/core/domain/exp_domain.py
--- a/core/domain/exp_domain.py
+++ b/core/domain/exp_domain.py
@@ -1162,6 +1162,31 @@ def validate(self, exp_param_specs_dict, allow_null_interaction):
         elif self.interaction.id is not None:
             self.interaction.validate(exp_param_specs_dict)
 
+    def can_undergo_classification(self):
+        """Checks whether the answers for this state satisfy the preconditions
+        for a ML model to be trained.
+
+        Returns:
+            bool: True, if the conditions are satisfied.
+        """
+        training_examples_count = 0
+        labels_count = 0
+        training_examples_count += len(
+            self.interaction.confirmed_unclassified_answers)
+        for answer_group in self.interaction.answer_groups:
+            classifier_rule_spec_index = (
+                answer_group.get_classifier_rule_index())
+            if classifier_rule_spec_index is not None:
+                classifier_rule_spec = answer_group.rule_specs[
+                    classifier_rule_spec_index]
+                training_examples_count += len(
+                    classifier_rule_spec.inputs['training_data'])
+                labels_count += 1
+        if ((training_examples_count >= feconf.MIN_TOTAL_TRAINING_EXAMPLES) and
+                (labels_count >= feconf.MIN_ASSIGNED_LABELS)):
+            return True
+        return False
+
     def update_content(self, content_list):
         # TODO(sll): Must sanitize all content in RTE component attrs.
         self.content = [Content.from_dict(content_list[0])]
diff --git a/feconf.py b/feconf.py
--- a/feconf.py
+++ b/feconf.py
@@ -75,6 +75,12 @@
     'TextInput': 'LDAStringClassifier'
 }
 
+# The minimum number of training samples required for training a classifier.
+MIN_TOTAL_TRAINING_EXAMPLES = 50
+
+# The minimum number of assigned labels required for training a classifier.
+MIN_ASSIGNED_LABELS = 2
+
 # Default label for classification algorithms.
 DEFAULT_CLASSIFIER_LABEL = '_default'
 


Test patch:
diff --git a/core/domain/exp_domain_test.py b/core/domain/exp_domain_test.py
--- a/core/domain/exp_domain_test.py
+++ b/core/domain/exp_domain_test.py
@@ -2036,6 +2036,29 @@ def _get_default_state_dict(content_str, dest_name):
 class StateOperationsUnitTests(test_utils.GenericTestBase):
     """Test methods operating on states."""
 
+    def test_can_undergo_classification(self):
+        """Test the can_undergo_classification() function."""
+        exploration_id = 'eid'
+        test_exp_filepath = os.path.join(
+            feconf.TESTS_DATA_DIR, 'string_classifier_test.yaml')
+        yaml_content = utils.get_file_contents(test_exp_filepath)
+        assets_list = []
+        exp_services.save_new_exploration_from_yaml_and_assets(
+            feconf.SYSTEM_COMMITTER_ID, yaml_content, exploration_id,
+            assets_list)
+
+        exploration = exp_services.get_exploration_by_id(exploration_id)
+        state_with_training_data = exploration.states['Home']
+        state_without_training_data = exploration.states['End']
+
+        # A state with 786 training examples.
+        self.assertTrue(
+            state_with_training_data.can_undergo_classification())
+
+        # A state with no training examples.
+        self.assertFalse(
+            state_without_training_data.can_undergo_classification())
+
     def test_delete_state(self):
         """Test deletion of states."""
         exploration = exp_domain.Exploration.create_default_exploration('eid')