Repository: oppia/oppia

Issue:
Create storage and domain layer classes for TrainClassifierJobModel
This implements the first part of Milestone 1 for the migration of ML to the VM.
We need to create a database schema for the TrainClassifierJobModel. We need to have the following attributes:
**job_request_id**: String. Job request ID. To be uniquely generated for each Job request.
**exp_id**: String. The exploration ID.
**state_name**: String. The state name.
**training_data**: An array of label objects. [labels]. To be fetched and updated at the time of retrieval.
    label: Object consists of two properties:
        label_name: The name of label
            answers: An array of answers associated with the label. [answers]
                answer: A string.
**algorithm_id**: String. The id of the classifying algorithm to be used.
**exp_version**: String. The version of exploration.
**status**: the status of classification job. (PENDING, COMPLETED. By default it’s PENDING)

Write tests for ensuring the proper storage of data.
CC @anmolshkl  @giritheja , PTAL. Add whatever else is required.
Create storage and domain layer classes for TrainClassifierJobModel
This implements the first part of Milestone 1 for the migration of ML to the VM.
We need to create a database schema for the TrainClassifierJobModel. We need to have the following attributes:
**job_request_id**: String. Job request ID. To be uniquely generated for each Job request.
**exp_id**: String. The exploration ID.
**state_name**: String. The state name.
**training_data**: An array of label objects. [labels]. To be fetched and updated at the time of retrieval.
    label: Object consists of two properties:
        label_name: The name of label
            answers: An array of answers associated with the label. [answers]
                answer: A string.
**algorithm_id**: String. The id of the classifying algorithm to be used.
**exp_version**: String. The version of exploration.
**status**: the status of classification job. (PENDING, COMPLETED. By default it’s PENDING)

Write tests for ensuring the proper storage of data.
CC @anmolshkl  @giritheja , PTAL. Add whatever else is required.


Discussion:
@pranavsid98 thanks for creating the issue! @seanlip I think training data can be stored as a list of dicts, for example - [{answer_group_index: 2, answers: [a2, a3, a4]}, {answer_group_index: 4, answers: [a1, a6, a5]} ..... ]. 

@pranavsid98 also, I think that we should just use answer_group_index instead of labels (check the doc for the detailed comment).
List of dicts seems fine  to me. Thanks!
@pranavsid98 thanks for creating the issue! @seanlip I think training data can be stored as a list of dicts, for example - [{answer_group_index: 2, answers: [a2, a3, a4]}, {answer_group_index: 4, answers: [a1, a6, a5]} ..... ]. 

@pranavsid98 also, I think that we should just use answer_group_index instead of labels (check the doc for the detailed comment).
List of dicts seems fine  to me. Thanks!

Resolving patch:
diff --git a/core/domain/classifier_domain.py b/core/domain/classifier_domain.py
--- a/core/domain/classifier_domain.py
+++ b/core/domain/classifier_domain.py
@@ -140,7 +140,7 @@ def validate(self):
 
         if not isinstance(self.exp_version_when_created, int):
             raise utils.ValidationError(
-                'Expected exp_version_when_created to be a int, received %s' %
+                'Expected exp_version_when_created to be an int, received %s' %
                 self.exp_version_when_created)
 
         if not isinstance(self.state_name, basestring):
@@ -154,10 +154,9 @@ def validate(self):
                 self.algorithm_id)
         utils.require_valid_name(
             self.algorithm_id, 'the algorithm id')
-        algorithm_ids = []
-        for classifier_details in (
-                feconf.INTERACTION_CLASSIFIER_MAPPING.values()):
-            algorithm_ids.append(classifier_details['algorithm_id'])
+        algorithm_ids = [
+            classifier_details['algorithm_id'] for classifier_details in
+            feconf.INTERACTION_CLASSIFIER_MAPPING.values()]
         if self.algorithm_id not in algorithm_ids:
             raise utils.ValidationError(
                 'Invalid algorithm id: %s' % self.algorithm_id)
@@ -170,3 +169,189 @@ def validate(self):
             classifier_registry.Registry.get_classifier_by_algorithm_id(
                 self.algorithm_id))
         classifier_class.validate(self.classifier_data)
+
+
+class ClassifierTrainingJob(object):
+    """Domain object for a classifier training job.
+
+    A classifier training job is an abstraction of a request made by Oppia
+    for training a classifier model using certain dataset and a particular ML
+    algorithm denoted by the algorithm id. The classifier training jobs are
+    then picked up by the Virtual Machine (VM) through APIs exposed by Oppia.
+    The training_data is populated lazily when the job is fetched from the
+    database upon the request from the VM.
+
+    Attributes:
+        job_id: str. The unique id of the classifier training job.
+        algorithm_id: str. The id of the algorithm that will be used for
+            generating the classifier.
+        exp_id: str. The id of the exploration that contains the state
+            for which the classifier will be generated.
+        exp_version_when_created: str. The version of the exploration when
+            the training job was generated.
+        state_name: str. The name of the state for which the classifier will be
+            generated.
+        status: str. The status of the training job request. This can be either
+            NEW (default value), FAILED, PENDING or COMPLETE.
+        training_data: list(dict). The training data that is used for training
+            the classifier. This field is populated lazily when the job request
+            is picked up by the VM. The list contains dicts where each dict
+            represents a single training data group, for example:
+            training_data = [
+                {
+                    'answer_group_index': 1,
+                    'answers': ['a1', 'a2']
+                },
+                {
+                    'answer_group_index': 2,
+                    'answers': ['a2', 'a3']
+                }
+            ]
+
+    """
+
+    def __init__(self, job_id, algorithm_id, exp_id, exp_version_when_created,
+                 state_name, status, training_data):
+        """Constructs a ClassifierTrainingJob domain object.
+
+        Args:
+        job_id: str. The unique id of the classifier training job.
+        algorithm_id: str. The id of the algorithm that will be used for
+            generating the classifier.
+        exp_id: str. The id of the exploration id that contains the state
+            for which classifier will be generated.
+        exp_version_when_created: str. The version of the exploration when
+            the training job was generated.
+        state_name: str. The name of the state for which the classifier will be
+            generated.
+        status: str. The status of the training job request. This can be either
+            NEW (default), PENDING (when a job has been picked up) or COMPLETE.
+        training_data: list. The training data that is used for training the
+            classifier. This is populated lazily when the job request is picked
+            up by the VM. The list contains dicts where each dict
+            represents a single training data group, for example:
+            training_data = [
+                {
+                    'answer_group_index': 1,
+                    'answers': ['a1', 'a2']
+                },
+                {
+                    'answer_group_index': 2,
+                    'answers': ['a2', 'a3']
+                }
+            ]
+        """
+        self._job_id = job_id
+        self._algorithm_id = algorithm_id
+        self._exp_id = exp_id
+        self._exp_version_when_created = exp_version_when_created
+        self._state_name = state_name
+        self._status = status
+        self._training_data = copy.deepcopy(training_data)
+
+    @property
+    def job_id(self):
+        return self._job_id
+
+    @property
+    def algorithm_id(self):
+        return self._algorithm_id
+
+    @property
+    def exp_id(self):
+        return self._exp_id
+
+    @property
+    def exp_version_when_created(self):
+        return self._exp_version_when_created
+
+    @property
+    def state_name(self):
+        return self._state_name
+
+    @property
+    def status(self):
+        return self._status
+
+    @property
+    def training_data(self):
+        return self._training_data
+
+    def to_dict(self):
+        """Constructs a dict representation of training job domain object.
+
+        Returns:
+            A dict representation of training job domain object.
+        """
+
+        return {
+            'job_id': self._job_id,
+            'algorithm_id': self._algorithm_id,
+            'exp_id': self._exp_id,
+            'exp_version_when_created': self._exp_version_when_created,
+            'state_name': self._state_name,
+            'status': self._status,
+            'training_data': self._training_data
+        }
+
+    def validate(self):
+        """Validates the training job before it is saved to storage."""
+
+        algorithm_ids = []
+        if not isinstance(self.job_id, basestring):
+            raise utils.ValidationError(
+                'Expected id to be a string, received %s' % self.job_id)
+
+        if not isinstance(self.exp_id, basestring):
+            raise utils.ValidationError(
+                'Expected exp_id to be a string, received %s' % self.exp_id)
+
+        if not isinstance(self.exp_version_when_created, int):
+            raise utils.ValidationError(
+                'Expected exp_version_when_created to be an int, received %s' %
+                self.exp_version_when_created)
+
+        if not isinstance(self.state_name, basestring):
+            raise utils.ValidationError(
+                'Expected state to be a string, received %s' % self.state_name)
+        utils.require_valid_name(self.state_name, 'the state name')
+
+        if self.status not in feconf.ALLOWED_TRAINING_JOB_STATUSES:
+            raise utils.ValidationError(
+                'Expected status to be in %s, received %s' %
+                feconf.ALLOWED_TRAINING_JOB_STATUSES,
+                self.exp_version_when_created)
+
+        if not isinstance(self.algorithm_id, basestring):
+            raise utils.ValidationError(
+                'Expected algorithm_id to be a string, received %s' %
+                self.algorithm_id)
+
+        algorithm_ids = [
+            classifier_details['algorithm_id'] for classifier_details in
+            feconf.INTERACTION_CLASSIFIER_MAPPING.values()]
+        if self.algorithm_id not in algorithm_ids:
+            raise utils.ValidationError(
+                'Invalid algorithm id: %s' % self.algorithm_id)
+
+        if not isinstance(self.training_data, list):
+            raise utils.ValidationError(
+                'Expected training_data to be a list, received %s' %(
+                    self.training_data))
+
+        for grouped_answers in self.training_data:
+            if 'answer_group_index' not in grouped_answers:
+                raise utils.ValidationError(
+                    'Expected answer_group_index to be a key in training_data',
+                    'list item')
+            if 'answers' not in grouped_answers:
+                raise utils.ValidationError(
+                    'Expected answers to be a key in training_data list item')
+            if not isinstance(grouped_answers['answer_group_index'], int):
+                raise utils.ValidationError(
+                    'Expected answer_group_index to be an int, received %s' %
+                    grouped_answers['answer_group_index'])
+            if not isinstance(grouped_answers['answers'], list):
+                raise utils.ValidationError(
+                    'Expected answers to be a list, received %s' %
+                    grouped_answers['answers'])
diff --git a/core/storage/classifier/gae_models.py b/core/storage/classifier/gae_models.py
--- a/core/storage/classifier/gae_models.py
+++ b/core/storage/classifier/gae_models.py
@@ -17,9 +17,11 @@
 from core.platform import models
 
 import feconf
+import utils
 
 from google.appengine.ext import ndb
 
+
 (base_models,) = models.Registry.import_models([models.NAMES.base_model])
 
 # Available choices of algorithms for classification.
@@ -83,3 +85,92 @@ def create(
 
         classifier_data_model_instance.put()
         return instance_id
+
+
+class ClassifierTrainingJobModel(base_models.BaseModel):
+    """Model for storing classifier training jobs.
+
+    The id of instances of this class has the form
+    {{exp_id}}.{{random_hash_of_16_chars}}
+    """
+
+    # The ID of the algorithm used to create the model.
+    algorithm_id = ndb.StringProperty(required=True, choices=ALGORITHM_CHOICES,
+                                      indexed=True)
+    # The exploration_id of the exploration to whose state the model belongs.
+    exp_id = ndb.StringProperty(required=True, indexed=True)
+    # The exploration version at the time this training job was created.
+    exp_version_when_created = ndb.IntegerProperty(required=True, indexed=True)
+    # The name of the state to which the model belongs.
+    state_name = ndb.StringProperty(required=True, indexed=True)
+    # The status of the training job. It can be either NEW, COMPLETE or PENDING
+    status = ndb.StringProperty(required=True,
+                                choices=feconf.ALLOWED_TRAINING_JOB_STATUSES,
+                                default=feconf.TRAINING_JOB_STATUS_PENDING,
+                                indexed=True)
+    # The training data which is to be populated when retrieving the job.
+    # The list contains dicts where each dict represents a single training
+    # data group.
+    training_data = ndb.JsonProperty(default=None)
+
+    @classmethod
+    def _generate_id(cls, exp_id):
+        """Generates a unique id for the training job of the form
+        {{exp_id}}.{{random_hash_of_16_chars}}
+
+        Args:
+            exp_id: str. ID of the exploration.
+
+        Returns:
+            ID of the new ClassifierTrainingJobModel instance.
+
+        Raises:
+            Exception: The id generator for ClassifierTrainingJobModel is
+            producing too many collisions.
+        """
+
+        for _ in range(base_models.MAX_RETRIES):
+            new_id = '%s.%s' % (
+                exp_id,
+                utils.convert_to_hash(
+                    str(utils.get_random_int(base_models.RAND_RANGE)),
+                    base_models.ID_LENGTH))
+            if not cls.get_by_id(new_id):
+                return new_id
+
+        raise Exception(
+            'The id generator for ClassifierTrainingJobModel is producing '
+            'too many collisions.')
+
+    @classmethod
+    def create(
+            cls, algorithm_id, exp_id, exp_version_when_created, training_data,
+            state_name, status=feconf.TRAINING_JOB_STATUS_NEW):
+        """Creates a new ClassifierTrainingJobModel entry.
+
+        Args:
+            algorithm_id: str. ID of the algorithm used to generate the model.
+            exp_id: str. ID of the exploration.
+            exp_version_when_created: int. The exploration version at the time
+                this training job was created.
+            state_name: str. The name of the state to which the classifier
+                belongs.
+            status: str. The status of the training job (NEW by default).
+            training_data: dict. The data used in training phase.
+
+        Returns:
+            ID of the new ClassifierModel entry.
+
+        Raises:
+            Exception: A model with the same ID already exists.
+        """
+
+        instance_id = cls._generate_id(exp_id)
+        training_job_instance = cls(
+            id=instance_id, algorithm_id=algorithm_id, exp_id=exp_id,
+            exp_version_when_created=exp_version_when_created,
+            state_name=state_name, status=status, training_data=training_data
+            )
+
+        training_job_instance.put()
+        return instance_id
diff --git a/feconf.py b/feconf.py
--- a/feconf.py
+++ b/feconf.py
@@ -78,6 +78,18 @@
     }
 }
 
+TRAINING_JOB_STATUS_COMPLETE = 'COMPLETE'
+TRAINING_JOB_STATUS_FAILED = 'FAILED'
+TRAINING_JOB_STATUS_NEW = 'NEW'
+TRAINING_JOB_STATUS_PENDING = 'PENDING'
+
+ALLOWED_TRAINING_JOB_STATUSES = [
+    TRAINING_JOB_STATUS_COMPLETE,
+    TRAINING_JOB_STATUS_FAILED,
+    TRAINING_JOB_STATUS_NEW,
+    TRAINING_JOB_STATUS_PENDING
+]
+
 # The minimum number of training samples required for training a classifier.
 MIN_TOTAL_TRAINING_EXAMPLES = 50
 


Test patch:
diff --git a/core/domain/classifier_domain_test.py b/core/domain/classifier_domain_test.py
--- a/core/domain/classifier_domain_test.py
+++ b/core/domain/classifier_domain_test.py
@@ -113,7 +113,7 @@ def test_validation(self):
             classifier_dict['classifier_data'],
             classifier_dict['data_schema_version'])
         with self.assertRaisesRegexp(utils.ValidationError, (
-            'Expected exp_version_when_created to be a int')):
+            'Expected exp_version_when_created to be an int')):
             classifier.validate()
 
         # Verify valdation error is raised when invalid state_name is provided.
@@ -161,3 +161,111 @@ def test_validation(self):
         with self.assertRaisesRegexp(utils.ValidationError, (
             'Expected classifier_data to be a dict')):
             classifier.validate()
+
+
+class ClassifierTrainingJobDomainTests(test_utils.GenericTestBase):
+    """Test the ClassifierTrainingJob domain."""
+
+    def _get_training_job_from_dict(self, training_job_dict):
+        training_job = classifier_domain.ClassifierTrainingJob(
+            training_job_dict['job_id'],
+            training_job_dict['algorithm_id'],
+            training_job_dict['exp_id'],
+            training_job_dict['exp_version_when_created'],
+            training_job_dict['state_name'],
+            training_job_dict['status'],
+            training_job_dict['training_data'])
+
+        return training_job
+
+    def test_to_dict(self):
+        expected_training_job_dict = {
+            'job_id': 'exp_id1.SOME_RANDOM_STRING',
+            'algorithm_id': 'LDAStringClassifier',
+            'exp_id': 'exp_id1',
+            'exp_version_when_created': 1,
+            'state_name': 'a state name',
+            'status': 'NEW',
+            'training_data': [
+                {
+                    'answer_group_index': 1,
+                    'answers': ['a1', 'a2']
+                },
+                {
+                    'answer_group_index': 2,
+                    'answers': ['a2', 'a3']
+                }
+            ]
+        }
+        observed_training_job = self._get_training_job_from_dict(
+            expected_training_job_dict)
+        self.assertDictEqual(expected_training_job_dict,
+                             observed_training_job.to_dict())
+
+    def test_validation(self):
+        """Tests to verify validate method of ClassifierTrainingJob domain."""
+
+        # Verify no errors are raised for correct data.
+        training_data = [
+            {
+                'answer_group_index': 1,
+                'answers': ['a1', 'a2']
+            },
+            {
+                'answer_group_index': 2,
+                'answers': ['a2', 'a3']
+            }
+        ]
+        training_job_dict = {
+            'job_id': 'exp_id1.SOME_RANDOM_STRING',
+            'exp_id': 'exp_id1',
+            'exp_version_when_created': 1,
+            'state_name': 'some state',
+            'algorithm_id': 'LDAStringClassifier',
+            'training_data': training_data,
+            'status': 'NEW'
+        }
+        training_job = self._get_training_job_from_dict(training_job_dict)
+        training_job.validate()
+
+        # Verify validation error is raised when int is provided for instance id
+        # instead of string.
+        training_job_dict['job_id'] = 1
+        training_job = self._get_training_job_from_dict(training_job_dict)
+        with self.assertRaisesRegexp(utils.ValidationError, (
+            'Expected id to be a string')):
+            training_job.validate()
+
+        # Verify validation error is raised when string is provided for
+        # exp_version_when_created instead of int.
+        training_job_dict['job_id'] = 'exp_id1.SOME_RANDOM_STRING'
+        training_job_dict['exp_version_when_created'] = 'abc'
+        training_job = self._get_training_job_from_dict(training_job_dict)
+        with self.assertRaisesRegexp(utils.ValidationError, (
+            'Expected exp_version_when_created to be an int')):
+            training_job.validate()
+
+        # Verify validation error is raised when invalid state_name is provided.
+        training_job_dict['exp_version_when_created'] = 1
+        training_job_dict['state_name'] = 'A string #'
+        training_job = self._get_training_job_from_dict(training_job_dict)
+        with self.assertRaisesRegexp(utils.ValidationError, (
+            'Invalid character # in the state name')):
+            training_job.validate()
+
+        # Verify validation error is raised when invalid algorithm_id is
+        # provided.
+        training_job_dict['state_name'] = 'a state name'
+        training_job_dict['algorithm_id'] = 'abc'
+        training_job = self._get_training_job_from_dict(training_job_dict)
+        with self.assertRaisesRegexp(utils.ValidationError, (
+            'Invalid algorithm id')):
+            training_job.validate()
+
+        # Verify validation error is raised when dict is provided for list.
+        training_job_dict['algorithm_id'] = 'LDAStringClassifier'
+        training_job_dict['training_data'] = {}
+        training_job = self._get_training_job_from_dict(training_job_dict)
+        with self.assertRaisesRegexp(utils.ValidationError, (
+            'Expected training_data to be a list')):
+            training_job.validate()
diff --git a/core/storage/classifier/gae_models_test.py b/core/storage/classifier/gae_models_test.py
--- a/core/storage/classifier/gae_models_test.py
+++ b/core/storage/classifier/gae_models_test.py
@@ -16,6 +16,7 @@
 
 from core.platform import models
 from core.tests import test_utils
+import feconf
 
 (classifier_models,) = models.Registry.import_models(
     [models.NAMES.classifier])
@@ -52,3 +53,40 @@ def test_create_new_classifier_runs_successfully(self):
         self.assertEqual(classifier.algorithm_id, 'LDAStringClassifier')
         self.assertEqual(classifier.classifier_data, {'alpha': 1.0})
         self.assertEqual(classifier.data_schema_version, 1)
+
+
+class ClassifierTrainingJobModelUnitTests(test_utils.GenericTestBase):
+    """Test the ClassifierTrainingJobModel class."""
+
+    def setUp(self):
+        super(ClassifierTrainingJobModelUnitTests, self).setUp()
+        classifier_models.ClassifierTrainingJobModel.create(
+            'LDAStringClassifier', 'exp_id1', 1,
+            [{'answer_group_index': 1, 'answers': ['a1', 'a2']}],
+            'state_name1')
+        classifier_models.ClassifierTrainingJobModel.create(
+            'LDAStringClassifier', 'exp_id2', 2,
+            [{'answer_group_index': 1, 'answers': ['a1', 'a2']}],
+            'state_name1')
+        classifier_models.ClassifierTrainingJobModel.create(
+            'LDAStringClassifier', 'exp_id3', 3,
+            [{'answer_group_index': 1, 'answers': ['a1', 'a2']}],
+            'state_name1')
+
+    def test_create_and_get_new_training_job_runs_successfully(self):
+        job_id = classifier_models.ClassifierTrainingJobModel.create(
+            'LDAStringClassifier', 'exp_id1', 1,
+            [{'answer_group_index': 1, 'answers': ['a1', 'a2']}],
+            'state_name2')
+
+        training_job = (
+            classifier_models.ClassifierTrainingJobModel.get(job_id))
+
+        self.assertEqual(training_job.algorithm_id, 'LDAStringClassifier')
+        self.assertEqual(training_job.exp_id, 'exp_id1')
+        self.assertEqual(training_job.exp_version_when_created, 1)
+        self.assertEqual(training_job.state_name, 'state_name2')
+        self.assertEqual(training_job.status,
+                         feconf.TRAINING_JOB_STATUS_NEW)
+        self.assertEqual(training_job.training_data,
+                         [{'answer_group_index': 1, 'answers': ['a1', 'a2']}])