Repository: oppia/oppia

Issue:
Modify the from_backend_dict method
Right now, the following classes have two separate methods called `from_backend_dict()` and `from_dict()`:
- [ ] `core.domain.stats_domain.Playthrough`
- [ ] `core.domain.stats_domain.ExplorationIssue`

`from_dict()` takes a dictionary and creates an instance based on its values.  
`from_backend_dict()` takes a dictionary and validates it before creating an instance based on its values and an automatically generated ID.

Put simply, `from_backend_dict()` does not require an actual ID, whereas `from_dict()` does.

We should consolidate this into one `from_dict()` method with a flag `id_required=True`
Modify the from_backend_dict method
Right now, the following classes have two separate methods called `from_backend_dict()` and `from_dict()`:
- [ ] `core.domain.stats_domain.Playthrough`
- [ ] `core.domain.stats_domain.ExplorationIssue`

`from_dict()` takes a dictionary and creates an instance based on its values.  
`from_backend_dict()` takes a dictionary and validates it before creating an instance based on its values and an automatically generated ID.

Put simply, `from_backend_dict()` does not require an actual ID, whereas `from_dict()` does.

We should consolidate this into one `from_dict()` method with a flag `id_required=True`


Discussion:
I don't like the use of a dummy ID. Can you add links to cases when it is actually needed?

In general we should never be using "fake" data in the main code.
@seanlip 

So this is the case:

When there is incoming playthrough data from the front end, it is in a dict form, and since the playthrough doesnt exist in the datastore yet, it does not have an ID. In our controller, before the playthrough is stored, a lot of operations are performed (finding the issue for the playthrough to be inserted, if it already exists) and the like. Going by our practice of not using dicts directly while performing operations, I converted it to a temporary instance that doesnt have any significance.

Is that bad?
It's good to use domain objects, but not great to use a temporary/fake ID. Some alternatives:

- Does your domain object actually need an ID field? If not, remove it from there and have it only be a storage model property.
- Use an ID value of None while the playthrough still does not have an ID assigned. In the domain object, document the significance of this value.
@brianrodri are you still working on this?
@brianrodri ping!
Hi @brianrodri, are you still working on this? Thanks! 
Heya, not for a while since I'm currently on vacation. I'll take a look
again in about a week, so feel free to reassign if that's too late.
No problem, sounds good! Thanks for the update.
Hey @brianrodri any updates on this?
De-assigning for now. Please re-assign if you'd like to continue work.
I'll take this issue
I don't like the use of a dummy ID. Can you add links to cases when it is actually needed?

In general we should never be using "fake" data in the main code.
@seanlip 

So this is the case:

When there is incoming playthrough data from the front end, it is in a dict form, and since the playthrough doesnt exist in the datastore yet, it does not have an ID. In our controller, before the playthrough is stored, a lot of operations are performed (finding the issue for the playthrough to be inserted, if it already exists) and the like. Going by our practice of not using dicts directly while performing operations, I converted it to a temporary instance that doesnt have any significance.

Is that bad?
It's good to use domain objects, but not great to use a temporary/fake ID. Some alternatives:

- Does your domain object actually need an ID field? If not, remove it from there and have it only be a storage model property.
- Use an ID value of None while the playthrough still does not have an ID assigned. In the domain object, document the significance of this value.
@brianrodri are you still working on this?
@brianrodri ping!
Hi @brianrodri, are you still working on this? Thanks! 
Heya, not for a while since I'm currently on vacation. I'll take a look
again in about a week, so feel free to reassign if that's too late.
No problem, sounds good! Thanks for the update.
Hey @brianrodri any updates on this?
De-assigning for now. Please re-assign if you'd like to continue work.
I'll take this issue

Resolving patch:
diff --git a/core/controllers/editor.py b/core/controllers/editor.py
--- a/core/controllers/editor.py
+++ b/core/controllers/editor.py
@@ -572,7 +572,7 @@ def post(self, exp_id):
         """Handles POST requests."""
         exp_issue_dict = self.payload.get('exp_issue_dict')
         try:
-            unused_exp_issue = stats_domain.ExplorationIssue.from_backend_dict(
+            unused_exp_issue = stats_domain.ExplorationIssue.from_dict(
                 exp_issue_dict)
         except utils.ValidationError as e:
             raise self.PageNotFoundException(e)
diff --git a/core/controllers/reader.py b/core/controllers/reader.py
--- a/core/controllers/reader.py
+++ b/core/controllers/reader.py
@@ -424,7 +424,7 @@ def post(self, exploration_id):
         """
         playthrough_data = self.payload.get('playthrough_data')
         try:
-            unused_playthrough = stats_domain.Playthrough.from_backend_dict(
+            unused_playthrough = stats_domain.Playthrough.from_dict(
                 playthrough_data)
         except utils.ValidationError as e:
             raise self.InvalidInputException(e)
diff --git a/core/domain/stats_domain.py b/core/domain/stats_domain.py
--- a/core/domain/stats_domain.py
+++ b/core/domain/stats_domain.py
@@ -544,41 +544,20 @@ def to_dict(self):
         }
 
     @classmethod
-    def from_dict(cls, playthrough_dict):
-        """Returns a Playthrough object from a dict.
-
-        Args:
-            playthrough_dict: dict. A dict mapping of all fields of Playthrough
-                object.
-
-        Returns:
-            Playthrough. The corresponding Playthrough domain object.
-        """
-        actions = [
-            LearnerAction.from_dict(action_dict)
-            for action_dict in playthrough_dict['actions']]
-        return cls(
-            playthrough_dict['exp_id'],
-            playthrough_dict['exp_version'],
-            playthrough_dict['issue_type'],
-            playthrough_dict['issue_customization_args'],
-            actions)
-
-    @classmethod
-    def from_backend_dict(cls, playthrough_data):
+    def from_dict(cls, playthrough_data):
         """Checks whether the playthrough dict has the correct keys and then
         returns a domain object instance.
 
         Args:
-            playthrough_data: dict. Dict representing a playthrough.
+            playthrough_data: dict. A dict mapping of all fields of Playthrough
+                object.
 
         Returns:
-            Playthrough. A playthrough domain object.
+            Playthrough. The corresponding Playthrough domain object.
         """
         playthrough_properties = [
             'exp_id', 'exp_version', 'issue_type',
             'issue_customization_args', 'actions']
-
         for playthrough_property in playthrough_properties:
             if playthrough_property not in playthrough_data:
                 raise utils.ValidationError(
@@ -686,50 +665,35 @@ def to_dict(self):
         }
 
     @classmethod
-    def from_dict(cls, issue_dict):
-        """Returns an ExplorationIssue object from a dict.
-
-        Args:
-            issue_dict: dict. A dict mapping of all fields of ExplorationIssue
-                object.
-
-        Returns:
-            ExplorationIssue. The corresponding ExplorationIssue domain object.
-        """
-        return cls(
-            issue_dict['issue_type'],
-            issue_dict['issue_customization_args'],
-            issue_dict['playthrough_ids'],
-            issue_dict['schema_version'],
-            issue_dict['is_valid'])
-
-    @classmethod
-    def from_backend_dict(cls, exp_issue_dict):
+    def from_dict(cls, exp_issue_dict):
         """Checks whether the exploration issue dict has the correct keys and
         then returns a domain object instance.
 
         Args:
-            exp_issue_dict: dict. Dict representing an exploration issue.
+            exp_issue_dict: dict. A dict mapping of all fields of
+                ExplorationIssue object.
 
         Returns:
-            ExplorationIssue. The exploration issue domain object.
+            ExplorationIssue. The corresponding ExplorationIssue domain object.
         """
         exp_issue_properties = [
             'issue_type', 'schema_version', 'issue_customization_args',
             'playthrough_ids', 'is_valid']
-
         for exp_issue_property in exp_issue_properties:
             if exp_issue_property not in exp_issue_dict:
                 raise utils.ValidationError(
-                    '%s not in exploration issue dict.' % (exp_issue_property))
+                    '%s not in exploration issue dict.' % (
+                        exp_issue_property))
 
-        dummy_exp_issue = cls(
+        exp_issue = cls(
             exp_issue_dict['issue_type'],
-            exp_issue_dict['issue_customization_args'], [],
-            exp_issue_dict['schema_version'], exp_issue_dict['is_valid'])
+            exp_issue_dict['issue_customization_args'],
+            exp_issue_dict['playthrough_ids'],
+            exp_issue_dict['schema_version'],
+            exp_issue_dict['is_valid'])
 
-        dummy_exp_issue.validate()
-        return dummy_exp_issue
+        exp_issue.validate()
+        return exp_issue
 
     @classmethod
     def update_exp_issue_from_model(cls, issue_dict):


Test patch:
diff --git a/core/domain/stats_domain_test.py b/core/domain/stats_domain_test.py
--- a/core/domain/stats_domain_test.py
+++ b/core/domain/stats_domain_test.py
@@ -34,6 +34,7 @@
 
 class ExplorationStatsTests(test_utils.GenericTestBase):
     """Tests the ExplorationStats domain object."""
+
     def setUp(self):
         super(ExplorationStatsTests, self).setUp()
 
@@ -203,6 +204,7 @@ def test_to_frontend_dict(self):
 
 class StateStatsTests(test_utils.GenericTestBase):
     """Tests the StateStats domain object."""
+
     def setUp(self):
         super(StateStatsTests, self).setUp()
 
@@ -338,6 +340,7 @@ def test_to_frontend_dict(self):
 
 class ExplorationIssuesTests(test_utils.GenericTestBase):
     """Tests the ExplorationIssues domain object."""
+
     def setUp(self):
         super(ExplorationIssuesTests, self).setUp()
 
@@ -451,6 +454,7 @@ def test_validate_unresolved_issues(self):
 
 class PlaythroughTests(test_utils.GenericTestBase):
     """Tests the Playthrough domain object."""
+
     def setUp(self):
         super(PlaythroughTests, self).setUp()
 
@@ -524,6 +528,7 @@ def test_to_dict(self):
                 }])
 
     def test_from_dict(self):
+        """Test the from_dict() method."""
         playthrough_dict = {
             'exp_id': 'exp_id1',
             'exp_version': 1,
@@ -573,8 +578,8 @@ def test_from_dict(self):
                 'schema_version': 1
             })
 
-    def test_from_backend_dict(self):
-        """Test the from_backend_dict() method."""
+    def test_from_dict_raises_exception_when_miss_exp_id(self):
+        """Test the from_dict() method."""
         # Test that a playthrough dict without 'exp_id' key raises exception.
         playthrough_dict = {
             'exp_version': 1,
@@ -585,7 +590,7 @@ def test_from_backend_dict(self):
         with self.assertRaisesRegexp(
             utils.ValidationError,
             'exp_id not in playthrough data dict.'):
-            stats_domain.Playthrough.from_backend_dict(playthrough_dict)
+            stats_domain.Playthrough.from_dict(playthrough_dict)
 
     def test_validate_with_string_exp_version(self):
         self.playthrough.exp_version = '1'
@@ -645,21 +650,13 @@ def test_validate_non_dict_issue_customization_args(self):
 
 class ExplorationIssueTests(test_utils.GenericTestBase):
     """Tests the ExplorationIssue domain object."""
+
     def setUp(self):
         super(ExplorationIssueTests, self).setUp()
 
         self.exp_issue = stats_domain.ExplorationIssue(
             'EarlyQuit', {}, [], 1, True)
 
-    def _dummy_convert_issue_v1_dict_to_v2_dict(self, issue_dict):
-        """A test implementation of schema conversion function."""
-        issue_dict['schema_version'] = 2
-        if issue_dict['issue_type'] == 'EarlyQuit':
-            issue_dict['issue_type'] = 'EarlyQuit1'
-            issue_dict['issue_customization_args']['new_key'] = 5
-
-        return issue_dict
-
     def test_to_dict(self):
         exp_issue = stats_domain.ExplorationIssue('EarlyQuit', {}, [], 1, True)
         exp_issue_dict = exp_issue.to_dict()
@@ -680,8 +677,34 @@ def test_to_dict(self):
                 'is_valid': True
             })
 
-    def test_from_backend_dict(self):
-        """Test the from_backend_dict() method."""
+    def test_from_dict(self):
+        expected_customization_args = {
+            'time_spent_in_exp_in_msecs': {
+                'value': 0
+            },
+            'state_name': {
+                'value': ''
+            }
+        }
+        exp_issue = stats_domain.ExplorationIssue.from_dict({
+            'issue_type': 'EarlyQuit',
+            'issue_customization_args': expected_customization_args,
+            'playthrough_ids': [],
+            'schema_version': 1,
+            'is_valid': True
+        })
+        exp_issue_dict = exp_issue.to_dict()
+        self.assertEqual(
+            exp_issue_dict, {
+                'issue_type': 'EarlyQuit',
+                'issue_customization_args': expected_customization_args,
+                'playthrough_ids': [],
+                'schema_version': 1,
+                'is_valid': True
+            })
+
+    def test_from_dict_raises_exception(self):
+        """Test the from_dict() method."""
         # Test that an exploration issue dict without 'issue_type' key raises
         # exception.
         exp_issue_dict = {
@@ -693,60 +716,7 @@ def test_from_backend_dict(self):
         with self.assertRaisesRegexp(
             utils.ValidationError,
             'issue_type not in exploration issue dict.'):
-            stats_domain.ExplorationIssue.from_backend_dict(exp_issue_dict)
-
-    def test_update_exp_issue_from_model(self):
-        """Test the migration of exploration issue domain objects."""
-        exp_issue = stats_domain.ExplorationIssue('EarlyQuit', {}, [], 1, True)
-        exp_issue_dict = exp_issue.to_dict()
-        stats_models.ExplorationIssuesModel.create(
-            'exp_id', 1, [exp_issue_dict])
-
-        exp_issues_model = stats_models.ExplorationIssuesModel.get_model(
-            'exp_id', 1)
-
-        current_issue_schema_version_swap = self.swap(
-            stats_models, 'CURRENT_ISSUE_SCHEMA_VERSION', 2)
-        convert_issue_dict_swap = self.swap(
-            stats_domain.ExplorationIssue,
-            '_convert_issue_v1_dict_to_v2_dict',
-            self._dummy_convert_issue_v1_dict_to_v2_dict)
-
-        with convert_issue_dict_swap, current_issue_schema_version_swap:
-            exp_issue_from_model = stats_services.get_exp_issues_from_model(
-                exp_issues_model)
-
-        self.assertEqual(
-            exp_issue_from_model.unresolved_issues[0].issue_type, 'EarlyQuit1')
-        self.assertEqual(
-            exp_issue_from_model.unresolved_issues[0].issue_customization_args[
-                'new_key'], 5)
-
-        # For other issue types, no changes happen during migration.
-        exp_issue1 = stats_domain.ExplorationIssue(
-            'MultipleIncorrectSubmissions', {}, [], 1, True)
-        exp_issue_dict1 = exp_issue1.to_dict()
-
-        stats_models.ExplorationIssuesModel.create(
-            'exp_id_1', 1, [exp_issue_dict1])
-
-        exp_issues_model1 = stats_models.ExplorationIssuesModel.get_model(
-            'exp_id_1', 1)
-
-        current_issue_schema_version_swap = self.swap(
-            stats_models, 'CURRENT_ISSUE_SCHEMA_VERSION', 2)
-        convert_issue_dict_swap = self.swap(
-            stats_domain.ExplorationIssue,
-            '_convert_issue_v1_dict_to_v2_dict',
-            self._dummy_convert_issue_v1_dict_to_v2_dict)
-
-        with convert_issue_dict_swap, current_issue_schema_version_swap:
-            exp_issue_from_model1 = stats_services.get_exp_issues_from_model(
-                exp_issues_model1)
-
-        self.assertEqual(
-            exp_issue_from_model1.unresolved_issues[0].issue_type,
-            'MultipleIncorrectSubmissions')
+            stats_domain.ExplorationIssue.from_dict(exp_issue_dict)
 
     def test_cannot_update_exp_issue_from_invalid_schema_version_model(self):
         exp_issue = stats_domain.ExplorationIssue('EarlyQuit', {}, [], 4, True)
@@ -822,6 +792,7 @@ def test_validate_playthrough_id_type(self):
 
 class LearnerActionTests(test_utils.GenericTestBase):
     """Tests the LearnerAction domain object."""
+
     def setUp(self):
         super(LearnerActionTests, self).setUp()