Repository: oppia/oppia

Issue:
Schema validation error on uploading an exploration YAML
<!--
  - Thanks for taking the time to report a bug in the Oppia project.
  - Before filing a new issue, please do a quick search to check that it hasn't
  - already been filed on the [issue tracker](https://github.com/oppia/oppia/issues)._
  -->

**Describe the bug**
A clear and concise description of what the bug is.

**To Reproduce**
Steps to reproduce the behavior:
 1. Set `ALLOW_YAML_FILE_UPLOAD` to true in `constants.ts`
 2. Go to the creator dashboard.
 3. Upload a test exploration YAML file from [here](https://github.com/oppia/oppia/tree/develop/data/explorations).
 4. See error.

**Observed behavior**
Valid exploration YAML does not get uploaded.

**Expected behavior**
Valid exploration YAML should be uploaded correctly.

**Screenshots**

https://user-images.githubusercontent.com/11008603/150233493-27963be2-18ac-4c2d-a184-c39cd3268f09.mp4





Discussion:
Looks like a schema validation issue.
/cc @vojtechjelinek @Nik-09 
@Nik-09 Can you take this up, please?
Sure I will take this one.
Thank you.
Hi @kevintab95,
I have to discuss a few points before solving this issue,

Currently, a frontend YAML file that we upload is operated with [JSON stringified](https://github.com/oppia/oppia/blob/526968ed846ecc1ae057af0297b3a08ec55a65a8/core/templates/components/entity-creation-services/exploration-creation.service.ts#L87) which results in the conversion of the YAML file to an empty dict
JSON.Strignify(yaml_content) —> {}
Due to this operation, an error was encountered which is clearly visible in the video.

The possible solution I have is – 

 1. Load YAML and then perform JSON.stringify() in frontend [(reference)](https://www.quora.com/How-can-I-convert-YAML-into-JSON-in-JavaScript-frontend-code-not-Node)
2. Load YAML and the perform json.dumps() in backend

In the first solution, I am not able to use YAML.parse() method can you help me whether we use this method for conversion?

Also if there is any point missing in the above statement can I go with the second solution, i.e., perform every operation in the backend?

Looking forward to your feedback.
Thank you.

Hm, I did a git blame and found the following implementation:

https://github.com/oppia/oppia/blob/b7cf4135e54a31877dae1a35e1bf9d2ad408750d/core/templates/components/entity-creation-services/exploration-creation.service.ts#L78-L84

Do you think reverting back to this implementation would work?
> Hi @kevintab95, I have to discuss a few points before solving this issue,
> 
> Currently, a frontend YAML file that we upload is operated with [JSON stringified](https://github.com/oppia/oppia/blob/526968ed846ecc1ae057af0297b3a08ec55a65a8/core/templates/components/entity-creation-services/exploration-creation.service.ts#L87) which results in the conversion of the YAML file to an empty dict JSON.Strignify(yaml_content) —> {} Due to this operation, an error was encountered which is clearly visible in the video.
> 
> The possible solution I have is –
> 
>     1. Load YAML and then perform JSON.stringify() in frontend [(reference)](https://www.quora.com/How-can-I-convert-YAML-into-JSON-in-JavaScript-frontend-code-not-Node)
> 
>     2. Load YAML and the perform json.dumps() in backend
> 
> 
> In the first solution, I am not able to use YAML.parse() method can you help me whether we use this method for conversion?
> 
> Also if there is any point missing in the above statement can I go with the second solution, i.e., perform every operation in the backend?
> 
> Looking forward to your feedback. Thank you.

Hmm, but we don't directly stringify the yaml, but we stringify a JSON that contains string yaml, right? Not sure why it turns to `{}` then :thinking: 

I'm fine with performing all the operations in the backend

Resolving patch:
diff --git a/core/controllers/creator_dashboard.py b/core/controllers/creator_dashboard.py
--- a/core/controllers/creator_dashboard.py
+++ b/core/controllers/creator_dashboard.py
@@ -352,7 +352,7 @@ class UploadExplorationHandler(base.BaseHandler):
     @acl_decorators.can_upload_exploration
     def post(self):
         """Handles POST requests."""
-        yaml_content = self.normalized_payload.get('yaml_file')
+        yaml_content = self.normalized_request.get('yaml_file')
 
         new_exploration_id = exp_fetchers.get_new_exploration_id()
         if constants.ALLOW_YAML_FILE_UPLOAD:


Test patch:
diff --git a/core/controllers/creator_dashboard_test.py b/core/controllers/creator_dashboard_test.py
--- a/core/controllers/creator_dashboard_test.py
+++ b/core/controllers/creator_dashboard_test.py
@@ -17,12 +17,13 @@
 from __future__ import annotations
 
 import logging
+import os
 
 from core import feconf
+from core import python_utils
 from core.constants import constants
 from core.controllers import creator_dashboard
 from core.domain import collection_services
-from core.domain import exp_fetchers
 from core.domain import exp_services
 from core.domain import feedback_domain
 from core.domain import feedback_services
@@ -500,6 +501,12 @@ def test_creator_dashboard_page(self):
 
 
 class CreationButtonsTests(test_utils.GenericTestBase):
+    with python_utils.open_file(
+        os.path.join(
+            feconf.SAMPLE_EXPLORATIONS_DIR, 'welcome', 'welcome.yaml'),
+        'rb', encoding=None
+    ) as f:
+        raw_yaml = f.read()
 
     def setUp(self):
         super(CreationButtonsTests, self).setUp()
@@ -542,14 +549,14 @@ def test_can_upload_exploration(self):
             self.assertEqual(explorations_list, [])
             exp_a_id = self.post_json(
                 feconf.UPLOAD_EXPLORATION_URL,
-                {'yaml_file': self.SAMPLE_YAML_CONTENT},
-                csrf_token=csrf_token
+                {},
+                csrf_token=csrf_token,
+                upload_files=((
+                'yaml_file', 'unused_filename', self.raw_yaml),)
             )[creator_dashboard.EXPLORATION_ID_KEY]
             explorations_list = self.get_json(
                 feconf.CREATOR_DASHBOARD_DATA_URL)['explorations_list']
-            exploration = exp_fetchers.get_exploration_by_id(exp_a_id)
             self.assertEqual(explorations_list[0]['id'], exp_a_id)
-            self.assertEqual(exploration.to_yaml(), self.SAMPLE_YAML_CONTENT)
             self.logout()
 
     def test_can_not_upload_exploration_when_server_does_not_allow_file_upload(
@@ -559,9 +566,10 @@ def test_can_not_upload_exploration_when_server_does_not_allow_file_upload(
         csrf_token = self.get_new_csrf_token()
         self.post_json(
             feconf.UPLOAD_EXPLORATION_URL,
-            {'yaml_file': self.SAMPLE_YAML_CONTENT},
+            {},
             csrf_token=csrf_token,
+            upload_files=((
+                'yaml_file', 'unused_filename', self.raw_yaml),),
             expected_status_int=400
         )
-
         self.logout()