Repository: oppia/oppia

Issue:
Extra Lines and List Items Added on Save Draft (Develop branch)
**Describe the bug**
On an instance working off `develop` branch, Save Draft causes blank bullet points (or blank lines, if no bullet points are in the text) added to the generic card text, e.g., text inside the Introduction Card.

It looks like the `/createhandler/data` call goes out with the correct `"new value":{"html"` in the payload, but the Response comes back with an extra blank bullet point (or line) in the `"content"..."html"`

**To Reproduce**
Steps to reproduce the behavior:

In the Creator Dashboard, click on "Create" button, then click into the introduction card to activate the editor. In the editor's top menu pane, click on Insert/Remove Billeted List. Back in the text area, type some text in the line next to the bullet, e.g., "Line 1"

Click Save Content
Click Save Draft and confirm Save Draft

**Observed behavior**
The re-displayed Card shows two bullet points: one with no text and the second one with the "Line 1" text

**Expected behavior**
After save, the content of the card should be preserved as it was being saved.

**Desktop (please complete the following information; delete this section if the issue does not arise on desktop):**
 - OS: Ubuntu
 - Browser chrome
 - Version latest


Extra Lines and List Items Added on Save Draft (Develop branch)
**Describe the bug**
On an instance working off `develop` branch, Save Draft causes blank bullet points (or blank lines, if no bullet points are in the text) added to the generic card text, e.g., text inside the Introduction Card.

It looks like the `/createhandler/data` call goes out with the correct `"new value":{"html"` in the payload, but the Response comes back with an extra blank bullet point (or line) in the `"content"..."html"`

**To Reproduce**
Steps to reproduce the behavior:

In the Creator Dashboard, click on "Create" button, then click into the introduction card to activate the editor. In the editor's top menu pane, click on Insert/Remove Billeted List. Back in the text area, type some text in the line next to the bullet, e.g., "Line 1"

Click Save Content
Click Save Draft and confirm Save Draft

**Observed behavior**
The re-displayed Card shows two bullet points: one with no text and the second one with the "Line 1" text

**Expected behavior**
After save, the content of the card should be preserved as it was being saved.

**Desktop (please complete the following information; delete this section if the issue does not arise on desktop):**
 - OS: Ubuntu
 - Browser chrome
 - Version latest




Discussion:

Resolving patch:
diff --git a/core/domain/html_validation_service.py b/core/domain/html_validation_service.py
--- a/core/domain/html_validation_service.py
+++ b/core/domain/html_validation_service.py
@@ -405,7 +405,22 @@ def convert_to_ckeditor(html_data):
     for tag_name in list_tags:
         for tag in soup.findAll(name=tag_name):
             for child in tag.children:
-                if child.name not in ['li', 'pre', 'ol', 'ul']:
+                # The html formed after inserting a list element is:
+                # <ul>
+                #  <li> Item1 </li>
+                #  <li> Item2 </li>
+                # </ul>
+                # The new line is being treated as a tag child by beautifulSoup
+                # and so there is an extra li being produced which is not
+                # required. So, we remove the extra new lines here since
+                # they are added due to html being stored as a string.
+                # CKEditor adds margin between list elements and the \n
+                # character is of no significance therefore. Further if
+                # user enters any newline it will be stored as <li>&nbsp;</li>
+                # so removing the \n will not affect these lines.
+                if child == '\n':
+                    child.replaceWith('')
+                elif child.name not in ['li', 'pre', 'ol', 'ul']:
                     new_parent = soup.new_tag('li')
                     next_sib = list(child.next_siblings)
                     child.wrap(new_parent)
@@ -484,7 +499,23 @@ def convert_to_ckeditor(html_data):
     # (It may be generated by copy paste).
     for content in soup.contents:
         if not content.name:
-            content.wrap(soup.new_tag('p'))
+            # When user enters a string and then presses enter, a new paragraph
+            # is created by ckeditor by default. This complete html data is
+            # stored as:
+            # <p>Para1</p>
+            # <p>Para2</p>
+            # The extra newline here is just in the storage representation but
+            # is not useful as ckeditor adds a margin between two paragraphs
+            # by default. On the other hand if user themselves enters a new line
+            # by pressing enter, it will be stored as <p>&nbsp;</p> instead of
+            # just a \n character. So, removing \n will not remove the new lines
+            # inserted by user. Also we are not replacing \n with <p>&nbsp;</p>
+            # since this is not entered by user. It is just due to the html
+            # being stored as a string.
+            if content == '\n':
+                content.replaceWith('')
+            else:
+                content.wrap(soup.new_tag('p'))
 
     return unicode(soup).replace('<br/>', '<br>')
 


Test patch:
diff --git a/core/domain/html_validation_service_test.py b/core/domain/html_validation_service_test.py
--- a/core/domain/html_validation_service_test.py
+++ b/core/domain/html_validation_service_test.py
@@ -840,6 +840,12 @@ def test_convert_to_ckeditor(self):
             'expected_output': (
                 '<ol><li><p>This is a p in ol</p><p> or ul</p></li></ol>'
             )
+        }, {
+            'html_content': '<ul>\n<li>Item</li>\n</ul>',
+            'expected_output': '<ul><li>Item</li></ul>'
+        }, {
+            'html_content': '<p>Para1</p>\n<p>Para2</p>',
+            'expected_output': '<p>Para1</p><p>Para2</p>'
         }]
 
         for test_case in test_cases: