Repository: oppia/oppia

Issue:
pylatexenc metadata folder name change
<!--
  - Thanks for taking the time to report a bug in the Oppia project.
  - Before filing a new issue, please do a quick search to check that it hasn't
  - already been filed on the [issue tracker](https://github.com/oppia/oppia/issues)._
  -->

**Describe the bug**
Running `scripts.start` errors out because of unexpected name for the `pylatexenc` library.

**To Reproduce**
Steps to reproduce the behavior:
 1. Run `scripts.start`
 2. See error

**Observed behavior**
The script errors because of the installed `pylatexenc-2.6-py2.7.egg-info` folder doesn't match expected name.

**Expected behavior**
Expected `scripts.start` to install all dependencies and start the app. [https://www.python.org/dev/peps/pep-0376/#id16](https://www.python.org/dev/peps/pep-0376/#id16) may provide additional context to accepted metadata folder names.

**Error Message**
```
Exception: The python library pylatexenc was installed without the correct metadata folders which may indicate that the convention for naming the metadata folders have changed. Please go to `scripts/install_backend_python_libs` and modify our assumptions in the _get_possible_normalized_metadata_directory_names function for what metadata directory names can be.
```

**Desktop (please complete the following information; delete this section if the issue does not arise on desktop):**
 - OS: MacOS Big Sur
 - Browser N/A
 - Browser-version N/A

pylatexenc metadata folder name change
<!--
  - Thanks for taking the time to report a bug in the Oppia project.
  - Before filing a new issue, please do a quick search to check that it hasn't
  - already been filed on the [issue tracker](https://github.com/oppia/oppia/issues)._
  -->

**Describe the bug**
Running `scripts.start` errors out because of unexpected name for the `pylatexenc` library.

**To Reproduce**
Steps to reproduce the behavior:
 1. Run `scripts.start`
 2. See error

**Observed behavior**
The script errors because of the installed `pylatexenc-2.6-py2.7.egg-info` folder doesn't match expected name.

**Expected behavior**
Expected `scripts.start` to install all dependencies and start the app. [https://www.python.org/dev/peps/pep-0376/#id16](https://www.python.org/dev/peps/pep-0376/#id16) may provide additional context to accepted metadata folder names.

**Error Message**
```
Exception: The python library pylatexenc was installed without the correct metadata folders which may indicate that the convention for naming the metadata folders have changed. Please go to `scripts/install_backend_python_libs` and modify our assumptions in the _get_possible_normalized_metadata_directory_names function for what metadata directory names can be.
```

**Desktop (please complete the following information; delete this section if the issue does not arise on desktop):**
 - OS: MacOS Big Sur
 - Browser N/A
 - Browser-version N/A



Discussion:
The linked PEP-0376 lists `docutils-0.5-py2.6.egg-info` as a valid metadata folder name. Note the `-py2.6` suffix to the version string, which is what appears to be anomalous in this metadata folder name
@DubeySandeep PTAL
Also @vojtechjelinek PTAL
I will prepare a PR for this.
The linked PEP-0376 lists `docutils-0.5-py2.6.egg-info` as a valid metadata folder name. Note the `-py2.6` suffix to the version string, which is what appears to be anomalous in this metadata folder name
@DubeySandeep PTAL
Also @vojtechjelinek PTAL
I will prepare a PR for this.

Resolving patch:
diff --git a/scripts/install_backend_python_libs.py b/scripts/install_backend_python_libs.py
--- a/scripts/install_backend_python_libs.py
+++ b/scripts/install_backend_python_libs.py
@@ -390,6 +390,8 @@ def _get_possible_normalized_metadata_directory_names(
     """
     # Some metadata folders replace the hyphens in the library name with
     # underscores.
+    # TODO(#11474): The '-py2.7' suffix might be used in some metadata directory
+    # names, this will need to be changed after the Python 3 migration.
     return {
         normalize_directory_name(
             '%s-%s.dist-info' % (library_name, version_string)),
@@ -401,6 +403,11 @@ def _get_possible_normalized_metadata_directory_names(
         normalize_directory_name(
             '%s-%s.egg-info' % (
                 library_name.replace('-', '_'), version_string)),
+        normalize_directory_name(
+            '%s-%s-py2.7.egg-info' % (library_name, version_string)),
+        normalize_directory_name(
+            '%s-%s-py2.7.egg-info' % (
+                library_name.replace('-', '_'), version_string)),
     }
 
 


Test patch:
diff --git a/scripts/install_backend_python_libs_test.py b/scripts/install_backend_python_libs_test.py
--- a/scripts/install_backend_python_libs_test.py
+++ b/scripts/install_backend_python_libs_test.py
@@ -514,7 +514,7 @@ def mock_rm(path):
             paths_to_delete.append(
                 path[len(common.THIRD_PARTY_PYTHON_LIBS_DIR) + 1:])
 
-        def mock_is_dir(path): # pylint: disable=unused-argument
+        def mock_is_dir(unused_path):
             return True
 
         def mock_get_mismatches():
@@ -577,8 +577,41 @@ def mock_validate_metadata_directories():
                 u'google_cloud_datastore-1.13.0.dist-info'
             ])
 
+    def test_correct_metadata_directory_names_do_not_throw_error(self):
+        def mock_find_distributions(unused_paths):
+            return [
+                Distribution('dependency-1', '1.5.1', {}),
+                Distribution('dependency2', '5.0.0', {}),
+                Distribution('dependency-5', '0.5.3', {}),
+                Distribution('dependency6', '0.5.3', {
+                    'direct_url.json': json.dumps({
+                        'url': 'git://github.com/oppia/dependency6',
+                        'vcs_info': {'vcs': 'git', 'commit_id': 'z' * 40},
+                    })
+                }),
+            ]
+
+        def mock_list_dir(unused_path):
+            return [
+                'dependency-1-1.5.1.dist-info',
+                'dependency2-5.0.0.egg-info',
+                'dependency-5-0.5.3-py2.7.egg-info',
+                'dependency_6-0.5.3-py2.7.egg-info',
+            ]
+
+        def mock_is_dir(unused_path):
+            return True
+
+        swap_find_distributions = self.swap(
+            pkg_resources, 'find_distributions', mock_find_distributions)
+        swap_list_dir = self.swap(os, 'listdir', mock_list_dir)
+        swap_is_dir = self.swap(os.path, 'isdir', mock_is_dir)
+
+        with swap_find_distributions, swap_list_dir, swap_is_dir:
+            install_backend_python_libs.validate_metadata_directories()
+
     def test_exception_raised_when_metadata_directory_names_are_missing(self):
-        def mock_find_distributions(paths): # pylint: disable=unused-argument
+        def mock_find_distributions(unused_paths):
             return [
                 Distribution('dependency1', '1.5.1', {}),
                 Distribution('dependency2', '5.0.0', {}),
@@ -591,7 +624,7 @@ def mock_find_distributions(paths): # pylint: disable=unused-argument
                 }),
             ]
 
-        def mock_list_dir(path): # pylint: disable=unused-argument
+        def mock_list_dir(unused_path):
             return [
                 'dependency1-1.5.1.dist-info',
                 'dependency1',
@@ -601,7 +634,7 @@ def mock_list_dir(path): # pylint: disable=unused-argument
                 'dependency5-0.5.3.metadata',
             ]
 
-        def mock_is_dir(path): # pylint: disable=unused-argument
+        def mock_is_dir(unused_path):
             return True
         swap_find_distributions = self.swap(
             pkg_resources, 'find_distributions', mock_find_distributions)