Repository: oppia/oppia

Issue:
[BUG]: E2E Tests Incorrectly Look Up Chromedriver Version for Chrome >=115.*
### Describe the bug

In `scripts/servers.py`, we access `https://chromedriver.storage.googleapis.com/LATEST_RELEASE_[the major, minor, and patch parts of the chrome version]` to look up the associated chromedriver version. For example, given a Chrome version of 115.0.5790.114, we would retrieve the chromedriver version from https://chromedriver.storage.googleapis.com/LATEST_RELEASE_115.0.5790.

https://github.com/oppia/oppia/blob/9235fdd5005fc7534a18880240da5190dd0c7208/scripts/servers.py#L553-L588

However, for Chrome versions starting with 115, Chrome and Chromedriver have the same version numbers ([docs](https://chromedriver.chromium.org/downloads/version-selection)), In other words, for any given Chrome version, the compatible Chromedriver version has the same version number. We need to update `scripts.servers.get_chrome_version` to account for this.

As a temporary workaround, developers can manually pass their Chrome version using the `--chrome_driver_version` flag to `run_e2e_tests.py` if they have a version of Chrome >= 115.

https://github.com/oppia/oppia/blob/9235fdd5005fc7534a18880240da5190dd0c7208/scripts/run_e2e_tests.py#L76-L77

### URL of the page where the issue is observed.

_No response_

### Steps To Reproduce

_No response_

### Expected Behavior

_No response_

### Screenshots/Videos

_No response_

### What device are you using?

_No response_

### Operating System

_No response_

### What browsers are you seeing the problem on?

_No response_

### Browser version

_No response_

### Additional context

_No response_

### Tips for developers

Before tackling the bug, please use `git bisect` (see https://git-scm.com/docs/git-bisect) to investigate which PR caused it (you only need to go back as far as commit https://github.com/oppia/oppia/commits/9a334e9). If you find the PR, leave a comment on this issue pointing to it, or just say that it happened before commit 9a334e9 if you could reproduce it there. This will help us fix the issue by reverting the faulty PR.

Also, if this is your first issue, please make sure to follow https://github.com/oppia/oppia/wiki/Contributing-code-to-Oppia#choosing-a-good-first-issue and https://github.com/oppia/oppia/wiki/Contributing-code-to-Oppia#setting-things-up before claiming it. Thanks!

[BUG]: E2E Tests Incorrectly Look Up Chromedriver Version for Chrome >=115.*
### Describe the bug

In `scripts/servers.py`, we access `https://chromedriver.storage.googleapis.com/LATEST_RELEASE_[the major, minor, and patch parts of the chrome version]` to look up the associated chromedriver version. For example, given a Chrome version of 115.0.5790.114, we would retrieve the chromedriver version from https://chromedriver.storage.googleapis.com/LATEST_RELEASE_115.0.5790.

https://github.com/oppia/oppia/blob/9235fdd5005fc7534a18880240da5190dd0c7208/scripts/servers.py#L553-L588

However, for Chrome versions starting with 115, Chrome and Chromedriver have the same version numbers ([docs](https://chromedriver.chromium.org/downloads/version-selection)), In other words, for any given Chrome version, the compatible Chromedriver version has the same version number. We need to update `scripts.servers.get_chrome_version` to account for this.

As a temporary workaround, developers can manually pass their Chrome version using the `--chrome_driver_version` flag to `run_e2e_tests.py` if they have a version of Chrome >= 115.

https://github.com/oppia/oppia/blob/9235fdd5005fc7534a18880240da5190dd0c7208/scripts/run_e2e_tests.py#L76-L77

### URL of the page where the issue is observed.

_No response_

### Steps To Reproduce

_No response_

### Expected Behavior

_No response_

### Screenshots/Videos

_No response_

### What device are you using?

_No response_

### Operating System

_No response_

### What browsers are you seeing the problem on?

_No response_

### Browser version

_No response_

### Additional context

_No response_

### Tips for developers

Before tackling the bug, please use `git bisect` (see https://git-scm.com/docs/git-bisect) to investigate which PR caused it (you only need to go back as far as commit https://github.com/oppia/oppia/commits/9a334e9). If you find the PR, leave a comment on this issue pointing to it, or just say that it happened before commit 9a334e9 if you could reproduce it there. This will help us fix the issue by reverting the faulty PR.

Also, if this is your first issue, please make sure to follow https://github.com/oppia/oppia/wiki/Contributing-code-to-Oppia#choosing-a-good-first-issue and https://github.com/oppia/oppia/wiki/Contributing-code-to-Oppia#setting-things-up before claiming it. Thanks!



Discussion:

Resolving patch:
diff --git a/scripts/servers.py b/scripts/servers.py
--- a/scripts/servers.py
+++ b/scripts/servers.py
@@ -550,9 +550,10 @@ def print_proc_output() -> None:
         yield proc
 
 
-def get_chrome_version() -> str:
-    """Returns the version of Chrome installed on the system."""
-
+def get_chromedriver_version() -> str:
+    """Returns the version of Chromedriver compatible with the installed version
+    of Chrome.
+    """
     # Although there are spaces between Google and Chrome in the path, we
     # don't need to escape them for Popen (as opposed to on the terminal, in
     # which case we would need to escape them for the command to run).
@@ -577,15 +578,23 @@ def get_chrome_version() -> str:
             'https://chromedriver.chromium.org/downloads/version-selection'
             % chrome_command.replace(' ', r'\ ')) from e
 
-    installed_version_parts = b''.join(re.findall(rb'[0-9.]', output))
-    installed_version = '.'.join(
-        installed_version_parts.decode('utf-8').split('.')[:-1])
-    response = utils.url_open(
-        'https://chromedriver.storage.googleapis.com/LATEST_RELEASE_%s' % (
-            installed_version))
-    chrome_version: str = response.read().decode('utf-8')
+    installed_version_bytes = b''.join(re.findall(rb'[0-9.]', output))
+    installed_version_parts = installed_version_bytes.decode('utf-8').split('.')
+    # For Chrome versions 115 and above, compatible Chromedriver and Chrome
+    # versions have the same version numbers. For earlier versions, we use a
+    # Google API to find the compatible Chromedriver version. See
+    # https://chromedriver.chromium.org/downloads/version-selection for details.
+    if int(installed_version_parts[0]) >= 115:
+        chromedriver_version: str = '.'.join(installed_version_parts)
+    else:
+        response = utils.url_open(
+            'https://chromedriver.storage.googleapis.com/LATEST_RELEASE_%s' % (
+                '.'.join(installed_version_parts[:-1])
+            )
+        )
+        chromedriver_version = response.read().decode('utf-8')
 
-    return chrome_version
+    return chromedriver_version
 
 
 @contextlib.contextmanager
@@ -683,7 +692,7 @@ def managed_webdriverio_server(
         raise ValueError('Sharding instance should be larger than 0')
 
     if chrome_version is None:
-        chrome_version = get_chrome_version()
+        chrome_version = get_chromedriver_version()
 
     if mobile:
         os.environ['MOBILE'] = 'true'


Test patch:
diff --git a/scripts/servers_test.py b/scripts/servers_test.py
--- a/scripts/servers_test.py
+++ b/scripts/servers_test.py
@@ -31,6 +31,7 @@
 import time
 from urllib import request as urlrequest
 
+from core import utils
 from core.tests import test_utils
 from scripts import common
 from scripts import scripts_test_utils
@@ -1098,3 +1099,42 @@ def test_managed_acceptance_test_server_with_invalid_suite(self) -> None:
                 servers.managed_acceptance_tests_server(
                     suite_name=suite_name,
                     stdout=subprocess.PIPE))
+
+
+class GetChromedriverVersionTests(test_utils.TestBase):
+
+    def test_chrome_before_115_queries_api(self) -> None:
+        def mock_check_output(_: List[str]) -> bytes:
+            return b'Google Chrome 72.0.3626.123\n'
+        def mock_url_open(_: str) -> io.BytesIO:
+            return io.BytesIO(b'72.0.3626.69')
+        expected_url = (
+            'https://chromedriver.storage.googleapis.com/'
+            'LATEST_RELEASE_72.0.3626')
+
+        check_output_swap = self.swap(
+            subprocess, 'check_output', mock_check_output)
+        url_open_swap = self.swap_with_checks(
+            utils, 'url_open', mock_url_open, expected_args=[(expected_url,)])
+
+        with check_output_swap, url_open_swap:
+            self.assertEqual(
+                servers.get_chromedriver_version(),
+                '72.0.3626.69',
+            )
+
+    def test_chrome_115_and_later_returns_chrome_version(self) -> None:
+        def mock_check_output(_: List[str]) -> bytes:
+            return b'Google Chrome 115.0.3626.123\n'
+        def mock_url_open(_: str) -> None:
+            raise AssertionError('url_open should not be called.')
+
+        check_output_swap = self.swap(
+            subprocess, 'check_output', mock_check_output)
+        url_open_swap = self.swap(utils, 'url_open', mock_url_open)
+
+        with check_output_swap, url_open_swap:
+            self.assertEqual(
+                servers.get_chromedriver_version(),
+                '115.0.3626.123',
+            )