Repository: oppia/oppia

Issue:
E2E Flake: Unable to bind localhost:21054
Flakes seen:
* [ ] Failed on `develop` in CircleCI job [44869](https://circleci.com/gh/oppia/oppia/44869) in the accessibility suite
* [ ] A similar flake occurred in CircleCI job [44699](https://circleci.com/gh/oppia/oppia/44699) on `develop` (efd3492) in the E2E file upload features suite
* [ ] Observed on develop (ca5d54a) in CircleCI job [46819](https://circleci.com/gh/oppia/oppia/46819) with message `Unable to bind localhost:17260Unable to bind localhost:17260`
* [ ] Observed on develop (44dc800) in CircleCI job [47768](https://circleci.com/gh/oppia/oppia/47768) with message `Unable to bind localhost:16405Unable to bind localhost:16405`
* [ ] Observed on develop (b46d573) in CircleCI job [57247](https://app.circleci.com/pipelines/github/oppia/oppia/10212/workflows/4fbfd05b-ba54-4917-9d43-29656acd810a/jobs/57247)

Log: 

```
[01:54:03] I/launcher - Running 1 instances of WebDriver
[01:54:03] I/hosted - Using the selenium server at http://localhost:4444/wd/hub
Started
Jasmine started
Traceback (most recent call last):
Traceback (most recent call last):
  File "/home/circleci/oppia_tools/google_appengine_1.9.67/google_appengine/_python_runtime.py", line 103, in <module>
  File "/home/circleci/oppia_tools/google_appengine_1.9.67/google_appengine/_python_runtime.py", line 103, in <module>
        _run_file(__file__, globals())
_run_file(__file__, globals())
  File "/home/circleci/oppia_tools/google_appengine_1.9.67/google_appengine/_python_runtime.py", line 97, in _run_file
  File "/home/circleci/oppia_tools/google_appengine_1.9.67/google_appengine/_python_runtime.py", line 97, in _run_file
        execfile(_PATHS.script_file(script_name), globals_)
execfile(_PATHS.script_file(script_name), globals_)
  File "/home/circleci/oppia_tools/google_appengine_1.9.67/google_appengine/google/appengine/tools/devappserver2/python/runtime/runtime.py", line 199, in <module>
  File "/home/circleci/oppia_tools/google_appengine_1.9.67/google_appengine/google/appengine/tools/devappserver2/python/runtime/runtime.py", line 199, in <module>
        main()
main()
  File "/home/circleci/oppia_tools/google_appengine_1.9.67/google_appengine/google/appengine/tools/devappserver2/python/runtime/runtime.py", line 188, in main
  File "/home/circleci/oppia_tools/google_appengine_1.9.67/google_appengine/google/appengine/tools/devappserver2/python/runtime/runtime.py", line 188, in main
        server.start()
server.start()
  File "/home/circleci/oppia_tools/google_appengine_1.9.67/google_appengine/google/appengine/tools/devappserver2/wsgi_server.py", line 396, in start
  File "/home/circleci/oppia_tools/google_appengine_1.9.67/google_appengine/google/appengine/tools/devappserver2/wsgi_server.py", line 396, in start
        self._start_all_fixed_port(host_ports)
self._start_all_fixed_port(host_ports)
  File "/home/circleci/oppia_tools/google_appengine_1.9.67/google_appengine/google/appengine/tools/devappserver2/wsgi_server.py", line 433, in _start_all_fixed_port
  File "/home/circleci/oppia_tools/google_appengine_1.9.67/google_appengine/google/appengine/tools/devappserver2/wsgi_server.py", line 433, in _start_all_fixed_port
        raise BindError('Unable to bind %s:%s' % self.bind_addr)
raise BindError('Unable to bind %s:%s' % self.bind_addr)
google.appengine.tools.devappserver2.wsgi_servergoogle.appengine.tools.devappserver2.wsgi_server..BindErrorBindError: : Unable to bind localhost:21054Unable to bind localhost:21054

F
  screenreader and keyboard user accessibility features
    ✗ should skip to the main content element
      - Failed: Timed out waiting for asynchronous Angular tasks to finish after 180 seconds. This may be because the current page is not an Angular application. Please see the FAQ for more details: https://github.com/angular/protractor/blob/master/docs/timeouts.md#waiting-for-angular
      While waiting for element with locator - Locator: By(css selector, [ng-show="loadingMessage"]). 
      The following tasks were pending:
       - $http: /userinfohandler

.  Cache Slugs
    ✓ should check that errors get logged for missing resources



Failures:
1) screenreader and keyboard user accessibility features should skip to the main content element
  Message:
    Failed: Timed out waiting for asynchronous Angular tasks to finish after 180 seconds. This may be because the current page is not an Angular application. Please see the FAQ for more details: https://github.com/angular/protractor/blob/master/docs/timeouts.md#waiting-for-angular
    While waiting for element with locator - Locator: By(css selector, [ng-show="loadingMessage"]). 
    The following tasks were pending:
     - $http: /userinfohandler
  Stack:
    ScriptTimeoutError: asynchronous script timeout: result was not received in 180 seconds
      (Session info: chrome=81.0.4044.92)
      (Driver info: chromedriver=2.41.578700 (2f1ed5f9343c13f73144538f15c00b370eda6706),platform=Linux 4.15.0-1052-aws x86_64)
        at Object.checkLegacyResponse (/home/circleci/oppia/node_modules/selenium-webdriver/lib/error.js:546:15)
        at parseHttpResponse (/home/circleci/oppia/node_modules/selenium-webdriver/lib/http.js:509:13)
        at doSend.then.response (/home/circleci/oppia/node_modules/selenium-webdriver/lib/http.js:441:30)
        at process._tickCallback (internal/process/next_tick.js:68:7)
    From: Task: Protractor.waitForAngular() - Locator: By(css selector, [ng-show="loadingMessage"])
        at thenableWebDriverProxy.schedule (/home/circleci/oppia/node_modules/selenium-webdriver/lib/webdriver.js:807:17)
        at ProtractorBrowser.executeAsyncScript_ (/home/circleci/oppia/node_modules/protractor/built/browser.js:425:28)
        at angularAppRoot.then (/home/circleci/oppia/node_modules/protractor/built/browser.js:456:33)
        at ManagedPromise.invokeCallback_ (/home/circleci/oppia/node_modules/selenium-webdriver/lib/promise.js:1376:14)
        at TaskQueue.execute_ (/home/circleci/oppia/node_modules/selenium-webdriver/lib/promise.js:3084:14)
        at TaskQueue.executeNext_ (/home/circleci/oppia/node_modules/selenium-webdriver/lib/promise.js:3067:27)
        at asyncRun (/home/circleci/oppia/node_modules/selenium-webdriver/lib/promise.js:2927:27)
        at /home/circleci/oppia/node_modules/selenium-webdriver/lib/promise.js:668:7
        at process._tickCallback (internal/process/next_tick.js:68:7)
    From: Task: <anonymous>
        at pollCondition (/home/circleci/oppia/node_modules/selenium-webdriver/lib/promise.js:2195:19)
        at /home/circleci/oppia/node_modules/selenium-webdriver/lib/promise.js:2191:7
        at new ManagedPromise (/home/circleci/oppia/node_modules/selenium-webdriver/lib/promise.js:1077:7)
        at ControlFlow.promise (/home/circleci/oppia/node_modules/selenium-webdriver/lib/promise.js:2505:12)
        at /home/circleci/oppia/node_modules/selenium-webdriver/lib/promise.js:2190:22
        at TaskQueue.execute_ (/home/circleci/oppia/node_modules/selenium-webdriver/lib/promise.js:3084:14)
        at TaskQueue.executeNext_ (/home/circleci/oppia/node_modules/selenium-webdriver/lib/promise.js:3067:27)
        at asyncRun (/home/circleci/oppia/node_modules/selenium-webdriver/lib/promise.js:2927:27)
        at /home/circleci/oppia/node_modules/selenium-webdriver/lib/promise.js:668:7
        at process._tickCallback (internal/process/next_tick.js:68:7)
    From: Task: Page takes more than 15 secs to load
        at scheduleWait (/home/circleci/oppia/node_modules/selenium-webdriver/lib/promise.js:2188:20)
        at ControlFlow.wait (/home/circleci/oppia/node_modules/selenium-webdriver/lib/promise.js:2517:12)
        at thenableWebDriverProxy.wait (/home/circleci/oppia/node_modules/selenium-webdriver/lib/webdriver.js:934:29)
        at Object.pageToFullyLoad (/home/circleci/oppia/core/tests/protractor_utils/waitFor.js:61:25)
        at LibraryPage.get (/home/circleci/oppia/core/tests/protractor_utils/LibraryPage.js:81:20)
        at UserContext.<anonymous> (/home/circleci/oppia/core/tests/protractor/accessibility.js:33:17)
        at /home/circleci/oppia/node_modules/jasminewd2/index.js:112:25
        at new ManagedPromise (/home/circleci/oppia/node_modules/selenium-webdriver/lib/promise.js:1077:7)
        at ControlFlow.promise (/home/circleci/oppia/node_modules/selenium-webdriver/lib/promise.js:2505:12)
        at schedulerExecute (/home/circleci/oppia/node_modules/jasminewd2/index.js:95:18)
    From: Task: Run it("should skip to the main content element") in control flow
        at UserContext.<anonymous> (/home/circleci/oppia/node_modules/jasminewd2/index.js:94:19)
        at /home/circleci/oppia/node_modules/jasminewd2/index.js:64:48
        at ControlFlow.emit (/home/circleci/oppia/node_modules/selenium-webdriver/lib/events.js:62:21)
        at ControlFlow.shutdown_ (/home/circleci/oppia/node_modules/selenium-webdriver/lib/promise.js:2674:10)
        at shutdownTask_.MicroTask (/home/circleci/oppia/node_modules/selenium-webdriver/lib/promise.js:2599:53)
    From asynchronous test: 
    Error
        at Suite.<anonymous> (/home/circleci/oppia/core/tests/protractor/accessibility.js:32:3)
        at Object.<anonymous> (/home/circleci/oppia/core/tests/protractor/accessibility.js:25:1)
        at Module._compile (internal/modules/cjs/loader.js:778:30)
        at Object.Module._extensions..js (internal/modules/cjs/loader.js:789:10)
        at Module.load (internal/modules/cjs/loader.js:653:32)
        at tryModuleLoad (internal/modules/cjs/loader.js:593:12)

2 specs, 1 failure
Finished in 185.332 seconds

**************************************************
*                    Failures                    *
**************************************************

1) screenreader and keyboard user accessibility features should skip to the main content element
  - Failed: Timed out waiting for asynchronous Angular tasks to finish after 180 seconds. This may be because the current page is not an Angular application. Please see the FAQ for more details: https://github.com/angular/protractor/blob/master/docs/timeouts.md#waiting-for-angular
  While waiting for element with locator - Locator: By(css selector, [ng-show="loadingMessage"]). 
  The following tasks were pending:
   - $http: /userinfohandler

Executed 2 of 2 specs (1 FAILED) in 3 mins 5 secs.
[01:57:09] I/launcher - 0 instance(s) of WebDriver still running
[01:57:09] I/launcher - chrome #01 failed 1 test(s)
[01:57:09] I/launcher - overall: 1 failed spec(s)
[01:57:09] E/launcher - Process exited with error code 1
Killing /usr/local/bin/python /home/circleci/oppia/../oppia_tools/google_appengine_1.9.67/google_appengine/dev_appserver.py --host 0.0.0.0 --port 9001 --clear_datastore=yes --dev_appserver_log_level=critical --log_level=critical --skip_sdk_update_check=true app.yaml ...
Killing java -Dwebdriver.chrome.driver=/home/circleci/oppia/node_modules/webdriver-manager/downloads/chromedriver_2.41 -Dwebdriver.gecko.driver=/home/circleci/oppia/node_modules/webdriver-manager/downloads/geckodriver_0.26.0 -jar /home/circleci/oppia/node_modules/webdriver-manager/downloads/selenium-server-standalone-4.0.0-alpha-1.jar -role node -servlet org.openqa.grid.web.servlet.LifecycleServlet -registerCycle 0 -port 4444 ...

Exited with code exit status 1
```
E2E Flake: Unable to bind localhost:21054
Flakes seen:
* [ ] Failed on `develop` in CircleCI job [44869](https://circleci.com/gh/oppia/oppia/44869) in the accessibility suite
* [ ] A similar flake occurred in CircleCI job [44699](https://circleci.com/gh/oppia/oppia/44699) on `develop` (efd3492) in the E2E file upload features suite
* [ ] Observed on develop (ca5d54a) in CircleCI job [46819](https://circleci.com/gh/oppia/oppia/46819) with message `Unable to bind localhost:17260Unable to bind localhost:17260`
* [ ] Observed on develop (44dc800) in CircleCI job [47768](https://circleci.com/gh/oppia/oppia/47768) with message `Unable to bind localhost:16405Unable to bind localhost:16405`
* [ ] Observed on develop (b46d573) in CircleCI job [57247](https://app.circleci.com/pipelines/github/oppia/oppia/10212/workflows/4fbfd05b-ba54-4917-9d43-29656acd810a/jobs/57247)

Log: 

```
[01:54:03] I/launcher - Running 1 instances of WebDriver
[01:54:03] I/hosted - Using the selenium server at http://localhost:4444/wd/hub
Started
Jasmine started
Traceback (most recent call last):
Traceback (most recent call last):
  File "/home/circleci/oppia_tools/google_appengine_1.9.67/google_appengine/_python_runtime.py", line 103, in <module>
  File "/home/circleci/oppia_tools/google_appengine_1.9.67/google_appengine/_python_runtime.py", line 103, in <module>
        _run_file(__file__, globals())
_run_file(__file__, globals())
  File "/home/circleci/oppia_tools/google_appengine_1.9.67/google_appengine/_python_runtime.py", line 97, in _run_file
  File "/home/circleci/oppia_tools/google_appengine_1.9.67/google_appengine/_python_runtime.py", line 97, in _run_file
        execfile(_PATHS.script_file(script_name), globals_)
execfile(_PATHS.script_file(script_name), globals_)
  File "/home/circleci/oppia_tools/google_appengine_1.9.67/google_appengine/google/appengine/tools/devappserver2/python/runtime/runtime.py", line 199, in <module>
  File "/home/circleci/oppia_tools/google_appengine_1.9.67/google_appengine/google/appengine/tools/devappserver2/python/runtime/runtime.py", line 199, in <module>
        main()
main()
  File "/home/circleci/oppia_tools/google_appengine_1.9.67/google_appengine/google/appengine/tools/devappserver2/python/runtime/runtime.py", line 188, in main
  File "/home/circleci/oppia_tools/google_appengine_1.9.67/google_appengine/google/appengine/tools/devappserver2/python/runtime/runtime.py", line 188, in main
        server.start()
server.start()
  File "/home/circleci/oppia_tools/google_appengine_1.9.67/google_appengine/google/appengine/tools/devappserver2/wsgi_server.py", line 396, in start
  File "/home/circleci/oppia_tools/google_appengine_1.9.67/google_appengine/google/appengine/tools/devappserver2/wsgi_server.py", line 396, in start
        self._start_all_fixed_port(host_ports)
self._start_all_fixed_port(host_ports)
  File "/home/circleci/oppia_tools/google_appengine_1.9.67/google_appengine/google/appengine/tools/devappserver2/wsgi_server.py", line 433, in _start_all_fixed_port
  File "/home/circleci/oppia_tools/google_appengine_1.9.67/google_appengine/google/appengine/tools/devappserver2/wsgi_server.py", line 433, in _start_all_fixed_port
        raise BindError('Unable to bind %s:%s' % self.bind_addr)
raise BindError('Unable to bind %s:%s' % self.bind_addr)
google.appengine.tools.devappserver2.wsgi_servergoogle.appengine.tools.devappserver2.wsgi_server..BindErrorBindError: : Unable to bind localhost:21054Unable to bind localhost:21054

F
  screenreader and keyboard user accessibility features
    ✗ should skip to the main content element
      - Failed: Timed out waiting for asynchronous Angular tasks to finish after 180 seconds. This may be because the current page is not an Angular application. Please see the FAQ for more details: https://github.com/angular/protractor/blob/master/docs/timeouts.md#waiting-for-angular
      While waiting for element with locator - Locator: By(css selector, [ng-show="loadingMessage"]). 
      The following tasks were pending:
       - $http: /userinfohandler

.  Cache Slugs
    ✓ should check that errors get logged for missing resources



Failures:
1) screenreader and keyboard user accessibility features should skip to the main content element
  Message:
    Failed: Timed out waiting for asynchronous Angular tasks to finish after 180 seconds. This may be because the current page is not an Angular application. Please see the FAQ for more details: https://github.com/angular/protractor/blob/master/docs/timeouts.md#waiting-for-angular
    While waiting for element with locator - Locator: By(css selector, [ng-show="loadingMessage"]). 
    The following tasks were pending:
     - $http: /userinfohandler
  Stack:
    ScriptTimeoutError: asynchronous script timeout: result was not received in 180 seconds
      (Session info: chrome=81.0.4044.92)
      (Driver info: chromedriver=2.41.578700 (2f1ed5f9343c13f73144538f15c00b370eda6706),platform=Linux 4.15.0-1052-aws x86_64)
        at Object.checkLegacyResponse (/home/circleci/oppia/node_modules/selenium-webdriver/lib/error.js:546:15)
        at parseHttpResponse (/home/circleci/oppia/node_modules/selenium-webdriver/lib/http.js:509:13)
        at doSend.then.response (/home/circleci/oppia/node_modules/selenium-webdriver/lib/http.js:441:30)
        at process._tickCallback (internal/process/next_tick.js:68:7)
    From: Task: Protractor.waitForAngular() - Locator: By(css selector, [ng-show="loadingMessage"])
        at thenableWebDriverProxy.schedule (/home/circleci/oppia/node_modules/selenium-webdriver/lib/webdriver.js:807:17)
        at ProtractorBrowser.executeAsyncScript_ (/home/circleci/oppia/node_modules/protractor/built/browser.js:425:28)
        at angularAppRoot.then (/home/circleci/oppia/node_modules/protractor/built/browser.js:456:33)
        at ManagedPromise.invokeCallback_ (/home/circleci/oppia/node_modules/selenium-webdriver/lib/promise.js:1376:14)
        at TaskQueue.execute_ (/home/circleci/oppia/node_modules/selenium-webdriver/lib/promise.js:3084:14)
        at TaskQueue.executeNext_ (/home/circleci/oppia/node_modules/selenium-webdriver/lib/promise.js:3067:27)
        at asyncRun (/home/circleci/oppia/node_modules/selenium-webdriver/lib/promise.js:2927:27)
        at /home/circleci/oppia/node_modules/selenium-webdriver/lib/promise.js:668:7
        at process._tickCallback (internal/process/next_tick.js:68:7)
    From: Task: <anonymous>
        at pollCondition (/home/circleci/oppia/node_modules/selenium-webdriver/lib/promise.js:2195:19)
        at /home/circleci/oppia/node_modules/selenium-webdriver/lib/promise.js:2191:7
        at new ManagedPromise (/home/circleci/oppia/node_modules/selenium-webdriver/lib/promise.js:1077:7)
        at ControlFlow.promise (/home/circleci/oppia/node_modules/selenium-webdriver/lib/promise.js:2505:12)
        at /home/circleci/oppia/node_modules/selenium-webdriver/lib/promise.js:2190:22
        at TaskQueue.execute_ (/home/circleci/oppia/node_modules/selenium-webdriver/lib/promise.js:3084:14)
        at TaskQueue.executeNext_ (/home/circleci/oppia/node_modules/selenium-webdriver/lib/promise.js:3067:27)
        at asyncRun (/home/circleci/oppia/node_modules/selenium-webdriver/lib/promise.js:2927:27)
        at /home/circleci/oppia/node_modules/selenium-webdriver/lib/promise.js:668:7
        at process._tickCallback (internal/process/next_tick.js:68:7)
    From: Task: Page takes more than 15 secs to load
        at scheduleWait (/home/circleci/oppia/node_modules/selenium-webdriver/lib/promise.js:2188:20)
        at ControlFlow.wait (/home/circleci/oppia/node_modules/selenium-webdriver/lib/promise.js:2517:12)
        at thenableWebDriverProxy.wait (/home/circleci/oppia/node_modules/selenium-webdriver/lib/webdriver.js:934:29)
        at Object.pageToFullyLoad (/home/circleci/oppia/core/tests/protractor_utils/waitFor.js:61:25)
        at LibraryPage.get (/home/circleci/oppia/core/tests/protractor_utils/LibraryPage.js:81:20)
        at UserContext.<anonymous> (/home/circleci/oppia/core/tests/protractor/accessibility.js:33:17)
        at /home/circleci/oppia/node_modules/jasminewd2/index.js:112:25
        at new ManagedPromise (/home/circleci/oppia/node_modules/selenium-webdriver/lib/promise.js:1077:7)
        at ControlFlow.promise (/home/circleci/oppia/node_modules/selenium-webdriver/lib/promise.js:2505:12)
        at schedulerExecute (/home/circleci/oppia/node_modules/jasminewd2/index.js:95:18)
    From: Task: Run it("should skip to the main content element") in control flow
        at UserContext.<anonymous> (/home/circleci/oppia/node_modules/jasminewd2/index.js:94:19)
        at /home/circleci/oppia/node_modules/jasminewd2/index.js:64:48
        at ControlFlow.emit (/home/circleci/oppia/node_modules/selenium-webdriver/lib/events.js:62:21)
        at ControlFlow.shutdown_ (/home/circleci/oppia/node_modules/selenium-webdriver/lib/promise.js:2674:10)
        at shutdownTask_.MicroTask (/home/circleci/oppia/node_modules/selenium-webdriver/lib/promise.js:2599:53)
    From asynchronous test: 
    Error
        at Suite.<anonymous> (/home/circleci/oppia/core/tests/protractor/accessibility.js:32:3)
        at Object.<anonymous> (/home/circleci/oppia/core/tests/protractor/accessibility.js:25:1)
        at Module._compile (internal/modules/cjs/loader.js:778:30)
        at Object.Module._extensions..js (internal/modules/cjs/loader.js:789:10)
        at Module.load (internal/modules/cjs/loader.js:653:32)
        at tryModuleLoad (internal/modules/cjs/loader.js:593:12)

2 specs, 1 failure
Finished in 185.332 seconds

**************************************************
*                    Failures                    *
**************************************************

1) screenreader and keyboard user accessibility features should skip to the main content element
  - Failed: Timed out waiting for asynchronous Angular tasks to finish after 180 seconds. This may be because the current page is not an Angular application. Please see the FAQ for more details: https://github.com/angular/protractor/blob/master/docs/timeouts.md#waiting-for-angular
  While waiting for element with locator - Locator: By(css selector, [ng-show="loadingMessage"]). 
  The following tasks were pending:
   - $http: /userinfohandler

Executed 2 of 2 specs (1 FAILED) in 3 mins 5 secs.
[01:57:09] I/launcher - 0 instance(s) of WebDriver still running
[01:57:09] I/launcher - chrome #01 failed 1 test(s)
[01:57:09] I/launcher - overall: 1 failed spec(s)
[01:57:09] E/launcher - Process exited with error code 1
Killing /usr/local/bin/python /home/circleci/oppia/../oppia_tools/google_appengine_1.9.67/google_appengine/dev_appserver.py --host 0.0.0.0 --port 9001 --clear_datastore=yes --dev_appserver_log_level=critical --log_level=critical --skip_sdk_update_check=true app.yaml ...
Killing java -Dwebdriver.chrome.driver=/home/circleci/oppia/node_modules/webdriver-manager/downloads/chromedriver_2.41 -Dwebdriver.gecko.driver=/home/circleci/oppia/node_modules/webdriver-manager/downloads/geckodriver_0.26.0 -jar /home/circleci/oppia/node_modules/webdriver-manager/downloads/selenium-server-standalone-4.0.0-alpha-1.jar -role node -servlet org.openqa.grid.web.servlet.LifecycleServlet -registerCycle 0 -port 4444 ...

Exited with code exit status 1
```


Discussion:
A similar flake occurred in CircleCI job [44699](https://circleci.com/gh/oppia/oppia/44699) on `develop` (efd3492) in the E2E file upload features suite
One general observation, based on the reported errors, it seems like most of them (if not all) seem to occur in Circle. I suspect that Circle is having issues with running tests in parallel -- I have seen this occur when there are multiple instances of the e2e script running on an env.

/cc @seanlip @U8NWXD @DubeySandeep @nithusha21 @kevinlee12 
Should we set sharding to false? (it forces the test to run one at a time).
I'm not opposed. You can try?
I dug into this a little, and here's what I think is happening:

* When we run the e2e tests using GAE `dev_appserver.py` to launch the site locally. This involves repeatedly using `devappserver2/http_runtime.py` to start runtime instances that serve the site's content.
* `http_runtime.py`chooses an unused port (line 347) and stores it in the environment variable `PORT` when it starts a WSGI server as a subprocess by calling `devappserver2/python/runtime/runtime.py`.
* The WSGI server binds to the port in the `PORT` environment variable. When we see this flake, it's because the binding here fails.

My guess is that if we are running multiple site instances at once (e.g. when Kevin was running the tests in separate terminal windows or when we shard the tests), one instance binds to a port in-between another instance getting a port in `http_runtime.py` and binding to it in `python/runtime/runtime.py`. If this is right, then this race condition means we cannot reliably use sharding. I'm creating a PR to disable sharding.

Note that the paths I mention in this comment are relative to `oppia_tools/google_appengine_1.9.67/google_appengine/google/appengine/tools`
Ooh, yes, that looks like a good analysis, thanks @U8NWXD!

I think one alternative might be to have a sort of retry mechanism (i.e. if we get a BindError, we try again) -- sort of like what happens when we're waiting for the port to be open. But I think disabling sharding is a good short-term fix. Will this impact the speed of running the tests, though?
@seanlip unfortunately yes, this will make the tests run more slowly. However, I think it's worth it since this flaked 6 times over July 4 and 5, so we're losing a lot of time to reruns anyway.

I was also hoping for a retry mechanism, but since GAE binds many times in a single test to different ports, I think GAE is binding to a new port every time we make a request to the web server. The logical place to put the retry then would be in the GAE code that binds the port, but that would require applying a custom patch to GAE, which I think would be better to avoid, right?
Yeah, I agree with avoiding a custom patch. 

I'm a bit confused about why this occurs with sharding but not without sharding. I mean -- the GAE server won't really know how many clients are spun up, right? The server should be able to take pings from multiple clients at a time. Though perhaps this assumption is less true on the dev server since usually multiple client requests won't come in simultaneously in that context.
@seanlip maybe I don't understand how sharding works then. Does sharding involve multiple servers being spun up, or just multiple clients? I assumed we were running multiple servers, but if not, then I share your confusion
Pretty sure it's just multiple clients, and all protractor tests run against a single server. If you search for "sharding" on this page https://www.protractortest.org/#/style-guide (which has other useful tips!) it says that "The files are executed across different browsers as they become available."
OK, then sharding probably isn't the issue. I'll dig some more
This still happens
This still happens
A similar flake occurred in CircleCI job [44699](https://circleci.com/gh/oppia/oppia/44699) on `develop` (efd3492) in the E2E file upload features suite
One general observation, based on the reported errors, it seems like most of them (if not all) seem to occur in Circle. I suspect that Circle is having issues with running tests in parallel -- I have seen this occur when there are multiple instances of the e2e script running on an env.

/cc @seanlip @U8NWXD @DubeySandeep @nithusha21 @kevinlee12 
Should we set sharding to false? (it forces the test to run one at a time).
I'm not opposed. You can try?
I dug into this a little, and here's what I think is happening:

* When we run the e2e tests using GAE `dev_appserver.py` to launch the site locally. This involves repeatedly using `devappserver2/http_runtime.py` to start runtime instances that serve the site's content.
* `http_runtime.py`chooses an unused port (line 347) and stores it in the environment variable `PORT` when it starts a WSGI server as a subprocess by calling `devappserver2/python/runtime/runtime.py`.
* The WSGI server binds to the port in the `PORT` environment variable. When we see this flake, it's because the binding here fails.

My guess is that if we are running multiple site instances at once (e.g. when Kevin was running the tests in separate terminal windows or when we shard the tests), one instance binds to a port in-between another instance getting a port in `http_runtime.py` and binding to it in `python/runtime/runtime.py`. If this is right, then this race condition means we cannot reliably use sharding. I'm creating a PR to disable sharding.

Note that the paths I mention in this comment are relative to `oppia_tools/google_appengine_1.9.67/google_appengine/google/appengine/tools`
Ooh, yes, that looks like a good analysis, thanks @U8NWXD!

I think one alternative might be to have a sort of retry mechanism (i.e. if we get a BindError, we try again) -- sort of like what happens when we're waiting for the port to be open. But I think disabling sharding is a good short-term fix. Will this impact the speed of running the tests, though?
@seanlip unfortunately yes, this will make the tests run more slowly. However, I think it's worth it since this flaked 6 times over July 4 and 5, so we're losing a lot of time to reruns anyway.

I was also hoping for a retry mechanism, but since GAE binds many times in a single test to different ports, I think GAE is binding to a new port every time we make a request to the web server. The logical place to put the retry then would be in the GAE code that binds the port, but that would require applying a custom patch to GAE, which I think would be better to avoid, right?
Yeah, I agree with avoiding a custom patch. 

I'm a bit confused about why this occurs with sharding but not without sharding. I mean -- the GAE server won't really know how many clients are spun up, right? The server should be able to take pings from multiple clients at a time. Though perhaps this assumption is less true on the dev server since usually multiple client requests won't come in simultaneously in that context.
@seanlip maybe I don't understand how sharding works then. Does sharding involve multiple servers being spun up, or just multiple clients? I assumed we were running multiple servers, but if not, then I share your confusion
Pretty sure it's just multiple clients, and all protractor tests run against a single server. If you search for "sharding" on this page https://www.protractortest.org/#/style-guide (which has other useful tips!) it says that "The files are executed across different browsers as they become available."
OK, then sharding probably isn't the issue. I'll dig some more
This still happens
This still happens

Resolving patch:
diff --git a/scripts/run_portserver.py b/scripts/run_portserver.py
new file mode 100644
--- /dev/null
+++ b/scripts/run_portserver.py
@@ -0,0 +1,531 @@
+# Copyright 2020 The Oppia Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+# Adapted from portserver.py in the python_portpicker project (commit
+# bf6250b121cd610c3280cd748b5702a2be040f41) at
+# https://github.com/google/python_portpicker/blob/main/src/portserver.py,
+# which is Copyright 2015 Google Inc. All Rights Reserved. The following
+# changes were made by The Oppia Authors to this file when it was
+# originally introduced. More may have been made since.
+#
+# * We used threading instead of asyncio for compatibility with Python 2.
+#   This change involved adding a Server class to replace asyncio's
+#   event loop.
+# * We re-wrote the logger format string to be compatible with Python 2.
+#   We kept the same general string structure.
+# * We renamed _handle_port_request to handle_port_request and removed
+#   the original handle_port_request function from the
+#   _PortServerRequestHandler class. We also made _handle_port_request
+#   return the request response instead of writing it to a provided
+#   writer.
+# * We re-wrote the main function to use the Server class instead of
+#   asyncio. In doing so we also simplified the main function by
+#   removing the signal.SIGUSR1 handler.
+# * We changed the default portserver address to `portserver.sock` to
+#   avoid problems introduced by using an address beginning with a null
+#   byte.
+
+"""A server to hand out network ports to applications running on one host.
+Typical usage:
+ 1) Run one instance of this process on each of your unittest farm hosts.
+ 2) Set the PORTSERVER_ADDRESS environment variable in your test runner
+    environment to let the portpicker library know to use a port server
+    rather than attempt to find ports on its own.
+$ /path/to/portserver.py &
+$ export PORTSERVER_ADDRESS=portserver.sock
+$ # ... launch a bunch of tests that use portpicker ...
+"""
+
+from __future__ import absolute_import  # pylint: disable=import-only-modules
+from __future__ import unicode_literals  # pylint: disable=import-only-modules
+
+import argparse
+import collections
+import logging
+import os
+import socket
+import sys
+import threading
+
+import python_utils
+
+_PROTOCOLS = [(socket.SOCK_STREAM, socket.IPPROTO_TCP),
+              (socket.SOCK_DGRAM, socket.IPPROTO_UDP)]
+
+
+def _get_process_command_line(pid):
+    """Get the command for a process.
+
+    Args:
+        pid: int. The process ID.
+
+    Returns:
+        str. The command that started the process.
+    """
+    try:
+        with python_utils.open_file(
+            '/proc/{}/cmdline'.format(pid), 'rt'
+        ) as f:
+            return f.read()
+    except IOError:
+        return ''
+
+
+def _get_process_start_time(pid):
+    """Get the start time for a process.
+
+    Args:
+        pid: int. The process ID.
+
+    Returns:
+        str. The time when the process started.
+    """
+    try:
+        with python_utils.open_file(
+            '/proc/{}/stat'.format(pid), 'rt'
+        ) as f:
+            return int(f.readline().split()[21])
+    except IOError:
+        return 0
+
+
+def _bind(port, socket_type, socket_protocol):
+    """Try to bind to a socket of the specified type, protocol, and port.
+    For the port to be considered available, the kernel must support at least
+    one of (IPv6, IPv4), and the port must be available on each supported
+    family.
+
+    Args:
+        port: int. The port number to bind to, or 0 to have the OS pick
+            a free port.
+        socket_type: int. The type of the socket (e.g.:
+            socket.SOCK_STREAM).
+        socket_protocol: int. The protocol of the socket (e.g.:
+            socket.IPPROTO_TCP).
+
+    Returns:
+        int|None. The port number on success or None on failure.
+    """
+    got_socket = False
+    for family in (socket.AF_INET6, socket.AF_INET):
+        try:
+            sock = socket.socket(family, socket_type, socket_protocol)
+            got_socket = True
+        except socket.error:
+            continue
+        try:
+            sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
+            sock.bind(('', port))
+            if socket_type == socket.SOCK_STREAM:
+                sock.listen(1)
+            port = sock.getsockname()[1]
+        except socket.error:
+            return None
+        finally:
+            sock.close()
+    return port if got_socket else None
+
+
+def _is_port_free(port):
+    """Check if specified port is free.
+
+    Args:
+        port: int. Port to check.
+
+    Returns:
+        bool. Whether the port is free to use for both TCP and UDP.
+    """
+    return _bind(port, *_PROTOCOLS[0]) and _bind(port, *_PROTOCOLS[1])
+
+
+def _should_allocate_port(pid):
+    """Determine whether to allocate a port for a process id.
+
+    Args:
+        pid: int. The process ID.
+
+    Returns:
+        bool. Whether or not to allocate a port to the process.
+    """
+    if pid <= 0:
+        logging.info('Not allocating a port to invalid pid')
+        return False
+    if pid == 1:
+        # The client probably meant to send us its parent pid but
+        # had been reparented to init.
+        logging.info('Not allocating a port to init.')
+        return False
+    try:
+        os.kill(pid, 0)
+    except OSError:
+        logging.info('Not allocating a port to a non-existent process')
+        return False
+    return True
+
+
+class _PortInfo(python_utils.OBJECT):
+    """Container class for information about a given port assignment.
+
+    Attributes:
+      port: int. Port number.
+      pid: int. Process id or 0 if unassigned.
+      start_time: int. Time in seconds since the epoch that the process
+          started.
+    """
+
+    __slots__ = ('port', 'pid', 'start_time')
+
+    def __init__(self, port):
+        self.port = port
+        self.pid = 0
+        self.start_time = 0
+
+
+class _PortPool(python_utils.OBJECT):
+    """Manage available ports for processes.
+
+    Ports are reclaimed when the reserving process exits and the reserved port
+    is no longer in use.  Only ports which are free for both TCP and UDP will be
+    handed out.  It is easier to not differentiate between protocols.
+    The pool must be pre-seeded with add_port_to_free_pool() calls
+    after which get_port_for_process() will allocate and reclaim ports.
+    The len() of a _PortPool returns the total number of ports being managed.
+
+    Attributes:
+      ports_checked_for_last_request: int. The number of ports examined
+          in order to return from the most recent get_port_for_process()
+          request.  A high number here likely means the number of
+          available ports with no active process using them is getting
+          low.
+    """
+
+    def __init__(self):
+        self._port_queue = collections.deque()
+        self.ports_checked_for_last_request = 0
+
+    def num_ports(self):
+        """Get the number of ports in the pool.
+
+        Returns:
+            int. The number of ports in the pool.
+        """
+        return len(self._port_queue)
+
+    def get_port_for_process(self, pid):
+        """Allocates a port for the given process.
+
+        Args:
+            pid: int. ID for process to allocate port to.
+
+        Returns:
+            int. Allocated port or 0 if none could be allocated.
+        """
+        if not self._port_queue:
+            raise RuntimeError('No ports being managed.')
+
+        # Avoid an infinite loop if all ports are currently assigned.
+        check_count = 0
+        max_ports_to_test = len(self._port_queue)
+        while check_count < max_ports_to_test:
+            # Get the next candidate port and move it to the back of the queue.
+            candidate = self._port_queue.pop()
+            self._port_queue.appendleft(candidate)
+            check_count += 1
+            if (candidate.start_time == 0
+                    or candidate.start_time
+                    != _get_process_start_time(candidate.pid)):
+                if _is_port_free(candidate.port):
+                    candidate.pid = pid
+                    candidate.start_time = _get_process_start_time(pid)
+                    if not candidate.start_time:
+                        logging.info('Can\'t read start time for pid %d.', pid)
+                    self.ports_checked_for_last_request = check_count
+                    return candidate.port
+                else:
+                    logging.info(
+                        'Port %d unexpectedly in use, last owning pid %d.',
+                        candidate.port, candidate.pid)
+
+        logging.info('All ports in use.')
+        self.ports_checked_for_last_request = check_count
+        return 0
+
+    def add_port_to_free_pool(self, port):
+        """Add a new port to the free pool for allocation.
+
+        Args:
+            port: int. The port number to add to the pool.
+        """
+        if port < 1 or port > 65535:
+            raise ValueError(
+                'Port must be in the [1, 65535] range, not %d.' % port)
+        port_info = _PortInfo(port=port)
+        self._port_queue.append(port_info)
+
+
+class _PortServerRequestHandler(python_utils.OBJECT):
+    """A class to handle port allocation and status requests.
+
+    Allocates ports to process ids via the dead simple port server protocol
+    when the handle_port_request asyncio.coroutine handler has been registered.
+    Statistics can be logged using the dump_stats method.
+    """
+
+    def __init__(self, ports_to_serve):
+        """Initialize a new port server.
+
+        Args:
+            ports_to_serve: list(int). A sequence of unique port numbers
+                to test and offer up to clients.
+        """
+        self._port_pool = _PortPool()
+        self._total_allocations = 0
+        self._denied_allocations = 0
+        self._client_request_errors = 0
+        for port in ports_to_serve:
+            self._port_pool.add_port_to_free_pool(port)
+
+    def handle_port_request(self, client_data):
+        """Given a port request body, parse it and respond appropriately.
+
+        Args:
+            client_data: bytes. The request bytes from the client.
+
+        Returns:
+            str. The response to return to the client.
+        """
+        try:
+            pid = int(client_data)
+        except ValueError as error:
+            self._client_request_errors += 1
+            logging.warning('Could not parse request: %s', error)
+            return
+
+        logging.info('Request on behalf of pid %d.', pid)
+        logging.info('cmdline: %s', _get_process_command_line(pid))
+
+        if not _should_allocate_port(pid):
+            self._denied_allocations += 1
+            return
+
+        port = self._port_pool.get_port_for_process(pid)
+        if port > 0:
+            self._total_allocations += 1
+            logging.debug('Allocated port %d to pid %d', port, pid)
+            return '{:d}\n'.format(port).encode(encoding='utf-8')
+        else:
+            self._denied_allocations += 1
+            logging.info('Denied allocation to pid %d', pid)
+            return ''
+
+    def dump_stats(self):
+        """Logs statistics of our operation."""
+        logging.info('Dumping statistics:')
+        stats = []
+        stats.append(
+            'client-request-errors {}'.format(self._client_request_errors))
+        stats.append('denied-allocations {}'.format(self._denied_allocations))
+        stats.append('num-ports-managed {}'.format(self._port_pool.num_ports()))
+        stats.append('num-ports-checked-for-last-request {}'.format(
+            self._port_pool.ports_checked_for_last_request))
+        stats.append('total-allocations {}'.format(self._total_allocations))
+        for stat in stats:
+            logging.info(stat)
+
+
+def _parse_command_line():
+    """Configure and parse our command line flags.
+
+    Returns:
+        Namespace. The parsed arguments.
+    """
+    parser = argparse.ArgumentParser()
+    parser.add_argument(
+        '--portserver_static_pool',
+        type=str,
+        default='15000-24999',
+        help='Comma separated N-P Range(s) of ports to manage (inclusive).')
+    parser.add_argument(
+        '--portserver_unix_socket_address',
+        type=str,
+        default='portserver.sock',
+        help='Address of AF_UNIX socket on which to listen (first @ is a NUL).')
+    return parser.parse_args(args=sys.argv[1:])
+
+
+def _parse_port_ranges(pool_str):
+    """Given a 'N-P,X-Y' description of port ranges, return a set of ints.
+
+    Args:
+        pool_str: str. The N-P,X-Y description of port ranges.
+
+    Returns:
+        set(int). The port numbers in the port ranges.
+    """
+    ports = set()
+    for range_str in pool_str.split(','):
+        try:
+            a, b = range_str.split('-', 1)
+            start, end = int(a), int(b)
+        except ValueError:
+            logging.error('Ignoring unparsable port range %r.', range_str)
+            continue
+        if start < 1 or end > 65535:
+            logging.error('Ignoring out of bounds port range %r.', range_str)
+            continue
+        ports.update(set(python_utils.RANGE(start, end + 1)))
+    return ports
+
+
+class Server(python_utils.OBJECT):
+    """Manages the portserver server.
+
+    Attributes:
+        max_backlog: int. The maximum number of pending requests to hold
+            at a time.
+        message_size: int. Maximum number of bytes to read from each
+            connection to read the request.
+    """
+
+    max_backlog = 5
+    message_size = 1024
+
+    def __init__(self, handler, socket_path):
+        """Runs the portserver
+
+        Args:
+            handler: Callable. Function that accepts a port allocation
+                request string and returns the allocated port number.
+            socket_path: str. Path to socket file.
+        """
+        self.socket_path = socket_path
+        self.socket = self._start_server(self.socket_path)
+        self.handler = handler
+
+    def run(self):
+        """Run the server in an infinite loop.
+
+        Spawns a thread to handle each connection to the socket. Uses
+        the handle_connection function to handle each connection.
+        """
+        while True:
+            connection, _ = self.socket.accept()
+            thread = threading.Thread(
+                target=Server.handle_connection,
+                args=(connection, self.handler),
+            )
+            thread.start()
+
+    def close(self):
+        """Gracefully shut down the server.
+
+        Shutting down the server involves closing the socket and
+        removing the socket file.
+        """
+        try:
+            self.socket.shutdown(socket.SHUT_RDWR)
+        except socket.error:
+            pass
+        finally:
+            self.socket.close()
+        if not self.socket_path.startswith('\0'):
+            os.remove(self.socket_path)
+
+    @staticmethod
+    def handle_connection(connection, handler):
+        """Handle a socket connection.
+
+        Reads the request from the socket connection and passes it to
+        the handler.
+
+        Args:
+            connection: Socket. The connection socket to read the
+                request from.
+            handler: Callable. The handler function that will handle the
+                request. Should accept a string with the PID of the
+                requesting process and return allocated socket.
+        """
+        request = connection.recv(Server.message_size)
+        response = handler(request)
+        connection.sendall(response)
+        connection.close()
+
+    def _start_server(self, path):
+        """Start the server bound to a socket file.
+
+        Args:
+            path: str. Path to socket file. No such file should exist,
+                and a new one will be created.
+
+        Returns:
+            Socket. A new socket object bound to the socket file.
+        """
+        sock = self._get_socket()
+        try:
+            sock.bind(path)
+        except socket.error as err:
+            raise RuntimeError(
+                'Failed to bind socket {}. Error: {}'.format(
+                    path, err)
+            )
+        sock.listen(self.max_backlog)
+        return sock
+
+    def _get_socket(self):
+        """Get a new socket.
+
+        Returns:
+            Socket. A new socket object. If UNIX sockets are supported,
+            such a socket will be created. Otherwise, an AF_INET socket
+            is created.
+        """
+        if hasattr(socket, 'AF_UNIX'):
+            return socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
+        else:
+            # Fallback to AF_INET if this is not unix.
+            return socket.socket(socket.AF_INET, socket.SOCK_STREAM)
+
+
+def main():
+    """Runs the portserver until ctrl-C, then shuts it down."""
+    config = _parse_command_line()
+    ports_to_serve = _parse_port_ranges(config.portserver_static_pool)
+    if not ports_to_serve:
+        logging.error(
+            'No ports. Invalid port ranges in --portserver_static_pool?'
+        )
+        sys.exit(1)
+
+    request_handler = _PortServerRequestHandler(ports_to_serve)
+
+    server = Server(
+        request_handler.handle_port_request,
+        config.portserver_unix_socket_address.replace('@', '\0', 1),
+    )
+    logging.info(
+        'Serving portserver on %s',
+        config.portserver_unix_socket_address
+    )
+    try:
+        server.run()
+    except KeyboardInterrupt:
+        logging.info('Stopping portserver due to ^C.')
+
+    server.close()
+    request_handler.dump_stats()
+    logging.info('Shutting down portserver.')
+
+
+if __name__ == '__main__':
+    main()


Test patch:
diff --git a/scripts/run_e2e_tests.py b/scripts/run_e2e_tests.py
--- a/scripts/run_e2e_tests.py
+++ b/scripts/run_e2e_tests.py
@@ -22,6 +22,7 @@
 import contextlib
 import os
 import re
+import signal
 import subprocess
 import sys
 import time
@@ -37,6 +38,11 @@
 OPPIA_SERVER_PORT = 8181
 PROTRACTOR_BIN_PATH = os.path.join(
     common.NODE_MODULES_PATH, 'protractor', 'bin', 'protractor')
+# Path relative to current working directory where portserver socket
+# file will be created.
+PORTSERVER_SOCKET_FILEPATH = os.path.join(
+    os.getcwd(), 'portserver.socket')
+KILL_PORTSERVER_TIMEOUT_SECS = 10
 
 CONSTANT_FILE_PATH = os.path.join(common.CURR_DIR, 'assets', 'constants.ts')
 FECONF_FILE_PATH = os.path.join('feconf.py')
@@ -436,6 +442,7 @@ def start_google_app_engine_server(dev_mode_setting, log_level):
         '--log_level=%s --skip_sdk_update_check=true %s' % (
             common.CURRENT_PYTHON_BIN, common.GOOGLE_APP_ENGINE_HOME,
             GOOGLE_APP_ENGINE_PORT, log_level, log_level, app_yaml_filepath),
+        env={'PORTSERVER_ADDRESS': PORTSERVER_SOCKET_FILEPATH},
         shell=True)
     SUBPROCESSES.append(p)
 
@@ -468,6 +475,54 @@ def get_chrome_driver_version():
     return chrome_driver_version
 
 
+def start_portserver():
+    """Start a portserver in a subprocess.
+
+    The portserver listens at PORTSERVER_SOCKET_FILEPATH and allocates free
+    ports to clients. This prevents race conditions when two clients
+    request ports in quick succession. The local Google App Engine
+    server that we use to serve the development version of Oppia uses
+    python_portpicker, which is compatible with the portserver this
+    function starts, to request ports.
+
+    By "compatible" we mean that python_portpicker requests a port by
+    sending a request consisting of the PID of the requesting process
+    and expects a response consisting of the allocated port number. This
+    is the interface provided by this portserver.
+
+    Returns:
+        subprocess.Popen. The Popen subprocess object.
+    """
+    process = subprocess.Popen([
+        'python', '-m',
+        '.'.join(['scripts', 'run_portserver']),
+        '--portserver_unix_socket_address',
+        PORTSERVER_SOCKET_FILEPATH,
+    ])
+    return process
+
+
+def cleanup_portserver(portserver_process):
+    """Shut down the portserver.
+
+    We wait KILL_PORTSERVER_TIMEOUT_SECS seconds for the portserver to
+    shut down after sending CTRL-C (SIGINT). The portserver is configured
+    to shut down cleanly upon receiving this signal. If the server fails
+    to shut down, we kill the process.
+
+    Args:
+        portserver_process: subprocess.Popen. The Popen subprocess
+            object for the portserver.
+    """
+    portserver_process.send_signal(signal.SIGINT)
+    for _ in python_utils.RANGE(KILL_PORTSERVER_TIMEOUT_SECS):
+        time.sleep(1)
+        if not portserver_process.poll():
+            break
+    if portserver_process.poll():
+        portserver_process.kill()
+
+
 def main(args=None):
     """Run the scripts to start end-to-end tests."""
 
@@ -494,6 +549,8 @@ def main(args=None):
     python_utils.PRINT('\n\nCHROMEDRIVER VERSION: %s\n\n' % version)
     start_webdriver_manager(version)
 
+    portserver_process = start_portserver()
+    atexit.register(cleanup_portserver, portserver_process)
     start_google_app_engine_server(dev_mode, parsed_args.server_log_level)
 
     wait_for_port_to_be_open(WEB_DRIVER_PORT)
diff --git a/scripts/run_e2e_tests_test.py b/scripts/run_e2e_tests_test.py
--- a/scripts/run_e2e_tests_test.py
+++ b/scripts/run_e2e_tests_test.py
@@ -22,6 +22,7 @@
 import functools
 import os
 import re
+import signal
 import subprocess
 import sys
 import time
@@ -40,15 +41,60 @@
 
 
 class MockProcessClass(python_utils.OBJECT):
-    def __init__(self):
-        pass
 
-    kill_count = 0
+    def __init__(self, clean_shutdown=True):
+        """Create a mock process object.
+
+        Attributes:
+            poll_count: int. The number of times poll() has been called.
+            signals_received: list(int). List of received signals (as
+                ints) in order of receipt.
+            kill_count: int. Number of times kill() has been called.
+            poll_return: bool. The return value for poll().
+            clean_shutdown: bool. Whether to shut down when signal.SIGINT
+                signal is received.
+
+        Args:
+            clean_shutdown: bool. Whether to shut down when SIGINT received.
+        """
+        self.poll_count = 0
+        self.signals_received = []
+        self.kill_count = 0
+        self.poll_return = True
+        self.clean_shutdown = clean_shutdown
 
-    # pylint: disable=missing-docstring
     def kill(self):
-        MockProcessClass.kill_count += 1
-    # pylint: enable=missing-docstring
+        """Increment kill_count.
+
+        Mocks the process being killed.
+        """
+        self.kill_count += 1
+
+    def poll(self):
+        """Increment poll_count.
+
+        Mocks checking whether the process is still alive.
+
+        Returns:
+            bool. The value of self.poll_return, which mocks whether the
+            process is still alive.
+        """
+        self.poll_count += 1
+        return self.poll_return
+
+    def send_signal(self, signal_number):
+        """Append signal to self.signals_received.
+
+        Mocks receiving a process signal. If a SIGINT signal is received
+        (e.g. from ctrl-C) and self.clean_shutdown is True, then we set
+        self.poll_return to False to mimic the process shutting down.
+
+        Args:
+            signal_number: int. The number of the received signal.
+        """
+        self.signals_received.append(signal_number)
+        if signal_number == signal.SIGINT and self.clean_shutdown:
+            self.poll_return = False
 
 
 class RunE2ETestsTests(test_utils.GenericTestBase):
@@ -69,7 +115,7 @@ def mock_check_call(unused_commands):
         def mock_build_main(args):
             pass
 
-        def mock_popen(args, shell):
+        def mock_popen(args, env, shell):
             return
         # pylint: enable=unused-argument
 
@@ -695,7 +741,13 @@ def test_start_google_app_engine_server_in_dev_mode(self):
                 'app_dev.yaml'))
         popen_swap = self.popen_swap(
             expected_args=[(expected_command,)],
-            expected_kwargs=[{'shell': True}])
+            expected_kwargs=[{
+                'env': {
+                    'PORTSERVER_ADDRESS':
+                        run_e2e_tests.PORTSERVER_SOCKET_FILEPATH,
+                },
+                'shell': True,
+            }])
         with popen_swap:
             run_e2e_tests.start_google_app_engine_server(True, 'critical')
 
@@ -710,7 +762,13 @@ def test_start_google_app_engine_server_in_prod_mode(self):
                 'app.yaml'))
         popen_swap = self.popen_swap(
             expected_args=[(expected_command,)],
-            expected_kwargs=[{'shell': True}])
+            expected_kwargs=[{
+                'env': {
+                    'PORTSERVER_ADDRESS':
+                        run_e2e_tests.PORTSERVER_SOCKET_FILEPATH,
+                },
+                'shell': True,
+            }])
         with popen_swap:
             run_e2e_tests.start_google_app_engine_server(False, 'critical')
 
@@ -730,13 +788,15 @@ def mock_is_oppia_server_already_running(*unused_args):
 
     def test_start_tests_when_no_other_instance_running(self):
 
+        mock_process = MockProcessClass()
+
         def mock_is_oppia_server_already_running(*unused_args):
             return False
 
         def mock_setup_and_install_dependencies(unused_arg):
             return
 
-        def mock_register(unused_func):
+        def mock_register(unused_func, unused_arg=None):
             return
 
         def mock_cleanup():
@@ -764,7 +824,7 @@ def mock_get_e2e_test_parameters(
         def mock_popen(unused_commands):
             def mock_communicate():
                 return
-            result = MockProcessClass()
+            result = mock_process
             result.communicate = mock_communicate # pylint: disable=attribute-defined-outside-init
             result.returncode = 0 # pylint: disable=attribute-defined-outside-init
             return result
@@ -788,7 +848,10 @@ def mock_get_chrome_driver_version():
             mock_setup_and_install_dependencies, expected_args=[(False,)])
 
         register_swap = self.swap_with_checks(
-            atexit, 'register', mock_register, expected_args=[(mock_cleanup,)])
+            atexit, 'register', mock_register, expected_args=[
+                (mock_cleanup,),
+                (run_e2e_tests.cleanup_portserver, mock_process),
+            ])
 
         cleanup_swap = self.swap(run_e2e_tests, 'cleanup', mock_cleanup)
         build_swap = self.swap_with_checks(
@@ -815,10 +878,21 @@ def mock_get_chrome_driver_version():
             run_e2e_tests, 'get_e2e_test_parameters',
             mock_get_e2e_test_parameters, expected_args=[(3, 'full', True)])
         popen_swap = self.swap_with_checks(
-            subprocess, 'Popen', mock_popen, expected_args=[([
-                common.NODE_BIN_PATH, '--unhandled-rejections=strict',
-                run_e2e_tests.PROTRACTOR_BIN_PATH,
-                'commands'],)])
+            subprocess, 'Popen', mock_popen, expected_args=[
+                ([
+                    'python', '-m',
+                    'scripts.run_portserver',
+                    '--portserver_unix_socket_address',
+                    run_e2e_tests.PORTSERVER_SOCKET_FILEPATH,
+                ],),
+                ([
+                    common.NODE_BIN_PATH,
+                    '--unhandled-rejections=strict',
+                    run_e2e_tests.PROTRACTOR_BIN_PATH,
+                    'commands',
+                ],),
+            ],
+        )
         exit_swap = self.swap_with_checks(
             sys, 'exit', mock_exit, expected_args=[(0,)])
         with check_swap, setup_and_install_swap, register_swap, cleanup_swap:
@@ -831,13 +905,15 @@ def mock_get_chrome_driver_version():
 
     def test_start_tests_skip_build(self):
 
+        mock_process = MockProcessClass()
+
         def mock_is_oppia_server_already_running(*unused_args):
             return False
 
         def mock_setup_and_install_dependencies(unused_arg):
             return
 
-        def mock_register(unused_func):
+        def mock_register(unused_func, unused_arg=None):
             return
 
         def mock_cleanup():
@@ -865,7 +941,7 @@ def mock_get_e2e_test_parameters(
         def mock_popen(unused_commands):
             def mock_communicate():
                 return
-            result = MockProcessClass()
+            result = mock_process
             result.communicate = mock_communicate # pylint: disable=attribute-defined-outside-init
             result.returncode = 0 # pylint: disable=attribute-defined-outside-init
             return result
@@ -887,7 +963,10 @@ def mock_get_chrome_driver_version():
             run_e2e_tests, 'setup_and_install_dependencies',
             mock_setup_and_install_dependencies, expected_args=[(True,)])
         register_swap = self.swap_with_checks(
-            atexit, 'register', mock_register, expected_args=[(mock_cleanup,)])
+            atexit, 'register', mock_register, expected_args=[
+                (mock_cleanup,),
+                (run_e2e_tests.cleanup_portserver, mock_process),
+            ])
         cleanup_swap = self.swap(run_e2e_tests, 'cleanup', mock_cleanup)
         modify_constants_swap = self.swap_with_checks(
             build, 'modify_constants', mock_modify_constants,
@@ -913,10 +992,21 @@ def mock_get_chrome_driver_version():
             run_e2e_tests, 'get_e2e_test_parameters',
             mock_get_e2e_test_parameters, expected_args=[(3, 'full', True)])
         popen_swap = self.swap_with_checks(
-            subprocess, 'Popen', mock_popen, expected_args=[([
-                common.NODE_BIN_PATH, '--unhandled-rejections=strict',
-                run_e2e_tests.PROTRACTOR_BIN_PATH,
-                'commands'],)])
+            subprocess, 'Popen', mock_popen, expected_args=[
+                ([
+                    'python', '-m',
+                    'scripts.run_portserver',
+                    '--portserver_unix_socket_address',
+                    run_e2e_tests.PORTSERVER_SOCKET_FILEPATH,
+                ],),
+                ([
+                    common.NODE_BIN_PATH,
+                    '--unhandled-rejections=strict',
+                    run_e2e_tests.PROTRACTOR_BIN_PATH,
+                    'commands'
+                ],),
+            ],
+        )
         exit_swap = self.swap_with_checks(
             sys, 'exit', mock_exit, expected_args=[(0,)])
         with check_swap, setup_and_install_swap, register_swap, cleanup_swap:
@@ -950,13 +1040,15 @@ def mock_popen(unused_commands, stdout):
 
     def test_start_tests_in_debug_mode(self):
 
+        mock_process = MockProcessClass()
+
         def mock_is_oppia_server_already_running(*unused_args):
             return False
 
         def mock_setup_and_install_dependencies(unused_arg):
             return
 
-        def mock_register(unused_func):
+        def mock_register(unused_func, unused_arg=None):
             return
 
         def mock_cleanup():
@@ -984,7 +1076,7 @@ def mock_get_e2e_test_parameters(
         def mock_popen(unused_commands):
             def mock_communicate():
                 return
-            result = MockProcessClass()
+            result = mock_process
             result.communicate = mock_communicate # pylint: disable=attribute-defined-outside-init
             result.returncode = 0 # pylint: disable=attribute-defined-outside-init
             return result
@@ -1008,7 +1100,10 @@ def mock_get_chrome_driver_version():
             mock_setup_and_install_dependencies, expected_args=[(False,)])
 
         register_swap = self.swap_with_checks(
-            atexit, 'register', mock_register, expected_args=[(mock_cleanup,)])
+            atexit, 'register', mock_register, expected_args=[
+                (mock_cleanup,),
+                (run_e2e_tests.cleanup_portserver, mock_process),
+            ])
 
         cleanup_swap = self.swap(run_e2e_tests, 'cleanup', mock_cleanup)
         build_swap = self.swap_with_checks(
@@ -1035,10 +1130,22 @@ def mock_get_chrome_driver_version():
             run_e2e_tests, 'get_e2e_test_parameters',
             mock_get_e2e_test_parameters, expected_args=[(3, 'full', True)])
         popen_swap = self.swap_with_checks(
-            subprocess, 'Popen', mock_popen, expected_args=[([
-                common.NODE_BIN_PATH, '--inspect-brk',
-                '--unhandled-rejections=strict',
-                run_e2e_tests.PROTRACTOR_BIN_PATH, 'commands'],)])
+            subprocess, 'Popen', mock_popen, expected_args=[
+                ([
+                    'python', '-m',
+                    'scripts.run_portserver',
+                    '--portserver_unix_socket_address',
+                    run_e2e_tests.PORTSERVER_SOCKET_FILEPATH,
+                ],),
+                ([
+                    common.NODE_BIN_PATH,
+                    '--inspect-brk',
+                    '--unhandled-rejections=strict',
+                    run_e2e_tests.PROTRACTOR_BIN_PATH,
+                    'commands',
+                ],),
+            ],
+        )
         exit_swap = self.swap_with_checks(
             sys, 'exit', mock_exit, expected_args=[(0,)])
         with check_swap, setup_and_install_swap, register_swap, cleanup_swap:
@@ -1051,13 +1158,15 @@ def mock_get_chrome_driver_version():
 
     def test_start_tests_in_with_chromedriver_flag(self):
 
+        mock_process = MockProcessClass()
+
         def mock_is_oppia_server_already_running(*unused_args):
             return False
 
         def mock_setup_and_install_dependencies(unused_arg):
             return
 
-        def mock_register(unused_func):
+        def mock_register(unused_func, unused_arg=None):
             return
 
         def mock_cleanup():
@@ -1085,7 +1194,7 @@ def mock_get_e2e_test_parameters(
         def mock_popen(unused_commands):
             def mock_communicate():
                 return
-            result = MockProcessClass()
+            result = mock_process
             result.communicate = mock_communicate # pylint: disable=attribute-defined-outside-init
             result.returncode = 0 # pylint: disable=attribute-defined-outside-init
             return result
@@ -1109,7 +1218,10 @@ def mock_get_chrome_driver_version():
             mock_setup_and_install_dependencies, expected_args=[(False,)])
 
         register_swap = self.swap_with_checks(
-            atexit, 'register', mock_register, expected_args=[(mock_cleanup,)])
+            atexit, 'register', mock_register, expected_args=[
+                (mock_cleanup,),
+                (run_e2e_tests.cleanup_portserver, mock_process),
+            ])
 
         cleanup_swap = self.swap(run_e2e_tests, 'cleanup', mock_cleanup)
         build_swap = self.swap_with_checks(
@@ -1136,9 +1248,21 @@ def mock_get_chrome_driver_version():
             run_e2e_tests, 'get_e2e_test_parameters',
             mock_get_e2e_test_parameters, expected_args=[(3, 'full', True)])
         popen_swap = self.swap_with_checks(
-            subprocess, 'Popen', mock_popen, expected_args=[([
-                common.NODE_BIN_PATH, '--unhandled-rejections=strict',
-                run_e2e_tests.PROTRACTOR_BIN_PATH, 'commands'],)])
+            subprocess, 'Popen', mock_popen, expected_args=[
+                ([
+                    'python', '-m',
+                    'scripts.run_portserver',
+                    '--portserver_unix_socket_address',
+                    run_e2e_tests.PORTSERVER_SOCKET_FILEPATH,
+                ],),
+                ([
+                    common.NODE_BIN_PATH,
+                    '--unhandled-rejections=strict',
+                    run_e2e_tests.PROTRACTOR_BIN_PATH,
+                    'commands',
+                ],),
+            ],
+        )
         exit_swap = self.swap_with_checks(
             sys, 'exit', mock_exit, expected_args=[(0,)])
         with check_swap, setup_and_install_swap, register_swap, cleanup_swap:
@@ -1172,3 +1296,26 @@ def test_update_community_dashboard_status_with_dashboard_disabled(self):
         with swap_inplace_replace:
             run_e2e_tests.update_community_dashboard_status_in_feconf_file(
                 run_e2e_tests.FECONF_FILE_PATH, False)
+
+    def test_cleanup_portserver_when_server_shuts_down_cleanly(self):
+        process = MockProcessClass(clean_shutdown=True)
+        run_e2e_tests.cleanup_portserver(process)
+        self.assertEqual(process.kill_count, 0)
+        # Server gets polled twice. Once to break out of wait loop and
+        # again to check that the process shut down and does not need to
+        # be killed.
+        self.assertEqual(process.poll_count, 2)
+        self.assertEqual(process.signals_received, [signal.SIGINT])
+
+    def test_cleanup_portserver_when_server_shutdown_fails(self):
+        process = MockProcessClass(clean_shutdown=False)
+        run_e2e_tests.cleanup_portserver(process)
+        self.assertEqual(process.kill_count, 1)
+        # Server gets polled 11 times. 1 for each second of the wait
+        # loop and again to see that the process did not shut down and
+        # therefore needs to be killed.
+        self.assertEqual(
+            process.poll_count,
+            run_e2e_tests.KILL_PORTSERVER_TIMEOUT_SECS + 1
+        )
+        self.assertEqual(process.signals_received, [signal.SIGINT])