Repository: actions/runner

Issue #3393: GitHub Actions Runner: 'Object reference not set to an instance of an object' Error"
Created: 2024-07-17T09:12:58Z
Describe the bug
The GitHub Actions runner application repeatedly encounters an error stating "An error occurred: Object reference not set to an instance of an object." This issue persists even after restarting the service. There are no clear indicators in the system logs that point to the cause of this error. This is the first time this error has been encountered.
To Reproduce
Steps to reproduce the behavior:

Start the GitHub Actions runner service on the machine.
Observe the runner's log output.
The error "An error occurred: Object reference not set to an instance of an object" appears shortly after the service starts.

Expected behavior
The GitHub Actions runner should start without encountering a null reference exception and be ready to execute workflows without errors.
Runner Version and Platform
Version of your runner: 2.317.0
OS of the machine running the runner: Linux (ubuntu)
What's not working?
The runner fails to operate normally due to a recurring error message: "An error occurred: Object reference not set to an instance of an object." This prevents the runner from executing any workflows.
● actions.runner.puk.UKLENOAPIPI001.service - GitHub Actions Runner (puk.UKLENOAPIPI001)
Loaded: loaded (/etc/systemd/system/actions.runner.puk.UKLENOAPIPI001.service; enabled; vendor preset: enabled)
Active: active (running) since Wed 2024-07-17 08:48:58 UTC; 13min ago
Main PID: 2129042 (runsvc.sh)
Tasks: 8 (limit: 19044)
Memory: 8.7M
CPU: 1min 39.466s
CGroup: /system.slice/actions.runner.puk.UKLENOAPIPI001.service
├─2129042 /bin/bash /home/infraeno/actions-runner/runsvc.sh
└─2129044 ./externals/node16/bin/node ./bin/RunnerService.js
Jul 17 08:56:09 UKLENOAPIPI001 runsvc.sh[2129044]: Runner listener exited with error code 2
Jul 17 08:56:09 UKLENOAPIPI001 runsvc.sh[2129044]: Runner listener exit with retryable error, re-launch runner in 5 seconds.
Jul 17 08:56:14 UKLENOAPIPI001 runsvc.sh[2129044]: Starting Runner listener with startup type: service
Jul 17 08:56:14 UKLENOAPIPI001 runsvc.sh[2129044]: Started listener process, pid: 2130576
Jul 17 08:56:15 UKLENOAPIPI001 runsvc.sh[2129044]: √ Connected to GitHub
Jul 17 08:56:16 UKLENOAPIPI001 runsvc.sh[2129044]: Current runner version: '2.317.0'
Jul 17 08:56:16 UKLENOAPIPI001 runsvc.sh[2129044]: 2024-07-17 08:56:16Z: Listening for Jobs
Jul 17 09:02:35 UKLENOAPIPI001 runsvc.sh[2129044]: An error occurred: Object reference not set to an instance of an object.
Jul 17 09:02:35 UKLENOAPIPI001 runsvc.sh[2129044]: Runner listener exited with error code 2
Jul 17 09:02:35 UKLENOAPIPI001 runsvc.sh[2129044]: Runner listener exit with retryable error, re-launch runner in 5 seconds.
Job Log Output
Not applicable, as the error occurs at the service startup, before any job execution.
Runner and Worker's Diagnostic Logs
Diagnostic logs from the _diag folder might provide more insight, but specific log entries related to the error have not been provided. Users experiencing this issue are encouraged to check their runner's _diag folder for logs prefixed with Runner_ and Worker_ for more details and include relevant excerpts when reporting the issue.

Duplicate issue #3371: Runner stuck at `BrokerMigration message received. Polling Broker for messages...`
Created: 2024-07-04T11:53:18Z
Describe the bug
As of ~last night (4th of July CEST) self hosted runners do not pick up any new jobs. Runners do emit a message ~every minute:
Runner stuck at `BrokerMigration message received. Polling Broker for messages...`

This is similar to what's reported here: ChristopherHX/github-act-runner#186 (comment).
Expected behavior
Runners should pick up jobs
Runner Version and Platform
Runner
v2.317.0

OS:
Distributor ID:	Ubuntu
Description:	Ubuntu 22.04.4 LTS
Release:	22.04
Codename:	jammy

Runner and Worker's Diagnostic Logs
[2024-07-04 11:43:31Z INFO GitHubActionsService] AAD Correlation ID for this token request: Unknown
[2024-07-04 11:43:31Z INFO MessageListener] Session created.
[2024-07-04 11:43:31Z INFO Terminal] WRITE LINE: Current runner version: '2.317.0'
[2024-07-04 11:43:31Z INFO Terminal] WRITE LINE: 2024-07-04 11:43:31Z: Listening for Jobs
[2024-07-04 11:43:31Z INFO JobDispatcher] Set runner/worker IPC timeout to 30 seconds.
[2024-07-04 11:43:31Z INFO MessageListener] BrokerMigration message received. Polling Broker for messages...
[2024-07-04 11:43:31Z INFO RSAFileKeyManager] Loading RSA key parameters from file /home/bittide/Downloads/actions-runner-1/.credentials_rsaparams
[2024-07-04 11:43:31Z INFO GitHubActionsService] AAD Correlation ID for this token request: Unknown
[2024-07-04 11:44:22Z INFO MessageListener] BrokerMigration message received. Polling Broker for messages...
[2024-07-04 11:45:12Z INFO MessageListener] BrokerMigration message received. Polling Broker for messages...
[2024-07-04 11:46:02Z INFO MessageListener] BrokerMigration message received. Polling Broker for messages...
[2024-07-04 11:46:53Z INFO MessageListener] BrokerMigration message received. Polling Broker for messages...
[2024-07-04 11:47:43Z INFO MessageListener] BrokerMigration message received. Polling Broker for messages...
[2024-07-04 11:48:33Z INFO MessageListener] BrokerMigration message received. Polling Broker for messages...
[2024-07-04 11:49:23Z INFO MessageListener] BrokerMigration message received. Polling Broker for messages...
[2024-07-04 11:50:13Z INFO MessageListener] BrokerMigration message received. Polling Broker for messages...
[2024-07-04 11:51:04Z INFO MessageListener] BrokerMigration message received. Polling Broker for messages...

Question
Is there anything we can do to get more debug information out of the runners?

Closing comment (2024-07-18T08:07:33Z): Actually, I think this particular problem might be related to #3393. I went through all the logs again, and we hit the same error as in #3393 on our runners:
[2024-07-17 12:59:03Z INFO GitHubActionsService] Starting operation Location.GetConnectionData
[2024-07-17 12:59:03Z INFO GitHubActionsService] Finished operation Location.GetConnectionData
[2024-07-17 12:59:03Z INFO GitHubActionsService] Starting operation Location.GetConnectionData
[2024-07-17 12:59:03Z INFO GitHubActionsService] Finished operation Location.GetConnectionData
[2024-07-17 12:59:03Z INFO RSAEncryptedFileKeyManager] Loading RSA key parameters from file C:\Buildserver\actions-runner\.credentials_rsaparams
[2024-07-17 12:59:03Z INFO RSAEncryptedFileKeyManager] Loading RSA key parameters from file C:\Buildserver\actions-runner\.credentials_rsaparams
[2024-07-17 12:59:03Z INFO GitHubActionsService] AAD Correlation ID for this token request: Unknown
[2024-07-17 12:59:04Z INFO JobDispatcher] Shutting down JobDispatcher. Make sure all WorkerDispatcher has finished.
[2024-07-17 12:59:04Z INFO Runner] Deleting Runner Session...
[2024-07-17 12:59:05Z ERR  Terminal] WRITE ERROR: An error occurred: Object reference not set to an instance of an object.
[2024-07-17 12:59:05Z ERR  Listener] System.NullReferenceException: Object reference not set to an instance of an object.
   at GitHub.Runner.Listener.JobDispatcher.Run(AgentJobRequestMessage jobRequestMessage, Boolean runOnce)
   at GitHub.Runner.Listener.Runner.RunAsync(RunnerSettings settings, Boolean runOnce)
   at GitHub.Runner.Listener.Runner.RunAsync(RunnerSettings settings, Boolean runOnce)
   at GitHub.Runner.Listener.Runner.RunAsync(RunnerSettings settings, Boolean runOnce)
   at GitHub.Runner.Listener.Runner.ExecuteCommand(CommandSettings command)
   at GitHub.Runner.Listener.Program.MainAsync(IHostContext context, String[] args)

After hitting these errors, the runner was effectively dead. Removing and redeploying the runner seems to have resolved the problem.