Bug: Intermittent services/sync/tests/unit/test_errorhandler_1.js | single tracking bug
Status: REOPENED
Priority: P5
Created: 2023-11-10 07:37:35+00:00
Resolved: 2025-03-03 06:50:47+00:00

**Filed by:** ctuns [at] mozilla.com
**Parsed log:** https://treeherder.mozilla.org/logviewer?job_id=435742331&repo=mozilla-central
**Full log:** https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/L8l5uXnoQBSZHa7DLjSgHw/runs/0/artifacts/public/logs/live_backing.log

---
```
[task 2023-11-10T07:14:10.495Z] 07:14:10     INFO -  TEST-PASS | services/sync/tests/unit/test_errorhandler_1.js | test_sync_non_network_error - [test_sync_non_network_error : 212] true == true
[task 2023-11-10T07:14:10.495Z] 07:14:10     INFO -  PID 15816 | 1699600450313	Sync.BulkKeyBundle	INFO	BulkKeyBundle being created for crypto
[task 2023-11-10T07:14:10.495Z] 07:14:10     INFO -  PID 15816 | 1699600450313	Sync.LogManager	TRACE	finished copy to: /var/folders/wz/qq1sb2011z75sg933d5njjr8000014/T/xpc-profile-ulb14ov_/weave/logs/error-sync-1699600450313.txt
[task 2023-11-10T07:14:10.495Z] 07:14:10     INFO -  PID 15816 | 1699600450313	Sync.LogManager	TRACE	onCopyComplete: 1699600450313
[task 2023-11-10T07:14:10.495Z] 07:14:10     INFO -  PID 15816 | 1699600450313	Sync.LogManager	TRACE	Running cleanup.
[task 2023-11-10T07:14:10.495Z] 07:14:10     INFO -  PID 15816 | 1699600450313	Sync.LogManager	DEBUG	Log cleanup threshold time: 1698736450313
[task 2023-11-10T07:14:10.495Z] 07:14:10     INFO -  PID 15816 | 1699600450314	Sync.SyncAuthManager	TRACE	_ensureValidToken already has one
[task 2023-11-10T07:14:10.495Z] 07:14:10     INFO -  PID 15816 | 1699600450314	Sync.Resource	TRACE	HTTP Header authorization: ***** (suppressed)
[task 2023-11-10T07:14:10.495Z] 07:14:10     INFO -  PID 15816 | 1699600450314	Sync.Resource	TRACE	HTTP Header content-type: text/plain
[task 2023-11-10T07:14:10.496Z] 07:14:10     INFO -  PID 15816 | 1699600450314	Sync.Resource	TRACE	HTTP Header user-agent: XPCShell/1 (Intel Mac OS X 10.15) FxSync/1.123.0.20160315.desktop
[task 2023-11-10T07:14:10.496Z] 07:14:10     INFO -  PID 15816 | 1699600450314	Sync.Resource	DEBUG	PUT Length: 377
[task 2023-11-10T07:14:10.496Z] 07:14:10     INFO -  PID 15816 | 1699600450314	Sync.Resource	TRACE	PUT Body: {"payload":"{\\"ciphertext\\":\\"0Q5DPE1aOXb/0lt0mO5l7ZV7FB7jGfpx8ZMJy/pnd5w5wI+NPO3XgPUYfoSEmajOV8L0ZTPBlPEuojI+98IgdfRZYl5UA2YN+hgVNz6/YAoQWxOUcuEkg1LRCL06m6cfc23UQf1F4p2ukUzSDC3IpHeCK8BAH6bxfn50oakx1xgdVa8XfkJ8nxbzP2BdSTPlhc4wDdnE2rzZQ/c5Glm0gA==\\",\\"IV\\":\\"g/POCy83fpiuYlKTATzw2g==\\",\\"hmac\\":\\"1f2240c6338baf9876806eb723636bd462ad15d86eace8cef297c9e9da7d885b\\"}","id":"keys"}
[task 2023-11-10T07:14:10.496Z] 07:14:10     INFO -  PID 15816 | 1699600450314	Sync.LogManager	DEBUG	Done deleting files.
[task 2023-11-10T07:14:10.496Z] 07:14:10     INFO -  PID 15816 | console.error: "Sync encountered an error - see about:sync-log for the log file."
[task 2023-11-10T07:14:10.496Z] 07:14:10     INFO -  PID 15816 | Updating collection crypto to 1699600450.31
[task 2023-11-10T07:14:10.496Z] 07:14:10     INFO -  PID 15816 | 1699600450314	Sync.Resource	DEBUG	PUT success 200 http://localhost:57129/1.1/johndoe/storage/crypto/keys
[task 2023-11-10T07:14:10.496Z] 07:14:10     INFO -  PID 15816 | 1699600450314	Sync.Resource	TRACE	PUT body: 1699600450.31
[task 2023-11-10T07:14:10.496Z] 07:14:10     INFO -  PID 15816 | 1699600450314	Sync.Service	DEBUG	User-Agent: XPCShell/1 (Intel Mac OS X 10.15) FxSync/1.123.0.20160315.desktop
[task 2023-11-10T07:14:10.496Z] 07:14:10     INFO -  PID 15816 | 1699600450314	Sync.Service	INFO	Starting sync at 2023-11-10 07:14:10 in browser session RbolX8oBmQP6
[task 2023-11-10T07:14:10.496Z] 07:14:10     INFO -  PID 15816 | 1699600450314	Sync.Service	TRACE	In sync: no need to login.
[task 2023-11-10T07:14:10.496Z] 07:14:10     INFO -  PID 15816 | 1699600450314	Sync.Service	TRACE	Event: weave:service:sync:start
[task 2023-11-10T07:14:10.496Z] 07:14:10     INFO -  PID 15816 | 1699600450315	Sync.SyncScheduler	TRACE	Handling weave:service:sync:start
[task 2023-11-10T07:14:10.496Z] 07:14:10     INFO -  PID 15816 | 1699600450315	Sync.SyncScheduler	DEBUG	Clearing sync triggers and the global score.
[task 2023-11-10T07:14:10.497Z] 07:14:10     INFO -  PID 15816 | 1699600450315	Toolkit.Telemetry	TRACE	TelemetryEnvironment::observe - aTopic: nsPref:changed, aData: services.sync.nextSync
[task 2023-11-10T07:14:10.497Z] 07:14:10     INFO -  PID 15816 | 1699600450315	Sync.Telemetry	TRACE	observed weave:service:sync:start {}
[task 2023-11-10T07:14:10.497Z] 07:14:10     INFO -  PID 15816 | 1699600450315	Sync.Status	INFO	Resetting Status.
[task 2023-11-10T07:14:10.497Z] 07:14:10     INFO -  PID 15816 | 1699600450315	Sync.Status	DEBUG	Status.service: success.status_ok => success.status_ok
[task 2023-11-10T07:14:10.497Z] 07:14:10     INFO -  PID 15816 | 1699600450315	Sync.Service	TRACE	In _fetchInfo: http://localhost:57129/1.1/johndoe/info/collections
[task 2023-11-10T07:14:10.497Z] 07:14:10     INFO -  PID 15816 | 1699600450315	Sync.SyncAuthManager	TRACE	_ensureValidToken already has one
[task 2023-11-10T07:14:10.497Z] 07:14:10     INFO -  PID 15816 | 1699600450315	Sync.Resource	TRACE	HTTP Header authorization: ***** (suppressed)
[task 2023-11-10T07:14:10.497Z] 07:14:10     INFO -  PID 15816 | 1699600450315	Sync.Resource	TRACE	HTTP Header user-agent: XPCShell/1 (Intel Mac OS X 10.15) FxSync/1.123.0.20160315.desktop
[task 2023-11-10T07:14:10.497Z] 07:14:10     INFO -  PID 15816 | 1699600450315	Sync.Resource	DEBUG	GET success 200 http://localhost:57129/1.1/johndoe/info/collections
[task 2023-11-10T07:14:10.497Z] 07:14:10     INFO -  PID 15816 | 1699600450315	Sync.Resource	TRACE	GET body: {"crypto":1699600450.31,"clients":1699600450.31}
[task 2023-11-10T07:14:10.497Z] 07:14:10     INFO -  PID 15816 | 1699600450315	Sync.Service	DEBUG	Fetching server configuration: http://localhost:57129/1.1/johndoe/info/configuration
[task 2023-11-10T07:14:10.497Z] 07:14:10     INFO -  PID 15816 | 1699600450315	Sync.SyncAuthManager	TRACE	_ensureValidToken already has one
[task 2023-11-10T07:14:10.497Z] 07:14:10     INFO -  PID 15816 | 1699600450316	Sync.Resource	TRACE	HTTP Header authorization: ***** (suppressed)
[task 2023-11-10T07:14:10.497Z] 07:14:10     INFO -  PID 15816 | 1699600450316	Sync.Resource	TRACE	HTTP Header user-agent: XPCShell/1 (Intel Mac OS X 10.15) FxSync/1.123.0.20160315.desktop
[task 2023-11-10T07:14:10.497Z] 07:14:10     INFO -  PID 15816 | 1699600450316	Sync.Resource	DEBUG	GET fail 404 http://localhost:57129/1.1/johndoe/info/configuration
[task 2023-11-10T07:14:10.498Z] 07:14:10     INFO -  PID 15816 | 1699600450316	Sync.Resource	TRACE	GET body: <html>                    <head><title>404 Not Found</title></head>                    <body>                      <h1>404 Not Found</h1>                      <p>                        <span style='font-family: monospace;'>&#47;&#49;&#46;&#49;&#47;&#106;&#111;&#104;&#110;&#100;&#111;&#101;&#47;&#105;&#110;&#102;&#111;&#47;&#99;&#111;&#110;&#102;&#105;&#103;&#117;&#114;&#97;&#116;&#105;&#111;&#110;</span> was not found.                      </p>                    </body>                  </html>
[task 2023-11-10T07:14:10.498Z] 07:14:10     INFO -  PID 15816 | 1699600450316	Sync.Resource	WARN	GET request to http://localhost:57129/1.1/johndoe/info/configuration failed with status 404
[task 2023-11-10T07:14:10.498Z] 07:14:10     INFO -  PID 15816 | 1699600450316	Sync.Service	DEBUG	info/configuration returned 404 - using default upload semantics
[task 2023-11-10T07:14:10.498Z] 07:14:10     INFO -  PID 15816 | 1699600450316	Sync.Service	TRACE	info/configuration for this server
[task 2023-11-10T07:14:10.498Z] 07:14:10     INFO -  PID 15816 | 1699600450316	Sync.Service	DEBUG	Fetching global metadata record
[task 2023-11-10T07:14:10.498Z] 07:14:10     INFO -  PID 15816 | 1699600450316	Sync.Service	DEBUG	Weave Version: 1.123.0 Local Storage: 5 Remote Storage: 5
[task 2023-11-10T07:14:10.498Z] 07:14:10     INFO -  PID 15816 | 1699600450316	Sync.Service	DEBUG	Fetching and verifying -- or generating -- symmetric keys.
[task 2023-11-10T07:14:10.498Z] 07:14:10     INFO -  PID 15816 | 1699600450316	Sync.Service	INFO	Testing info/collections: {"crypto":1699600450.31,"clients":1699600450.31}
[task 2023-11-10T07:14:10.498Z] 07:14:10     INFO -  PID 15816 | 1699600450316	Sync.CollectionKeyManager	INFO	Testing for updateNeeded. Last modified: 1699600450.31
[task 2023-11-10T07:14:10.498Z] 07:14:10     INFO -  PID 15816 | 1699600450316	Sync.Synchronizer	DEBUG	Refreshing client list.
[task 2023-11-10T07:14:10.498Z] 07:14:10     INFO -  PID 15816 | 1699600450316	Sync.Engine.Clients	TRACE	Event: weave:engine:sync:start
[task 2023-11-10T07:14:10.499Z] 07:14:10     INFO -  PID 15816 | 1699600450316	Sync.Telemetry	TRACE	observed weave:engine:sync:start clients
[task 2023-11-10T07:14:10.499Z] 07:14:10     INFO -  PID 15816 | 1699600450316	Sync.Engine.Clients.Tracker	TRACE	Clearing changed ID list
[task 2023-11-10T07:14:10.499Z] 07:14:10     INFO -  PID 15816 | 1699600450316	Sync.Engine.Clients	DEBUG	Resetting clients last sync time
[task 2023-11-10T07:14:10.499Z] 07:14:10     INFO -  PID 15816 | 1699600450316	Toolkit.Telemetry	TRACE	TelemetryEnvironment::observe - aTopic: nsPref:changed, aData: services.sync.clients.lastSync
[task 2023-11-10T07:14:10.499Z] 07:14:10     INFO -  PID 15816 | 1699600450316	Sync.Engine.Clients	TRACE	Downloading & applying server changes
[task 2023-11-10T07:14:10.499Z] 07:14:10     INFO -  PID 15816 | 1699600450316	Sync.Collection	TRACE	Performing batched GET: {"limit":1000,"offset":null}
[task 2023-11-10T07:14:10.499Z] 07:14:10     INFO -  PID 15816 | 1699600450317	Sync.SyncAuthManager	TRACE	_ensureValidToken already has one
[task 2023-11-10T07:14:10.499Z] 07:14:10     INFO -  PID 15816 | 1699600450317	Sync.Collection	TRACE	HTTP Header authorization: ***** (suppressed)
[task 2023-11-10T07:14:10.499Z] 07:14:10     INFO -  PID 15816 | 1699600450317	Sync.Collection	TRACE	HTTP Header user-agent: XPCShell/1 (Intel Mac OS X 10.15) FxSync/1.123.0.20160315.desktop
[task 2023-11-10T07:14:10.499Z] 07:14:10     INFO -  PID 15816 | 1699600450317	Sync.Test.Server	INFO	Records: 1, nextOffset: undefined
[task 2023-11-10T07:14:10.499Z] 07:14:10     INFO -  PID 15816 | 1699600450317	Sync.Collection	DEBUG	GET success 200 http://localhost:57129/1.1/johndoe/storage/clients?full=1&limit=1000
[task 2023-11-10T07:14:10.499Z] 07:14:10     INFO -  PID 15816 | 1699600450317	Sync.Collection	TRACE	GET body: [{"id":"LzO-Yru8yO9P","modified":1699600450.31,"payload":"{\\"ciphertext\\":\\"Vt2SMgYLX0tJiOZeQzoSNfH/6ublRoZDYAzLnxQPkYGGA5JTxXBnhWjFWDTSRGBhLiDM2fJaKpS89tLuEUbWAyxULqB2oDaGGw3mzSvDW91rdLoV3E50xWNgRZoCTFtM4eYenZu1BkfeCVW1ATk2OwO1UFv+VwpRGul/OypQRwk/Rq8dyfiBT/VdiyCDEUBrz13rol17BqSDireW/7BipW2piuaIP88I6PTjWY3u12AYVyGPf1AmAN+Gx78mgqrA/TFkaATiReUZozGvqzrZkvB3n4oOlyctodxmnBOBPm4=\\",\\"IV\\":\\"sDSJEs/kFi1DUk2gOnRhAw==\\",\\"hmac\\":\\"44642051eabd0306ab87d0fc917b9620b09d3c078dbd5105020a0c8cd2e0fca0\\"}"}]
[task 2023-11-10T07:14:10.500Z] 07:14:10     INFO -  PID 15816 | 1699600450318	Sync.Engine.Clients	TRACE	Skipping reconciled incoming item LzO-Yru8yO9P
[task 2023-11-10T07:14:10.500Z] 07:14:10     INFO -  PID 15816 | 1699600450318	Toolkit.Telemetry	TRACE	TelemetryEnvironment::observe - aTopic: nsPref:changed, aData: services.sync.clients.lastSync
[task 2023-11-10T07:14:10.500Z] 07:14:10     INFO -  PID 15816 | 1699600450318	Sync.Engine.Clients	INFO	Records: 0 applied, 0 successfully, 0 failed to apply, 0 newly failed to apply, 1 reconciled.
[task 2023-11-10T07:14:10.500Z] 07:14:10     INFO -  PID 15816 | 1699600450318	Sync.ErrorHandler	TRACE	Handling weave:engine:sync:applied
[task 2023-11-10T07:14:10.500Z] 07:14:10     INFO -  PID 15816 | 1699600450318	Sync.SyncScheduler	TRACE	Handling weave:engine:sync:applied
[task 2023-11-10T07:14:10.500Z] 07:14:10     INFO -  PID 15816 | 1699600450318	Sync.SyncScheduler	TRACE	Engine clients successfully applied 0 items.
[task 2023-11-10T07:14:10.500Z] 07:14:10     INFO -  PID 15816 | 1699600450318	Sync.Telemetry	TRACE	observed weave:engine:sync:applied clients
[task 2023-11-10T07:14:10.500Z] 07:14:10     INFO -  PID 15816 | 1699600450318	Sync.Engine.Clients	DEBUG	Refreshing the known stale clients list
[task 2023-11-10T07:14:10.500Z] 07:14:10     INFO -  PID 15816 | 1699600450318	Sync.Engine.Clients	TRACE	Moving /var/folders/wz/qq1sb2011z75sg933d5njjr8000014/T/xpc-profile-ulb14ov_/weave/commands.json to /var/folders/wz/qq1sb2011z75sg933d5njjr8000014/T/xpc-profile-ulb14ov_/weave/commands-syncing.json
[task 2023-11-10T07:14:10.500Z] 07:14:10     INFO -  PID 15816 | 1699600450318	Sync.Engine.Clients	TRACE	Loading json from disk: /var/folders/wz/qq1sb2011z75sg933d5njjr8000014/T/xpc-profile-ulb14ov_/weave/commands-syncing.json
[task 2023-11-10T07:14:10.500Z] 07:14:10     INFO -  PID 15816 | 1699600450318	Sync.Engine.Clients	TRACE	Uploading local changes to server.
[task 2023-11-10T07:14:10.500Z] 07:14:10     INFO -  PID 15816 | 1699600450318	Sync.Engine.Clients	INFO	Uploading 0 outgoing records
[task 2023-11-10T07:14:10.500Z] 07:14:10     INFO -  PID 15816 | 1699600450318	Sync.Engine.Clients	TRACE	Finishing up sync
[task 2023-11-10T07:14:10.500Z] 07:14:10     INFO -  PID 15816 | 1699600450318	Sync.Engine.Clients	TRACE	Event: weave:engine:sync:finish
[task 2023-11-10T07:14:10.501Z] 07:14:10     INFO -  PID 15816 | 1699600450318	Sync.SyncScheduler	TRACE	Handling weave:engine:sync:finish
[task 2023-11-10T07:14:10.501Z] 07:14:10     INFO -  PID 15816 | 1699600450318	Sync.Telemetry	TRACE	observed weave:engine:sync:finish clients
[task 2023-11-10T07:14:10.501Z] 07:14:10     INFO -  PID 15816 | 1699600450318	Sync.Synchronizer	INFO	Updating enabled engines: 1 clients.
[task 2023-11-10T07:14:10.501Z] 07:14:10     INFO -  PID 15816 | 1699600450318	Sync.Synchronizer	INFO	Syncing all enabled engines.
[task 2023-11-10T07:14:10.501Z] 07:14:10     INFO -  PID 15816 | 1699600450318	Sync.Engine.Catapult	TRACE	Event: weave:engine:sync:start
[task 2023-11-10T07:14:10.501Z] 07:14:10     INFO -  PID 15816 | 1699600450318	Sync.Telemetry	TRACE	observed weave:engine:sync:start catapult
[task 2023-11-10T07:14:10.501Z] 07:14:10     INFO -  PID 15816 | 1699600450318	Sync.Engine.Catapult	TRACE	Event: weave:engine:sync:finish
[task 2023-11-10T07:14:10.501Z] 07:14:10     INFO -  PID 15816 | 1699600450318	Sync.SyncScheduler	TRACE	Handling weave:engine:sync:finish
[task 2023-11-10T07:14:10.501Z] 07:14:10     INFO -  PID 15816 | 1699600450318	Sync.Telemetry	TRACE	observed weave:engine:sync:finish catapult
[task 2023-11-10T07:14:10.501Z] 07:14:10     INFO -  PID 15816 | 1699600450318	Sync.Doctor	INFO	Skipping check of catapult - disabled via preferences
[task 2023-11-10T07:14:10.501Z] 07:14:10     INFO -  PID 15816 | 1699600450318	Sync.Doctor	INFO	Skipping validation: no engines qualify
[task 2023-11-10T07:14:10.501Z] 07:14:10     INFO -  PID 15816 | 1699600450318	Sync.Status	DEBUG	Status.sync: success.sync => success.sync
[task 2023-11-10T07:14:10.501Z] 07:14:10     INFO -  PID 15816 | 1699600450318	Sync.Status	DEBUG	Status.service: success.status_ok => success.status_ok
[task 2023-11-10T07:14:10.501Z] 07:14:10     INFO -  PID 15816 | 1699600450318	Sync.Synchronizer	INFO	Sync completed at 2023-11-10 07:14:10 after 0.00 secs.
[task 2023-11-10T07:14:10.501Z] 07:14:10     INFO -  PID 15816 | 1699600450318	Sync.Declined	DEBUG	Handling remote declined: []
[task 2023-11-10T07:14:10.501Z] 07:14:10     INFO -  PID 15816 | 1699600450318	Sync.Declined	DEBUG	Handling local declined: []
[task 2023-11-10T07:14:10.502Z] 07:14:10     INFO -  PID 15816 | 1699600450318	Sync.Declined	DEBUG	Declined changed? false
[task 2023-11-10T07:14:10.502Z] 07:14:10     INFO -  PID 15816 | 1699600450318	Sync.Service	INFO	No change to declined engines. Not reuploading meta/global.
[task 2023-11-10T07:14:10.502Z] 07:14:10     INFO -  PID 15816 | 1699600450318	Sync.Service	TRACE	Event: weave:service:sync:finish
[task 2023-11-10T07:14:10.502Z] 07:14:10     INFO -  PID 15816 | 1699600450318	Sync.ErrorHandler	TRACE	Handling weave:service:sync:finish
[task 2023-11-10T07:14:10.502Z] 07:14:10     INFO -  PID 15816 | 1699600450318	Sync.ErrorHandler	TRACE	Status.service is success.status_ok
[task 2023-11-10T07:14:10.502Z] 07:14:10     INFO -  PID 15816 | 1699600450318	Sync.ErrorHandler	TRACE	Clearing lastSyncReassigned.
[task 2023-11-10T07:14:10.502Z] 07:14:10     INFO -  PID 15816 | 1699600450318	Sync.LogManager	DEBUG	Flushing file log
[task 2023-11-10T07:14:10.502Z] 07:14:10     INFO -  PID 15816 | 1699600450318	Sync.LogManager	TRACE	Beginning stream copy to success-sync-1699600450318.txt: 1699600450318
[task 2023-11-10T07:14:10.502Z] 07:14:10     INFO -  PID 15816 | 1699600450318	Sync.SyncScheduler	TRACE	Handling weave:service:sync:finish
[task 2023-11-10T07:14:10.502Z] 07:14:10     INFO -  PID 15816 | 1699600450318	Sync.SyncScheduler	TRACE	Adjusting syncInterval to singleDeviceInterval.
[task 2023-11-10T07:14:10.502Z] 07:14:10     INFO -  PID 15816 | 1699600450318	Sync.SyncScheduler	TRACE	clients: score: 0
[task 2023-11-10T07:14:10.502Z] 07:14:10     INFO -  PID 15816 | 1699600450318	Sync.SyncScheduler	TRACE	catapult: score: 0
[task 2023-11-10T07:14:10.502Z] 07:14:10     INFO -  PID 15816 | 1699600450318	Sync.SyncScheduler	TRACE	Global score updated: 0
[task 2023-11-10T07:14:10.502Z] 07:14:10     INFO -  PID 15816 | 1699600450318	Sync.SyncScheduler	DEBUG	Next sync in 3600000 ms. (why=schedule)
[task 2023-11-10T07:14:10.502Z] 07:14:10     INFO -  PID 15816 | 1699600450318	Toolkit.Telemetry	TRACE	TelemetryEnvironment::observe - aTopic: nsPref:changed, aData: services.sync.nextSync
[task 2023-11-10T07:14:10.503Z] 07:14:10     INFO -  PID 15816 | 1699600450318	Sync.Telemetry	TRACE	observed weave:service:sync:finish {}
[task 2023-11-10T07:14:10.503Z] 07:14:10     INFO -  PID 15816 | 1699600450318	Sync.Telemetry	TRACE	shouldSubmitForDataChange - no need to submit
[task 2023-11-10T07:14:10.503Z] 07:14:10     INFO -  PID 15816 | 1699600450318	Sync.Telemetry	TRACE	Not sending telemetry ping for self-hosted Sync user
[task 2023-11-10T07:14:10.503Z] 07:14:10  WARNING -  TEST-UNEXPECTED-FAIL | services/sync/tests/unit/test_errorhandler_1.js | test_sync_non_network_error - [test_sync_non_network_error : 217] "success.sync" == "error.sync.reason.credentials_changed"
[task 2023-11-10T07:14:10.503Z] 07:14:10     INFO -  /opt/worker/tasks/task_169959808697579/build/tests/xpcshell/tests/services/sync/tests/unit/test_errorhandler_1.js:test_sync_non_network_error:217
[task 2023-11-10T07:14:10.503Z] 07:14:10     INFO -  /opt/worker/tasks/task_169959808697579/build/tests/xpcshell/head.js:_do_main:245
[task 2023-11-10T07:14:10.503Z] 07:14:10     INFO -  /opt/worker/tasks/task_169959808697579/build/tests/xpcshell/head.js:_execute_test:592
[task 2023-11-10T07:14:10.503Z] 07:14:10     INFO -  -e:null:1
[task 2023-11-10T07:14:10.503Z] 07:14:10     INFO -  exiting test
[task 2023-11-10T07:14:10.503Z] 07:14:10     INFO -  Unexpected exception NS_ERROR_ABORT:
[task 2023-11-10T07:14:10.503Z] 07:14:10     INFO -  _abort_failed_test@/opt/worker/tasks/task_169959808697579/build/tests/xpcshell/head.js:865:20
[task 2023-11-10T07:14:10.503Z] 07:14:10     INFO -  do_report_result@/opt/worker/tasks/task_169959808697579/build/tests/xpcshell/head.js:977:5
[task 2023-11-10T07:14:10.503Z] 07:14:10     INFO -  Assert<@/opt/worker/tasks/task_169959808697579/build/tests/xpcshell/head.js:70:21
[task 2023-11-10T07:14:10.503Z] 07:14:10     INFO -  Assert.prototype.report@resource://testing-common/Assert.sys.mjs:244:10
[task 2023-11-10T07:14:10.503Z] 07:14:10     INFO -  equal@resource://testing-common/Assert.sys.mjs:286:8
[task 2023-11-10T07:14:10.503Z] 07:14:10     INFO -  test_sync_non_network_error@/opt/worker/tasks/task_169959808697579/build/tests/xpcshell/tests/services/sync/tests/unit/test_errorhandler_1.js:217:10
[task 2023-11-10T07:14:10.503Z] 07:14:10     INFO -  _do_main@/opt/worker/tasks/task_169959808697579/build/tests/xpcshell/head.js:245:6
[task 2023-11-10T07:14:10.504Z] 07:14:10     INFO -  _execute_test@/opt/worker/tasks/task_169959808697579/build/tests/xpcshell/head.js:592:5
[task 2023-11-10T07:14:10.504Z] 07:14:10     INFO -  @-e:1:1
[task 2023-11-10T07:14:10.504Z] 07:14:10     INFO -  exiting test
[task 2023-11-10T07:14:10.504Z] 07:14:10     INFO -  PID 15816 | 1699600450319	Toolkit.Telemetry	TRACE	TelemetryEnvironment::observe - aTopic: nsPref:changed, aData: services.sync.engine.catapult
[task 2023-11-10T07:14:10.504Z] 07:14:10     INFO -  PID 15816 | 1699600450319	Sync.Service	TRACE	Status for catapult engine changed.
[task 2023-11-10T07:14:10.504Z] 07:14:10     INFO -  PID 15816 | 1699600450319	Toolkit.Telemetry	TRACE	TelemetryEnvironment::observe - aTopic: nsPref:changed, aData: services.sync.engineStatusChanged.catapult
[task 2023-11-10T07:14:10.504Z] 07:14:10     INFO -  PID 15816 | 1699600450319	Toolkit.Telemetry	TRACE	TelemetryEnvironment::observe - aTopic: nsPref:changed, aData: services.sync.catapult.lastSync
[task 2023-11-10T07:14:10.504Z] 07:14:10     INFO -  PID 15816 | 1699600450319	Toolkit.Telemetry	TRACE	TelemetryEnvironment::observe - aTopic: nsPref:changed, aData: services.sync.clients.devices.mobile
[task 2023-11-10T07:14:10.504Z] 07:14:10     INFO -  PID 15816 | 1699600450319	Toolkit.Telemetry	TRACE	TelemetryEnvironment::observe - aTopic: nsPref:changed, aData: services.sync.engine.bookmarks.validation.maxRecords
[task 2023-11-10T07:14:10.504Z] 07:14:10     INFO -  PID 15816 | 1699600450319	Toolkit.Telemetry	TRACE	TelemetryEnvironment::observe - aTopic: nsPref:changed, aData: services.sync.client.syncID
[task 2023-11-10T07:14:10.504Z] 07:14:10     INFO -  PID 15816 | 1699600450319	Toolkit.Telemetry	TRACE	TelemetryEnvironment::observe - aTopic: nsPref:changed, aData: services.sync.clients.lastRecordUpload
[task 2023-11-10T07:14:10.504Z] 07:14:10     INFO -  PID 15816 | 1699600450319	Toolkit.Telemetry	TRACE	TelemetryEnvironment::observe - aTopic: nsPref:changed, aData: services.sync.clients.devices.desktop
[task 2023-11-10T07:14:10.504Z] 07:14:10     INFO -  PID 15816 | 1699600450319	Toolkit.Telemetry	TRACE	TelemetryEnvironment::observe - aTopic: nsPref:changed, aData: services.sync.lastversion
[task 2023-11-10T07:14:10.504Z] 07:14:10     INFO -  PID 15816 | 1699600450319	Toolkit.Telemetry	TRACE	TelemetryEnvironment::observe - aTopic: nsPref:changed, aData: services.sync.clients.lastSync
[task 2023-11-10T07:14:10.505Z] 07:14:10     INFO -  PID 15816 | 1699600450319	Toolkit.Telemetry	TRACE	TelemetryEnvironment::observe - aTopic: nsPref:changed, aData: services.sync.engine.bookmarks.validation.interval
[task 2023-11-10T07:14:10.505Z] 07:14:10     INFO -  PID 15816 | 1699600450319	Toolkit.Telemetry	TRACE	TelemetryEnvironment::observe - aTopic: nsPref:changed, aData: services.sync.nextSync
[task 2023-11-10T07:14:10.505Z] 07:14:10     INFO -  PID 15816 | 1699600450319	Toolkit.Telemetry	TRACE	TelemetryEnvironment::observe - aTopic: nsPref:changed, aData: services.sync.clients.syncID
[task 2023-11-10T07:14:10.505Z] 07:14:10     INFO -  PID 15816 | 1699600450319	Toolkit.Telemetry	TRACE	TelemetryEnvironment::observe - aTopic: nsPref:changed, aData: services.sync.client.GUID
[task 2023-11-10T07:14:10.505Z] 07:14:10     INFO -  PID 15816 | 1699600450319	Toolkit.Telemetry	TRACE	TelemetryEnvironment::observe - aTopic: nsPref:changed, aData: services.sync.username
[task 2023-11-10T07:14:10.505Z] 07:14:10     INFO -  PID 15816 | 1699600450319	Toolkit.Telemetry	TRACE	TelemetryEnvironment::observe - aTopic: nsPref:changed, aData: services.sync.log.logger.engine
[task 2023-11-10T07:14:10.505Z] 07:14:10     INFO -  PID 15816 | 1699600450319	Toolkit.Telemetry	TRACE	TelemetryEnvironment::observe - aTopic: nsPref:changed, aData: services.sync.log.logger
[task 2023-11-10T07:14:10.505Z] 07:14:10     INFO -  PID 15816 | 1699600450319	Toolkit.Telemetry	TRACE	TelemetryEnvironment::observe - aTopic: nsPref:changed, aData: services.sync.engine.bookmarks.validation.percentageChance
[task 2023-11-10T07:14:10.505Z] 07:14:10     INFO -  PID 15816 | 1699600450319	Toolkit.Telemetry	TRACE	TelemetryEnvironment::observe - aTopic: nsPref:changed, aData: services.sync.catapult.syncID
[task 2023-11-10T07:14:10.505Z] 07:14:10     INFO -  PID 15816 | 1699600450319	Toolkit.Telemetry	TRACE	TelemetryEnvironment::observe - aTopic: nsPref:changed, aData: services.sync.lastPing
[task 2023-11-10T07:14:10.505Z] 07:14:10     INFO -  PID 15816 | 1699600450319	Toolkit.Telemetry	TRACE	TelemetryEnvironment::observe - aTopic: nsPref:changed, aData: services.sync.declinedEngines
[task 2023-11-10T07:14:10.505Z] 07:14:10     INFO -  PID 15816 | 1699600450320	Toolkit.Telemetry	TRACE	TelemetryEnvironment::observe - aTopic: nsPref:changed, aData: services.sync.lastSync
[task 2023-11-10T07:14:10.506Z] 07:14:10     INFO -  PID 15816 | 1699600450320	Sync.SyncScheduler	DEBUG	Clearing sync triggers and the global score.
[task 2023-11-10T07:14:10.506Z] 07:14:10     INFO -  PID 15816 | 1699600450320	Toolkit.Telemetry	TRACE	TelemetryEnvironment::observe - aTopic: nsPref:changed, aData: services.sync.nextSync
[task 2023-11-10T07:14:10.506Z] 07:14:10     INFO -  PID 15816 | !!! error running onStopped callback: TypeError: callback is not a function
[task 2023-11-10T07:14:10.506Z] 07:14:10     INFO -  <<<<<<<
[task 2023-11-10T07:14:10.506Z] 07:14:10     INFO -  TEST-START | toolkit/components/antitracking/test/xpcshell/test_staticPartition_font.js
```