The queue *does* care, though, as far as whether it should automatically retry the task.  To do otherwise would leave retrying to the job graph, which is probably not the right place.

I think you'll want active notifications of both transient and permanent failures, though, rather than a worker quietly re-trying tasks that it believes to have failed transiently.  That will give us visibility into transient failures and also allow transient failures related to a particular worker (e.g., failing HDD) to escape that worker and be retried on another.