taskcluster/taskcluster-queue — pull request #18 by jonasfj (2014-07-23 22:24:10+00:00)
Title: Refactor status structure (okay, well the API too)

The refactor changes a lot of things that have been long due:

**In the task definition:**
* Scopes on tasks (person who submits task must also have the scopes)
* Removed `timeout` (it can now be configured on the queue)

**In the task status structure**
* Removed `routing` the routing key is always present anyways
* Changes claims to exist on runs, and not on tasks
* Only property that changes on task-level is `retriesLeft` (formerly `retries`)

**Overview of API**
* Most end-points (except `/status`) require dynamic scopes
* A worker cannot modify runs claimed by other worker types (if scopes are assigned correctly)
* All (or just about all) end-points are idempotent and can be repeated with success
* Everything is routed through the queue, so that:
* We can add properties to the return format and upgrade old data on-the-fly
* We don't have magic URLs to construct
* We don't have magic URLs to remember
* We have more auxiliary end-points on the queue, for doing things using CURL and an authentication proxy

** Artifacts API**
* A log is an artifact
* Artifacts named `public/...` does not require authentication (all other artifacts does)
* We redirect all requests to `GET` artifacts (to signed URLs, unless it's an error artifact)
* Artifacts have an expiration date on (files in S3 and Azure are deleted by a cron job)
* Logs are stored in `public/logs/...` unless they are private in which case you assign them another name
(The task-inspector will only ever want to inspect public logs anyways)
* When Azure artifacts (live logs) are created an SAS is returned

**Creation API**
* Scope is required to post a task to a given `provisionerId` and `workerType`
* `deadline` must be less than 7 days from now (we check for less than 8, but the docs will say 7) .
* Task scopes must be a subset of the clients scopes, (note `queue:get:*` is a subset of `queue:*`).
* `taskId` is now determined by the poster of the task
* Task creation, defintion/scheduling is idempotent :)

**Worker API**
* Distinction between claims and reclaims
* Claims requires `queue:assume:worker-type:/`
* Operations on a runs requires `queue:assume:worker-id:/`
* No `result.json` structure or `logs.json` structure is uploaded anywhere
* When a task is completed the queue is still informed of `success`, generally interpreted as is exitcode zero

I think that is about it... give or take... maybe I forgot something. Take a look at `v1.js` that is the new API also `artifacts.js` contains the artifacts API... We can add support for additional kinds of artifacts later.