Document the HTTP transport protocol

Signed-off-by: Shawn O. Pearce <spearce@spearce.org>

---

Changes:
 - fix mis-spelling 'paramterized'
 - fix mis-spelling 'mangement' (spotted by Junio)
 - fix missing ABNF reference for smart replies (spotted by Sverre, Junio)
normalize indentation with protcol-common.txt

Indent client/server query examples with 3 spaces.

Indent ABNF rules with 2 spaces.
capitalize key words according to RFC 2119
normalize rules with RFC 5234

Drop LF, SP which are defined in RFC 5234.

Replace HT with HTAB (also defined in the RFC).

Use '/' instead of '|', as the RFC does.
drop rules, etc. common to the pack protocol

Use obj-id in lieu of id (defined as 40*HEX).

Use zero-id in lieu of 40*"0".

Use refname in lieu of name (not defined).

Drop section on capabilities, since they are already available in
protocol-capabilities.txt.

---

pkt-line format section was dropped in response to Junio's comments:

  Message-ID: <7vskdss3ei.fsf@alter.siamese.dyndns.org>

  > +pkt-line Format
  > +---------------
  > ...
  > +Examples (as C-style strings):
  > +
  > +  pkt-line          actual value
  > +  ---------------------------------
  > +  "0006a\n"         "a\n"
  > +  "0005a"           "a"
  > +  "000bfoobar\n"    "foobar\n"
  > +  "0004"            ""
  > +
  > +A pkt-line with a length of 0 ("0000") is a special case and MUST
  > +be treated as a message break or terminator in the payload.

  Isn't this "MUST be" wrong?

  It is not an advice to the implementors, but the protocol specification
  itself defines what the flush packet means.  IOW, "The author of this
  specification, Shawn, MUST treat a flush packet as a message break or
  terminator in the payload, when designing this protocol."

Capabilities was dropped by Nguyễn:

  Message-ID: <1377092713-25434-1-git-send-email-pclouds@gmail.com>
fix capabilities definition

For "capabilties^{}" lines, replace obj-id with zero-id.

Define capabilities using rules from pack-protocol.txt - ideally, this
should be shifted to protocol-common.txt.
reword 404 para

---

Reflowed to 72 chars.

Based on:

  Message-ID: <20091016142135.GR10505@spearce.org>

  Mike Hommey <mh@glandium.org> wrote:
  > On Thu, Oct 15, 2009 at 10:59:25PM -0700, H. Peter Anvin wrote:
  > > On 10/10/2009 03:12 AM, Antti-Juhani Kaijanaho wrote:
  > > > On 2009-10-09, Junio C Hamano <gitster@pobox.com> wrote:
  > > >>> +If there is no repository at $GIT_URL, the server MUST respond with
  > > >>> +the '404 Not Found' HTTP status code.
  > > >>
  > > >> We may also want to add
  > > >>
  > > >>     If there is no object at $GIT_URL/some/path, the server MUST respond
  > > >>     with the '404 Not Found' HTTP status code.
  > > >>
  > > >> to help dumb clients.
  > > >
  > > > In both cases - is it really necessary to forbid the use of 410 (Gone)?

  My original text got taken a bit out of context here.  I guess MUST
  was too strong of a word.  I more ment something like:

    If there is no repository at $GIT_URL, the server MUST NOT respond
    with '200 OK' and a valid info/refs response.  A server SHOULD
    respond with '404 Not Found', '410 Gone', or any other suitable
    HTTP status code which does not imply the resource exists as
    requested.
weaken cookie support para

---

Based on the discussion in
  <20091009195035.GA15153@coredump.intra.peff.net>,
  <20091015165228.GO10505@spearce.org> (patch),
  <20091015173902.GA22262@sigill.intra.peff.net> (agreement)

  Message-ID: <20091015165228.GO10505@spearce.org>

  I weakend the sections on cookies:

  + Authentication
  + --------------
  ....
  + Servers SHOULD NOT require HTTP cookies for the purposes of
  + authentication or access control.

  and that's all we say on the matter.  I took out the Servers MUST
  NOT line under session state.
mention different variations around $GIT_URL

Based on

  Message-ID: <loom.20091009T104530-586@post.gmane.org>

  Shawn O. Pearce <spearce <at> spearce.org> writes:

  > +URL Format
  > +----------
  > +
  > +URLs for Git repositories accessed by HTTP use the standard HTTP
  > +URL syntax documented by RFC 1738, so they are of the form:
  > +
  > +  http://<host>:<port>/<path>
  > +
  > +Within this documentation the placeholder $GIT_URL will stand for
  > +the http:// repository URL entered by the end-user.

  It's worth making clear here that $GIT_URL will be the path to the repository,
  rather than necessarily just the host upon which the server sits. Perhaps
  including an example, like http://example:8080/repos/example.git
  would make it clearer that there can be a path (and so leading to
  a request like http://example:8080/repos/example.git/info/refs?service=...

  It's also worth clarifying, therefore, that multiple repositories can be served
  by the same process (as with the git server today) by using different path(s).
  And for those that are interested in submodules, it's worth confirming that
  http://example/repos/master.git/child.git/info/refs?service= will ensure
  that the repository is the 'child' git rather than anything else.

The submodule example (/master.git/child.git) seems potentially
confusing - it suggests a setup where the server has a route to a git
repo (child.git) with a parent path containing another git repo
(master.git). It is excluded lest we be seen as encouraging such
mind-boggling setups.

While providing an example $GIT_URL containing a '?' (the catch-all
gateway one), also mention a possible contradiction between the
exactly-one-param requirement and the http client implementation in Git.
reword dumb client info/refs para

---

Notes:
 - said "request to" instead of Junio's "request against", for
   consistency with the rest of the document.
 - said "E.g." instead of "I.e." since it's an example request and
   response

Based on:

  Message-ID: <7vskdss3ei.fsf@alter.siamese.dyndns.org>

  > +Dumb Clients
  > +~~~~~~~~~~~~
  > +
  > +HTTP clients that only support the "dumb" protocol MUST discover
  > +references by making a request for the special info/refs file of
  > +the repository.
  > +
  > +Dumb HTTP clients MUST NOT include search/query parameters when
  > +fetching the info/refs file.  (That is, '?' must not appear in the
  > +requested URL.)

  It is unclear if '?' can be part of $GIT_URL. E.g.

      $ wget http://example.xz/serve.cgi?path=git.git/info/refs
      $ git clone http://example.xz/serve.cgi?path=git.git

  It might be clearer to just say

      Dumb HTTP clients MUST make a GET request against $GIT_URL/info/refs,
      without any search/query parameters.  I.e.

          C: GET $GIT_URL/info/refs HTTP/1.0

  to also exclude methods other than GET.
also mention missing objects

Junio's suggested text was not used, but the issue (no object found) was
addressed.

Based on:

  Message-ID: <7vskdss3ei.fsf@alter.siamese.dyndns.org>

  > +General Request Processing
  > +--------------------------
  > +
  > +Except where noted, all standard HTTP behavior SHOULD be assumed
  > +by both client and server.  This includes (but is not necessarily
  > +limited to):
  > +
  > +If there is no repository at $GIT_URL, the server MUST respond with
  > +the '404 Not Found' HTTP status code.

  We may also want to add

      If there is no object at $GIT_URL/some/path, the server MUST respond
      with the '404 Not Found' HTTP status code.

  to help dumb clients.
fix examples

 - add LF for responses
 - add pkt-line lengths for smart
 - add flush-pkt (0000) for smart
 - drop SP that followed NUL before capability list
reword 'remote repository' lines

Based on:

  Message-ID: <7vskdss3ei.fsf@alter.siamese.dyndns.org>

  > +Smart Service git-upload-pack
  > +------------------------------
  > +This service reads from the remote repository.

  The wording "remote repository" felt confusing.  I know it is "from the
  repository served by the server", but if it were named without
  "upload-pack", I might have mistaken that you are allowing to proxy a
  request to access a third-party repository by this server.  The same
  comment applies to the git-receive-pack service.
reword smart server response para

Based on:

  Message-ID: <7vskdss3ei.fsf@alter.siamese.dyndns.org>

  > +Smart Server Response
  > +^^^^^^^^^^^^^^^^^^^^^
  > +
  > +Smart servers MUST respond with the smart server reply format.
  > +If the server does not recognize the requested service name, or the
  > +requested service name has been disabled by the server administrator,
  > +the server MUST respond with the '403 Forbidden' HTTP status code.

  This is a bit confusing.

  If you as a server administrator want to disable the smart upload-pack for
  one repository (but not for other repositories), you would not be able to
  force smart clients to fall back to the dumb protocol by giving "403" for
  that repository.

  Maybe in 2 years somebody smarter than us will have invented a more
  efficient git-upload-pack-2 service, which is the only fetch protocol his
  server supports other than dumb.  If your v1 smart client asks for the
  original git-upload-pack service and gets a "403", you won't be able to
  fall back to "dumb".

  The solution for such cases likely is to pretend as if you are a dumb
  server for the smart request.  That unfortunately means that the first
  sentence is misleading, and the second sentence is also an inappropriate
  advice.
shift dumb server response details from client section to server

Stuff like abnf, caching. This is in line with smart client-server
equivalent.
nguyen's changes