package.el makes the request to whatever URL is contained in `package-archives`.  There is no hard-coded `/packages/` prefix in package.el, and some archives (such as Org's) use a different one.  Org's ELPA is at `http://orgmode.org/elpa/`.

I think, if you add `("localhost" . "http://localhost:9191/")` to `package-archives`, package.el will fetch `http://localhost:9191/archive-contents` just fine.

I agree that we should definitely keep the index, but I think we should really make it optional.  Small archives for unit tests really don't need an index on the disk.