Tech evaluation: Object storage using presigned URLs

This is a follow-up tech evaluation from #355 

@ayufan thanks for your [input on slack](https://gitlab.slack.com/archives/C1BSEQ138/p1585668949053900)!(copying here)

- We need to use pre-signed URLs from GitLab, that way we don’t need any credentials on Pages, and whether the .zip is used can be controlled by Rails exclusively, the link would have an encoded and Rails controlled expiry date
- If serving from .zip I think we need to likely define the maximum archive size that we can support, likely filtering the relevant files (public/ only folder), and holding that somewhere in memory. I would assume that we could likely configure how many files-in-archives/archives we cache and allow this to be configured and optimised towards cache-hit-ratio, likely GitLab.com would allow to use a ton of memory if needed
- I would likely break the support for Content-Range if serving files as I don’t think that this is cheaply possible with .zip
- GitLab Workhorse does have OpenArchive that supports local and remote archive just it is not performance optimised: the HTTP requests are badly aligned and this will likely need to be somehow improved, so just copy-pasting will not give a great performance yet

---
- [ ] @vshushlin started a [discussion](https://gitlab.com/gitlab-org/gitlab-pages/-/merge_requests/258#note_309921953):  (+7 comments)

    > Oh, I thought that https://gitlab.com/gitlab-org/gitlab-pages/-/merge_requests/136/ has some object storage implementation while it only has serving from zip files from disc :see_no_evil: 
    > 
    > I have a very simple idea for alternative PoC:
    > 
    > 1. We can copy(and maybe slightly modify) https://gitlab.com/gitlab-org/gitlab/-/blob/84c0ffe12646b9bae1fdf2e576cde7f01f8ded73/lib/api/job_artifacts.rb#L75-94 to pages API https://gitlab.com/gitlab-org/gitlab/-/blob/75f8d42bb443d0a6101a9c2f6b65c607cd95efd4/lib/api/internal/pages.rb#L19
    > 1. Then we can return the `job_id` in the API
    > 1. Pages will get specific file and proxy it to user.
    > 1. Later we can add cache for it.
    > 
    > Alternatively, we can get the whole artifacts zip archive and use the current zip reading code, then we'll need to cache those files.
    > 
    > I don't think that adding some object-storage specific code to pages is a good idea. We already have it in the workhorse, we can just use the API. It's slower, but much simplier.

### Diagram/proposal 

![Screen_Shot_2020-06-26_at_5.09.00_pm](/uploads/6c277237392a3ce3844fe3e1551f980f/Screen_Shot_2020-06-26_at_5.09.00_pm.png)

```
sequenceDiagram
  participant U as User
  participant P as gitlab-pages
  participant G as gitlab-workhorse and rails
  participant OS as Object Storage
  U->>P: 1. username.gitlab.io/index.html
  P->>G: 2. GET /api/v4/internal/pages?host=username.gitlab.io
  G->>P: 3. {... lookup_paths: [{source: {type: "zip", path: "presignedURL"}],...}
  loop zipartifacts
    P->>P: 4. reader:= OpenArchive(presignedURL)
    P->>OS: 5. GET presignedURL
    OS->>P: 6. .zip file
    P->>P: 7. reader.find(public/index.html)
    P->>P: 8. go func(){ cache({host,reader}) } ()
  end
  P->>U: 9. username.gitlab.io/index.html
```

### Proposal 

In this PoC we will hardcode the returning value from `/api/v4/internal/pages` to reduce the scope. I will use minio which is already supported in the GDK. I'll also shamelessly steal and slightly modify the `zipartifacts` package from workhorse.

To address https://gitlab.com/gitlab-org/gitlab-pages/-/issues/377#note_367358348 the source type should be `"zip"` so that Pages can serve from `.zip` regardless of the path (pre-signed URL or disk path).

### Outcomes

We have now https://gitlab.com/groups/gitlab-org/-/epics/3901 and https://gitlab.com/groups/gitlab-org/-/epics/3902 with parent https://gitlab.com/groups/gitlab-org/-/epics/1316 to track all future efforts.

#### Rails

1. Allow deploying Pages as `.zip` archives with a `max_archive_size`. https://gitlab.com/gitlab-org/gitlab/-/issues/208135
1. On deploy ->check size -> store `public.zip` either on disk or in object storage depending on the features enabled. also tracked in https://gitlab.com/gitlab-org/gitlab/-/issues/208135
1. Update `/api/v4/internal/pages` -> return a "source"."type":“zip” with a path https://gitlab.com/gitlab-org/gitlab/-/issues/225840 e.g.
```json
{
	"lookup_paths" : [
	{
		"source": {
			"type": "zip",
			"path": "https://presigned.url/public.zip",
			"_":"or from disk path"
			"path": "/shared/pages/domain/project/public.zip"
	}
]
```

#### Pages (Go)

0. extract the `resolvePath` logic from disk serving into its own package so it can be shared. https://gitlab.com/gitlab-org/gitlab-pages/-/issues/421
1. Add package `zip` with `zip/reader` https://gitlab.com/gitlab-org/gitlab/-/issues/28784
2. Add `zip` serving to Pages - this allows serving from disk or pre-signed URLs from object storage https://gitlab.com/gitlab-org/gitlab/-/issues/28784
3. Implement a zip reader caching mechanism https://gitlab.com/gitlab-org/gitlab-pages/-/issues/422
4. Add metrics for zip serving https://gitlab.com/gitlab-org/gitlab-pages/-/issues/423

* while testing I hit https://gitlab.com/gitlab-org/gitlab-pages/-/issues/371 so I think it would be valuable to work on that issue first.

Story points: 1.0