Repository: fsouza/fake-gcs-server
Pull request #35 (2019-06-05 02:05:44)

Add optional custom external URL
Add the ability to specify a different base server URL. This would be used by https://github.com/teone/gc-fake-storage to customize the ``Location`` header for resumable uploads.

When I use the Python client to upload a large file to ``gc-fake-storage``, it uses the resumable upload endpoint, which is supported by ``fake-gcs-server`` 🎉 . The raw request, to ``gc-fake-storage`` running in a Docker container called ``gcs-emulator`` on port 4443, looks something like this:

```
POST /upload/storage/v1/bucket_name/o?uploadType=resumable 
HTTP/1.1
Host: gcs-emulator:4443
User-Agent: python-requests/2.22.0
Accept-Encoding: gzip, deflate
Accept: */*
Connection: keep-alive
content-type: application/json; charset=UTF-8
x-upload-content-type: text/plain
Content-Length: 236

{"name": "v1/file.bin/file.bin/8D1AEB53DC42444C8C46ABD7FD697AFD/file.bin", "contentType": "text/plain", "metadata": {"original_size": "369412", "original_md5_hash": "8d1aeb53dc42444c8c46abd7fd697afd"}, "contentEncoding": "gzip"}'
```

``gc-fake-storage`` serves on ``0.0.0.0:4443`` inside the Docker container, and the raw response looks like:

```
HTTP/1.1 200 OK

Location: https://[::]:4443/upload/resumable/c3acc487c6f5a58295bb058a2099f88c
Date: Tue, 04 Jun 2019 22:06:16 GMT
Content-Length: 82
Content-Type: text/plain; charset=utf-8
```

``[::]`` is the IPv6 equivalent shorthand for IPv4's ``0.0.0.0``. The Python client attempts to ``POST`` the first chunk of the file to the ``Location`` URL, but that is on the requester's container, not the ``gc-fake-server`` container, and the request fails. 

In order for this to work, the ``Location`` header would need to be:

```
Location: https://gc-fake-storage:4443/upload/resumable/c3acc487c6f5a58295bb058a2099f88c
```

This PR adds a way to customize what ``Server.URL()`` returns, which can eventually be used in ``gc-fake-storage`` to set the external-facing domain for resumable uploads.

I also considered dropping the domain name from the resumable upload response, so the ``Location`` header looks like:

```
Location: /upload/resumable/c3acc487c6f5a58295bb058a2099f88c
```

This may also work, but causes tests to fail if done unconditionally. A similar option could be used to drop the domain if requested.

A third option is to use the ``Host`` from the request, but this also breaks tests.


Patch:
diff --git a/fakestorage/server.go b/fakestorage/server.go
index 725230c..de659c0 100644
--- a/fakestorage/server.go
+++ b/fakestorage/server.go
@@ -23,11 +23,12 @@ import (
 //
 // It provides a fake implementation of the Google Cloud Storage API.
 type Server struct {
-	backend   backend.Storage
-	uploads   sync.Map
-	transport http.RoundTripper
-	ts        *httptest.Server
-	mux       *mux.Router
+	backend     backend.Storage
+	uploads     sync.Map
+	transport   http.RoundTripper
+	ts          *httptest.Server
+	mux         *mux.Router
+	externalURL string
 }
 
 // NewServer creates a new instance of the server, pre-loaded with the given
@@ -58,6 +59,10 @@ type Options struct {
 	// when set to true, the server will not actually start a TCP listener,
 	// client requests will get processed by an internal mocked transport.
 	NoListener bool
+
+	// optional external URL, such as http://gcs.example.com
+	// returned as Location header for resumable uploads
+	ExternalURL string
 }
 
 // NewServerWithOptions creates a new server with custom options
@@ -70,6 +75,9 @@ func NewServerWithOptions(options Options) (*Server, error) {
 		s.setTransportToMux()
 		return s, nil
 	}
+	if options.ExternalURL != "" {
+		s.externalURL = options.ExternalURL
+	}
 
 	s.ts = httptest.NewUnstartedServer(s.mux)
 	if options.Port != 0 {
@@ -153,6 +161,9 @@ func (s *Server) Stop() {
 
 // URL returns the server URL.
 func (s *Server) URL() string {
+	if s.externalURL != "" {
+		return s.externalURL
+	}
 	if s.ts != nil {
 		return s.ts.URL
 	}


Test patch:
diff --git a/fakestorage/server_test.go b/fakestorage/server_test.go
index f8a2c44..dea6485 100644
--- a/fakestorage/server_test.go
+++ b/fakestorage/server_test.go
@@ -38,6 +38,19 @@ func TestNewServerNoListener(t *testing.T) {
 	}
 }
 
+func TestNewServerExternalHost(t *testing.T) {
+	t.Parallel()
+	server, err := NewServerWithOptions(Options{ExternalURL: "https://gcs.example.com"})
+	if err != nil {
+		t.Fatal(err)
+	}
+	defer server.Stop()
+	url := server.URL()
+	if url != "https://gcs.example.com" {
+		t.Errorf("wrong url returned\n want %q\ngot  %q", server.externalURL, url)
+	}
+}
+
 func TestDownloadObject(t *testing.T) {
 	objs := []Object{
 		{BucketName: "some-bucket", Name: "files/txt/text-01.txt", Content: []byte("something")},