Repository: restic/restic
Pull request #4410 (2023-07-17 07:25:43)

add --stdin-from-command flag to backup command
What does this PR change? What problem does it solve?
-----------------------------------------------------

This pull request is a continuation of #4254.

This fixes [#4251](https://github.com/restic/restic/issues/4251) by providing the requested `--stdin-from-command` flag to the `restic backup` command. It skips snapshot creation in case the given commands fails as determined by `exec.Cmd.Wait()` and returns the stderr of the given command as an error message back to the user.

I have not written any tests, nor documentation because I wanted to get some feedback about the path taken here wrt. upstream inclusion first. I am seeing various test failures in existing tests but these seem to be unrelated and might just occur because I'm building/testing the project within a container. Using these changes locally does look promising though:

```
$ ./restic backup --stdin-from-command --repo ./backup -- pg_dump -h foobar
enter password for repository:
repository 55e7d5a0 opened (version 2, compression level auto)
using parent snapshot cc6e1dec
error: read /stdin: no data read
Fatal: unable to save snapshot: pg_dump: error: connection to database "root" failed: could not translate host name "foobar" to address: No address associated with hostname
```

Was the change previously discussed in an issue or on the forum?
----------------------------------------------------------------

Closes [#4251](https://github.com/restic/restic/issues/4251)

Checklist
---------

-   [x]  I have read the [contribution guidelines](https://github.com/restic/restic/blob/master/CONTRIBUTING.md#providing-patches).
-   [x]  I have [enabled maintainer edits](https://help.github.com/en/github/collaborating-with-issues-and-pull-requests/allowing-changes-to-a-pull-request-branch-created-from-a-fork).
-   [x]  I have added tests for all code changes.
-   [x]  I have added documentation for relevant changes (in the manual).
-   [x]  There's a new file in `changelog/unreleased/` that describes the changes for our users (see [template](https://github.com/restic/restic/blob/master/changelog/TEMPLATE)).
-   [x]  I have run `gofmt` on the code in all commits.
-   [x]  All commit messages are formatted in the same style as [the other commits in the repo](https://github.com/restic/restic/blob/master/CONTRIBUTING.md#git-commits).


Enrico204: I have rebased the code to the latest `master`, and in few days I will proceed to fix it based on changes requested in the old PR :+1: 
rawtaz: Is there other software that has this option name (`--stdin-from-command`)? It's quite long, feels like `--stdin-command` is more concise without losing any semantics.
Enrico204: > Is there other software that has this option name (`--stdin-from-command`)? It's quite long, feels like `--stdin-command` is more concise without losing any semantics.

Actually, `content-from-command` may be better, that is also the name it has in Borg backup. See https://github.com/restic/restic/pull/4254#discussion_r1160749426
rawtaz: Nah, there's already two `--stdin-*` options, no point in deviating from that prefix.
Enrico204: @MichaelEischer I've implemented your suggestions from the old pull request. I implemented a new `io.ReadCloser` instead of a new `fs.FS`, calling `cmd.Wait()` on `io.ReadCloser.Close()`. The archiver is back to the original code.

I'm testing the code now :-)
meise: Thank you for continuing with this feature. I tested your branch and it still creates snapshots when subcommand exists with non zero:

```
$ restic-stdin-command/restic backup --stdin-from-command -- pg_dump -h foobar  
repository 12ba8696 opened (version 2, compression level auto)
using parent snapshot 75e9f4c7
subprocess pg_dump: pg_dump: error: could not translate host name "foobar" to address: Name or service not known
error: read /stdin: no data read

Files:           0 new,     0 changed,     0 unmodified
Dirs:            0 new,     0 changed,     0 unmodified
Added to the repository: 0 B   (0 B   stored)

processed 0 files, 0 B in 0:00
snapshot d044b2eb saved
Warning: at least one source file could not be read

$ echo $?
3
```

```
$ pg_dump -h foobar; echo $?
pg_dump: error: could not translate host name "foobar" to address: Name or service not known
1
```
Enrico204: > Thank you for continuing with this feature. I tested your branch and it still creates snapshots when subcommand exists with non zero:
> 
> ```
> $ restic-stdin-command/restic backup --stdin-from-command -- pg_dump -h foobar  
> repository 12ba8696 opened (version 2, compression level auto)
> using parent snapshot 75e9f4c7
> subprocess pg_dump: pg_dump: error: could not translate host name "foobar" to address: Name or service not known
> error: read /stdin: no data read
> 
> Files:           0 new,     0 changed,     0 unmodified
> Dirs:            0 new,     0 changed,     0 unmodified
> Added to the repository: 0 B   (0 B   stored)
> 
> processed 0 files, 0 B in 0:00
> snapshot d044b2eb saved
> Warning: at least one source file could not be read
> 
> $ echo $?
> 3
> ```
> 
> ```
> $ pg_dump -h foobar; echo $?
> pg_dump: error: could not translate host name "foobar" to address: Name or service not known
> 1
> ```

Uh, I was about to do the same test :-D

Thanks for your report, I'll investigate and fix :+1: 
Enrico204: The problem was that the error returned in file reading or closing was logged, but the snapshot itself was continuing unaffected (because the logic assumed that an error while reading one file should not abort the snapshot).

The `ReadCloserCommand` now returns a "fatal" error, and the error is checked in the `cmd_backup.go` file: if it is "fatal", the snapshot is aborted. Otherwise, the behavior is the same as before: log the error, and exit with code 3 after completing the snapshot.

What do you think?
meise: > What do you think?

I'm not sure about the `command terminated with no output` fatal error. The current behaviour in stdin mode is to log `no data read` and exit with status code `3` when reading no data.

I like your current implementation more, because having a snapshot with no data makes no sense for me. But it might be behaving differently than we have in existing implementations.
Enrico204: > > What do you think?
> 
> I'm not sure about the `command terminated with no output` fatal error. The current behaviour in stdin mode is to log `no data read` and exit with status code `3` when reading no data.
> 
> I like your current implementation more, because having a snapshot with no data makes no sense for me. But it might be behaving differently than we have in existing implementations.

You are right, it is a different behavior. I assumed that this is expected, given that we are explicitly asking for monitoring the command.

On second thought, the best way is to check the command exit code also in `ReadCloserCommand.Read()`, and return a fatal error only when there is an EOF and the exit code is non-zero. By doing so, if the command exits with `0` and there is no output, the snapshot is created with a warning (which is the current behavior for files and stdin), while the snapshot is not created if the exit code is non-zero (which is expected).

I'll modify the code asap :-)
Enrico204: Ok, now the code should stop the snapshot creation by looking only at the exit code. If there is no output, the snapshot is still created with a warning (and exit code 3), which is the same behavior that other snapshot options have (`--stdin`, but also normal file backup). Let me know if there is something we can improve :-)
meise: Works for me. Very good!

I think there should now be some feedback on the code itself.
Enrico204: I rebased the branch to the latest `master` :-)
MichaelEischer: @Enrico204 Thanks for finishing the PR, and sorry for the late feedback. I've somehow missed that the review comments have been addressed in the meantime.

I've reworked the code a bit: the command startup is now handled by `fs.CommandReader` (that's the new name for the helper in `fs`). Instead of using canceling the backup indirectly using a context, `arch.Error` now just returns an error if it is Fatal. This also allows for a nicer error message.
Enrico204: No problem at all :-)

I will compile this branch and I will test it as soon as possible!
Enrico204: I am testing the code since a few weeks: so far, no issues :-)

Patch:
diff --git a/changelog/unreleased/issue-4251 b/changelog/unreleased/issue-4251
new file mode 100644
index 000000000..31be52401
--- /dev/null
+++ b/changelog/unreleased/issue-4251
@@ -0,0 +1,14 @@
+Enhancement: Support reading backup from a program's standard output
+
+When reading data from stdin, the `backup` command could not verify whether the
+corresponding command completed successfully.
+
+The `backup` command now supports starting an arbitrary command and sourcing
+the backup content from its standard output. This enables restic to verify that
+the command completes with exit code zero. A non-zero exit code causes the
+backup to fail.
+
+Example: `restic backup --stdin-from-command mysqldump [...]`
+
+https://github.com/restic/restic/issues/4251
+https://github.com/restic/restic/pull/4410
diff --git a/cmd/restic/cmd_backup.go b/cmd/restic/cmd_backup.go
index 9499701aa..a2b81a759 100644
--- a/cmd/restic/cmd_backup.go
+++ b/cmd/restic/cmd_backup.go
@@ -97,6 +97,7 @@ type BackupOptions struct {
 	ExcludeLargerThan string
 	Stdin             bool
 	StdinFilename     string
+	StdinCommand      bool
 	Tags              restic.TagLists
 	Host              string
 	FilesFrom         []string
@@ -134,6 +135,7 @@ func init() {
 	f.StringVar(&backupOptions.ExcludeLargerThan, "exclude-larger-than", "", "max `size` of the files to be backed up (allowed suffixes: k/K, m/M, g/G, t/T)")
 	f.BoolVar(&backupOptions.Stdin, "stdin", false, "read backup from stdin")
 	f.StringVar(&backupOptions.StdinFilename, "stdin-filename", "stdin", "`filename` to use when reading from stdin")
+	f.BoolVar(&backupOptions.StdinCommand, "stdin-from-command", false, "execute command and store its stdout")
 	f.Var(&backupOptions.Tags, "tag", "add `tags` for the new snapshot in the format `tag[,tag,...]` (can be specified multiple times)")
 	f.UintVar(&backupOptions.ReadConcurrency, "read-concurrency", 0, "read `n` files concurrently (default: $RESTIC_READ_CONCURRENCY or 2)")
 	f.StringVarP(&backupOptions.Host, "host", "H", "", "set the `hostname` for the snapshot manually. To prevent an expensive rescan use the \"parent\" flag")
@@ -287,7 +289,7 @@ func (opts BackupOptions) Check(gopts GlobalOptions, args []string) error {
 		}
 	}
 
-	if opts.Stdin {
+	if opts.Stdin || opts.StdinCommand {
 		if len(opts.FilesFrom) > 0 {
 			return errors.Fatal("--stdin and --files-from cannot be used together")
 		}
@@ -298,7 +300,7 @@ func (opts BackupOptions) Check(gopts GlobalOptions, args []string) error {
 			return errors.Fatal("--stdin and --files-from-raw cannot be used together")
 		}
 
-		if len(args) > 0 {
+		if len(args) > 0 && !opts.StdinCommand {
 			return errors.Fatal("--stdin was specified and files/dirs were listed as arguments")
 		}
 	}
@@ -366,7 +368,7 @@ func collectRejectFuncs(opts BackupOptions, targets []string) (fs []RejectFunc,
 
 // collectTargets returns a list of target files/dirs from several sources.
 func collectTargets(opts BackupOptions, args []string) (targets []string, err error) {
-	if opts.Stdin {
+	if opts.Stdin || opts.StdinCommand {
 		return nil, nil
 	}
 
@@ -592,16 +594,24 @@ func runBackup(ctx context.Context, opts BackupOptions, gopts GlobalOptions, ter
 		defer localVss.DeleteSnapshots()
 		targetFS = localVss
 	}
-	if opts.Stdin {
+
+	if opts.Stdin || opts.StdinCommand {
 		if !gopts.JSON {
 			progressPrinter.V("read data from stdin")
 		}
 		filename := path.Join("/", opts.StdinFilename)
+		var source io.ReadCloser = os.Stdin
+		if opts.StdinCommand {
+			source, err = fs.NewCommandReader(ctx, args, globalOptions.stderr)
+			if err != nil {
+				return err
+			}
+		}
 		targetFS = &fs.Reader{
 			ModTime:    timeStamp,
 			Name:       filename,
 			Mode:       0644,
-			ReadCloser: os.Stdin,
+			ReadCloser: source,
 		}
 		targets = []string{filename}
 	}
@@ -630,7 +640,13 @@ func runBackup(ctx context.Context, opts BackupOptions, gopts GlobalOptions, ter
 	success := true
 	arch.Error = func(item string, err error) error {
 		success = false
-		return progressReporter.Error(item, err)
+		reterr := progressReporter.Error(item, err)
+		// If we receive a fatal error during the execution of the snapshot,
+		// we abort the snapshot.
+		if reterr == nil && errors.IsFatal(err) {
+			reterr = err
+		}
+		return reterr
 	}
 	arch.CompleteItem = progressReporter.CompleteItem
 	arch.StartFile = progressReporter.StartFile
diff --git a/doc/040_backup.rst b/doc/040_backup.rst
index c917c3c29..acafe2694 100644
--- a/doc/040_backup.rst
+++ b/doc/040_backup.rst
@@ -489,35 +489,71 @@ particular note are::
   - file ownership and ACLs on Windows
   - the "hidden" flag on Windows
 
+
+Reading data from a command
+***************************
+
+Sometimes, it can be useful to directly save the output of a program, for example,
+``mysqldump`` so that the SQL can later be restored. Restic supports this mode
+of operation; just supply the option ``--stdin-from-command`` when using the
+``backup`` action, and write the command in place of the files/directories:
+
+.. code-block:: console
+
+    $ restic -r /srv/restic-repo backup --stdin-from-command mysqldump [...]
+
+This command creates a new snapshot based on the standard output of ``mysqldump``.
+By default, the command's standard output is saved in a file named ``stdin``.
+A different name can be specified with ``--stdin-filename``:
+
+.. code-block:: console
+
+    $ restic -r /srv/restic-repo backup --stdin-filename production.sql --stdin-from-command mysqldump [...]
+
+Restic uses the command exit code to determine whether the command succeeded. A
+non-zero exit code from the command causes restic to cancel the backup. This causes
+restic to fail with exit code 1. No snapshot will be created in this case.
+
+
 Reading data from stdin
 ***********************
 
-Sometimes it can be nice to directly save the output of a program, e.g.
-``mysqldump`` so that the SQL can later be restored. Restic supports
-this mode of operation, just supply the option ``--stdin`` to the
-``backup`` command like this:
+.. warning::
+
+    Restic cannot detect if data read from stdin is complete or not. As explained
+    below, this can cause incomplete backup unless additional checks (outside of
+    restic) are configured. If possible, use ``--stdin-from-command`` instead.
+
+Alternatively, restic supports reading arbitrary data directly from the standard
+input. Use the option ``--stdin`` of the ``backup`` command as  follows:
 
 .. code-block:: console
 
-    $ set -o pipefail
-    $ mysqldump [...] | restic -r /srv/restic-repo backup --stdin
+    # Will not notice failures, see the warning below
+    $ gzip bigfile.dat | restic -r /srv/restic-repo backup --stdin
 
-This creates a new snapshot of the output of ``mysqldump``. You can then
-use e.g. the fuse mounting option (see below) to mount the repository
-and read the file.
+This creates a new snapshot of the content of ``bigfile.dat``.
+As for ``--stdin-from-command``, the default file name is ``stdin``; a
+different name can be specified with ``--stdin-filename``.
 
-By default, the file name ``stdin`` is used, a different name can be
-specified with ``--stdin-filename``, e.g. like this:
+**Important**: while it is possible to pipe a command output to restic using
+``--stdin``, doing so is discouraged as it will mask errors from the
+command, leading to corrupted backups. For example, in the following code
+block, if ``mysqldump`` fails to connect to the MySQL database, the restic
+backup will nevertheless succeed in creating an _empty_ backup:
 
 .. code-block:: console
 
-    $ mysqldump [...] | restic -r /srv/restic-repo backup --stdin --stdin-filename production.sql
+    # Will not notice failures, read the warning above
+    $ mysqldump [...] | restic -r /srv/restic-repo backup --stdin
 
-The option ``pipefail`` is highly recommended so that a non-zero exit code from
-one of the programs in the pipe (e.g. ``mysqldump`` here) makes the whole chain
-return a non-zero exit code. Refer to the `Use the Unofficial Bash Strict Mode
-<http://redsymbol.net/articles/unofficial-bash-strict-mode/>`__ for more
-details on this.
+A simple solution is to use ``--stdin-from-command`` (see above). If you
+still need to use the ``--stdin`` flag, you must use the shell option ``set -o pipefail``
+(so that a non-zero exit code from one of the programs in the pipe makes the
+whole chain return a non-zero exit code) and you must check the exit code of
+the pipe and act accordingly (e.g., remove the last backup). Refer to the
+`Use the Unofficial Bash Strict Mode <http://redsymbol.net/articles/unofficial-bash-strict-mode/>`__
+for more details on this.
 
 
 Tags for backup
diff --git a/internal/archiver/file_saver.go b/internal/archiver/file_saver.go
index 0742c8b57..724f5e620 100644
--- a/internal/archiver/file_saver.go
+++ b/internal/archiver/file_saver.go
@@ -2,6 +2,7 @@ package archiver
 
 import (
 	"context"
+	"fmt"
 	"io"
 	"os"
 	"sync"
@@ -146,7 +147,7 @@ func (s *FileSaver) saveFile(ctx context.Context, chnker *chunker.Chunker, snPat
 				panic("completed twice")
 			}
 			isCompleted = true
-			fnr.err = err
+			fnr.err = fmt.Errorf("failed to save %v: %w", target, err)
 			fnr.node = nil
 			fnr.stats = ItemStats{}
 			finish(fnr)
diff --git a/internal/fs/fs_reader_command.go b/internal/fs/fs_reader_command.go
new file mode 100644
index 000000000..3830e5811
--- /dev/null
+++ b/internal/fs/fs_reader_command.go
@@ -0,0 +1,97 @@
+package fs
+
+import (
+	"bufio"
+	"context"
+	"fmt"
+	"io"
+	"os/exec"
+
+	"github.com/restic/restic/internal/errors"
+)
+
+// CommandReader wrap a command such that its standard output can be read using
+// a io.ReadCloser. Close() waits for the command to terminate, reporting
+// any error back to the caller.
+type CommandReader struct {
+	cmd    *exec.Cmd
+	stdout io.ReadCloser
+
+	// cmd.Wait() must only be called once. Prevent duplicate executions in
+	// Read() and Close().
+	waitHandled bool
+
+	// alreadyClosedReadErr is the error that we should return if we try to
+	// read the pipe again after closing. This works around a Read() call that
+	// is issued after a previous Read() with `io.EOF` (but some bytes were
+	// read in the past).
+	alreadyClosedReadErr error
+}
+
+func NewCommandReader(ctx context.Context, args []string, logOutput io.Writer) (*CommandReader, error) {
+	// Prepare command and stdout
+	command := exec.CommandContext(ctx, args[0], args[1:]...)
+	stdout, err := command.StdoutPipe()
+	if err != nil {
+		return nil, fmt.Errorf("failed to setup stdout pipe: %w", err)
+	}
+
+	// Use a Go routine to handle the stderr to avoid deadlocks
+	stderr, err := command.StderrPipe()
+	if err != nil {
+		return nil, fmt.Errorf("failed to setup stderr pipe: %w", err)
+	}
+	go func() {
+		sc := bufio.NewScanner(stderr)
+		for sc.Scan() {
+			_, _ = fmt.Fprintf(logOutput, "subprocess %v: %v\n", command.Args[0], sc.Text())
+		}
+	}()
+
+	if err := command.Start(); err != nil {
+		return nil, fmt.Errorf("failed to start command: %w", err)
+	}
+
+	return &CommandReader{
+		cmd:    command,
+		stdout: stdout,
+	}, nil
+}
+
+// Read populate the array with data from the process stdout.
+func (fp *CommandReader) Read(p []byte) (int, error) {
+	if fp.alreadyClosedReadErr != nil {
+		return 0, fp.alreadyClosedReadErr
+	}
+	b, err := fp.stdout.Read(p)
+
+	// If the error is io.EOF, the program terminated. We need to check the
+	// exit code here because, if the program terminated with no output, the
+	// error in `Close()` is ignored.
+	if errors.Is(err, io.EOF) {
+		fp.waitHandled = true
+		// check if the command terminated successfully, If not return the error.
+		if errw := fp.wait(); errw != nil {
+			err = errw
+		}
+	}
+	fp.alreadyClosedReadErr = err
+	return b, err
+}
+
+func (fp *CommandReader) wait() error {
+	err := fp.cmd.Wait()
+	if err != nil {
+		// Use a fatal error to abort the snapshot.
+		return errors.Fatal(fmt.Errorf("command failed: %w", err).Error())
+	}
+	return nil
+}
+
+func (fp *CommandReader) Close() error {
+	if fp.waitHandled {
+		return nil
+	}
+
+	return fp.wait()
+}


Test patch:
diff --git a/cmd/restic/cmd_backup_integration_test.go b/cmd/restic/cmd_backup_integration_test.go
index 742b6ff6c..c60e9c543 100644
--- a/cmd/restic/cmd_backup_integration_test.go
+++ b/cmd/restic/cmd_backup_integration_test.go
@@ -568,3 +568,72 @@ func linkEqual(source, dest []string) bool {
 
 	return true
 }
+
+func TestStdinFromCommand(t *testing.T) {
+	env, cleanup := withTestEnvironment(t)
+	defer cleanup()
+
+	testSetupBackupData(t, env)
+	opts := BackupOptions{
+		StdinCommand:  true,
+		StdinFilename: "stdin",
+	}
+
+	testRunBackup(t, filepath.Dir(env.testdata), []string{"python", "-c", "import sys; print('something'); sys.exit(0)"}, opts, env.gopts)
+	testListSnapshots(t, env.gopts, 1)
+
+	testRunCheck(t, env.gopts)
+}
+
+func TestStdinFromCommandNoOutput(t *testing.T) {
+	env, cleanup := withTestEnvironment(t)
+	defer cleanup()
+
+	testSetupBackupData(t, env)
+	opts := BackupOptions{
+		StdinCommand:  true,
+		StdinFilename: "stdin",
+	}
+
+	err := testRunBackupAssumeFailure(t, filepath.Dir(env.testdata), []string{"python", "-c", "import sys; sys.exit(0)"}, opts, env.gopts)
+	rtest.Assert(t, err != nil && err.Error() == "at least one source file could not be read", "No data error expected")
+	testListSnapshots(t, env.gopts, 1)
+
+	testRunCheck(t, env.gopts)
+}
+
+func TestStdinFromCommandFailExitCode(t *testing.T) {
+	env, cleanup := withTestEnvironment(t)
+	defer cleanup()
+
+	testSetupBackupData(t, env)
+	opts := BackupOptions{
+		StdinCommand:  true,
+		StdinFilename: "stdin",
+	}
+
+	err := testRunBackupAssumeFailure(t, filepath.Dir(env.testdata), []string{"python", "-c", "import sys; print('test'); sys.exit(1)"}, opts, env.gopts)
+	rtest.Assert(t, err != nil, "Expected error while backing up")
+
+	testListSnapshots(t, env.gopts, 0)
+
+	testRunCheck(t, env.gopts)
+}
+
+func TestStdinFromCommandFailNoOutputAndExitCode(t *testing.T) {
+	env, cleanup := withTestEnvironment(t)
+	defer cleanup()
+
+	testSetupBackupData(t, env)
+	opts := BackupOptions{
+		StdinCommand:  true,
+		StdinFilename: "stdin",
+	}
+
+	err := testRunBackupAssumeFailure(t, filepath.Dir(env.testdata), []string{"python", "-c", "import sys; sys.exit(1)"}, opts, env.gopts)
+	rtest.Assert(t, err != nil, "Expected error while backing up")
+
+	testListSnapshots(t, env.gopts, 0)
+
+	testRunCheck(t, env.gopts)
+}
diff --git a/internal/fs/fs_reader_command_test.go b/internal/fs/fs_reader_command_test.go
new file mode 100644
index 000000000..a9028544c
--- /dev/null
+++ b/internal/fs/fs_reader_command_test.go
@@ -0,0 +1,48 @@
+package fs_test
+
+import (
+	"bytes"
+	"context"
+	"io"
+	"strings"
+	"testing"
+
+	"github.com/restic/restic/internal/fs"
+	"github.com/restic/restic/internal/test"
+)
+
+func TestCommandReaderSuccess(t *testing.T) {
+	reader, err := fs.NewCommandReader(context.TODO(), []string{"true"}, io.Discard)
+	test.OK(t, err)
+
+	_, err = io.Copy(io.Discard, reader)
+	test.OK(t, err)
+
+	test.OK(t, reader.Close())
+}
+
+func TestCommandReaderFail(t *testing.T) {
+	reader, err := fs.NewCommandReader(context.TODO(), []string{"false"}, io.Discard)
+	test.OK(t, err)
+
+	_, err = io.Copy(io.Discard, reader)
+	test.Assert(t, err != nil, "missing error")
+}
+
+func TestCommandReaderInvalid(t *testing.T) {
+	_, err := fs.NewCommandReader(context.TODO(), []string{"w54fy098hj7fy5twijouytfrj098y645wr"}, io.Discard)
+	test.Assert(t, err != nil, "missing error")
+}
+
+func TestCommandReaderOutput(t *testing.T) {
+	reader, err := fs.NewCommandReader(context.TODO(), []string{"echo", "hello world"}, io.Discard)
+	test.OK(t, err)
+
+	var buf bytes.Buffer
+
+	_, err = io.Copy(&buf, reader)
+	test.OK(t, err)
+	test.OK(t, reader.Close())
+
+	test.Equals(t, "hello world", strings.TrimSpace(buf.String()))
+}