Repository: pgjdbc/pgjdbc
Pull request #1659 (2019-12-23 08:07:24)

feat: add caller push of binary data (rebase of #953)
### All Submissions:
Rebase and fix of #953.

* [x] Have you followed the guidelines in our [Contributing](https://github.com/pgjdbc/pgjdbc/blob/master/CONTRIBUTING.md) document?
* [x] Have you checked to ensure there aren't other open [Pull Requests](../../pulls) for the same update/change?

<!-- You can erase any parts of this template not applicable to your Pull Request. -->

### New Feature Submissions:

1. [x] Does your submission pass tests?
2. [x] Does mvn checkstyle:check pass ?
3. [x] Have you added your new test classes to an existing test suite in alphabetical order?

### Changes to Existing Features:

* [ ] Does this break existing behaviour? If so please explain.
* [x] Have you added an explanation of what your changes do and why you'd like us to include them?
* [x] Have you written new tests for your core changes, as applicable?
* [ ] Have you successfully run tests with your changes locally?


paplorinc: @davecramer, addressed your concerns from https://github.com/pgjdbc/pgjdbc/pull/1656#discussion_r360183530 (pushed it again properly).
cc: @tomdcc 
davecramer: Thanks!
[WARNING] /home/travis/build/pgjdbc/pgjdbc/pgjdbc/target/gen-src/org/postgresql/util/ByteStreamWriter.java:62: warning: no @return
 is one of the travis errors
paplorinc: @davecramer, thanks for your quick review - one test (`testFastCloses`) seems to fail for a possibly unrelated reason - is it known flaky or should I investigate?
davecramer: ya, it's flaky, no need to muck with that one, unless you feel so inclined but I'd prefer if that was fixed in another PR
paplorinc: Is there anything else you would like us to do here?
davecramer: Nope, looks good, I just want to see if I can get travis green. Either way
I will push this later today

Thanks again

Dave Cramer


On Mon, 23 Dec 2019 at 14:41, Lőrinc Pap <notifications@github.com> wrote:

> Is there anything else you would like us to do here?
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> <https://github.com/pgjdbc/pgjdbc/pull/1659?email_source=notifications&email_token=AADDH5XXAT4KT6LU67YDTU3Q2EH7DA5CNFSM4J6REELKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEHRZU2I#issuecomment-568564329>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AADDH5S2AXILHMSDLL25LFLQ2EH7DANCNFSM4J6REELA>
> .
>

codecov-io: # [Codecov](https://codecov.io/gh/pgjdbc/pgjdbc/pull/1659?src=pr&el=h1) Report
> Merging [#1659](https://codecov.io/gh/pgjdbc/pgjdbc/pull/1659?src=pr&el=desc) into [master](https://codecov.io/gh/pgjdbc/pgjdbc/commit/a44ab4becd9e5c198bfbce84fc87da6d01e5f25a?src=pr&el=desc) will **increase** coverage by `0.09%`.
> The diff coverage is `87.71%`.



```diff
@@             Coverage Diff              @@
##             master    #1659      +/-   ##
============================================
+ Coverage     68.79%   68.89%   +0.09%     
- Complexity     4080     4104      +24     
============================================
  Files           181      183       +2     
  Lines         16957    17014      +57     
  Branches       2789     2794       +5     
============================================
+ Hits          11666    11721      +55     
- Misses         4011     4015       +4     
+ Partials       1280     1278       -2
```

paplorinc: Thanks @davecramer and @tomdcc.
Happy Christmas!

Patch:
diff --git a/pgjdbc/src/main/java/org/postgresql/core/FixedLengthOutputStream.java b/pgjdbc/src/main/java/org/postgresql/core/FixedLengthOutputStream.java
new file mode 100644
index 00000000..a2f4a254
--- /dev/null
+++ b/pgjdbc/src/main/java/org/postgresql/core/FixedLengthOutputStream.java
@@ -0,0 +1,52 @@
+/*
+ * Copyright (c) 2020, PostgreSQL Global Development Group
+ * See the LICENSE file in the project root for more information.
+ */
+
+package org.postgresql.core;
+
+import java.io.IOException;
+import java.io.OutputStream;
+
+/**
+ * A stream that refuses to write more than a maximum number of bytes.
+ */
+public class FixedLengthOutputStream extends OutputStream {
+
+  private final int size;
+  private final OutputStream target;
+  private int written;
+
+  public FixedLengthOutputStream(int size, OutputStream target) {
+    this.size = size;
+    this.target = target;
+  }
+
+  @Override
+  public void write(int b) throws IOException {
+    verifyAllowed(1);
+    written++;
+    target.write(b);
+  }
+
+  public void write(byte[] buf, int offset, int len) throws IOException {
+    if ((offset < 0) || (len < 0) || ((offset + len) > buf.length)) {
+      throw new IndexOutOfBoundsException();
+    } else if (len == 0) {
+      return;
+    }
+    verifyAllowed(len);
+    target.write(buf, offset, len);
+    written += len;
+  }
+
+  public int remaining() {
+    return size - written;
+  }
+
+  private void verifyAllowed(int wanted) throws IOException {
+    if (remaining() < wanted) {
+      throw new IOException("Attempt to write more than the specified " + size + " bytes");
+    }
+  }
+}
diff --git a/pgjdbc/src/main/java/org/postgresql/core/PGStream.java b/pgjdbc/src/main/java/org/postgresql/core/PGStream.java
index af7f227a..439d5193 100644
--- a/pgjdbc/src/main/java/org/postgresql/core/PGStream.java
+++ b/pgjdbc/src/main/java/org/postgresql/core/PGStream.java
@@ -5,6 +5,7 @@
 
 package org.postgresql.core;
 
+import org.postgresql.util.ByteStreamWriter;
 import org.postgresql.util.GT;
 import org.postgresql.util.HostSpec;
 import org.postgresql.util.PSQLException;
@@ -311,6 +312,32 @@ public class PGStream implements Closeable, Flushable {
     }
   }
 
+  /**
+   * Send a fixed-size array of bytes to the backend. If {@code length < siz}, pad with zeros. If
+   * {@code length > siz}, truncate the array.
+   *
+   * @param writer the stream writer to invoke to send the bytes
+   * @throws IOException if an I/O error occurs
+   */
+  public void send(ByteStreamWriter writer) throws IOException {
+    final FixedLengthOutputStream fixedLengthStream = new FixedLengthOutputStream(writer.getLength(), pgOutput);
+    try {
+      writer.writeTo(new ByteStreamWriter.ByteStreamTarget() {
+        @Override
+        public OutputStream getOutputStream() {
+          return fixedLengthStream;
+        }
+      });
+    } catch (IOException ioe) {
+      throw ioe;
+    } catch (Exception re) {
+      throw new IOException("Error writing bytes to stream", re);
+    }
+    for (int i = fixedLengthStream.remaining(); i > 0; i--) {
+      pgOutput.write(0);
+    }
+  }
+
   /**
    * Receives a single character from the backend, without advancing the current protocol stream
    * position.
diff --git a/pgjdbc/src/main/java/org/postgresql/core/ParameterList.java b/pgjdbc/src/main/java/org/postgresql/core/ParameterList.java
index de73c512..9c25868a 100644
--- a/pgjdbc/src/main/java/org/postgresql/core/ParameterList.java
+++ b/pgjdbc/src/main/java/org/postgresql/core/ParameterList.java
@@ -6,6 +6,8 @@
 
 package org.postgresql.core;
 
+import org.postgresql.util.ByteStreamWriter;
+
 import java.io.InputStream;
 import java.sql.SQLException;
 
@@ -122,6 +124,16 @@ public interface ParameterList {
    */
   void setBytea(int index, InputStream stream) throws SQLException;
 
+  /**
+   * Binds a binary bytea value stored as a ByteStreamWriter. The parameter's type is implicitly set to
+   * 'bytea'. The stream should remain valid until query execution has completed.
+   *
+   * @param index the 1-based parameter index to bind.
+   * @param writer a writer that can write the bytes for the parameter
+   * @throws SQLException on error or if <code>index</code> is out of range
+   */
+  void setBytea(int index, ByteStreamWriter writer) throws SQLException;
+
   /**
    * Binds a text value stored as an InputStream that is a valid UTF-8 byte stream.
    * Any byte-order marks (BOM) in the stream are passed to the backend.
diff --git a/pgjdbc/src/main/java/org/postgresql/core/v3/CompositeParameterList.java b/pgjdbc/src/main/java/org/postgresql/core/v3/CompositeParameterList.java
index 438c3121..0fabdc30 100644
--- a/pgjdbc/src/main/java/org/postgresql/core/v3/CompositeParameterList.java
+++ b/pgjdbc/src/main/java/org/postgresql/core/v3/CompositeParameterList.java
@@ -7,6 +7,7 @@
 package org.postgresql.core.v3;
 
 import org.postgresql.core.ParameterList;
+import org.postgresql.util.ByteStreamWriter;
 import org.postgresql.util.GT;
 import org.postgresql.util.PSQLException;
 import org.postgresql.util.PSQLState;
@@ -107,6 +108,11 @@ class CompositeParameterList implements V3ParameterList {
     subparams[sub].setBytea(index - offsets[sub], stream);
   }
 
+  public void setBytea(int index, ByteStreamWriter writer) throws SQLException {
+    int sub = findSubParam(index);
+    subparams[sub].setBytea(index - offsets[sub], writer);
+  }
+
   public void setText(int index, InputStream stream) throws SQLException {
     int sub = findSubParam(index);
     subparams[sub].setText(index - offsets[sub], stream);
diff --git a/pgjdbc/src/main/java/org/postgresql/core/v3/SimpleParameterList.java b/pgjdbc/src/main/java/org/postgresql/core/v3/SimpleParameterList.java
index 8d53eafa..1ce49996 100644
--- a/pgjdbc/src/main/java/org/postgresql/core/v3/SimpleParameterList.java
+++ b/pgjdbc/src/main/java/org/postgresql/core/v3/SimpleParameterList.java
@@ -14,6 +14,7 @@ import org.postgresql.geometric.PGbox;
 import org.postgresql.geometric.PGpoint;
 import org.postgresql.jdbc.UUIDArrayAssistant;
 import org.postgresql.util.ByteConverter;
+import org.postgresql.util.ByteStreamWriter;
 import org.postgresql.util.GT;
 import org.postgresql.util.PSQLException;
 import org.postgresql.util.PSQLState;
@@ -146,6 +147,11 @@ class SimpleParameterList implements V3ParameterList {
     bind(index, new StreamWrapper(stream), Oid.BYTEA, BINARY);
   }
 
+  @Override
+  public void setBytea(int index, ByteStreamWriter writer) throws SQLException {
+    bind(index, writer, Oid.BYTEA, BINARY);
+  }
+
   @Override
   public void setText(int index, InputStream stream) throws SQLException {
     bind(index, new StreamWrapper(stream), Oid.TEXT, TEXT);
@@ -291,6 +297,14 @@ class SimpleParameterList implements V3ParameterList {
     pgStream.sendStream(wrapper.getStream(), wrapper.getLength());
   }
 
+  //
+  // byte stream writer support
+  //
+
+  private static void streamBytea(PGStream pgStream, ByteStreamWriter writer) throws IOException {
+    pgStream.send(writer);
+  }
+
   public int[] getTypeOIDs() {
     return paramTypes;
   }
@@ -352,6 +366,11 @@ class SimpleParameterList implements V3ParameterList {
       return ((StreamWrapper) paramValues[index]).getLength();
     }
 
+    // Binary-format bytea?
+    if (paramValues[index] instanceof ByteStreamWriter) {
+      return ((ByteStreamWriter) paramValues[index]).getLength();
+    }
+
     // Already encoded?
     if (encoded[index] == null) {
       // Encode value and compute actual length using UTF-8.
@@ -381,6 +400,12 @@ class SimpleParameterList implements V3ParameterList {
       return;
     }
 
+    // Streamed bytea?
+    if (paramValues[index] instanceof ByteStreamWriter) {
+      streamBytea(pgStream, (ByteStreamWriter) paramValues[index]);
+      return;
+    }
+
     // Encoded string.
     if (encoded[index] == null) {
       encoded[index] = Utils.encodeUTF8((String) paramValues[index]);
diff --git a/pgjdbc/src/main/java/org/postgresql/jdbc/PgPreparedStatement.java b/pgjdbc/src/main/java/org/postgresql/jdbc/PgPreparedStatement.java
index 1dc994e8..fd25a280 100644
--- a/pgjdbc/src/main/java/org/postgresql/jdbc/PgPreparedStatement.java
+++ b/pgjdbc/src/main/java/org/postgresql/jdbc/PgPreparedStatement.java
@@ -18,6 +18,7 @@ import org.postgresql.core.v3.BatchedQuery;
 import org.postgresql.largeobject.LargeObject;
 import org.postgresql.largeobject.LargeObjectManager;
 import org.postgresql.util.ByteConverter;
+import org.postgresql.util.ByteStreamWriter;
 import org.postgresql.util.GT;
 import org.postgresql.util.HStoreConverter;
 import org.postgresql.util.PGBinaryObject;
@@ -363,6 +364,10 @@ class PgPreparedStatement extends PgStatement implements PreparedStatement {
     preparedParameters.setBytea(parameterIndex, copy, 0, x.length);
   }
 
+  private void setByteStreamWriter(int parameterIndex, ByteStreamWriter x) throws SQLException {
+    preparedParameters.setBytea(parameterIndex, x);
+  }
+
   public void setDate(int parameterIndex, java.sql.Date x) throws SQLException {
     setDate(parameterIndex, x, null);
   }
@@ -930,6 +935,8 @@ class PgPreparedStatement extends PgStatement implements PreparedStatement {
       setDouble(parameterIndex, (Double) x);
     } else if (x instanceof byte[]) {
       setBytes(parameterIndex, (byte[]) x);
+    } else if (x instanceof ByteStreamWriter) {
+      setByteStreamWriter(parameterIndex, (ByteStreamWriter) x);
     } else if (x instanceof java.sql.Date) {
       setDate(parameterIndex, (java.sql.Date) x);
     } else if (x instanceof Time) {


Test patch:
diff --git a/pgjdbc/src/main/java/org/postgresql/util/ByteBufferByteStreamWriter.java b/pgjdbc/src/main/java/org/postgresql/util/ByteBufferByteStreamWriter.java
new file mode 100644
index 00000000..be25be54
--- /dev/null
+++ b/pgjdbc/src/main/java/org/postgresql/util/ByteBufferByteStreamWriter.java
@@ -0,0 +1,48 @@
+/*
+ * Copyright (c) 2020, PostgreSQL Global Development Group
+ * See the LICENSE file in the project root for more information.
+ */
+
+package org.postgresql.util;
+
+import java.io.IOException;
+import java.nio.ByteBuffer;
+import java.nio.channels.Channels;
+import java.nio.channels.WritableByteChannel;
+
+/**
+ * A {@link ByteStreamWriter} that writes a {@link ByteBuffer java.nio.ByteBuffer} to a byte array
+ * parameter.
+ */
+public class ByteBufferByteStreamWriter implements ByteStreamWriter {
+
+  private final ByteBuffer buf;
+  private final int length;
+
+  /**
+   * Construct the writer with the given {@link ByteBuffer}
+   *
+   * @param buf the buffer to use.
+   */
+  public ByteBufferByteStreamWriter(ByteBuffer buf) {
+    this.buf = buf;
+    this.length = buf.remaining();
+  }
+
+  @Override
+  public int getLength() {
+    return length;
+  }
+
+  @Override
+  public void writeTo(ByteStreamTarget target) throws IOException {
+    // this _does_ involve some copying to a temporary buffer, but that's unavoidable
+    // as OutputStream itself only accepts single bytes or heap allocated byte arrays
+    WritableByteChannel c = Channels.newChannel(target.getOutputStream());
+    try {
+      c.write(buf);
+    } finally {
+      c.close();
+    }
+  }
+}
diff --git a/pgjdbc/src/main/java/org/postgresql/util/ByteStreamWriter.java b/pgjdbc/src/main/java/org/postgresql/util/ByteStreamWriter.java
new file mode 100644
index 00000000..798f938a
--- /dev/null
+++ b/pgjdbc/src/main/java/org/postgresql/util/ByteStreamWriter.java
@@ -0,0 +1,66 @@
+/*
+ * Copyright (c) 2020, PostgreSQL Global Development Group
+ * See the LICENSE file in the project root for more information.
+ */
+
+package org.postgresql.util;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+
+/**
+ * A class that can be used to set a byte array parameter by writing to an OutputStream.
+ *
+ * <p>The intended use case is wanting to write data to a byte array parameter that is stored off
+ * heap in a direct memory pool or in some other form that is inconvenient to assemble into a single
+ * heap-allocated buffer.</p>
+ * <p> Users should write their own implementation depending on the
+ * original data source. The driver provides a built-in implementation supporting the {@link
+ * java.nio.ByteBuffer} class, see {@link ByteBufferByteStreamWriter}.</p>
+ * <p> Intended usage is to simply pass in an instance using
+ * {@link java.sql.PreparedStatement#setObject(int, Object)}:</p>
+ * <pre>
+ *     int bufLength = someBufferObject.length();
+ *     preparedStatement.setObject(1, new MyByteStreamWriter(bufLength, someBufferObject));
+ * </pre>
+ * <p>The length must be known ahead of the stream being written to. </p>
+ * <p>This provides the application more control over memory management than calling
+ * {@link java.sql.PreparedStatement#setBinaryStream(int, InputStream)} as with the latter the
+ * caller has no control over the buffering strategy. </p>
+ */
+public interface ByteStreamWriter {
+
+  /**
+   * Returns the length of the stream.
+   *
+   * <p> This must be known ahead of calling {@link #writeTo(ByteStreamTarget)}. </p>
+   *
+   * @return the number of bytes in the stream.
+   */
+  int getLength();
+
+  /**
+   * Write the data to the provided {@link OutputStream}.
+   *
+   * <p> Should not write more than {@link #getLength()} bytes. If attempted, the provided stream
+   * will throw an {@link java.io.IOException}. </p>
+   *
+   * @param target the stream to write the data to
+   * @throws IOException if the underlying stream throws or there is some other error.
+   */
+  void writeTo(ByteStreamTarget target) throws IOException;
+
+  /**
+   * Provides a target to write bytes to.
+   */
+  interface ByteStreamTarget {
+
+    /**
+     * Provides an output stream to write bytes to.
+     *
+     * @return an output stream
+     */
+    OutputStream getOutputStream();
+  }
+}
diff --git a/pgjdbc/src/test/java/org/postgresql/test/core/FixedLengthOutputStreamTest.java b/pgjdbc/src/test/java/org/postgresql/test/core/FixedLengthOutputStreamTest.java
new file mode 100644
index 00000000..60587951
--- /dev/null
+++ b/pgjdbc/src/test/java/org/postgresql/test/core/FixedLengthOutputStreamTest.java
@@ -0,0 +1,93 @@
+/*
+ * Copyright (c) 2020, PostgreSQL Global Development Group
+ * See the LICENSE file in the project root for more information.
+ */
+
+package org.postgresql.test.core;
+
+import static org.junit.Assert.assertArrayEquals;
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.fail;
+
+import org.postgresql.core.FixedLengthOutputStream;
+
+import org.junit.After;
+import org.junit.Before;
+import org.junit.Test;
+
+import java.io.ByteArrayOutputStream;
+import java.io.IOException;
+import java.sql.SQLException;
+
+public class FixedLengthOutputStreamTest {
+
+  private ByteArrayOutputStream targetStream;
+  private FixedLengthOutputStream fixedLengthStream;
+
+  @Before
+  public void setUp() throws Exception {
+    targetStream = new ByteArrayOutputStream();
+    fixedLengthStream = new FixedLengthOutputStream(10, targetStream);
+  }
+
+  @After
+  public void tearDown() throws SQLException {
+  }
+
+  private void verifyExpectedOutput(byte[] expected) {
+    assertArrayEquals("Incorrect data written to target stream",
+        expected, targetStream.toByteArray());
+  }
+
+  @Test
+  public void testSingleByteWrites() throws IOException {
+    fixedLengthStream.write((byte) 1);
+    assertEquals("Incorrect remaining value", 9, fixedLengthStream.remaining());
+    fixedLengthStream.write((byte) 2);
+    assertEquals("Incorrect remaining value", 8, fixedLengthStream.remaining());
+    verifyExpectedOutput(new byte[]{1, 2});
+  }
+
+  @Test
+  public void testMultipleByteWrites() throws IOException {
+    fixedLengthStream.write(new byte[]{1, 2, 3, 4});
+    assertEquals("Incorrect remaining value", 6, fixedLengthStream.remaining());
+    fixedLengthStream.write(new byte[]{5, 6, 7, 8});
+    assertEquals("Incorrect remaining value", 2, fixedLengthStream.remaining());
+    verifyExpectedOutput(new byte[]{1, 2, 3, 4, 5, 6, 7, 8});
+  }
+
+  @Test
+  public void testSingleByteOverLimit() throws IOException {
+    byte[] data = new byte[]{1, 2, 3, 4, 5, 6, 7, 8, 9, 0};
+    fixedLengthStream.write(data);
+    assertEquals("Incorrect remaining value", 0, fixedLengthStream.remaining());
+    try {
+      fixedLengthStream.write((byte) 'a');
+      fail("Expected exception not thrown");
+    } catch (IOException e) {
+      assertEquals("Incorrect exception message",
+          "Attempt to write more than the specified 10 bytes", e.getMessage());
+    }
+    assertEquals("Incorrect remaining value after exception",
+        0, fixedLengthStream.remaining());
+    verifyExpectedOutput(data);
+  }
+
+  @Test
+  public void testMultipleBytesOverLimit() throws IOException {
+    byte[] data = new byte[]{1, 2, 3, 4, 5, 6, 7, 8};
+    fixedLengthStream.write(data);
+    assertEquals(2, fixedLengthStream.remaining());
+    try {
+      fixedLengthStream.write(new byte[]{'a', 'b', 'c', 'd'});
+      fail("Expected exception not thrown");
+    } catch (IOException e) {
+      assertEquals("Incorrect exception message",
+          "Attempt to write more than the specified 10 bytes", e.getMessage());
+    }
+    assertEquals("Incorrect remaining value after exception",
+        2, fixedLengthStream.remaining());
+    verifyExpectedOutput(data);
+  }
+}
diff --git a/pgjdbc/src/test/java/org/postgresql/test/jdbc2/Jdbc2TestSuite.java b/pgjdbc/src/test/java/org/postgresql/test/jdbc2/Jdbc2TestSuite.java
index 4e01958d..6b2b8a83 100644
--- a/pgjdbc/src/test/java/org/postgresql/test/jdbc2/Jdbc2TestSuite.java
+++ b/pgjdbc/src/test/java/org/postgresql/test/jdbc2/Jdbc2TestSuite.java
@@ -16,10 +16,13 @@ import org.postgresql.jdbc.DeepBatchedInsertStatementTest;
 import org.postgresql.jdbc.NoColumnMetadataIssue1613Test;
 import org.postgresql.jdbc.PgSQLXMLTest;
 import org.postgresql.jdbc.PrimitiveArraySupportTest;
+import org.postgresql.test.core.FixedLengthOutputStreamTest;
 import org.postgresql.test.core.JavaVersionTest;
 import org.postgresql.test.core.LogServerMessagePropertyTest;
 import org.postgresql.test.core.NativeQueryBindLengthTest;
 import org.postgresql.test.core.OptionsPropertyTest;
+import org.postgresql.test.util.ByteBufferByteStreamWriterTest;
+import org.postgresql.test.util.ByteStreamWriterTest;
 import org.postgresql.test.util.ExpressionPropertiesTest;
 import org.postgresql.test.util.HostSpecTest;
 import org.postgresql.test.util.LruCacheTest;
@@ -80,6 +83,9 @@ import org.junit.runners.Suite;
     OidValueOfTest.class,
     OptionsPropertyTest.class,
     OuterJoinSyntaxTest.class,
+    FixedLengthOutputStreamTest.class,
+    ByteStreamWriterTest.class,
+    ByteBufferByteStreamWriterTest.class,
     ParameterStatusTest.class,
     ParserTest.class,
     PGPropertyTest.class,
diff --git a/pgjdbc/src/test/java/org/postgresql/test/util/ByteBufferByteStreamWriterTest.java b/pgjdbc/src/test/java/org/postgresql/test/util/ByteBufferByteStreamWriterTest.java
new file mode 100644
index 00000000..6893f6b8
--- /dev/null
+++ b/pgjdbc/src/test/java/org/postgresql/test/util/ByteBufferByteStreamWriterTest.java
@@ -0,0 +1,74 @@
+/*
+ * Copyright (c) 2020, PostgreSQL Global Development Group
+ * See the LICENSE file in the project root for more information.
+ */
+
+package org.postgresql.test.util;
+
+import static org.junit.Assert.assertArrayEquals;
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.fail;
+
+import org.postgresql.util.ByteBufferByteStreamWriter;
+import org.postgresql.util.ByteStreamWriter;
+
+import org.junit.Before;
+import org.junit.Test;
+
+import java.io.ByteArrayOutputStream;
+import java.io.IOException;
+import java.io.OutputStream;
+import java.nio.ByteBuffer;
+
+public class ByteBufferByteStreamWriterTest {
+
+  private ByteArrayOutputStream targetStream;
+  private byte[] data;
+  private ByteBufferByteStreamWriter writer;
+
+  @Before
+  public void setUp() throws Exception {
+    targetStream = new ByteArrayOutputStream();
+    data = new byte[] { 1, 2, 3, 4 };
+    ByteBuffer buffer = ByteBuffer.wrap(data);
+    writer = new ByteBufferByteStreamWriter(buffer);
+  }
+
+  @Test
+  public void testReportsLengthCorrectly() {
+    assertEquals("Incorrect length reported", 4, writer.getLength());
+  }
+
+  @Test
+  public void testCopiesDataCorrectly() throws IOException {
+    writer.writeTo(target(targetStream));
+    byte[] written = targetStream.toByteArray();
+    assertArrayEquals("Incorrect data written to target stream", data, written);
+  }
+
+  @Test
+  public void testPropagatesException() throws IOException {
+    final IOException e = new IOException("oh no");
+    OutputStream errorStream = new OutputStream() {
+      @Override
+      public void write(int b) throws IOException {
+        throw e;
+      }
+    };
+    try {
+      writer.writeTo(target(errorStream));
+      fail("No exception thrown");
+    } catch (IOException caught) {
+      assertEquals("Exception was thrown that wasn't the expected one", caught, e);
+    }
+  }
+
+  private static ByteStreamWriter.ByteStreamTarget target(final OutputStream stream) {
+    return new ByteStreamWriter.ByteStreamTarget() {
+      @Override
+      public OutputStream getOutputStream() {
+        return stream;
+      }
+    };
+  }
+}
diff --git a/pgjdbc/src/test/java/org/postgresql/test/util/ByteStreamWriterTest.java b/pgjdbc/src/test/java/org/postgresql/test/util/ByteStreamWriterTest.java
new file mode 100644
index 00000000..6f3d771e
--- /dev/null
+++ b/pgjdbc/src/test/java/org/postgresql/test/util/ByteStreamWriterTest.java
@@ -0,0 +1,214 @@
+/*
+ * Copyright (c) 2020, PostgreSQL Global Development Group
+ * See the LICENSE file in the project root for more information.
+ */
+
+package org.postgresql.test.util;
+
+import static org.junit.Assert.assertArrayEquals;
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.assertTrue;
+import static org.junit.Assert.fail;
+
+import org.postgresql.test.TestUtil;
+import org.postgresql.test.jdbc2.BaseTest4;
+import org.postgresql.util.ByteBufferByteStreamWriter;
+import org.postgresql.util.ByteStreamWriter;
+
+import org.junit.Test;
+
+import java.io.IOException;
+import java.nio.ByteBuffer;
+import java.sql.PreparedStatement;
+import java.sql.ResultSet;
+import java.sql.SQLException;
+import java.util.Random;
+
+public class ByteStreamWriterTest extends BaseTest4 {
+
+  @Override
+  public void setUp() throws Exception {
+    super.setUp();
+    assumeByteaSupported();
+    TestUtil.createTempTable(con, "images", "img bytea");
+  }
+
+  private ByteBuffer testData(int size) {
+    ByteBuffer data = ByteBuffer.allocate(size);
+    Random random = new Random(31459);
+    while (data.remaining() > 0) {
+      data.putLong(random.nextLong());
+    }
+    data.rewind();
+    return data;
+  }
+
+  private void insertStream(ByteBuffer testData) throws Exception {
+    insertStream(testData, null);
+  }
+
+  private void insertStream(ByteBuffer testData, Integer lengthOverride) throws Exception {
+    insertStream(new TestByteBufferByteStreamWriter(testData, lengthOverride));
+  }
+
+  private void insertStream(ByteStreamWriter writer) throws Exception {
+    PreparedStatement updatePS = con.prepareStatement(TestUtil.insertSQL("images", "img", "?"));
+    try {
+      updatePS.setObject(1, writer);
+      updatePS.executeUpdate();
+    } finally {
+      updatePS.close();
+    }
+  }
+
+  private void validateContent(ByteBuffer data) throws Exception {
+    validateContent(data.array());
+  }
+
+  private void validateContent(byte [] data) throws Exception {
+    PreparedStatement selectPS = con.prepareStatement(TestUtil.selectSQL("images", "img"));
+    try {
+      ResultSet rs = selectPS.executeQuery();
+      try {
+        rs.next();
+        byte[] actualData = rs.getBytes(1);
+        assertArrayEquals("Sent and received data are not the same", data, actualData);
+      } finally {
+        rs.close();
+      }
+    } finally {
+      selectPS.close();
+    }
+  }
+
+  @Test
+  public void testEmpty() throws Exception {
+    ByteBuffer testData = testData(0);
+    insertStream(testData);
+    validateContent(testData);
+  }
+
+  @Test
+  public void testLength2Kb() throws Exception {
+    ByteBuffer testData = testData(2 * 1024);
+    insertStream(testData);
+    validateContent(testData);
+  }
+
+  @Test
+  public void testLength10Kb() throws Exception {
+    ByteBuffer testData = testData(10 * 1024);
+    insertStream(testData);
+    validateContent(testData);
+  }
+
+  @Test
+  public void testLength100Kb() throws Exception {
+    ByteBuffer testData = testData(100 * 1024);
+    insertStream(testData);
+    validateContent(testData);
+  }
+
+  @Test
+  public void testLength200Kb() throws Exception {
+    ByteBuffer testData = testData(200 * 1024);
+    insertStream(testData);
+    validateContent(testData);
+  }
+
+  @Test
+  public void testLengthGreaterThanContent() throws Exception {
+    ByteBuffer testData = testData(8);
+    insertStream(testData, 10);
+    byte[] expectedData = new byte[10];
+    testData.rewind();
+    testData.get(expectedData, 0, 8);
+    // other two bytes are zeroed out, which the jvm does for us automatically
+    validateContent(expectedData);
+  }
+
+  @Test
+  public void testLengthLessThanContent() throws Exception {
+    ByteBuffer testData = testData(8);
+    try {
+      insertStream(testData, 4);
+      fail("did not throw exception when too much content");
+    } catch (SQLException e) {
+      Throwable cause = e.getCause();
+      assertTrue("cause wan't an IOException", cause instanceof IOException);
+      assertEquals("Incorrect exception message",
+          cause.getMessage(), "Attempt to write more than the specified 4 bytes");
+    }
+  }
+
+  @Test
+  public void testIOExceptionPassedThroughAsCause() throws Exception {
+    IOException e = new IOException("oh no");
+    try {
+      insertStream(new ExceptionThrowingByteStreamWriter(e));
+      fail("did not throw exception when IOException thrown");
+    } catch (SQLException sqle) {
+      Throwable cause = sqle.getCause();
+      assertEquals("Incorrect exception cause", e, cause);
+    }
+  }
+
+  @Test
+  public void testRuntimeExceptionPassedThroughAsIOException() throws Exception {
+    RuntimeException e = new RuntimeException("oh no");
+    try {
+      insertStream(new ExceptionThrowingByteStreamWriter(e));
+      fail("did not throw exception when RuntimeException thrown");
+    } catch (SQLException sqle) {
+      Throwable cause = sqle.getCause();
+      assertTrue("cause wan't an IOException", cause instanceof IOException);
+      assertEquals("Incorrect exception message",
+          cause.getMessage(), "Error writing bytes to stream");
+      Throwable nestedCause = cause.getCause();
+      assertEquals("Incorrect exception cause", e, nestedCause);
+    }
+  }
+
+  /**
+   * Allows testing where reported length doesn't match what the stream writer attempts
+   */
+  private static class TestByteBufferByteStreamWriter extends ByteBufferByteStreamWriter {
+
+    private final Integer lengthOverride;
+
+    private TestByteBufferByteStreamWriter(ByteBuffer buf, Integer lengthOverride) {
+      super(buf);
+      this.lengthOverride = lengthOverride;
+    }
+
+    @Override
+    public int getLength() {
+      return lengthOverride != null ? lengthOverride : super.getLength();
+    }
+  }
+
+  private static class ExceptionThrowingByteStreamWriter implements ByteStreamWriter {
+
+    private final Throwable cause;
+
+    private ExceptionThrowingByteStreamWriter(Throwable cause) {
+      assertTrue(cause instanceof RuntimeException || cause instanceof IOException);
+      this.cause = cause;
+    }
+
+    @Override
+    public int getLength() {
+      return 1;
+    }
+
+    @Override
+    public void writeTo(ByteStreamTarget target) throws IOException {
+      if (cause instanceof RuntimeException) {
+        throw (RuntimeException) cause;
+      } else if (cause instanceof IOException) {
+        throw (IOException) cause;
+      }
+    }
+  }
+
+}