Repository: triton-inference-server/server
Pull request: Support data compression in HTTP

Problem statement:
Support input/output compression
**Is your feature request related to a problem? Please describe.**
Some algorithms may require large amounts of input data, which could saturate network bandwidth for clients. (Generative or other algorithms may correspondingly return large amounts of output data.) It could be very useful in these cases to be able to compress the input and/or output, thereby trading CPU (for compressing/decompressing) with bandwidth, depending on which resource is the most limited.

**Describe the solution you'd like**
Consistent client/server options to enable compression for input and/or output, with different compression schemes and levels supported.

**Describe alternatives you've considered**
This could potentially be done manually, using a custom pre- or post-processor backend in an ensemble model that would understand how to (de)compress the data, with the reverse process implemented by the user along with their interface to the client. However, this would result in a lot of duplicated effort and possibly inefficiencies.

**Additional context**
N/A

Hints from issue discussion:
GRPC has some compression options. Would you just like those exposed? HTTP would need a custom solution on both client and server side.  Do you have any specific solution(s), compressions, etc. in mind?
Yes, exposing the gRPC compression options would be a great starting point. (We're not using the HTTP client right now, so that's lower priority.)
Exposed the gRPC compression options via PR #2628. The compression option for HTTP is still a TODO.

Commit message: Fix up

Patch to review:
diff --git a/Dockerfile.QA b/Dockerfile.QA
index 4668122db1..27b7108967 100644
--- a/Dockerfile.QA
+++ b/Dockerfile.QA
@@ -119,7 +119,10 @@ RUN mkdir -p qa/common && \
     cp /tmp/tritonbuild/tritonserver/build/test-util/install/bin/triton_repo_agent_test qa/L0_triton_repo_agent/. && \
     cp /tmp/tritonbuild/tritonserver/build/test-util/install/lib/libtritonrepoagent_relocation.so qa/L0_triton_repo_agent/. && \
     cp /tmp/tritonbuild/tritonserver/build/test-util/install/bin/rate_limiter_test qa/L0_rate_limiter/. && \
-    cp /tmp/tritonbuild/tritonserver/build/test-util/install/bin/async_work_queue_test qa/L0_async_work_queue/.
+    cp /tmp/tritonbuild/tritonserver/build/test-util/install/bin/async_work_queue_test qa/L0_async_work_queue/. && \
+    mkdir qa/L0_data_compression/models && \
+    cp -r /workspace/docs/examples/model_repository/simple qa/L0_data_compression/models && \
+    cp /tmp/tritonbuild/tritonserver/build/test-util/install/bin/data_compressor_test qa/L0_data_compression/.
 
 # caffe2plan will not exist if the build was done without TensorRT enabled
 RUN if [ -f /tmp/tritonbuild/tritonserver/build/test-util/install/bin/caffe2plan ]; then \
diff --git a/build/CMakeLists.txt b/build/CMakeLists.txt
index 434c2502dc..351bc9d988 100644
--- a/build/CMakeLists.txt
+++ b/build/CMakeLists.txt
@@ -531,6 +531,7 @@ ExternalProject_Add(test-util
     ${_CMAKE_ARGS_VCPKG_TARGET_TRIPLET}
     -DProtobuf_DIR:PATH=${_FINDPACKAGE_PROTOBUF_CONFIG_DIR}
     -DGTEST_ROOT:PATH=${CMAKE_CURRENT_BINARY_DIR}/googletest
+    -DLibevent_DIR:PATH=${CMAKE_CURRENT_BINARY_DIR}/libevent/install/lib/cmake/libevent
     -DCNMEM_PATH:PATH=${CMAKE_CURRENT_BINARY_DIR}/cnmem
     -DTRITON_COMMON_REPO_TAG:STRING=${TRITON_COMMON_REPO_TAG}
     -DTRITON_CORE_REPO_TAG:STRING=${TRITON_CORE_REPO_TAG}
diff --git a/qa/L0_data_compression/test.sh b/qa/L0_data_compression/test.sh
new file mode 100644
index 0000000000..543556e0ce
--- /dev/null
+++ b/qa/L0_data_compression/test.sh
@@ -0,0 +1,157 @@
+#!/bin/bash
+# Copyright (c) 2021, NVIDIA CORPORATION. All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+#  * Redistributions of source code must retain the above copyright
+#    notice, this list of conditions and the following disclaimer.
+#  * Redistributions in binary form must reproduce the above copyright
+#    notice, this list of conditions and the following disclaimer in the
+#    documentation and/or other materials provided with the distribution.
+#  * Neither the name of NVIDIA CORPORATION nor the names of its
+#    contributors may be used to endorse or promote products derived
+#    from this software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
+# EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+# PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL THE COPYRIGHT OWNER OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+# EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+# PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+# OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+REPO_VERSION=${NVIDIA_TRITON_SERVER_VERSION}
+if [ "$#" -ge 1 ]; then
+    REPO_VERSION=$1
+fi
+if [ -z "$REPO_VERSION" ]; then
+    echo -e "Repository version must be specified"
+    echo -e "\n***\n*** Test Failed\n***"
+    exit 1
+fi
+
+source ../common/util.sh
+
+RET=0
+
+TEST_LOG="./data_compressor_test.log"
+DATA_COMPRESSOR_TEST=./data_compressor_test
+
+
+export CUDA_VISIBLE_DEVICES=0
+
+rm -fr *.log *_data
+
+set +e
+
+echo "All work and no play makes Jack a dull boy" >> raw_data
+python3 validation.py generate_compressed_data
+
+$DATA_COMPRESSOR_TEST >>$TEST_LOG 2>&1
+if [ $? -ne 0 ]; then
+    echo -e "\n***\n*** Data Compression Test Failed\n***"
+    RET=1
+fi
+
+python3 validation.py validate_compressed_data
+if [ $? -ne 0 ]; then
+    echo -e "\n***\n*** Data Compression Failed\n***"
+    RET=1
+fi
+
+set -e
+
+# End-to-end testing with simple model
+SIMPLE_INFER_CLIENT_PY=../clients/simple_http_infer_client.py
+SIMPLE_INFER_CLIENT=../clients/simple_http_infer_client
+
+CLIENT_LOG=`pwd`/client.log
+DATADIR=`pwd`/models
+SERVER=/opt/tritonserver/bin/tritonserver
+SERVER_ARGS="--model-repository=$DATADIR"
+
+run_server
+if [ "$SERVER_PID" == "0" ]; then
+    echo -e "\n***\n*** Failed to start $SERVER\n***"
+    cat $SERVER_LOG
+    exit 1
+fi
+
+set +e
+# Test various combinations
+python $SIMPLE_INFER_CLIENT_PY -v --input-compression-algorithm deflate >> "${CLIENT_LOG}" 2>&1
+if [ $? -ne 0 ]; then
+    RET=1
+fi
+
+python $SIMPLE_INFER_CLIENT_PY -v --input-compression-algorithm gzip >> "${CLIENT_LOG}" 2>&1
+if [ $? -ne 0 ]; then
+    RET=1
+fi
+
+python $SIMPLE_INFER_CLIENT_PY -v --output-compression-algorithm deflate >> "${CLIENT_LOG}" 2>&1
+if [ $? -ne 0 ]; then
+    RET=1
+fi
+
+python $SIMPLE_INFER_CLIENT_PY -v --output-compression-algorithm gzip >> "${CLIENT_LOG}" 2>&1
+if [ $? -ne 0 ]; then
+    RET=1
+fi
+
+python $SIMPLE_INFER_CLIENT_PY -v --input-compression-algorithm deflate --output-compression-algorithm gzip >> "${CLIENT_LOG}" 2>&1
+if [ $? -ne 0 ]; then
+    RET=1
+fi
+
+python $SIMPLE_INFER_CLIENT_PY -v --input-compression-algorithm gzip --output-compression-algorithm deflate >> "${CLIENT_LOG}" 2>&1
+if [ $? -ne 0 ]; then
+    RET=1
+fi
+
+$SIMPLE_INFER_CLIENT -v -i deflate >> "${CLIENT_LOG}" 2>&1
+if [ $? -ne 0 ]; then
+    RET=1
+fi
+
+$SIMPLE_INFER_CLIENT -v -i gzip >> "${CLIENT_LOG}" 2>&1
+if [ $? -ne 0 ]; then
+    RET=1
+fi
+
+$SIMPLE_INFER_CLIENT -v -o deflate >> "${CLIENT_LOG}" 2>&1
+if [ $? -ne 0 ]; then
+    RET=1
+fi
+
+$SIMPLE_INFER_CLIENT -v -o gzip >> "${CLIENT_LOG}" 2>&1
+if [ $? -ne 0 ]; then
+    RET=1
+fi
+
+$SIMPLE_INFER_CLIENT -v -i deflate -o gzip >> "${CLIENT_LOG}" 2>&1
+if [ $? -ne 0 ]; then
+    RET=1
+fi
+
+$SIMPLE_INFER_CLIENT -v -i gzip -o deflate >> "${CLIENT_LOG}" 2>&1
+if [ $? -ne 0 ]; then
+    RET=1
+fi
+set -e
+
+if [ $RET -eq 0 ]; then
+    echo -e "\n***\n*** Test Passed\n***"
+else
+    cat $TEST_LOG
+    cat $SERVER_LOG
+    cat ${CLIENT_LOG}
+    echo -e "\n***\n*** Test FAILED\n***"
+fi
+
+exit $RET
diff --git a/qa/L0_data_compression/validation.py b/qa/L0_data_compression/validation.py
new file mode 100644
index 0000000000..2a04f7b8cc
--- /dev/null
+++ b/qa/L0_data_compression/validation.py
@@ -0,0 +1,55 @@
+# Copyright (c) 2021, NVIDIA CORPORATION. All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+#  * Redistributions of source code must retain the above copyright
+#    notice, this list of conditions and the following disclaimer.
+#  * Redistributions in binary form must reproduce the above copyright
+#    notice, this list of conditions and the following disclaimer in the
+#    documentation and/or other materials provided with the distribution.
+#  * Neither the name of NVIDIA CORPORATION nor the names of its
+#    contributors may be used to endorse or promote products derived
+#    from this software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
+# EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+# PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL THE COPYRIGHT OWNER OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+# EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+# PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+# OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+import sys
+
+def generate_compressed_data():
+    with open("raw_data", "rb") as f:
+        import zlib
+        import gzip
+        raw_data = f.read()
+        with open("deflate_compressed_data", "wb") as of:
+            of.write(zlib.compress(raw_data))
+        with open("gzip_compressed_data", "wb") as of:
+            of.write(gzip.compress(raw_data))
+
+def validate_compressed_data():
+    with open("raw_data", "rb") as f:
+        import zlib
+        import gzip
+        raw_data = f.read()
+        with open("generated_deflate_compressed_data", "rb") as cf:
+            decompressed_data = zlib.decompress(cf.read())
+            if decompressed_data != raw_data:
+                exit(1)
+        with open("generated_gzip_compressed_data", "rb") as cf:
+            decompressed_data = gzip.decompress(cf.read())
+            if decompressed_data != raw_data:
+                exit(1)
+
+
+if __name__ == '__main__':
+    globals()[sys.argv[1]]()
diff --git a/src/core/constants.h b/src/core/constants.h
index 4284b355d4..a8ae74b354 100644
--- a/src/core/constants.h
+++ b/src/core/constants.h
@@ -1,4 +1,4 @@
-// Copyright (c) 2018-2020, NVIDIA CORPORATION. All rights reserved.
+// Copyright (c) 2018-2021, NVIDIA CORPORATION. All rights reserved.
 //
 // Redistribution and use in source and binary forms, with or without
 // modification, are permitted provided that the following conditions
@@ -31,6 +31,8 @@ namespace nvidia { namespace inferenceserver {
 
 constexpr char kInferHeaderContentLengthHTTPHeader[] =
     "Inference-Header-Content-Length";
+constexpr char kAcceptEncodingHTTPHeader[] = "Accept-Encoding";
+constexpr char kContentEncodingHTTPHeader[] = "Content-Encoding";
 
 #ifdef TRITON_ENABLE_TENSORFLOW
 constexpr char kTensorFlowGraphDefPlatform[] = "tensorflow_graphdef";
diff --git a/src/servers/data_compressor.h b/src/servers/data_compressor.h
new file mode 100644
index 0000000000..d742b04804
--- /dev/null
+++ b/src/servers/data_compressor.h
@@ -0,0 +1,295 @@
+// Copyright (c) 2021, NVIDIA CORPORATION. All rights reserved.
+//
+// Redistribution and use in source and binary forms, with or without
+// modification, are permitted provided that the following conditions
+// are met:
+//  * Redistributions of source code must retain the above copyright
+//    notice, this list of conditions and the following disclaimer.
+//  * Redistributions in binary form must reproduce the above copyright
+//    notice, this list of conditions and the following disclaimer in the
+//    documentation and/or other materials provided with the distribution.
+//  * Neither the name of NVIDIA CORPORATION nor the names of its
+//    contributors may be used to endorse or promote products derived
+//    from this software without specific prior written permission.
+//
+// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
+// EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+// IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+// PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL THE COPYRIGHT OWNER OR
+// CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+// EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+// PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+// OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+#pragma once
+
+#include <cassert>
+#include <cstring>
+#include <iostream>
+#include <memory>
+#include <string>
+#include <vector>
+
+#include <event2/buffer.h>
+#include <zlib.h>
+#include "src/servers/common.h"
+#include "triton/core/tritonserver.h"
+
+namespace nvidia { namespace inferenceserver {
+class DataCompressor {
+ public:
+  enum class Type { GZIP, DEFLATE };
+
+  // Specialization where the source and destination buffer are stored as
+  // evbuffer
+  static TRITONSERVER_Error* CompressData(
+      const Type type, evbuffer* source, evbuffer* compressed_data)
+  {
+    size_t expected_compressed_size = evbuffer_get_length(source);
+    // nothing to be compressed
+    if (expected_compressed_size == 0) {
+      return TRITONSERVER_ErrorNew(
+          TRITONSERVER_ERROR_INVALID_ARG, "nothing to be compressed");
+    }
+
+    z_stream stream;
+    stream.zalloc = Z_NULL;
+    stream.zfree = Z_NULL;
+    stream.opaque = Z_NULL;
+    switch (type) {
+      case Type::GZIP:
+        if (deflateInit2(
+                &stream, Z_DEFAULT_COMPRESSION /* level */,
+                Z_DEFLATED /* method */, 15 | 16 /* windowBits */,
+                8 /* memLevel */, Z_DEFAULT_STRATEGY /* strategy */) != Z_OK) {
+          return TRITONSERVER_ErrorNew(
+              TRITONSERVER_ERROR_INTERNAL,
+              "failed to initialize state for gzip data compression");
+        }
+        break;
+      case Type::DEFLATE: {
+        if (deflateInit(&stream, Z_DEFAULT_COMPRESSION /* level */) != Z_OK) {
+          return TRITONSERVER_ErrorNew(
+              TRITONSERVER_ERROR_INTERNAL,
+              "failed to initialize state for deflate data compression");
+        }
+        break;
+      }
+    }
+    // ensure the internal state are cleaned up on function return
+    std::unique_ptr<z_stream, decltype(&deflateEnd)> managed_stream(
+        &stream, deflateEnd);
+
+    // Get the addr and size of each chunk of memory in 'source'
+    struct evbuffer_iovec* buffer_array = nullptr;
+    int buffer_count = evbuffer_peek(source, -1, NULL, NULL, 0);
+    if (buffer_count > 0) {
+      buffer_array = static_cast<struct evbuffer_iovec*>(
+          alloca(sizeof(struct evbuffer_iovec) * buffer_count));
+      if (evbuffer_peek(source, -1, NULL, buffer_array, buffer_count) !=
+          buffer_count) {
+        return TRITONSERVER_ErrorNew(
+            TRITONSERVER_ERROR_INTERNAL,
+            "unexpected error getting buffers to be compressed");
+      }
+    }
+    // Reserve the same size as source for compressed data, it is less likely
+    // that a negative compression happens.
+    struct evbuffer_iovec current_reserved_space;
+    RETURN_MSG_IF_ERR(
+        AllocEVBuffer(
+            expected_compressed_size, compressed_data, &current_reserved_space),
+        "unexpected error allocating output buffer for compression: ");
+    stream.next_out =
+        reinterpret_cast<unsigned char*>(current_reserved_space.iov_base);
+    stream.avail_out = expected_compressed_size;
+
+    // Compress until end of 'source'
+    for (int idx = 0; idx < buffer_count; ++idx) {
+      stream.next_in =
+          reinterpret_cast<unsigned char*>(buffer_array[idx].iov_base);
+      stream.avail_in = buffer_array[idx].iov_len;
+
+      // run deflate() on input until source has been read in
+      do {
+        // Need additional buffer
+        if (stream.avail_out == 0) {
+          RETURN_MSG_IF_ERR(
+              CommitEVBuffer(
+                  compressed_data, &current_reserved_space,
+                  expected_compressed_size),
+              "unexpected error comitting output buffer for compression: ");
+          RETURN_MSG_IF_ERR(
+              AllocEVBuffer(
+                  expected_compressed_size, compressed_data,
+                  &current_reserved_space),
+              "unexpected error allocating output buffer for compression: ");
+          stream.next_out =
+              reinterpret_cast<unsigned char*>(current_reserved_space.iov_base);
+          stream.avail_out = expected_compressed_size;
+        }
+        auto flush = ((idx + 1) != buffer_count) ? Z_NO_FLUSH : Z_FINISH;
+        auto ret = deflate(&stream, flush);
+        if (ret == Z_STREAM_ERROR) {
+          return TRITONSERVER_ErrorNew(
+              TRITONSERVER_ERROR_INTERNAL,
+              "encountered inconsistent stream state during compression");
+        }
+      } while (stream.avail_out == 0);
+    }
+    // Make sure the last buffer is committed
+    if (current_reserved_space.iov_base != nullptr) {
+      RETURN_MSG_IF_ERR(
+          CommitEVBuffer(
+              compressed_data, &current_reserved_space,
+              expected_compressed_size - stream.avail_out),
+          "unexpected error comitting output buffer for compression: ");
+    }
+    return nullptr;  // success
+  }
+
+  static TRITONSERVER_Error* DecompressData(
+      const Type type, evbuffer* source, evbuffer* decompressed_data)
+  {
+    size_t source_byte_size = evbuffer_get_length(source);
+    // nothing to be decompressed
+    if (evbuffer_get_length(source) == 0) {
+      return TRITONSERVER_ErrorNew(
+          TRITONSERVER_ERROR_INVALID_ARG, "nothing to be compressed");
+    }
+    // Set reasonable size for each output buffer to be allocated
+    size_t output_buffer_size = (source_byte_size > (1 << 20 /* 1MB */))
+                                    ? source_byte_size
+                                    : (1 << 20 /* 1MB */);
+
+    switch (type) {
+      case Type::GZIP:
+      case Type::DEFLATE:
+        // zlib can automatically detect compression type
+        {
+          z_stream stream;
+          stream.zalloc = Z_NULL;
+          stream.zfree = Z_NULL;
+          stream.opaque = Z_NULL;
+          stream.avail_in = 0;
+          stream.next_in = Z_NULL;
+
+          if (inflateInit2(&stream, 15 | 32) != Z_OK) {
+            return TRITONSERVER_ErrorNew(
+                TRITONSERVER_ERROR_INTERNAL,
+                "failed to initialize state for data decompression");
+          }
+          // ensure the internal state are cleaned up on function return
+          std::unique_ptr<z_stream, decltype(&inflateEnd)> managed_stream(
+              &stream, inflateEnd);
+
+          // Get the addr and size of each chunk of memory in 'source'
+          struct evbuffer_iovec* buffer_array = nullptr;
+          int buffer_count = evbuffer_peek(source, -1, NULL, NULL, 0);
+          if (buffer_count > 0) {
+            buffer_array = static_cast<struct evbuffer_iovec*>(
+                alloca(sizeof(struct evbuffer_iovec) * buffer_count));
+            if (evbuffer_peek(source, -1, NULL, buffer_array, buffer_count) !=
+                buffer_count) {
+              return TRITONSERVER_ErrorNew(
+                  TRITONSERVER_ERROR_INTERNAL,
+                  "unexpected error getting buffers to be decompressed");
+            }
+          }
+          // Reserve the same size as source for compressed data, it is less
+          // likely that a negative compression happens.
+          struct evbuffer_iovec current_reserved_space;
+          RETURN_MSG_IF_ERR(
+              AllocEVBuffer(
+                  output_buffer_size, decompressed_data,
+                  &current_reserved_space),
+              "unexpected error allocating output buffer for decompression: ");
+          stream.next_out =
+              reinterpret_cast<unsigned char*>(current_reserved_space.iov_base);
+          stream.avail_out = output_buffer_size;
+
+          // Compress until end of 'source'
+          for (int idx = 0; idx < buffer_count; ++idx) {
+            stream.next_in =
+                reinterpret_cast<unsigned char*>(buffer_array[idx].iov_base);
+            stream.avail_in = buffer_array[idx].iov_len;
+
+            // run inflate() on input until source has been read in
+            do {
+              // Need additional buffer
+              if (stream.avail_out == 0) {
+                RETURN_MSG_IF_ERR(
+                    CommitEVBuffer(
+                        decompressed_data, &current_reserved_space,
+                        output_buffer_size),
+                    "unexpected error comitting output buffer for "
+                    "decompression: ");
+                RETURN_MSG_IF_ERR(
+                    AllocEVBuffer(
+                        output_buffer_size, decompressed_data,
+                        &current_reserved_space),
+                    "unexpected error allocating output buffer for "
+                    "decompression: ");
+                stream.next_out = reinterpret_cast<unsigned char*>(
+                    current_reserved_space.iov_base);
+                stream.avail_out = output_buffer_size;
+              }
+              auto ret = inflate(&stream, Z_NO_FLUSH);
+              if (ret == Z_STREAM_ERROR) {
+                return TRITONSERVER_ErrorNew(
+                    TRITONSERVER_ERROR_INTERNAL,
+                    "encountered inconsistent stream state during "
+                    "decompression");
+              }
+            } while (stream.avail_out == 0);
+          }
+          // Make sure the last buffer is committed
+          if (current_reserved_space.iov_base != nullptr) {
+            RETURN_MSG_IF_ERR(
+                CommitEVBuffer(
+                    decompressed_data, &current_reserved_space,
+                    output_buffer_size - stream.avail_out),
+                "unexpected error comitting output buffer for compression: ");
+          }
+          break;
+        }
+    }
+    return nullptr;  // success
+  }
+
+ private:
+  static TRITONSERVER_Error* AllocEVBuffer(
+      const size_t byte_size, evbuffer* evb,
+      struct evbuffer_iovec* current_reserved_space)
+  {
+    // Reserve requested space in evbuffer...
+    if ((evbuffer_reserve_space(evb, byte_size, current_reserved_space, 1) !=
+         1) ||
+        (current_reserved_space->iov_len < byte_size)) {
+      return TRITONSERVER_ErrorNew(
+          TRITONSERVER_ERROR_INTERNAL,
+          std::string(
+              "failed to reserve " + std::to_string(byte_size) +
+              " bytes in evbuffer")
+              .c_str());
+    }
+    return nullptr;  // success
+  }
+
+  static TRITONSERVER_Error* CommitEVBuffer(
+      evbuffer* evb, struct evbuffer_iovec* current_reserved_space,
+      const size_t filled_byte_size)
+  {
+    current_reserved_space->iov_len = filled_byte_size;
+    if (evbuffer_commit_space(evb, current_reserved_space, 1) != 0) {
+      return TRITONSERVER_ErrorNew(
+          TRITONSERVER_ERROR_INTERNAL, "failed to commit allocated evbuffer");
+    }
+    current_reserved_space->iov_base = nullptr;
+    return nullptr;  // success
+  }
+};
+
+}}  // namespace nvidia::inferenceserver
diff --git a/src/servers/http_server.cc b/src/servers/http_server.cc
index 8c373571c2..c335f34346 100644
--- a/src/servers/http_server.cc
+++ b/src/servers/http_server.cc
@@ -1,4 +1,4 @@
-// Copyright (c) 2019-2020, NVIDIA CORPORATION. All rights reserved.
+// Copyright (c) 2019-2021, NVIDIA CORPORATION. All rights reserved.
 //
 // Redistribution and use in source and binary forms, with or without
 // modification, are permitted provided that the following conditions
@@ -39,6 +39,7 @@
 #include "src/core/model_config.h"
 #include "src/servers/classification.h"
 #include "src/servers/common.h"
+#include "src/servers/data_compressor.h"
 
 #define TRITONJSON_STATUSTYPE TRITONSERVER_Error*
 #define TRITONJSON_STATUSRETURN(M) \
@@ -2344,13 +2345,38 @@ HTTPAPIServer::HandleInfer(
       header_length = std::atoi(header_length_c_str);
     }
 
-    err = EVBufferToInput(
-        model_name, irequest, req->buffer_in, infer_request.get(),
-        header_length);
+    // Find Content-Encoding in header. Decompress if not empty
+    evbuffer* decompressed_buffer = nullptr;
+    const char* content_encoding_c_str =
+        evhtp_kv_find(req->headers_in, kContentEncodingHTTPHeader);
+    if (content_encoding_c_str != NULL) {
+      std::string content_encoding(content_encoding_c_str);
+      if (content_encoding == "deflate") {
+        decompressed_buffer = evbuffer_new();
+        err = DataCompressor::DecompressData(
+            DataCompressor::Type::DEFLATE, req->buffer_in, decompressed_buffer);
+      } else if (content_encoding == "gzip") {
+        decompressed_buffer = evbuffer_new();
+        err = DataCompressor::DecompressData(
+            DataCompressor::Type::GZIP, req->buffer_in, decompressed_buffer);
+      } else if (!content_encoding.empty()) {
+        err = TRITONSERVER_ErrorNew(
+            TRITONSERVER_ERROR_INVALID_ARG,
+            (std::string("unsupported Content-Encoding: ") + content_encoding)
+                .c_str());
+      }
+    }
+    if (err == nullptr) {
+      err = EVBufferToInput(
+          model_name, irequest,
+          (decompressed_buffer == nullptr) ? req->buffer_in
+                                           : decompressed_buffer,
+          infer_request.get(), header_length);
+    }
     if (err == nullptr) {
       err = TRITONSERVER_InferenceRequestSetReleaseCallback(
           irequest, InferRequestClass::InferRequestComplete,
-          nullptr /* request_release_userp */);
+          decompressed_buffer);
       if (err == nullptr) {
         err = TRITONSERVER_InferenceRequestSetResponseCallback(
             irequest, allocator_,
@@ -2471,6 +2497,11 @@ HTTPAPIServer::InferRequestClass::InferResponseComplete(
   HTTPAPIServer::InferRequestClass* infer_request =
       reinterpret_cast<HTTPAPIServer::InferRequestClass*>(userp);
 
+  // Always specify supported compression as Accept-Encoding
+  evhtp_headers_add_header(
+            infer_request->req_->headers_out,
+            evhtp_header_new(kAcceptEncodingHTTPHeader, "gzip, deflate", 1, 1));
+
   auto response_count = infer_request->IncrementResponseCount();
 
   // Defer to the callback with the final response
@@ -2720,16 +2751,17 @@ HTTPAPIServer::InferRequestClass::FinalizeResponse(
 
   RETURN_IF_ERR(response_json.Add("outputs", std::move(response_outputs)));
 
+  evbuffer* response_placeholder = evbuffer_new();
   // Write json metadata into response evbuffer
   triton::common::TritonJson::WriteBuffer buffer;
   RETURN_IF_ERR(response_json.Write(&buffer));
-  evbuffer_add(req_->buffer_out, buffer.Base(), buffer.Size());
+  evbuffer_add(response_placeholder, buffer.Base(), buffer.Size());
 
   // If there is binary data write it next in the appropriate
   // order... also need the HTTP header when returning binary data.
   if (!ordered_buffers.empty()) {
     for (evbuffer* b : ordered_buffers) {
-      evbuffer_add_buffer(req_->buffer_out, b);
+      evbuffer_add_buffer(response_placeholder, b);
     }
 
     evhtp_headers_add_header(
@@ -2738,6 +2770,53 @@ HTTPAPIServer::InferRequestClass::FinalizeResponse(
                                std::to_string(buffer.Size()).c_str(), 1, 1));
   }
 
+  evbuffer* response_body = response_placeholder;
+  // Find Accept-Encoding in header. Try to compress if found
+  const char* accept_encoding_c_str =
+      evhtp_kv_find(req_->headers_in, kAcceptEncodingHTTPHeader);
+  if (accept_encoding_c_str != NULL) {
+    // FIXME compress only work for the simplest case for now
+    // (one accept encoding specified)
+    std::string accept_encoding(accept_encoding_c_str);
+    if (accept_encoding == "deflate") {
+      auto compressed_buffer = evbuffer_new();
+      auto err = DataCompressor::CompressData(
+          DataCompressor::Type::DEFLATE, response_placeholder,
+          compressed_buffer);
+      if (err == nullptr) {
+        evhtp_headers_add_header(
+            req_->headers_out,
+            evhtp_header_new(kContentEncodingHTTPHeader, "deflate", 1, 1));
+        response_body = compressed_buffer;
+        evbuffer_free(response_placeholder);
+      } else {
+        // just log the compression error and return the uncompressed data
+        LOG_VERBOSE(1) << "unable to compress response: "
+                       << TRITONSERVER_ErrorMessage(err);
+        TRITONSERVER_ErrorDelete(err);
+        evbuffer_free(compressed_buffer);
+      }
+    } else if (accept_encoding == "gzip") {
+      auto compressed_buffer = evbuffer_new();
+      auto err = DataCompressor::CompressData(
+          DataCompressor::Type::GZIP, response_placeholder, compressed_buffer);
+      if (err == nullptr) {
+        evhtp_headers_add_header(
+            req_->headers_out,
+            evhtp_header_new(kContentEncodingHTTPHeader, "gzip", 1, 1));
+        response_body = compressed_buffer;
+        evbuffer_free(response_placeholder);
+      } else {
+        // just log the compression error and return the uncompressed data
+        LOG_VERBOSE(1) << "unable to compress response: "
+                       << TRITONSERVER_ErrorMessage(err);
+        TRITONSERVER_ErrorDelete(err);
+        evbuffer_free(compressed_buffer);
+      }
+    }
+  }
+  evbuffer_add_buffer(req_->buffer_out, response_body);
+
   return nullptr;  // success
 }
 
diff --git a/src/test/CMakeLists.txt b/src/test/CMakeLists.txt
index ccfb7e1e02..0c6a3ec7c5 100644
--- a/src/test/CMakeLists.txt
+++ b/src/test/CMakeLists.txt
@@ -246,6 +246,59 @@ install(
   RUNTIME DESTINATION bin
 )
 
+#
+# Unit test for DataCompressor
+#
+set(
+  DATA_COMPRESSOR_HDRS
+  ../servers/data_compressor.h
+  ../servers/common.h
+)
+
+set(
+  DATA_COMPRESSOR_TEST_SRCS
+  data_compressor_test.cc
+)
+
+set(
+  DATA_COMPRESSOR_TEST_HDRS
+  ${DATA_COMPRESSOR_HDRS}
+)
+
+find_package(GTest REQUIRED)
+find_package(Libevent CONFIG REQUIRED)
+include_directories(${LIBEVENT_INCLUDE_DIRS})
+add_executable(
+  data_compressor_test
+  ${DATA_COMPRESSOR_TEST_SRCS}
+  ${DATA_COMPRESSOR_TEST_HDRS}
+)
+set_target_properties(
+  data_compressor_test
+  PROPERTIES
+    SKIP_BUILD_RPATH TRUE
+    BUILD_WITH_INSTALL_RPATH TRUE
+    INSTALL_RPATH_USE_LINK_PATH FALSE
+    INSTALL_RPATH ""
+)
+target_include_directories(
+  data_compressor_test
+  PRIVATE ${GTEST_INCLUDE_DIR}
+)
+target_link_libraries(
+  data_compressor_test
+  PRIVATE triton-core-serverapi  # from repo-core
+  PRIVATE ${GTEST_LIBRARY}
+  PRIVATE ${GTEST_MAIN_LIBRARY}
+  PRIVATE -lpthread
+  PRIVATE -lz
+  PRIVATE ${LIBEVENT_LIBRARIES}
+)
+install(
+  TARGETS data_compressor_test
+  RUNTIME DESTINATION bin
+)
+
 # Generate C++ protobuf files for the following tests
 protobuf_generate_cpp(MODEL_CONFIG_PROTO_SRC MODEL_CONFIG_PROTO_HDR ../core/model_config.proto)
 
diff --git a/src/test/data_compressor_test.cc b/src/test/data_compressor_test.cc
new file mode 100644
index 0000000000..344ff2d262
--- /dev/null
+++ b/src/test/data_compressor_test.cc
@@ -0,0 +1,625 @@
+// Copyright (c) 2021, NVIDIA CORPORATION. All rights reserved.
+//
+// Redistribution and use in source and binary forms, with or without
+// modification, are permitted provided that the following conditions
+// are met:
+//  * Redistributions of source code must retain the above copyright
+//    notice, this list of conditions and the following disclaimer.
+//  * Redistributions in binary form must reproduce the above copyright
+//    notice, this list of conditions and the following disclaimer in the
+//    documentation and/or other materials provided with the distribution.
+//  * Neither the name of NVIDIA CORPORATION nor the names of its
+//    contributors may be used to endorse or promote products derived
+//    from this software without specific prior written permission.
+//
+// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
+// EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+// IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+// PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL THE COPYRIGHT OWNER OR
+// CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+// EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+// PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+// OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+#include "src/servers/data_compressor.h"
+
+// Undefine the FAIL() macro inside Triton code to avoid redefine error
+// from gtest. Okay as FAIL() is not used in data_compressor
+#ifdef FAIL
+#undef FAIL
+#endif
+
+#include "gtest/gtest.h"
+
+#include <event2/buffer.h>
+#include <chrono>
+#include <condition_variable>
+#include <fstream>
+#include <future>
+#include <limits>
+#include <mutex>
+#include <random>
+#include <string>
+#include <thread>
+#include <vector>
+
+namespace ni = nvidia::inferenceserver;
+
+namespace {
+
+struct TritonServerError {
+  TritonServerError(TRITONSERVER_Error_Code code, const char* msg)
+      : code_(code), msg_(msg)
+  {
+  }
+  TRITONSERVER_Error_Code code_;
+  std::string msg_;
+};
+
+void
+WriteEVBufferToFile(const std::string& file_name, evbuffer* evb)
+{
+  std::ofstream fs(file_name);
+  struct evbuffer_iovec* buffer_array = nullptr;
+  int buffer_count = evbuffer_peek(evb, -1, NULL, NULL, 0);
+  if (buffer_count > 0) {
+    buffer_array = static_cast<struct evbuffer_iovec*>(
+        alloca(sizeof(struct evbuffer_iovec) * buffer_count));
+    ASSERT_EQ(
+        evbuffer_peek(evb, -1, NULL, buffer_array, buffer_count), buffer_count)
+        << "unexpected error getting buffers for result";
+  }
+  for (int idx = 0; idx < buffer_count; ++idx) {
+    fs.write(
+        reinterpret_cast<const char*>(buffer_array[idx].iov_base),
+        buffer_array[idx].iov_len);
+  }
+}
+
+void
+EVBufferToContiguousBuffer(evbuffer* evb, std::vector<char>* buffer)
+{
+  *buffer = std::vector<char>(evbuffer_get_length(evb));
+  {
+    struct evbuffer_iovec* buffer_array = nullptr;
+    int buffer_count = evbuffer_peek(evb, -1, NULL, NULL, 0);
+    if (buffer_count > 0) {
+      buffer_array = static_cast<struct evbuffer_iovec*>(
+          alloca(sizeof(struct evbuffer_iovec) * buffer_count));
+      ASSERT_EQ(
+          evbuffer_peek(evb, -1, NULL, buffer_array, buffer_count),
+          buffer_count)
+          << "unexpected error getting buffers for result";
+    }
+    size_t offset = 0;
+    for (int idx = 0; idx < buffer_count; ++idx) {
+      memcpy(
+          buffer->data() + offset, buffer_array[idx].iov_base,
+          buffer_array[idx].iov_len);
+      offset += buffer_array[idx].iov_len;
+    }
+  }
+}
+
+}  // namespace
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+TRITONSERVER_Error*
+TRITONSERVER_ErrorNew(TRITONSERVER_Error_Code code, const char* msg)
+{
+  return reinterpret_cast<TRITONSERVER_Error*>(
+      new TritonServerError(code, msg));
+}
+
+TRITONSERVER_Error_Code
+TRITONSERVER_ErrorCode(TRITONSERVER_Error* error)
+{
+  return (reinterpret_cast<TritonServerError*>(error))->code_;
+}
+
+const char*
+TRITONSERVER_ErrorMessage(TRITONSERVER_Error* error)
+{
+  return (reinterpret_cast<TritonServerError*>(error))->msg_.c_str();
+}
+
+#ifdef __cplusplus
+}
+#endif
+
+namespace {
+
+class DataCompressorTest : public ::testing::Test {
+ public:
+  DataCompressorTest()
+      : raw_data_length_(0), deflate_compressed_length_(0),
+        gzip_compressed_length_(0)
+  {
+    std::vector<std::string> files{"raw_data", "deflate_compressed_data",
+                                   "gzip_compressed_data"};
+    for (const auto& file : files) {
+      std::fstream fs(file);
+      // get length of file
+      fs.seekg(0, fs.end);
+      int length = fs.tellg();
+      fs.seekg(0, fs.beg);
+
+      // allocate memory
+      char* data = nullptr;
+      if (file == "raw_data") {
+        raw_data_.reset(new char[length]);
+        data = raw_data_.get();
+        raw_data_length_ = length;
+      } else if (file == "deflate_compressed_data") {
+        deflate_compressed_data_.reset(new char[length]);
+        data = deflate_compressed_data_.get();
+        deflate_compressed_length_ = length;
+      } else {
+        gzip_compressed_data_.reset(new char[length]);
+        data = gzip_compressed_data_.get();
+        gzip_compressed_length_ = length;
+      }
+
+      fs.read(data, length);
+    }
+  }
+
+  std::unique_ptr<char[]> raw_data_;
+  size_t raw_data_length_;
+  std::unique_ptr<char[]> deflate_compressed_data_;
+  size_t deflate_compressed_length_;
+  std::unique_ptr<char[]> gzip_compressed_data_;
+  size_t gzip_compressed_length_;
+};
+
+TEST_F(DataCompressorTest, DeflateOneBuffer)
+{
+  // Convert the raw data into evbuffer format
+  auto source = evbuffer_new();
+  ASSERT_TRUE((source != nullptr)) << "Failed to create source evbuffer";
+  ASSERT_EQ(evbuffer_add(source, raw_data_.get(), raw_data_length_), 0)
+      << "Failed to initialize source evbuffer";
+
+  auto compressed = evbuffer_new();
+  ASSERT_TRUE((compressed != nullptr))
+      << "Failed to create compressed evbuffer";
+  auto decompressed = evbuffer_new();
+  ASSERT_TRUE((decompressed != nullptr))
+      << "Failed to create decompressed evbuffer";
+
+  auto err = ni::DataCompressor::CompressData(
+      ni::DataCompressor::Type::DEFLATE, source, compressed);
+  ASSERT_TRUE((err == nullptr))
+      << "Failed to compress data: " << TRITONSERVER_ErrorMessage(err);
+
+  err = ni::DataCompressor::DecompressData(
+      ni::DataCompressor::Type::DEFLATE, compressed, decompressed);
+  ASSERT_TRUE((err == nullptr))
+      << "Failed to decompress data: " << TRITONSERVER_ErrorMessage(err);
+
+  size_t destination_byte_size = evbuffer_get_length(decompressed);
+  ASSERT_EQ(destination_byte_size, raw_data_length_) << "Mismatched byte size";
+
+  std::vector<char> res;
+  EVBufferToContiguousBuffer(decompressed, &res);
+  for (size_t idx = 0; idx < raw_data_length_; ++idx) {
+    ASSERT_TRUE(raw_data_[idx] == res[idx]);
+  }
+}
+
+TEST_F(DataCompressorTest, GzipOneBuffer)
+{
+  // Convert the raw data into evbuffer format
+  auto source = evbuffer_new();
+  ASSERT_TRUE((source != nullptr)) << "Failed to create source evbuffer";
+  ASSERT_EQ(evbuffer_add(source, raw_data_.get(), raw_data_length_), 0)
+      << "Failed to initialize source evbuffer";
+
+  auto compressed = evbuffer_new();
+  ASSERT_TRUE((compressed != nullptr))
+      << "Failed to create compressed evbuffer";
+  auto decompressed = evbuffer_new();
+  ASSERT_TRUE((decompressed != nullptr))
+      << "Failed to create decompressed evbuffer";
+
+  auto err = ni::DataCompressor::CompressData(
+      ni::DataCompressor::Type::GZIP, source, compressed);
+  ASSERT_TRUE((err == nullptr))
+      << "Failed to compress data: " << TRITONSERVER_ErrorMessage(err);
+  err = ni::DataCompressor::DecompressData(
+      ni::DataCompressor::Type::GZIP, compressed, decompressed);
+  ASSERT_TRUE((err == nullptr))
+      << "Failed to decompress data: " << TRITONSERVER_ErrorMessage(err);
+
+  size_t destination_byte_size = evbuffer_get_length(decompressed);
+  ASSERT_EQ(destination_byte_size, raw_data_length_) << "Mismatched byte size";
+
+  std::vector<char> res;
+  EVBufferToContiguousBuffer(decompressed, &res);
+  for (size_t idx = 0; idx < raw_data_length_; ++idx) {
+    ASSERT_TRUE(raw_data_[idx] == res[idx]);
+  }
+}
+
+TEST_F(DataCompressorTest, DeflateTwoBuffer)
+{
+  // Convert the raw data into evbuffer format with two buffers
+  auto source = evbuffer_new();
+  ASSERT_TRUE((source != nullptr)) << "Failed to create source evbuffer";
+  size_t half_length = raw_data_length_ / 2;
+  ASSERT_EQ(evbuffer_add(source, raw_data_.get(), half_length), 0)
+      << "Failed to initialize source evbuffer";
+  // verify evbuffer has two extend
+  {
+    auto second_source = evbuffer_new();
+    ASSERT_EQ(
+        evbuffer_add(
+            second_source, raw_data_.get() + half_length,
+            raw_data_length_ - half_length),
+        0)
+        << "Failed to initialize source evbuffer";
+    ASSERT_EQ(evbuffer_add_buffer(source, second_source), 0)
+        << "Failed to initialize source evbuffer";
+    int buffer_count = evbuffer_peek(source, -1, NULL, NULL, 0);
+    ASSERT_EQ(buffer_count, 2) << "Expect two buffers as source";
+  }
+
+  auto compressed = evbuffer_new();
+  ASSERT_TRUE((compressed != nullptr))
+      << "Failed to create compressed evbuffer";
+  // Reconstruct the compressed buffer to be two buffers
+  if (evbuffer_peek(compressed, -1, NULL, NULL, 0) == 1) {
+    struct evbuffer_iovec* buffer_array = nullptr;
+    int buffer_count = evbuffer_peek(compressed, -1, NULL, NULL, 0);
+    if (buffer_count > 0) {
+      buffer_array = static_cast<struct evbuffer_iovec*>(
+          alloca(sizeof(struct evbuffer_iovec) * buffer_count));
+      ASSERT_EQ(
+          evbuffer_peek(compressed, -1, NULL, buffer_array, buffer_count),
+          buffer_count)
+          << "unexpected error getting buffers for result";
+    }
+
+    auto first_compressed = evbuffer_new();
+    auto second_compressed = evbuffer_new();
+    size_t half_length = buffer_array[0].iov_len / 2;
+    ASSERT_EQ(
+        evbuffer_add(first_compressed, buffer_array[0].iov_base, half_length),
+        0)
+        << "Failed to split compressed buffer";
+    ASSERT_EQ(
+        evbuffer_add(
+            second_compressed,
+            reinterpret_cast<char*>(buffer_array[0].iov_base) + half_length,
+            buffer_array[0].iov_len - half_length),
+        0)
+        << "Failed to split compressed buffer";
+    ASSERT_EQ(evbuffer_add_buffer(first_compressed, second_compressed), 0)
+        << "Failed to initialize source evbuffer";
+    compressed = first_compressed;
+  }
+
+  auto decompressed = evbuffer_new();
+  ASSERT_TRUE((decompressed != nullptr))
+      << "Failed to create decompressed evbuffer";
+
+  auto err = ni::DataCompressor::CompressData(
+      ni::DataCompressor::Type::DEFLATE, source, compressed);
+  ASSERT_TRUE((err == nullptr))
+      << "Failed to compress data: " << TRITONSERVER_ErrorMessage(err);
+  err = ni::DataCompressor::DecompressData(
+      ni::DataCompressor::Type::DEFLATE, compressed, decompressed);
+  ASSERT_TRUE((err == nullptr))
+      << "Failed to decompress data: " << TRITONSERVER_ErrorMessage(err);
+
+  size_t destination_byte_size = evbuffer_get_length(decompressed);
+  ASSERT_EQ(destination_byte_size, raw_data_length_) << "Mismatched byte size";
+
+  std::vector<char> res;
+  EVBufferToContiguousBuffer(decompressed, &res);
+  for (size_t idx = 0; idx < raw_data_length_; ++idx) {
+    ASSERT_TRUE(raw_data_[idx] == res[idx]);
+  }
+}
+
+TEST_F(DataCompressorTest, GzipTwoBuffer)
+{
+  // Convert the raw data into evbuffer format with two buffers
+  auto source = evbuffer_new();
+  ASSERT_TRUE((source != nullptr)) << "Failed to create source evbuffer";
+  size_t half_length = raw_data_length_ / 2;
+  ASSERT_EQ(evbuffer_add(source, raw_data_.get(), half_length), 0)
+      << "Failed to initialize source evbuffer";
+  // verify evbuffer has two extend
+  {
+    auto second_source = evbuffer_new();
+    ASSERT_EQ(
+        evbuffer_add(
+            second_source, raw_data_.get() + half_length,
+            raw_data_length_ - half_length),
+        0)
+        << "Failed to initialize source evbuffer";
+    ASSERT_EQ(evbuffer_add_buffer(source, second_source), 0)
+        << "Failed to initialize source evbuffer";
+    int buffer_count = evbuffer_peek(source, -1, NULL, NULL, 0);
+    ASSERT_EQ(buffer_count, 2) << "Expect two buffers as source";
+  }
+
+  auto compressed = evbuffer_new();
+  ASSERT_TRUE((compressed != nullptr))
+      << "Failed to create compressed evbuffer";
+  // Reconstruct the compressed buffer to be two buffers
+  if (evbuffer_peek(compressed, -1, NULL, NULL, 0) == 1) {
+    struct evbuffer_iovec* buffer_array = nullptr;
+    int buffer_count = evbuffer_peek(compressed, -1, NULL, NULL, 0);
+    if (buffer_count > 0) {
+      buffer_array = static_cast<struct evbuffer_iovec*>(
+          alloca(sizeof(struct evbuffer_iovec) * buffer_count));
+      ASSERT_EQ(
+          evbuffer_peek(compressed, -1, NULL, buffer_array, buffer_count),
+          buffer_count)
+          << "unexpected error getting buffers for result";
+    }
+
+    auto first_compressed = evbuffer_new();
+    auto second_compressed = evbuffer_new();
+    size_t half_length = buffer_array[0].iov_len / 2;
+    ASSERT_EQ(
+        evbuffer_add(first_compressed, buffer_array[0].iov_base, half_length),
+        0)
+        << "Failed to split compressed buffer";
+    ASSERT_EQ(
+        evbuffer_add(
+            second_compressed,
+            reinterpret_cast<char*>(buffer_array[0].iov_base) + half_length,
+            buffer_array[0].iov_len - half_length),
+        0)
+        << "Failed to split compressed buffer";
+    ASSERT_EQ(evbuffer_add_buffer(first_compressed, second_compressed), 0)
+        << "Failed to initialize source evbuffer";
+    compressed = first_compressed;
+  }
+
+  auto decompressed = evbuffer_new();
+  ASSERT_TRUE((decompressed != nullptr))
+      << "Failed to create decompressed evbuffer";
+
+  auto err = ni::DataCompressor::CompressData(
+      ni::DataCompressor::Type::GZIP, source, compressed);
+  ASSERT_TRUE((err == nullptr))
+      << "Failed to compress data: " << TRITONSERVER_ErrorMessage(err);
+  err = ni::DataCompressor::DecompressData(
+      ni::DataCompressor::Type::GZIP, compressed, decompressed);
+  ASSERT_TRUE((err == nullptr))
+      << "Failed to decompress data: " << TRITONSERVER_ErrorMessage(err);
+
+  size_t destination_byte_size = evbuffer_get_length(decompressed);
+  ASSERT_EQ(destination_byte_size, raw_data_length_) << "Mismatched byte size";
+
+  std::vector<char> res;
+  EVBufferToContiguousBuffer(decompressed, &res);
+  for (size_t idx = 0; idx < raw_data_length_; ++idx) {
+    ASSERT_TRUE(raw_data_[idx] == res[idx]);
+  }
+}
+
+TEST_F(DataCompressorTest, DeflateOneLargeBuffer)
+{
+  // Duplicate raw data 2^20 times
+  {
+    std::unique_ptr<char[]> extended_raw_data(
+        new char[raw_data_length_ * (1 << 20)]);
+    memcpy(extended_raw_data.get(), raw_data_.get(), raw_data_length_);
+    size_t filled_size = raw_data_length_;
+    for (size_t i = 1; i < 20; ++i) {
+      memcpy(
+          extended_raw_data.get() + filled_size, extended_raw_data.get(),
+          filled_size);
+      filled_size += filled_size;
+    }
+    raw_data_length_ = filled_size;
+    raw_data_.swap(extended_raw_data);
+  }
+  // Convert the raw data into evbuffer format
+  auto source = evbuffer_new();
+  ASSERT_TRUE((source != nullptr)) << "Failed to create source evbuffer";
+  ASSERT_EQ(evbuffer_add(source, raw_data_.get(), raw_data_length_), 0)
+      << "Failed to initialize source evbuffer";
+
+  auto compressed = evbuffer_new();
+  ASSERT_TRUE((compressed != nullptr))
+      << "Failed to create compressed evbuffer";
+  ASSERT_GE(raw_data_length_ / 2, evbuffer_get_length(compressed))
+      << "Compression should be desired for large data";
+
+  auto decompressed = evbuffer_new();
+  ASSERT_TRUE((decompressed != nullptr))
+      << "Failed to create decompressed evbuffer";
+
+  auto err = ni::DataCompressor::CompressData(
+      ni::DataCompressor::Type::DEFLATE, source, compressed);
+  ASSERT_TRUE((err == nullptr))
+      << "Failed to compress data: " << TRITONSERVER_ErrorMessage(err);
+
+  err = ni::DataCompressor::DecompressData(
+      ni::DataCompressor::Type::DEFLATE, compressed, decompressed);
+  ASSERT_TRUE((err == nullptr))
+      << "Failed to decompress data: " << TRITONSERVER_ErrorMessage(err);
+  size_t destination_byte_size = evbuffer_get_length(decompressed);
+  ASSERT_EQ(destination_byte_size, raw_data_length_) << "Mismatched byte size";
+
+  std::vector<char> res;
+  EVBufferToContiguousBuffer(decompressed, &res);
+  for (size_t idx = 0; idx < raw_data_length_; ++idx) {
+    ASSERT_TRUE(raw_data_[idx] == res[idx]);
+  }
+}
+
+TEST_F(DataCompressorTest, GzipOneLargeBuffer)
+{
+  // Duplicate raw data 2^20 times
+  {
+    std::unique_ptr<char[]> extended_raw_data(
+        new char[raw_data_length_ * (1 << 20)]);
+    memcpy(extended_raw_data.get(), raw_data_.get(), raw_data_length_);
+    size_t filled_size = raw_data_length_;
+    for (size_t i = 1; i < 20; ++i) {
+      memcpy(
+          extended_raw_data.get() + filled_size, extended_raw_data.get(),
+          filled_size);
+      filled_size += filled_size;
+    }
+    raw_data_length_ = filled_size;
+    raw_data_.swap(extended_raw_data);
+  }
+  // Convert the raw data into evbuffer format
+  auto source = evbuffer_new();
+  ASSERT_TRUE((source != nullptr)) << "Failed to create source evbuffer";
+  ASSERT_EQ(evbuffer_add(source, raw_data_.get(), raw_data_length_), 0)
+      << "Failed to initialize source evbuffer";
+
+  auto compressed = evbuffer_new();
+  ASSERT_TRUE((compressed != nullptr))
+      << "Failed to create compressed evbuffer";
+  ASSERT_GE(raw_data_length_ / 2, evbuffer_get_length(compressed))
+      << "Compression should be desired for large data";
+
+  auto decompressed = evbuffer_new();
+  ASSERT_TRUE((decompressed != nullptr))
+      << "Failed to create decompressed evbuffer";
+
+  auto err = ni::DataCompressor::CompressData(
+      ni::DataCompressor::Type::GZIP, source, compressed);
+  ASSERT_TRUE((err == nullptr))
+      << "Failed to compress data: " << TRITONSERVER_ErrorMessage(err);
+  err = ni::DataCompressor::DecompressData(
+      ni::DataCompressor::Type::GZIP, compressed, decompressed);
+  ASSERT_TRUE((err == nullptr))
+      << "Failed to decompress data: " << TRITONSERVER_ErrorMessage(err);
+
+  size_t destination_byte_size = evbuffer_get_length(decompressed);
+  ASSERT_EQ(destination_byte_size, raw_data_length_) << "Mismatched byte size";
+
+  std::vector<char> res;
+  EVBufferToContiguousBuffer(decompressed, &res);
+  for (size_t idx = 0; idx < raw_data_length_; ++idx) {
+    ASSERT_TRUE(raw_data_[idx] == res[idx]);
+  }
+}
+
+TEST_F(DataCompressorTest, DecompressDeflateBuffer)
+{
+  // Convert the compressed data into evbuffer format
+  auto source = evbuffer_new();
+  ASSERT_TRUE((source != nullptr)) << "Failed to create source evbuffer";
+  ASSERT_EQ(
+      evbuffer_add(
+          source, deflate_compressed_data_.get(), deflate_compressed_length_),
+      0)
+      << "Failed to initialize source evbuffer";
+  auto decompressed = evbuffer_new();
+  ASSERT_TRUE((decompressed != nullptr))
+      << "Failed to create decompressed evbuffer";
+
+  auto err = ni::DataCompressor::DecompressData(
+      ni::DataCompressor::Type::DEFLATE, source, decompressed);
+  ASSERT_TRUE((err == nullptr))
+      << "Failed to decompress data: " << TRITONSERVER_ErrorMessage(err);
+
+  size_t destination_byte_size = evbuffer_get_length(decompressed);
+  ASSERT_EQ(destination_byte_size, raw_data_length_) << "Mismatched byte size";
+
+  std::vector<char> res;
+  EVBufferToContiguousBuffer(decompressed, &res);
+  for (size_t idx = 0; idx < raw_data_length_; ++idx) {
+    ASSERT_TRUE(raw_data_[idx] == res[idx]);
+  }
+}
+
+TEST_F(DataCompressorTest, DecompressGzipBuffer)
+{
+  // Convert the compressed data into evbuffer format
+  auto source = evbuffer_new();
+  ASSERT_TRUE((source != nullptr)) << "Failed to create source evbuffer";
+  ASSERT_EQ(
+      evbuffer_add(
+          source, gzip_compressed_data_.get(), gzip_compressed_length_),
+      0)
+      << "Failed to initialize source evbuffer";
+  auto decompressed = evbuffer_new();
+  ASSERT_TRUE((decompressed != nullptr))
+      << "Failed to create decompressed evbuffer";
+
+  auto err = ni::DataCompressor::DecompressData(
+      ni::DataCompressor::Type::GZIP, source, decompressed);
+  ASSERT_TRUE((err == nullptr))
+      << "Failed to decompress data: " << TRITONSERVER_ErrorMessage(err);
+
+  size_t destination_byte_size = evbuffer_get_length(decompressed);
+  ASSERT_EQ(destination_byte_size, raw_data_length_) << "Mismatched byte size";
+
+  std::vector<char> res;
+  EVBufferToContiguousBuffer(decompressed, &res);
+  for (size_t idx = 0; idx < raw_data_length_; ++idx) {
+    ASSERT_TRUE(raw_data_[idx] == res[idx]);
+  }
+}
+
+TEST_F(DataCompressorTest, CompressDeflateBuffer)
+{
+  // Convert the raw data into evbuffer format
+  auto source = evbuffer_new();
+  ASSERT_TRUE((source != nullptr)) << "Failed to create source evbuffer";
+  ASSERT_EQ(evbuffer_add(source, raw_data_.get(), raw_data_length_), 0)
+      << "Failed to initialize source evbuffer";
+
+  auto compressed = evbuffer_new();
+  ASSERT_TRUE((compressed != nullptr))
+      << "Failed to create compressed evbuffer";
+
+  auto err = ni::DataCompressor::CompressData(
+      ni::DataCompressor::Type::DEFLATE, source, compressed);
+  ASSERT_TRUE((err == nullptr))
+      << "Failed to compress data: " << TRITONSERVER_ErrorMessage(err);
+
+  // Write compressed data to file which will be validated by other compression
+  // tool
+  WriteEVBufferToFile("generated_deflate_compressed_data", compressed);
+}
+
+TEST_F(DataCompressorTest, CompressGzipBuffer)
+{
+  // Convert the raw data into evbuffer format
+  auto source = evbuffer_new();
+  ASSERT_TRUE((source != nullptr)) << "Failed to create source evbuffer";
+  ASSERT_EQ(evbuffer_add(source, raw_data_.get(), raw_data_length_), 0)
+      << "Failed to initialize source evbuffer";
+
+  auto compressed = evbuffer_new();
+  ASSERT_TRUE((compressed != nullptr))
+      << "Failed to create compressed evbuffer";
+
+  auto err = ni::DataCompressor::CompressData(
+      ni::DataCompressor::Type::GZIP, source, compressed);
+  ASSERT_TRUE((err == nullptr))
+      << "Failed to compress data: " << TRITONSERVER_ErrorMessage(err);
+
+  // Write compressed data to file which will be validated by other compression
+  // tool
+  WriteEVBufferToFile("generated_gzip_compressed_data", compressed);
+}
+
+}  // namespace
+
+int
+main(int argc, char** argv)
+{
+  ::testing::InitGoogleTest(&argc, argv);
+  return RUN_ALL_TESTS();
+}


Code review comments:
diff_hunk: @@ -2471,6 +2497,11 @@ HTTPAPIServer::InferRequestClass::InferResponseComplete(
   HTTPAPIServer::InferRequestClass* infer_request =
       reinterpret_cast<HTTPAPIServer::InferRequestClass*>(userp);
 
+  // Always specify supported compression as Accept-Encoding
+  evhtp_headers_add_header(
path: src/servers/http_server.cc
text: @user1:
The response doesn't need the Accept-Encoding header

@author:
How would the client know what compression type is supported by the server? I thought it was communicated by "Accept-Encoding" from the endpoint

@user1:
https://developer.mozilla.org/en-US/docs/Web/HTTP/Content_negotiation#server-driven_content_negotiation

It looks like we should return 415 if client has Content-Encoding that Triton can support. That will tell the client that they need to try the request again with a different Content-Encoding. We don't need to automatically do the retry in our client libraries.


@author:
So only on client sending content with unsupported type, the server returns 415 and attach the supported type as `Accept-Encoding` in the response header?

@user1:
Client-side compression negitiation doesn't seem to be very well standardized. It should be ok to include Accept-Encoding in that error response.

diff_hunk: @@ -2738,6 +2770,53 @@ HTTPAPIServer::InferRequestClass::FinalizeResponse(
                                std::to_string(buffer.Size()).c_str(), 1, 1));
   }
 
+  evbuffer* response_body = response_placeholder;
+  // Find Accept-Encoding in header. Try to compress if found
+  const char* accept_encoding_c_str =
+      evhtp_kv_find(req_->headers_in, kAcceptEncodingHTTPHeader);
+  if (accept_encoding_c_str != NULL) {
+    // FIXME compress only work for the simplest case for now
path: src/servers/http_server.cc
text: @user1:
Should fix this