Repository: tensorflow/serving

Issue #1414: question about warm up 
Created: 2019-08-13T03:11:30Z
I've created warmup data following the official guideline as below, and I launched the  model_server with the option --enable_model_warmup=true, but I did not see any improvement of the latency for the first request and I did not see any logs about loading warmup data.
I tested the issue on both tensorflow-serving:1.12-gpu and tensorflow-sering:latest-gpu docker build.
https://www.tensorflow.org/serving/saved_model_warmup

Duplicate issue #1562: tf_serving_warmup_requests with model-warmup-options gives error
Created: 2020-02-28T19:22:52Z
Bug Report
If this is a bug report, please fill out the following form in full:
System information

OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Debian 4.9
TensorFlow Serving installed from (source or binary): source
TensorFlow Serving version: 2.1.0

Describe the problem
Describe the problem clearly here. Be sure to convey here why it's a bug in
TensorFlow Serving.
2020-02-28 19:15:21.150722: I tensorflow_serving/servables/tensorflow/saved_model_warmup.cc:117] Starting to read warmup data for model at /models/1/2/exported_models/12322323/123123123442/assets.extra/tf_serving_warmup_requests with model-warmup-options 
2020-02-28 19:15:21.150771: E tensorflow_serving/util/retrier.cc:37] Loading servable: {name: model-1-2 version: 123123123442} failed: Data loss: corrupted record at 0. Please verify your warmup data is in TFRecord format.

Exact Steps to Reproduce
Please include all steps necessary for someone to reproduce this issue on their
own machine. If not, skip this section.
docker run --rm -it \
  ...
  --model_config_file=/serving/models.conf \
  --enable_batching=true \
  --batching_parameters_file=/serving/batching_parameters.txt \
  --monitoring_config_file=/serving/monitoring.conf \
  --enable_model_warmup=true

here my tf_serving_warmup_requests file, which I copied into assets.extra:
import tensorflow as tf
from tensorflow.python.framework import tensor_util
from tensorflow_serving.apis import model_pb2
from tensorflow_serving.apis import predict_pb2
from tensorflow_serving.apis import prediction_log_pb2

def main():
    """Generate TFRecords for warming up."""
    with tf.python_io.TFRecordWriter("tf_serving_warmup_requests") as writer:
        predict_request = predict_pb2.PredictRequest()
        predict_request.model_spec.name = 'model-1-2'
        predict_request.model_spec.signature_name = 'serving_default'
        predict_request.inputs['price'].CopyFrom(tensor_util.make_tensor_proto([0.1], tf.float32))
        predict_request.inputs['height'].CopyFrom(tensor_util.make_tensor_proto([1], tf.int64))
        predict_request.inputs['domain'].CopyFrom(tensor_util.make_tensor_proto(["1".encode()], tf.string))

        log = prediction_log_pb2.PredictionLog(
            predict_log=prediction_log_pb2.PredictLog(request=predict_request))
        writer.write(log.SerializeToString())

if __name__ == "__main__":
    main()

Source code / logs
Include any logs or source code that would be helpful to diagnose the problem.
If including tracebacks, please include the full traceback. Large logs and files
should be attached. Try to provide a reproducible test case that is the bare
minimum necessary to generate the problem.
Other infos
when I disable enable_model_warmup ( --enable_model_warmup=false):
than I don't get this strange Data loss error and the model can be served without issues...

Closing comment (2020-02-28T19:37:24Z): @christisg @rmothukuru any idea since this is similar to #1414?