Issue 1:
I have a big problem, I use the tfrecord file to import data for my tensorflow program. But, when the program run a period of time， it displays the DataLossError:

System information

OS Platform and Distribution : Linux Ubuntu 14.04

TensorFlow installed from : Anaconda

TensorFlow version : 1.3.0

Python version: 2.7.13

CUDA/cuDNN version: 8.0 / 6.0

GPU model and memory: Pascal TITAN X

Describe the problem

2017-10-03 19:45:43.854601: W tensorflow/core/framework/op_kernel.cc:1192] Data loss: corrupted record at 12

Traceback (most recent call last):

File "east_quad_train_backup.py", line 416, in

tf.app.run(main=main, argv=[sys.argv[0]] + unparsed)

File "/home/t/anaconda2/lib/python2.7/site-packages/tensorflow/python/platform/app.py", line 48, in run

_sys.exit(main(_sys.argv[:1] + flags_passthrough))

File "east_quad_train_backup.py", line 330, in main

Training()

File "east_quad_train_backup.py", line 312, in Training

feed_dict={learning_rate: lr})

File "/home/t/anaconda2/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 895, in run

run_metadata_ptr)

File "/home/t/anaconda2/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1124, in _run

feed_dict_tensor, options, run_metadata)

File "/home/t/anaconda2/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1321, in _do_run

options, run_metadata)

File "/home/t/anaconda2/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1340, in _do_call

raise type(e)(node_def, op, message)

tensorflow.python.framework.errors_impl.DataLossError: corrupted record at 12

[[Node: IteratorGetNext = IteratorGetNextoutput_shapes=[[?,512,512,3], [?,128,128,9]], output_types=[DT_UINT8, DT_FLOAT], _device="/job:localhost/replica:0/task:0/cpu:0"]]

[[Node: gradients/Tile_grad/Shape/_23 = _HostRecvclient_terminated=false, recv_device="/job:localhost/replica:0/task:0/gpu:0", send_device="/job:localhost/replica:0/task:0/cpu:0", send_device_incarnation=1, tensor_name="edge_442_gradients/Tile_grad/Shape", tensor_type=DT_INT32, _device="/job:localhost/replica:0/task:0/gpu:0"]]

Caused by op u'IteratorGetNext', defined at:

File "east_quad_train_backup.py", line 416, in

tf.app.run(main=main, argv=[sys.argv[0]] + unparsed)

File "/home/t/anaconda2/lib/python2.7/site-packages/tensorflow/python/platform/app.py", line 48, in run

_sys.exit(main(_sys.argv[:1] + flags_passthrough))

File "east_quad_train_backup.py", line 330, in main

Training()

File "east_quad_train_backup.py", line 251, in Training

batch_image, batch_label = iterator.get_next()

File "/home/t/anaconda2/lib/python2.7/site-packages/tensorflow/contrib/data/python/ops/dataset_ops.py", line 304, in get_next

name=name))

File "/home/t/anaconda2/lib/python2.7/site-packages/tensorflow/python/ops/gen_dataset_ops.py", line 379, in iterator_get_next

output_shapes=output_shapes, name=name)

File "/home/t/anaconda2/lib/python2.7/site-packages/tensorflow/python/framework/op_def_library.py", line 767, in apply_op

op_def=op_def)

File "/home/t/anaconda2/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 2630, in create_op

original_op=self._default_original_op, op_def=op_def)

File "/home/t/anaconda2/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 1204, in init

self._traceback = self._graph._extract_stack() # pylint: disable=protected-access

DataLossError (see above for traceback): corrupted record at 12

[[Node: IteratorGetNext = IteratorGetNextoutput_shapes=[[?,512,512,3], [?,128,128,9]], output_types=[DT_UINT8, DT_FLOAT], _device="/job:localhost/replica:0/task:0/cpu:0"]]

[[Node: gradients/Tile_grad/Shape/_23 = _HostRecvclient_terminated=false, recv_device="/job:localhost/replica:0/task:0/gpu:0", send_device="/job:localhost/replica:0/task:0/cpu:0", send_device_incarnation=1, tensor_name="edge_442_gradients/Tile_grad/Shape", tensor_type=DT_INT32, _device="/job:localhost/replica:0/task:0/gpu:0"]]

Thanks anyone to answer this question.

Issue 2:
NOTE: Only file GitHub issues for bugs and feature requests. All other topics will be closed.

For general support from the community, see StackOverflow.

To make bugs and feature requests more easy to find and organize, we close issues that are deemed

out of scope for GitHub Issues and point people to StackOverflow.

For bugs or installation issues, please provide the following information.

The more information you provide, the more easily we will be able to offer

help and advice.

What related GitHub issues or StackOverflow threads have you found by searching the web for your problem?

Environment info

Operating System:

Installed version of CUDA and cuDNN:

(please attach the output of ls -l /path/to/cuda/lib/libcud*):

If installed from binary pip package, provide:

A link to the pip package you installed:

The output from python -c "import tensorflow; print(tensorflow.__version__)".

If installed from source, provide

The commit hash (git rev-parse HEAD)

The output of bazel version

If possible, provide a minimal reproducible example (We usually don't have time to read hundreds of lines of your code)

What other attempted solutions have you tried?

Logs or other output that would be helpful

(If logs are large, please upload as attachment or provide link).