Repository: tensorflow/tensor2tensor

Issue #529: Bug introduced in v1.3.0 causing training divergence
Created: 2018-01-19T18:26:10Z
I train with exactly the same setup (1GPU, transformer_big_single_gpu, batch_size=1500, en-cs dataset with 58M training sentences, checkpoints saved each hour) with T2T v1.2.9 and v1.3.0 (which is the version immediately following 1.2.9). While in v1.2.9 everything is OK, in v1.3.0 the training diverges after about one hour of training and BLEU drops to almost zero after about 5 hours of training. See the learning curves below (orange=v1.2.9, magenta=1.3.0, y-axis is dev-set BLEU as measured by t2t-bleu, x-axis is training time in hours):

I attach also training loss curves for these two experiments (but now orange=v1.3.0, gray=v1.2.9, so the colors don't match the previous graph, x-axis is now in steps):


This confirms the problem: the training loss in v1.3.0 starts slowly growing after 1h (10k steps) of training. So the bug is already in training, not in decoding.
I am not sure if it is significant, but v1.3.0 has lower training loss than v1.2.9 between 2k and 8k steps. This could mean that it is not a bug, but just faster training, which needs different optimal hyperparameters to prevent the divergence (e.g. higher a bigger batch size or more warmup steps) than v1.2.9. But until it is proven, I will call it a bug/problem.
I tried all versions between 1.3.1 and 1.4.2 and they show the same problem as 1.3.0.
Other users (e.g. in #525) report similar problems with different datasets (en-de, en-fr).

Duplicate issue #535: Can not continue training
Created: 2018-01-22T11:37:13Z
t2t_trainer fails to restore saved checkpoints. I run it first time and broke at some point, then run the same command again.
Here is the full output:
INFO:tensorflow:schuedule=continuous_train_and_eval
INFO:tensorflow:worker_gpu=1
INFO:tensorflow:sync=False
WARNING:tensorflow:Schedule=continuous_train_and_eval. Assuming that training is running on a single machine.
INFO:tensorflow:datashard_devices: ['gpu:0']
INFO:tensorflow:caching_devices: None
INFO:tensorflow:ps_devices: ['gpu:0']
INFO:tensorflow:Using config: {'_task_type': None, '_task_id': 0, '_cluster_spec': <tensorflow.python.training.server_lib.ClusterSpec object at 0x0000014E79D96A58>, '_master': '', '_num_ps_replicas': 0, '_num_worker_replicas': 0, '_environment': 'local', '_is_chief': True, '_evaluation_master': '', '_tf_config': gpu_options {
  per_process_gpu_memory_fraction: 1
}
, '_tf_random_seed': 1234, '_save_summary_steps': 100, '_save_checkpoints_secs': None, '_log_step_count_steps': 100, '_session_config': allow_soft_placement: true
, '_save_checkpoints_steps': 2000, '_keep_checkpoint_max': 20, '_keep_checkpoint_every_n_hours': 10000, '_model_dir': 't2t_out', 'use_tpu': False, 't2t_device_info': {'num_async_replicas': 1}, 'data_parallelism': <tensor2tensor.utils.expert_utils.Parallelism object at 0x0000014E782B74A8>}
WARNING:tensorflow:Estimator's model_fn (<function T2TModel.make_estimator_model_fn.<locals>.wrapping_model_fn at 0x0000014E782BA0D0>) includes params argument, but params are not passed to Estimator.
WARNING:tensorflow:From C:\Users\Artem\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\contrib\learn\python\learn\monitors.py:267: BaseMonitor.__init__ (from tensorflow.contrib.learn.python.learn.monitors) is deprecated and will be removed after 2016-12-05.
Instructions for updating:
Monitors are deprecated. Please use tf.train.SessionRunHook.
WARNING:tensorflow:Experiment.continuous_train_and_eval (from tensorflow.contrib.learn.python.learn.experiment) is experimental and may change or be removed at any time, and without warning.
INFO:tensorflow:Training model for 2000 steps
INFO:tensorflow:Reading data files from t2t_data\translate_enfr_wmt_small8k-train*
INFO:tensorflow:Calling model_fn.
WARNING:tensorflow:From C:\Users\Artem\AppData\Local\Programs\Python\Python36\lib\site-packages\tensor2tensor\layers\common_layers.py:496: calling reduce_mean (from tensorflow.python.ops.math_ops) with keep_dims is deprecated and will be removed in a future version.
Instructions for updating:
keep_dims is deprecated, use keepdims instead
WARNING:tensorflow:From C:\Users\Artem\AppData\Local\Programs\Python\Python36\lib\site-packages\tensor2tensor\layers\common_layers.py:1680: softmax_cross_entropy_with_logits (from tensorflow.python.ops.nn_ops) is deprecated and will be removed in a future version.
Instructions for updating:

Future major versions of TensorFlow will allow gradients to flow
into the labels input on backprop by default.

See tf.nn.softmax_cross_entropy_with_logits_v2.

INFO:tensorflow:Weight    transformer/body/decoder/layer_0/encdec_attention/layer_prepostprocess/layer_norm/layer_norm_bias     shape    (512,)                 size    512
INFO:tensorflow:Weight    transformer/body/decoder/layer_0/encdec_attention/layer_prepostprocess/layer_norm/layer_norm_scale    shape    (512,)                 size    512
INFO:tensorflow:Weight    transformer/body/decoder/layer_0/encdec_attention/multihead_attention/k/kernel        shape    (512, 512)             size    262144
INFO:tensorflow:Weight    transformer/body/decoder/layer_0/encdec_attention/multihead_attention/output_transform/kernel
shape    (512, 512)             size    262144
INFO:tensorflow:Weight    transformer/body/decoder/layer_0/encdec_attention/multihead_attention/q/kernel        shape    (512, 512)             size    262144
INFO:tensorflow:Weight    transformer/body/decoder/layer_0/encdec_attention/multihead_attention/v/kernel        shape    (512, 512)             size    262144
INFO:tensorflow:Weight    transformer/body/decoder/layer_0/ffn/conv1/bias                                       shape    (2048,)                size    2048
INFO:tensorflow:Weight    transformer/body/decoder/layer_0/ffn/conv1/kernel                                     shape    (512, 2048)            size    1048576
INFO:tensorflow:Weight    transformer/body/decoder/layer_0/ffn/conv2/bias                                       shape    (512,)                 size    512
INFO:tensorflow:Weight    transformer/body/decoder/layer_0/ffn/conv2/kernel                                     shape    (2048, 512)            size    1048576
INFO:tensorflow:Weight    transformer/body/decoder/layer_0/ffn/layer_prepostprocess/layer_norm/layer_norm_bias  shape    (512,)                 size    512
INFO:tensorflow:Weight    transformer/body/decoder/layer_0/ffn/layer_prepostprocess/layer_norm/layer_norm_scale shape    (512,)                 size    512
INFO:tensorflow:Weight    transformer/body/decoder/layer_0/self_attention/layer_prepostprocess/layer_norm/layer_norm_bias       shape    (512,)                 size    512
INFO:tensorflow:Weight    transformer/body/decoder/layer_0/self_attention/layer_prepostprocess/layer_norm/layer_norm_scale      shape    (512,)                 size    512
INFO:tensorflow:Weight    transformer/body/decoder/layer_0/self_attention/multihead_attention/k/kernel          shape    (512, 512)             size    262144
INFO:tensorflow:Weight    transformer/body/decoder/layer_0/self_attention/multihead_attention/output_transform/kernel
shape    (512, 512)             size    262144
INFO:tensorflow:Weight    transformer/body/decoder/layer_0/self_attention/multihead_attention/q/kernel          shape    (512, 512)             size    262144
INFO:tensorflow:Weight    transformer/body/decoder/layer_0/self_attention/multihead_attention/v/kernel          shape    (512, 512)             size    262144
INFO:tensorflow:Weight    transformer/body/decoder/layer_1/encdec_attention/layer_prepostprocess/layer_norm/layer_norm_bias     shape    (512,)                 size    512
INFO:tensorflow:Weight    transformer/body/decoder/layer_1/encdec_attention/layer_prepostprocess/layer_norm/layer_norm_scale    shape    (512,)                 size    512
INFO:tensorflow:Weight    transformer/body/decoder/layer_1/encdec_attention/multihead_attention/k/kernel        shape    (512, 512)             size    262144
INFO:tensorflow:Weight    transformer/body/decoder/layer_1/encdec_attention/multihead_attention/output_transform/kernel
shape    (512, 512)             size    262144
INFO:tensorflow:Weight    transformer/body/decoder/layer_1/encdec_attention/multihead_attention/q/kernel        shape    (512, 512)             size    262144
INFO:tensorflow:Weight    transformer/body/decoder/layer_1/encdec_attention/multihead_attention/v/kernel        shape    (512, 512)             size    262144
INFO:tensorflow:Weight    transformer/body/decoder/layer_1/ffn/conv1/bias                                       shape    (2048,)                size    2048
INFO:tensorflow:Weight    transformer/body/decoder/layer_1/ffn/conv1/kernel                                     shape    (512, 2048)            size    1048576
INFO:tensorflow:Weight    transformer/body/decoder/layer_1/ffn/conv2/bias                                       shape    (512,)                 size    512
INFO:tensorflow:Weight    transformer/body/decoder/layer_1/ffn/conv2/kernel                                     shape    (2048, 512)            size    1048576
INFO:tensorflow:Weight    transformer/body/decoder/layer_1/ffn/layer_prepostprocess/layer_norm/layer_norm_bias  shape    (512,)                 size    512
INFO:tensorflow:Weight    transformer/body/decoder/layer_1/ffn/layer_prepostprocess/layer_norm/layer_norm_scale shape    (512,)                 size    512
INFO:tensorflow:Weight    transformer/body/decoder/layer_1/self_attention/layer_prepostprocess/layer_norm/layer_norm_bias       shape    (512,)                 size    512
INFO:tensorflow:Weight    transformer/body/decoder/layer_1/self_attention/layer_prepostprocess/layer_norm/layer_norm_scale      shape    (512,)                 size    512
INFO:tensorflow:Weight    transformer/body/decoder/layer_1/self_attention/multihead_attention/k/kernel          shape    (512, 512)             size    262144
INFO:tensorflow:Weight    transformer/body/decoder/layer_1/self_attention/multihead_attention/output_transform/kernel
shape    (512, 512)             size    262144
INFO:tensorflow:Weight    transformer/body/decoder/layer_1/self_attention/multihead_attention/q/kernel          shape    (512, 512)             size    262144
INFO:tensorflow:Weight    transformer/body/decoder/layer_1/self_attention/multihead_attention/v/kernel          shape    (512, 512)             size    262144
INFO:tensorflow:Weight    transformer/body/decoder/layer_2/encdec_attention/layer_prepostprocess/layer_norm/layer_norm_bias     shape    (512,)                 size    512
INFO:tensorflow:Weight    transformer/body/decoder/layer_2/encdec_attention/layer_prepostprocess/layer_norm/layer_norm_scale    shape    (512,)                 size    512
INFO:tensorflow:Weight    transformer/body/decoder/layer_2/encdec_attention/multihead_attention/k/kernel        shape    (512, 512)             size    262144
INFO:tensorflow:Weight    transformer/body/decoder/layer_2/encdec_attention/multihead_attention/output_transform/kernel
shape    (512, 512)             size    262144
INFO:tensorflow:Weight    transformer/body/decoder/layer_2/encdec_attention/multihead_attention/q/kernel        shape    (512, 512)             size    262144
INFO:tensorflow:Weight    transformer/body/decoder/layer_2/encdec_attention/multihead_attention/v/kernel        shape    (512, 512)             size    262144
INFO:tensorflow:Weight    transformer/body/decoder/layer_2/ffn/conv1/bias                                       shape    (2048,)                size    2048
INFO:tensorflow:Weight    transformer/body/decoder/layer_2/ffn/conv1/kernel                                     shape    (512, 2048)            size    1048576
INFO:tensorflow:Weight    transformer/body/decoder/layer_2/ffn/conv2/bias                                       shape    (512,)                 size    512
INFO:tensorflow:Weight    transformer/body/decoder/layer_2/ffn/conv2/kernel                                     shape    (2048, 512)            size    1048576
INFO:tensorflow:Weight    transformer/body/decoder/layer_2/ffn/layer_prepostprocess/layer_norm/layer_norm_bias  shape    (512,)                 size    512
INFO:tensorflow:Weight    transformer/body/decoder/layer_2/ffn/layer_prepostprocess/layer_norm/layer_norm_scale shape    (512,)                 size    512
INFO:tensorflow:Weight    transformer/body/decoder/layer_2/self_attention/layer_prepostprocess/layer_norm/layer_norm_bias       shape    (512,)                 size    512
INFO:tensorflow:Weight    transformer/body/decoder/layer_2/self_attention/layer_prepostprocess/layer_norm/layer_norm_scale      shape    (512,)                 size    512
INFO:tensorflow:Weight    transformer/body/decoder/layer_2/self_attention/multihead_attention/k/kernel          shape    (512, 512)             size    262144
INFO:tensorflow:Weight    transformer/body/decoder/layer_2/self_attention/multihead_attention/output_transform/kernel
shape    (512, 512)             size    262144
INFO:tensorflow:Weight    transformer/body/decoder/layer_2/self_attention/multihead_attention/q/kernel          shape    (512, 512)             size    262144
INFO:tensorflow:Weight    transformer/body/decoder/layer_2/self_attention/multihead_attention/v/kernel          shape    (512, 512)             size    262144
INFO:tensorflow:Weight    transformer/body/decoder/layer_3/encdec_attention/layer_prepostprocess/layer_norm/layer_norm_bias     shape    (512,)                 size    512
INFO:tensorflow:Weight    transformer/body/decoder/layer_3/encdec_attention/layer_prepostprocess/layer_norm/layer_norm_scale    shape    (512,)                 size    512
INFO:tensorflow:Weight    transformer/body/decoder/layer_3/encdec_attention/multihead_attention/k/kernel        shape    (512, 512)             size    262144
INFO:tensorflow:Weight    transformer/body/decoder/layer_3/encdec_attention/multihead_attention/output_transform/kernel
shape    (512, 512)             size    262144
INFO:tensorflow:Weight    transformer/body/decoder/layer_3/encdec_attention/multihead_attention/q/kernel        shape    (512, 512)             size    262144
INFO:tensorflow:Weight    transformer/body/decoder/layer_3/encdec_attention/multihead_attention/v/kernel        shape    (512, 512)             size    262144
INFO:tensorflow:Weight    transformer/body/decoder/layer_3/ffn/conv1/bias                                       shape    (2048,)                size    2048
INFO:tensorflow:Weight    transformer/body/decoder/layer_3/ffn/conv1/kernel                                     shape    (512, 2048)            size    1048576
INFO:tensorflow:Weight    transformer/body/decoder/layer_3/ffn/conv2/bias                                       shape    (512,)                 size    512
INFO:tensorflow:Weight    transformer/body/decoder/layer_3/ffn/conv2/kernel                                     shape    (2048, 512)            size    1048576
INFO:tensorflow:Weight    transformer/body/decoder/layer_3/ffn/layer_prepostprocess/layer_norm/layer_norm_bias  shape    (512,)                 size    512
INFO:tensorflow:Weight    transformer/body/decoder/layer_3/ffn/layer_prepostprocess/layer_norm/layer_norm_scale shape    (512,)                 size    512
INFO:tensorflow:Weight    transformer/body/decoder/layer_3/self_attention/layer_prepostprocess/layer_norm/layer_norm_bias       shape    (512,)                 size    512
INFO:tensorflow:Weight    transformer/body/decoder/layer_3/self_attention/layer_prepostprocess/layer_norm/layer_norm_scale      shape    (512,)                 size    512
INFO:tensorflow:Weight    transformer/body/decoder/layer_3/self_attention/multihead_attention/k/kernel          shape    (512, 512)             size    262144
INFO:tensorflow:Weight    transformer/body/decoder/layer_3/self_attention/multihead_attention/output_transform/kernel
shape    (512, 512)             size    262144
INFO:tensorflow:Weight    transformer/body/decoder/layer_3/self_attention/multihead_attention/q/kernel          shape    (512, 512)             size    262144
INFO:tensorflow:Weight    transformer/body/decoder/layer_3/self_attention/multihead_attention/v/kernel          shape    (512, 512)             size    262144
INFO:tensorflow:Weight    transformer/body/decoder/layer_4/encdec_attention/layer_prepostprocess/layer_norm/layer_norm_bias     shape    (512,)                 size    512
INFO:tensorflow:Weight    transformer/body/decoder/layer_4/encdec_attention/layer_prepostprocess/layer_norm/layer_norm_scale    shape    (512,)                 size    512
INFO:tensorflow:Weight    transformer/body/decoder/layer_4/encdec_attention/multihead_attention/k/kernel        shape    (512, 512)             size    262144
INFO:tensorflow:Weight    transformer/body/decoder/layer_4/encdec_attention/multihead_attention/output_transform/kernel
shape    (512, 512)             size    262144
INFO:tensorflow:Weight    transformer/body/decoder/layer_4/encdec_attention/multihead_attention/q/kernel        shape    (512, 512)             size    262144
INFO:tensorflow:Weight    transformer/body/decoder/layer_4/encdec_attention/multihead_attention/v/kernel        shape    (512, 512)             size    262144
INFO:tensorflow:Weight    transformer/body/decoder/layer_4/ffn/conv1/bias                                       shape    (2048,)                size    2048
INFO:tensorflow:Weight    transformer/body/decoder/layer_4/ffn/conv1/kernel                                     shape    (512, 2048)            size    1048576
INFO:tensorflow:Weight    transformer/body/decoder/layer_4/ffn/conv2/bias                                       shape    (512,)                 size    512
INFO:tensorflow:Weight    transformer/body/decoder/layer_4/ffn/conv2/kernel                                     shape    (2048, 512)            size    1048576
INFO:tensorflow:Weight    transformer/body/decoder/layer_4/ffn/layer_prepostprocess/layer_norm/layer_norm_bias  shape    (512,)                 size    512
INFO:tensorflow:Weight    transformer/body/decoder/layer_4/ffn/layer_prepostprocess/layer_norm/layer_norm_scale shape    (512,)                 size    512
INFO:tensorflow:Weight    transformer/body/decoder/layer_4/self_attention/layer_prepostprocess/layer_norm/layer_norm_bias       shape    (512,)                 size    512
INFO:tensorflow:Weight    transformer/body/decoder/layer_4/self_attention/layer_prepostprocess/layer_norm/layer_norm_scale      shape    (512,)                 size    512
INFO:tensorflow:Weight    transformer/body/decoder/layer_4/self_attention/multihead_attention/k/kernel          shape    (512, 512)             size    262144
INFO:tensorflow:Weight    transformer/body/decoder/layer_4/self_attention/multihead_attention/output_transform/kernel
shape    (512, 512)             size    262144
INFO:tensorflow:Weight    transformer/body/decoder/layer_4/self_attention/multihead_attention/q/kernel          shape    (512, 512)             size    262144
INFO:tensorflow:Weight    transformer/body/decoder/layer_4/self_attention/multihead_attention/v/kernel          shape    (512, 512)             size    262144
INFO:tensorflow:Weight    transformer/body/decoder/layer_5/encdec_attention/layer_prepostprocess/layer_norm/layer_norm_bias     shape    (512,)                 size    512
INFO:tensorflow:Weight    transformer/body/decoder/layer_5/encdec_attention/layer_prepostprocess/layer_norm/layer_norm_scale    shape    (512,)                 size    512
INFO:tensorflow:Weight    transformer/body/decoder/layer_5/encdec_attention/multihead_attention/k/kernel        shape    (512, 512)             size    262144
INFO:tensorflow:Weight    transformer/body/decoder/layer_5/encdec_attention/multihead_attention/output_transform/kernel
shape    (512, 512)             size    262144
INFO:tensorflow:Weight    transformer/body/decoder/layer_5/encdec_attention/multihead_attention/q/kernel        shape    (512, 512)             size    262144
INFO:tensorflow:Weight    transformer/body/decoder/layer_5/encdec_attention/multihead_attention/v/kernel        shape    (512, 512)             size    262144
INFO:tensorflow:Weight    transformer/body/decoder/layer_5/ffn/conv1/bias                                       shape    (2048,)                size    2048
INFO:tensorflow:Weight    transformer/body/decoder/layer_5/ffn/conv1/kernel                                     shape    (512, 2048)            size    1048576
INFO:tensorflow:Weight    transformer/body/decoder/layer_5/ffn/conv2/bias                                       shape    (512,)                 size    512
INFO:tensorflow:Weight    transformer/body/decoder/layer_5/ffn/conv2/kernel                                     shape    (2048, 512)            size    1048576
INFO:tensorflow:Weight    transformer/body/decoder/layer_5/ffn/layer_prepostprocess/layer_norm/layer_norm_bias  shape    (512,)                 size    512
INFO:tensorflow:Weight    transformer/body/decoder/layer_5/ffn/layer_prepostprocess/layer_norm/layer_norm_scale shape    (512,)                 size    512
INFO:tensorflow:Weight    transformer/body/decoder/layer_5/self_attention/layer_prepostprocess/layer_norm/layer_norm_bias       shape    (512,)                 size    512
INFO:tensorflow:Weight    transformer/body/decoder/layer_5/self_attention/layer_prepostprocess/layer_norm/layer_norm_scale      shape    (512,)                 size    512
INFO:tensorflow:Weight    transformer/body/decoder/layer_5/self_attention/multihead_attention/k/kernel          shape    (512, 512)             size    262144
INFO:tensorflow:Weight    transformer/body/decoder/layer_5/self_attention/multihead_attention/output_transform/kernel
shape    (512, 512)             size    262144
INFO:tensorflow:Weight    transformer/body/decoder/layer_5/self_attention/multihead_attention/q/kernel          shape    (512, 512)             size    262144
INFO:tensorflow:Weight    transformer/body/decoder/layer_5/self_attention/multihead_attention/v/kernel          shape    (512, 512)             size    262144
INFO:tensorflow:Weight    transformer/body/decoder/layer_prepostprocess/layer_norm/layer_norm_bias              shape    (512,)                 size    512
INFO:tensorflow:Weight    transformer/body/decoder/layer_prepostprocess/layer_norm/layer_norm_scale             shape    (512,)                 size    512
INFO:tensorflow:Weight    transformer/body/encoder/layer_0/ffn/conv1/bias                                       shape    (2048,)                size    2048
INFO:tensorflow:Weight    transformer/body/encoder/layer_0/ffn/conv1/kernel                                     shape    (512, 2048)            size    1048576
INFO:tensorflow:Weight    transformer/body/encoder/layer_0/ffn/conv2/bias                                       shape    (512,)                 size    512
INFO:tensorflow:Weight    transformer/body/encoder/layer_0/ffn/conv2/kernel                                     shape    (2048, 512)            size    1048576
INFO:tensorflow:Weight    transformer/body/encoder/layer_0/ffn/layer_prepostprocess/layer_norm/layer_norm_bias  shape    (512,)                 size    512
INFO:tensorflow:Weight    transformer/body/encoder/layer_0/ffn/layer_prepostprocess/layer_norm/layer_norm_scale shape    (512,)                 size    512
INFO:tensorflow:Weight    transformer/body/encoder/layer_0/self_attention/layer_prepostprocess/layer_norm/layer_norm_bias       shape    (512,)                 size    512
INFO:tensorflow:Weight    transformer/body/encoder/layer_0/self_attention/layer_prepostprocess/layer_norm/layer_norm_scale      shape    (512,)                 size    512
INFO:tensorflow:Weight    transformer/body/encoder/layer_0/self_attention/multihead_attention/k/kernel          shape    (512, 512)             size    262144
INFO:tensorflow:Weight    transformer/body/encoder/layer_0/self_attention/multihead_attention/output_transform/kernel
shape    (512, 512)             size    262144
INFO:tensorflow:Weight    transformer/body/encoder/layer_0/self_attention/multihead_attention/q/kernel          shape    (512, 512)             size    262144
INFO:tensorflow:Weight    transformer/body/encoder/layer_0/self_attention/multihead_attention/v/kernel          shape    (512, 512)             size    262144
INFO:tensorflow:Weight    transformer/body/encoder/layer_1/ffn/conv1/bias                                       shape    (2048,)                size    2048
INFO:tensorflow:Weight    transformer/body/encoder/layer_1/ffn/conv1/kernel                                     shape    (512, 2048)            size    1048576
INFO:tensorflow:Weight    transformer/body/encoder/layer_1/ffn/conv2/bias                                       shape    (512,)                 size    512
INFO:tensorflow:Weight    transformer/body/encoder/layer_1/ffn/conv2/kernel                                     shape    (2048, 512)            size    1048576
INFO:tensorflow:Weight    transformer/body/encoder/layer_1/ffn/layer_prepostprocess/layer_norm/layer_norm_bias  shape    (512,)                 size    512
INFO:tensorflow:Weight    transformer/body/encoder/layer_1/ffn/layer_prepostprocess/layer_norm/layer_norm_scale shape    (512,)                 size    512
INFO:tensorflow:Weight    transformer/body/encoder/layer_1/self_attention/layer_prepostprocess/layer_norm/layer_norm_bias       shape    (512,)                 size    512
INFO:tensorflow:Weight    transformer/body/encoder/layer_1/self_attention/layer_prepostprocess/layer_norm/layer_norm_scale      shape    (512,)                 size    512
INFO:tensorflow:Weight    transformer/body/encoder/layer_1/self_attention/multihead_attention/k/kernel          shape    (512, 512)             size    262144
INFO:tensorflow:Weight    transformer/body/encoder/layer_1/self_attention/multihead_attention/output_transform/kernel
shape    (512, 512)             size    262144
INFO:tensorflow:Weight    transformer/body/encoder/layer_1/self_attention/multihead_attention/q/kernel          shape    (512, 512)             size    262144
INFO:tensorflow:Weight    transformer/body/encoder/layer_1/self_attention/multihead_attention/v/kernel          shape    (512, 512)             size    262144
INFO:tensorflow:Weight    transformer/body/encoder/layer_2/ffn/conv1/bias                                       shape    (2048,)                size    2048
INFO:tensorflow:Weight    transformer/body/encoder/layer_2/ffn/conv1/kernel                                     shape    (512, 2048)            size    1048576
INFO:tensorflow:Weight    transformer/body/encoder/layer_2/ffn/conv2/bias                                       shape    (512,)                 size    512
INFO:tensorflow:Weight    transformer/body/encoder/layer_2/ffn/conv2/kernel                                     shape    (2048, 512)            size    1048576
INFO:tensorflow:Weight    transformer/body/encoder/layer_2/ffn/layer_prepostprocess/layer_norm/layer_norm_bias  shape    (512,)                 size    512
INFO:tensorflow:Weight    transformer/body/encoder/layer_2/ffn/layer_prepostprocess/layer_norm/layer_norm_scale shape    (512,)                 size    512
INFO:tensorflow:Weight    transformer/body/encoder/layer_2/self_attention/layer_prepostprocess/layer_norm/layer_norm_bias       shape    (512,)                 size    512
INFO:tensorflow:Weight    transformer/body/encoder/layer_2/self_attention/layer_prepostprocess/layer_norm/layer_norm_scale      shape    (512,)                 size    512
INFO:tensorflow:Weight    transformer/body/encoder/layer_2/self_attention/multihead_attention/k/kernel          shape    (512, 512)             size    262144
INFO:tensorflow:Weight    transformer/body/encoder/layer_2/self_attention/multihead_attention/output_transform/kernel
shape    (512, 512)             size    262144
INFO:tensorflow:Weight    transformer/body/encoder/layer_2/self_attention/multihead_attention/q/kernel          shape    (512, 512)             size    262144
INFO:tensorflow:Weight    transformer/body/encoder/layer_2/self_attention/multihead_attention/v/kernel          shape    (512, 512)             size    262144
INFO:tensorflow:Weight    transformer/body/encoder/layer_3/ffn/conv1/bias                                       shape    (2048,)                size    2048
INFO:tensorflow:Weight    transformer/body/encoder/layer_3/ffn/conv1/kernel                                     shape    (512, 2048)            size    1048576
INFO:tensorflow:Weight    transformer/body/encoder/layer_3/ffn/conv2/bias                                       shape    (512,)                 size    512
INFO:tensorflow:Weight    transformer/body/encoder/layer_3/ffn/conv2/kernel                                     shape    (2048, 512)            size    1048576
INFO:tensorflow:Weight    transformer/body/encoder/layer_3/ffn/layer_prepostprocess/layer_norm/layer_norm_bias  shape    (512,)                 size    512
INFO:tensorflow:Weight    transformer/body/encoder/layer_3/ffn/layer_prepostprocess/layer_norm/layer_norm_scale shape    (512,)                 size    512
INFO:tensorflow:Weight    transformer/body/encoder/layer_3/self_attention/layer_prepostprocess/layer_norm/layer_norm_bias       shape    (512,)                 size    512
INFO:tensorflow:Weight    transformer/body/encoder/layer_3/self_attention/layer_prepostprocess/layer_norm/layer_norm_scale      shape    (512,)                 size    512
INFO:tensorflow:Weight    transformer/body/encoder/layer_3/self_attention/multihead_attention/k/kernel          shape    (512, 512)             size    262144
INFO:tensorflow:Weight    transformer/body/encoder/layer_3/self_attention/multihead_attention/output_transform/kernel
shape    (512, 512)             size    262144
INFO:tensorflow:Weight    transformer/body/encoder/layer_3/self_attention/multihead_attention/q/kernel          shape    (512, 512)             size    262144
INFO:tensorflow:Weight    transformer/body/encoder/layer_3/self_attention/multihead_attention/v/kernel          shape    (512, 512)             size    262144
INFO:tensorflow:Weight    transformer/body/encoder/layer_4/ffn/conv1/bias                                       shape    (2048,)                size    2048
INFO:tensorflow:Weight    transformer/body/encoder/layer_4/ffn/conv1/kernel                                     shape    (512, 2048)            size    1048576
INFO:tensorflow:Weight    transformer/body/encoder/layer_4/ffn/conv2/bias                                       shape    (512,)                 size    512
INFO:tensorflow:Weight    transformer/body/encoder/layer_4/ffn/conv2/kernel                                     shape    (2048, 512)            size    1048576
INFO:tensorflow:Weight    transformer/body/encoder/layer_4/ffn/layer_prepostprocess/layer_norm/layer_norm_bias  shape    (512,)                 size    512
INFO:tensorflow:Weight    transformer/body/encoder/layer_4/ffn/layer_prepostprocess/layer_norm/layer_norm_scale shape    (512,)                 size    512
INFO:tensorflow:Weight    transformer/body/encoder/layer_4/self_attention/layer_prepostprocess/layer_norm/layer_norm_bias       shape    (512,)                 size    512
INFO:tensorflow:Weight    transformer/body/encoder/layer_4/self_attention/layer_prepostprocess/layer_norm/layer_norm_scale      shape    (512,)                 size    512
INFO:tensorflow:Weight    transformer/body/encoder/layer_4/self_attention/multihead_attention/k/kernel          shape    (512, 512)             size    262144
INFO:tensorflow:Weight    transformer/body/encoder/layer_4/self_attention/multihead_attention/output_transform/kernel
shape    (512, 512)             size    262144
INFO:tensorflow:Weight    transformer/body/encoder/layer_4/self_attention/multihead_attention/q/kernel          shape    (512, 512)             size    262144
INFO:tensorflow:Weight    transformer/body/encoder/layer_4/self_attention/multihead_attention/v/kernel          shape    (512, 512)             size    262144
INFO:tensorflow:Weight    transformer/body/encoder/layer_5/ffn/conv1/bias                                       shape    (2048,)                size    2048
INFO:tensorflow:Weight    transformer/body/encoder/layer_5/ffn/conv1/kernel                                     shape    (512, 2048)            size    1048576
INFO:tensorflow:Weight    transformer/body/encoder/layer_5/ffn/conv2/bias                                       shape    (512,)                 size    512
INFO:tensorflow:Weight    transformer/body/encoder/layer_5/ffn/conv2/kernel                                     shape    (2048, 512)            size    1048576
INFO:tensorflow:Weight    transformer/body/encoder/layer_5/ffn/layer_prepostprocess/layer_norm/layer_norm_bias  shape    (512,)                 size    512
INFO:tensorflow:Weight    transformer/body/encoder/layer_5/ffn/layer_prepostprocess/layer_norm/layer_norm_scale shape    (512,)                 size    512
INFO:tensorflow:Weight    transformer/body/encoder/layer_5/self_attention/layer_prepostprocess/layer_norm/layer_norm_bias       shape    (512,)                 size    512
INFO:tensorflow:Weight    transformer/body/encoder/layer_5/self_attention/layer_prepostprocess/layer_norm/layer_norm_scale      shape    (512,)                 size    512
INFO:tensorflow:Weight    transformer/body/encoder/layer_5/self_attention/multihead_attention/k/kernel          shape    (512, 512)             size    262144
INFO:tensorflow:Weight    transformer/body/encoder/layer_5/self_attention/multihead_attention/output_transform/kernel
shape    (512, 512)             size    262144
INFO:tensorflow:Weight    transformer/body/encoder/layer_5/self_attention/multihead_attention/q/kernel          shape    (512, 512)             size    262144
INFO:tensorflow:Weight    transformer/body/encoder/layer_5/self_attention/multihead_attention/v/kernel          shape    (512, 512)             size    262144
INFO:tensorflow:Weight    transformer/body/encoder/layer_prepostprocess/layer_norm/layer_norm_bias              shape    (512,)                 size    512
INFO:tensorflow:Weight    transformer/body/encoder/layer_prepostprocess/layer_norm/layer_norm_scale             shape    (512,)                 size    512
INFO:tensorflow:Weight    transformer/body/target_space_embedding/kernel                                        shape    (32, 512)              size    16384
INFO:tensorflow:Weight    transformer/symbol_modality_8044_512/shared/weights_0                                 shape    (503, 512)             size    257536
INFO:tensorflow:Weight    transformer/symbol_modality_8044_512/shared/weights_10                                shape    (503, 512)             size    257536
INFO:tensorflow:Weight    transformer/symbol_modality_8044_512/shared/weights_11                                shape    (503, 512)             size    257536
INFO:tensorflow:Weight    transformer/symbol_modality_8044_512/shared/weights_12                                shape    (502, 512)             size    257024
INFO:tensorflow:Weight    transformer/symbol_modality_8044_512/shared/weights_13                                shape    (502, 512)             size    257024
INFO:tensorflow:Weight    transformer/symbol_modality_8044_512/shared/weights_14                                shape    (502, 512)             size    257024
INFO:tensorflow:Weight    transformer/symbol_modality_8044_512/shared/weights_15                                shape    (502, 512)             size    257024
INFO:tensorflow:Weight    transformer/symbol_modality_8044_512/shared/weights_1                                 shape    (503, 512)             size    257536
INFO:tensorflow:Weight    transformer/symbol_modality_8044_512/shared/weights_2                                 shape    (503, 512)             size    257536
INFO:tensorflow:Weight    transformer/symbol_modality_8044_512/shared/weights_3                                 shape    (503, 512)             size    257536
INFO:tensorflow:Weight    transformer/symbol_modality_8044_512/shared/weights_4                                 shape    (503, 512)             size    257536
INFO:tensorflow:Weight    transformer/symbol_modality_8044_512/shared/weights_5                                 shape    (503, 512)             size    257536
INFO:tensorflow:Weight    transformer/symbol_modality_8044_512/shared/weights_6                                 shape    (503, 512)             size    257536
INFO:tensorflow:Weight    transformer/symbol_modality_8044_512/shared/weights_7                                 shape    (503, 512)             size    257536
INFO:tensorflow:Weight    transformer/symbol_modality_8044_512/shared/weights_8                                 shape    (503, 512)             size    257536
INFO:tensorflow:Weight    transformer/symbol_modality_8044_512/shared/weights_9                                 shape    (503, 512)             size    257536
INFO:tensorflow:Trainable Variables Total size: 48238592
C:\Users\Artem\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\ops\gradients_impl.py:97: UserWarning: Converting sparse IndexedSlices to a dense Tensor of unknown shape. This may consume a large amount of memory.
  "Converting sparse IndexedSlices to a dense Tensor of unknown shape. "
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Create CheckpointSaverHook.
INFO:tensorflow:Graph was finalized.
2018-01-22 13:17:11.656498: I C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\platform\cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX AVX2
INFO:tensorflow:Restoring parameters from D:\nmt\t2t_out\model.ckpt-8001
2018-01-22 13:17:17.403394: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_4/ffn/conv2/bias/Adam not found in checkpoint
2018-01-22 13:17:17.404585: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_5/encdec_attention/multihead_attention/v/kernel/Adam not found in checkpoint
2018-01-22 13:17:17.405112: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_3/encdec_attention/multihead_attention/v/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:17.405174: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_5/encdec_attention/multihead_attention/q/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:17.407703: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_4/ffn/conv1/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:17.408954: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_5/ffn/conv1/bias/Adam not found in checkpoint
2018-01-22 13:17:17.409079: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_5/ffn/conv1/bias/Adam_1 not found in checkpoint
2018-01-22 13:17:17.409598: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_5/encdec_attention/multihead_attention/v/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:17.411983: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_4/ffn/conv1/kernel/Adam not found in checkpoint
2018-01-22 13:17:17.414939: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_5/ffn/conv1/kernel/Adam not found in checkpoint
2018-01-22 13:17:17.414939: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_5/ffn/conv1/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:17.414999: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_5/ffn/conv2/bias/Adam not found in checkpoint
2018-01-22 13:17:17.419790: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_4/ffn/conv1/bias/Adam_1 not found in checkpoint
2018-01-22 13:17:17.422136: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_5/ffn/conv2/bias/Adam_1 not found in checkpoint
2018-01-22 13:17:17.422234: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_5/ffn/conv2/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:17.423567: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_5/ffn/conv2/kernel/Adam not found in checkpoint
2018-01-22 13:17:17.429337: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_4/ffn/conv1/bias/Adam not found in checkpoint
2018-01-22 13:17:17.431840: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_5/ffn/layer_prepostprocess/layer_norm/layer_norm_bias/Adam_1 not found in checkpoint
2018-01-22 13:17:17.433034: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_5/ffn/layer_prepostprocess/layer_norm/layer_norm_bias/Adam not found in checkpoint
2018-01-22 13:17:17.433034: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_5/ffn/layer_prepostprocess/layer_norm/layer_norm_scale/Adam not found in checkpoint
2018-01-22 13:17:17.439612: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_4/encdec_attention/multihead_attention/v/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:17.441630: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_5/self_attention/layer_prepostprocess/layer_norm/layer_norm_bias/Adam not found in checkpoint
2018-01-22 13:17:17.441630: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_5/self_attention/layer_prepostprocess/layer_norm/layer_norm_bias/Adam_1 not found in checkpoint
2018-01-22 13:17:17.441643: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_5/ffn/layer_prepostprocess/layer_norm/layer_norm_scale/Adam_1 not found in checkpoint
2018-01-22 13:17:17.448279: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_4/encdec_attention/multihead_attention/v/kernel/Adam not found in checkpoint
2018-01-22 13:17:17.449924: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_5/self_attention/layer_prepostprocess/layer_norm/layer_norm_scale/Adam not found in checkpoint
2018-01-22 13:17:17.450334: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_5/self_attention/multihead_attention/k/kernel/Adam not found in checkpoint
2018-01-22 13:17:17.450775: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_5/self_attention/layer_prepostprocess/layer_norm/layer_norm_scale/Adam_1 not found in checkpoint
2018-01-22 13:17:17.455239: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_4/encdec_attention/multihead_attention/q/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:17.456216: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_5/self_attention/multihead_attention/k/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:17.456771: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_5/self_attention/multihead_attention/output_transform/kernel/Adam not found in checkpoint
2018-01-22 13:17:17.459532: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_5/self_attention/multihead_attention/output_transform/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:17.461418: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_4/encdec_attention/multihead_attention/q/kernel/Adam not found in checkpoint
2018-01-22 13:17:17.462788: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_5/self_attention/multihead_attention/q/kernel/Adam not found in checkpoint
2018-01-22 13:17:17.462791: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_5/self_attention/multihead_attention/q/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:17.468904: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_5/self_attention/multihead_attention/v/kernel/Adam not found in checkpoint
2018-01-22 13:17:17.470954: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_4/encdec_attention/multihead_attention/output_transform/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:17.471754: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_prepostprocess/layer_norm/layer_norm_bias/Adam not found in checkpoint
2018-01-22 13:17:17.471765: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_5/self_attention/multihead_attention/v/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:17.475190: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_prepostprocess/layer_norm/layer_norm_bias/Adam_1 not found in checkpoint
2018-01-22 13:17:17.476894: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_4/encdec_attention/multihead_attention/output_transform/kernel/Adam not found in checkpoint
2018-01-22 13:17:17.477741: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_prepostprocess/layer_norm/layer_norm_scale/Adam_1 not found in checkpoint
2018-01-22 13:17:17.477741: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_prepostprocess/layer_norm/layer_norm_scale/Adam not found in checkpoint
2018-01-22 13:17:17.480783: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_0/ffn/conv1/bias/Adam not found in checkpoint
2018-01-22 13:17:17.482274: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_4/encdec_attention/multihead_attention/k/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:17.482875: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_0/ffn/conv1/kernel/Adam not found in checkpoint
2018-01-22 13:17:17.483420: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_0/ffn/conv1/bias/Adam_1 not found in checkpoint
2018-01-22 13:17:17.490662: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_0/ffn/conv1/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:17.493225: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_4/encdec_attention/multihead_attention/k/kernel/Adam not found in checkpoint
2018-01-22 13:17:17.493274: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_0/ffn/conv2/bias/Adam_1 not found in checkpoint
2018-01-22 13:17:17.493374: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_0/ffn/conv2/bias/Adam not found in checkpoint
2018-01-22 13:17:17.499576: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_0/ffn/conv2/kernel/Adam not found in checkpoint
2018-01-22 13:17:17.501697: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_0/ffn/conv2/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:17.501712: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_0/ffn/layer_prepostprocess/layer_norm/layer_norm_bias/Adam not found in checkpoint
2018-01-22 13:17:17.502988: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_4/encdec_attention/layer_prepostprocess/layer_norm/layer_norm_scale/Adam_1 not found in checkpoint
2018-01-22 13:17:17.506421: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_0/ffn/layer_prepostprocess/layer_norm/layer_norm_bias/Adam_1 not found in checkpoint
2018-01-22 13:17:17.509271: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_0/ffn/layer_prepostprocess/layer_norm/layer_norm_scale/Adam not found in checkpoint
2018-01-22 13:17:17.509312: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_0/ffn/layer_prepostprocess/layer_norm/layer_norm_scale/Adam_1 not found in checkpoint
2018-01-22 13:17:17.509913: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_4/encdec_attention/layer_prepostprocess/layer_norm/layer_norm_scale/Adam not found in checkpoint
2018-01-22 13:17:17.511824: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_0/self_attention/layer_prepostprocess/layer_norm/layer_norm_bias/Adam not found in checkpoint
2018-01-22 13:17:17.514986: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_0/self_attention/layer_prepostprocess/layer_norm/layer_norm_scale/Adam not found in checkpoint
2018-01-22 13:17:17.514998: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_0/self_attention/layer_prepostprocess/layer_norm/layer_norm_bias/Adam_1 not found in checkpoint
2018-01-22 13:17:17.515345: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_4/encdec_attention/layer_prepostprocess/layer_norm/layer_norm_bias/Adam_1 not found in checkpoint
2018-01-22 13:17:17.521393: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_0/self_attention/layer_prepostprocess/layer_norm/layer_norm_scale/Adam_1 not found in checkpoint
2018-01-22 13:17:17.522820: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_0/self_attention/multihead_attention/k/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:17.523270: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_4/encdec_attention/layer_prepostprocess/layer_norm/layer_norm_bias/Adam not found in checkpoint
2018-01-22 13:17:17.523610: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_0/self_attention/multihead_attention/k/kernel/Adam not found in checkpoint
2018-01-22 13:17:17.529604: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_0/self_attention/multihead_attention/output_transform/kernel/Adam not found in checkpoint
2018-01-22 13:17:17.530765: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_0/self_attention/multihead_attention/output_transform/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:17.530765: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_0/self_attention/multihead_attention/q/kernel/Adam not found in checkpoint
2018-01-22 13:17:17.531185: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_3/self_attention/multihead_attention/v/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:17.535445: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_0/self_attention/multihead_attention/q/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:17.538752: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_0/self_attention/multihead_attention/v/kernel/Adam not found in checkpoint
2018-01-22 13:17:17.538752: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_3/self_attention/multihead_attention/v/kernel/Adam not found in checkpoint
2018-01-22 13:17:17.538752: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_0/self_attention/multihead_attention/v/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:17.541533: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_1/ffn/conv1/bias/Adam not found in checkpoint
2018-01-22 13:17:17.542217: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_1/ffn/conv1/bias/Adam_1 not found in checkpoint
2018-01-22 13:17:17.543246: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_1/ffn/conv1/kernel/Adam not found in checkpoint
2018-01-22 13:17:17.543371: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_3/self_attention/multihead_attention/q/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:17.550179: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_3/self_attention/multihead_attention/q/kernel/Adam not found in checkpoint
2018-01-22 13:17:17.555690: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_3/self_attention/multihead_attention/output_transform/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:17.564730: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_3/self_attention/multihead_attention/output_transform/kernel/Adam not found in checkpoint
2018-01-22 13:17:17.570283: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_3/self_attention/multihead_attention/k/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:17.573201: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_3/self_attention/multihead_attention/k/kernel/Adam not found in checkpoint
2018-01-22 13:17:17.577641: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_3/self_attention/layer_prepostprocess/layer_norm/layer_norm_scale/Adam_1 not found in checkpoint
2018-01-22 13:17:17.581388: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_3/self_attention/layer_prepostprocess/layer_norm/layer_norm_scale/Adam not found in checkpoint
2018-01-22 13:17:17.587002: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_3/self_attention/layer_prepostprocess/layer_norm/layer_norm_bias/Adam_1 not found in checkpoint
2018-01-22 13:17:17.591740: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_3/self_attention/layer_prepostprocess/layer_norm/layer_norm_bias/Adam not found in checkpoint
2018-01-22 13:17:17.597476: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_3/ffn/layer_prepostprocess/layer_norm/layer_norm_scale/Adam_1 not found in checkpoint
2018-01-22 13:17:17.601197: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_3/ffn/layer_prepostprocess/layer_norm/layer_norm_scale/Adam not found in checkpoint
2018-01-22 13:17:17.606455: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_3/ffn/layer_prepostprocess/layer_norm/layer_norm_bias/Adam_1 not found in checkpoint
2018-01-22 13:17:17.609365: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_3/ffn/layer_prepostprocess/layer_norm/layer_norm_bias/Adam not found in checkpoint
2018-01-22 13:17:17.614264: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_3/ffn/conv2/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:17.619225: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_3/ffn/conv2/kernel/Adam not found in checkpoint
2018-01-22 13:17:17.624495: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_3/ffn/conv2/bias/Adam_1 not found in checkpoint
2018-01-22 13:17:17.629087: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_3/ffn/conv2/bias/Adam not found in checkpoint
2018-01-22 13:17:17.634626: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_3/ffn/conv1/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:17.642459: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_3/ffn/conv1/kernel/Adam not found in checkpoint
2018-01-22 13:17:17.646465: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_3/ffn/conv1/bias/Adam_1 not found in checkpoint
2018-01-22 13:17:17.649546: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_3/ffn/conv1/bias/Adam not found in checkpoint
2018-01-22 13:17:17.790061: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_1/ffn/conv1/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:17.792451: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_1/ffn/conv2/bias/Adam not found in checkpoint
2018-01-22 13:17:17.796302: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_1/ffn/conv2/bias/Adam_1 not found in checkpoint
2018-01-22 13:17:17.797575: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_1/ffn/conv2/kernel/Adam not found in checkpoint
2018-01-22 13:17:17.798844: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_1/ffn/conv2/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:17.802113: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_1/ffn/layer_prepostprocess/layer_norm/layer_norm_bias/Adam not found in checkpoint
2018-01-22 13:17:17.803454: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_1/ffn/layer_prepostprocess/layer_norm/layer_norm_bias/Adam_1 not found in checkpoint
2018-01-22 13:17:17.807171: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_1/ffn/layer_prepostprocess/layer_norm/layer_norm_scale/Adam not found in checkpoint
2018-01-22 13:17:17.808727: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_1/ffn/layer_prepostprocess/layer_norm/layer_norm_scale/Adam_1 not found in checkpoint
2018-01-22 13:17:17.809421: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_1/self_attention/layer_prepostprocess/layer_norm/layer_norm_bias/Adam not found in checkpoint
2018-01-22 13:17:17.812806: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_1/self_attention/layer_prepostprocess/layer_norm/layer_norm_bias/Adam_1 not found in checkpoint
2018-01-22 13:17:17.814979: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_1/self_attention/layer_prepostprocess/layer_norm/layer_norm_scale/Adam not found in checkpoint
2018-01-22 13:17:17.816123: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_1/self_attention/layer_prepostprocess/layer_norm/layer_norm_scale/Adam_1 not found in checkpoint
2018-01-22 13:17:17.818362: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_1/self_attention/multihead_attention/k/kernel/Adam not found in checkpoint
2018-01-22 13:17:17.821462: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_1/self_attention/multihead_attention/k/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:17.822271: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_1/self_attention/multihead_attention/output_transform/kernel/Adam not found in checkpoint
2018-01-22 13:17:17.826808: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_1/self_attention/multihead_attention/output_transform/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:17.828386: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_1/self_attention/multihead_attention/q/kernel/Adam not found in checkpoint
2018-01-22 13:17:17.829838: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_1/self_attention/multihead_attention/q/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:17.833372: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_1/self_attention/multihead_attention/v/kernel/Adam not found in checkpoint
2018-01-22 13:17:17.834168: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_1/self_attention/multihead_attention/v/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:17.837818: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_2/ffn/conv1/bias/Adam not found in checkpoint
2018-01-22 13:17:17.839714: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_2/ffn/conv1/kernel/Adam not found in checkpoint
2018-01-22 13:17:17.839753: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_2/ffn/conv1/bias/Adam_1 not found in checkpoint
2018-01-22 13:17:17.846264: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_2/ffn/conv1/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:17.847915: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_2/ffn/conv2/bias/Adam not found in checkpoint
2018-01-22 13:17:17.848677: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_2/ffn/conv2/bias/Adam_1 not found in checkpoint
2018-01-22 13:17:17.852011: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_2/ffn/conv2/kernel/Adam not found in checkpoint
2018-01-22 13:17:17.852712: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_2/ffn/conv2/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:17.856625: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_2/ffn/layer_prepostprocess/layer_norm/layer_norm_bias/Adam not found in checkpoint
2018-01-22 13:17:17.858523: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_2/ffn/layer_prepostprocess/layer_norm/layer_norm_bias/Adam_1 not found in checkpoint
2018-01-22 13:17:17.859003: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_2/ffn/layer_prepostprocess/layer_norm/layer_norm_scale/Adam not found in checkpoint
2018-01-22 13:17:17.862149: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_2/ffn/layer_prepostprocess/layer_norm/layer_norm_scale/Adam_1 not found in checkpoint
2018-01-22 13:17:17.863801: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_2/self_attention/layer_prepostprocess/layer_norm/layer_norm_bias/Adam not found in checkpoint
2018-01-22 13:17:17.864441: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_2/self_attention/layer_prepostprocess/layer_norm/layer_norm_bias/Adam_1 not found in checkpoint
2018-01-22 13:17:17.869325: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_2/self_attention/layer_prepostprocess/layer_norm/layer_norm_scale/Adam not found in checkpoint
2018-01-22 13:17:17.872186: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_2/self_attention/multihead_attention/k/kernel/Adam not found in checkpoint
2018-01-22 13:17:17.873109: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_2/self_attention/layer_prepostprocess/layer_norm/layer_norm_scale/Adam_1 not found in checkpoint
2018-01-22 13:17:17.877862: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_2/self_attention/multihead_attention/k/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:17.878761: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_2/self_attention/multihead_attention/output_transform/kernel/Adam not found in checkpoint
2018-01-22 13:17:17.879991: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_2/self_attention/multihead_attention/output_transform/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:17.883401: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_2/self_attention/multihead_attention/q/kernel/Adam not found in checkpoint
2018-01-22 13:17:17.886680: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_2/self_attention/multihead_attention/v/kernel/Adam not found in checkpoint
2018-01-22 13:17:17.886750: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_2/self_attention/multihead_attention/q/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:17.893026: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_2/self_attention/multihead_attention/v/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:17.894789: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_3/ffn/conv1/bias/Adam not found in checkpoint
2018-01-22 13:17:17.898505: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_3/ffn/conv1/bias/Adam_1 not found in checkpoint
2018-01-22 13:17:17.903007: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_3/ffn/conv1/kernel/Adam not found in checkpoint
2018-01-22 13:17:17.906001: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_3/ffn/conv1/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:17.908031: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_3/ffn/conv2/bias/Adam not found in checkpoint
2018-01-22 13:17:17.911252: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_3/ffn/conv2/bias/Adam_1 not found in checkpoint
2018-01-22 13:17:17.911567: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_3/ffn/conv2/kernel/Adam not found in checkpoint
2018-01-22 13:17:17.916570: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_3/ffn/conv2/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:17.920467: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_3/ffn/layer_prepostprocess/layer_norm/layer_norm_bias/Adam_1 not found in checkpoint
2018-01-22 13:17:17.920530: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_3/ffn/layer_prepostprocess/layer_norm/layer_norm_bias/Adam not found in checkpoint
2018-01-22 13:17:17.926260: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_3/ffn/layer_prepostprocess/layer_norm/layer_norm_scale/Adam not found in checkpoint
2018-01-22 13:17:17.927769: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_3/ffn/layer_prepostprocess/layer_norm/layer_norm_scale/Adam_1 not found in checkpoint
2018-01-22 13:17:17.928822: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_3/self_attention/layer_prepostprocess/layer_norm/layer_norm_bias/Adam not found in checkpoint
2018-01-22 13:17:17.931706: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_3/self_attention/layer_prepostprocess/layer_norm/layer_norm_bias/Adam_1 not found in checkpoint
2018-01-22 13:17:17.932840: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_3/self_attention/layer_prepostprocess/layer_norm/layer_norm_scale/Adam not found in checkpoint
2018-01-22 13:17:17.933581: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_3/self_attention/layer_prepostprocess/layer_norm/layer_norm_scale/Adam_1 not found in checkpoint
2018-01-22 13:17:17.939270: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_3/self_attention/multihead_attention/k/kernel/Adam not found in checkpoint
2018-01-22 13:17:17.940407: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_3/self_attention/multihead_attention/k/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:17.942443: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_3/self_attention/multihead_attention/output_transform/kernel/Adam not found in checkpoint
2018-01-22 13:17:17.946737: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_3/self_attention/multihead_attention/q/kernel/Adam not found in checkpoint
2018-01-22 13:17:17.946738: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_3/self_attention/multihead_attention/output_transform/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:17.949017: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_3/self_attention/multihead_attention/q/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:17.952332: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_3/self_attention/multihead_attention/v/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:17.952820: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_3/self_attention/multihead_attention/v/kernel/Adam not found in checkpoint
2018-01-22 13:17:17.957323: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_4/ffn/conv1/bias/Adam not found in checkpoint
2018-01-22 13:17:17.959112: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_4/ffn/conv1/kernel/Adam not found in checkpoint
2018-01-22 13:17:17.959336: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_4/ffn/conv1/bias/Adam_1 not found in checkpoint
2018-01-22 13:17:17.963345: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_4/ffn/conv1/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:17.965715: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_4/ffn/conv2/bias/Adam not found in checkpoint
2018-01-22 13:17:17.968379: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_4/ffn/conv2/bias/Adam_1 not found in checkpoint
2018-01-22 13:17:17.971175: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_4/ffn/conv2/kernel/Adam not found in checkpoint
2018-01-22 13:17:17.972636: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_4/ffn/conv2/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:17.976259: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_4/ffn/layer_prepostprocess/layer_norm/layer_norm_bias/Adam not found in checkpoint
2018-01-22 13:17:17.977434: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_4/ffn/layer_prepostprocess/layer_norm/layer_norm_bias/Adam_1 not found in checkpoint
2018-01-22 13:17:17.978663: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_4/ffn/layer_prepostprocess/layer_norm/layer_norm_scale/Adam not found in checkpoint
2018-01-22 13:17:17.981294: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_4/ffn/layer_prepostprocess/layer_norm/layer_norm_scale/Adam_1 not found in checkpoint
2018-01-22 13:17:17.984580: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_4/self_attention/layer_prepostprocess/layer_norm/layer_norm_bias/Adam not found in checkpoint
2018-01-22 13:17:17.985962: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_4/self_attention/layer_prepostprocess/layer_norm/layer_norm_bias/Adam_1 not found in checkpoint
2018-01-22 13:17:17.987747: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_4/self_attention/layer_prepostprocess/layer_norm/layer_norm_scale/Adam not found in checkpoint
2018-01-22 13:17:17.991531: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_4/self_attention/layer_prepostprocess/layer_norm/layer_norm_scale/Adam_1 not found in checkpoint
2018-01-22 13:17:17.991668: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_4/self_attention/multihead_attention/k/kernel/Adam not found in checkpoint
2018-01-22 13:17:17.996451: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_4/self_attention/multihead_attention/k/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:17.998151: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_4/self_attention/multihead_attention/output_transform/kernel/Adam not found in checkpoint
2018-01-22 13:17:17.999581: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_4/self_attention/multihead_attention/output_transform/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:18.002221: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_4/self_attention/multihead_attention/q/kernel/Adam not found in checkpoint
2018-01-22 13:17:18.007001: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_4/self_attention/multihead_attention/q/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:18.007020: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_4/self_attention/multihead_attention/v/kernel/Adam not found in checkpoint
2018-01-22 13:17:18.010220: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_4/self_attention/multihead_attention/v/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:18.012379: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_5/ffn/conv1/bias/Adam_1 not found in checkpoint
2018-01-22 13:17:18.012379: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_5/ffn/conv1/bias/Adam not found in checkpoint
2018-01-22 13:17:18.015833: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_5/ffn/conv1/kernel/Adam not found in checkpoint
2018-01-22 13:17:18.017391: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_5/ffn/conv2/bias/Adam not found in checkpoint
2018-01-22 13:17:18.018924: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_5/ffn/conv1/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:18.022087: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_4/ffn/conv2/bias/Adam_1 not found in checkpoint
2018-01-22 13:17:18.024688: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_4/ffn/conv2/kernel/Adam not found in checkpoint
2018-01-22 13:17:18.025601: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_4/ffn/conv2/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:18.027646: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_4/ffn/layer_prepostprocess/layer_norm/layer_norm_bias/Adam not found in checkpoint
2018-01-22 13:17:18.029221: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_4/ffn/layer_prepostprocess/layer_norm/layer_norm_bias/Adam_1 not found in checkpoint
2018-01-22 13:17:18.032791: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_4/ffn/layer_prepostprocess/layer_norm/layer_norm_scale/Adam not found in checkpoint
2018-01-22 13:17:18.037021: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_4/ffn/layer_prepostprocess/layer_norm/layer_norm_scale/Adam_1 not found in checkpoint
2018-01-22 13:17:18.037798: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_4/self_attention/layer_prepostprocess/layer_norm/layer_norm_bias/Adam not found in checkpoint
2018-01-22 13:17:18.040490: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_4/self_attention/layer_prepostprocess/layer_norm/layer_norm_bias/Adam_1 not found in checkpoint
2018-01-22 13:17:18.042538: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_4/self_attention/layer_prepostprocess/layer_norm/layer_norm_scale/Adam_1 not found in checkpoint
2018-01-22 13:17:18.042825: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_4/self_attention/layer_prepostprocess/layer_norm/layer_norm_scale/Adam not found in checkpoint
2018-01-22 13:17:18.045877: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_4/self_attention/multihead_attention/k/kernel/Adam not found in checkpoint
2018-01-22 13:17:18.047365: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_4/self_attention/multihead_attention/k/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:18.048330: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_4/self_attention/multihead_attention/output_transform/kernel/Adam not found in checkpoint
2018-01-22 13:17:18.051341: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_4/self_attention/multihead_attention/output_transform/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:18.055458: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_4/self_attention/multihead_attention/q/kernel/Adam not found in checkpoint
2018-01-22 13:17:18.056833: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_4/self_attention/multihead_attention/q/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:18.058009: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_4/self_attention/multihead_attention/v/kernel/Adam not found in checkpoint
2018-01-22 13:17:18.059596: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_4/self_attention/multihead_attention/v/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:18.062791: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_5/encdec_attention/layer_prepostprocess/layer_norm/layer_norm_bias/Adam not found in checkpoint
2018-01-22 13:17:18.063072: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_5/encdec_attention/layer_prepostprocess/layer_norm/layer_norm_bias/Adam_1 not found in checkpoint
2018-01-22 13:17:18.065225: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_5/encdec_attention/layer_prepostprocess/layer_norm/layer_norm_scale/Adam not found in checkpoint
2018-01-22 13:17:18.067392: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_5/encdec_attention/multihead_attention/k/kernel/Adam not found in checkpoint
2018-01-22 13:17:18.068209: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_5/encdec_attention/layer_prepostprocess/layer_norm/layer_norm_scale/Adam_1 not found in checkpoint
2018-01-22 13:17:18.071974: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_5/encdec_attention/multihead_attention/k/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:18.073657: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_5/encdec_attention/multihead_attention/output_transform/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:18.074583: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_5/encdec_attention/multihead_attention/output_transform/kernel/Adam not found in checkpoint
2018-01-22 13:17:18.077361: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_5/encdec_attention/multihead_attention/q/kernel/Adam not found in checkpoint
2018-01-22 13:17:18.143139: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_5/ffn/conv2/bias/Adam_1 not found in checkpoint
2018-01-22 13:17:18.146081: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_5/ffn/conv2/kernel/Adam not found in checkpoint
2018-01-22 13:17:18.150439: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_5/ffn/conv2/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:18.152058: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_5/ffn/layer_prepostprocess/layer_norm/layer_norm_bias/Adam not found in checkpoint
2018-01-22 13:17:18.154038: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_5/ffn/layer_prepostprocess/layer_norm/layer_norm_bias/Adam_1 not found in checkpoint
2018-01-22 13:17:18.155796: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_5/ffn/layer_prepostprocess/layer_norm/layer_norm_scale/Adam not found in checkpoint
2018-01-22 13:17:18.159248: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_5/ffn/layer_prepostprocess/layer_norm/layer_norm_scale/Adam_1 not found in checkpoint
2018-01-22 13:17:18.161829: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_5/self_attention/layer_prepostprocess/layer_norm/layer_norm_bias/Adam not found in checkpoint
2018-01-22 13:17:18.165128: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_5/self_attention/layer_prepostprocess/layer_norm/layer_norm_bias/Adam_1 not found in checkpoint
2018-01-22 13:17:18.166899: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_5/self_attention/layer_prepostprocess/layer_norm/layer_norm_scale/Adam not found in checkpoint
2018-01-22 13:17:18.170958: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_5/self_attention/layer_prepostprocess/layer_norm/layer_norm_scale/Adam_1 not found in checkpoint
2018-01-22 13:17:18.172830: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_5/self_attention/multihead_attention/k/kernel/Adam not found in checkpoint
2018-01-22 13:17:18.173683: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_5/self_attention/multihead_attention/k/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:18.179480: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_5/self_attention/multihead_attention/output_transform/kernel/Adam not found in checkpoint
2018-01-22 13:17:18.181657: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_5/self_attention/multihead_attention/output_transform/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:18.181830: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_5/self_attention/multihead_attention/q/kernel/Adam not found in checkpoint
2018-01-22 13:17:18.185268: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_5/self_attention/multihead_attention/q/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:18.189373: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_5/self_attention/multihead_attention/v/kernel/Adam not found in checkpoint
2018-01-22 13:17:18.190019: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_5/self_attention/multihead_attention/v/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:18.191316: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_prepostprocess/layer_norm/layer_norm_bias/Adam not found in checkpoint
2018-01-22 13:17:18.194574: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_prepostprocess/layer_norm/layer_norm_bias/Adam_1 not found in checkpoint
2018-01-22 13:17:18.195347: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_prepostprocess/layer_norm/layer_norm_scale/Adam not found in checkpoint
2018-01-22 13:17:18.199076: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/encoder/layer_prepostprocess/layer_norm/layer_norm_scale/Adam_1 not found in checkpoint
2018-01-22 13:17:18.201614: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/target_space_embedding/kernel/Adam not found in checkpoint
2018-01-22 13:17:18.201630: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/target_space_embedding/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:18.205218: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/symbol_modality_8044_512/shared/weights_0/Adam not found in checkpoint
2018-01-22 13:17:18.207096: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/symbol_modality_8044_512/shared/weights_0/Adam_1 not found in checkpoint
2018-01-22 13:17:18.207790: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/symbol_modality_8044_512/shared/weights_1/Adam not found in checkpoint
2018-01-22 13:17:18.211089: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/symbol_modality_8044_512/shared/weights_1/Adam_1 not found in checkpoint
2018-01-22 13:17:18.212025: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/symbol_modality_8044_512/shared/weights_10/Adam not found in checkpoint
2018-01-22 13:17:18.222614: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/beta1_power not found in checkpoint
2018-01-22 13:17:18.225658: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/beta2_power not found in checkpoint
2018-01-22 13:17:18.228915: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_0/encdec_attention/layer_prepostprocess/layer_norm/layer_norm_bias/Adam not found in checkpoint
2018-01-22 13:17:18.229581: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_0/encdec_attention/layer_prepostprocess/layer_norm/layer_norm_bias/Adam_1 not found in checkpoint
2018-01-22 13:17:18.234493: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_0/encdec_attention/layer_prepostprocess/layer_norm/layer_norm_scale/Adam not found in checkpoint
2018-01-22 13:17:18.239674: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_0/encdec_attention/multihead_attention/k/kernel/Adam not found in checkpoint
2018-01-22 13:17:18.239694: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_0/encdec_attention/layer_prepostprocess/layer_norm/layer_norm_scale/Adam_1 not found in checkpoint
2018-01-22 13:17:18.242390: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_0/encdec_attention/multihead_attention/k/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:18.244430: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_0/encdec_attention/multihead_attention/output_transform/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:18.247286: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_0/encdec_attention/multihead_attention/output_transform/kernel/Adam not found in checkpoint
2018-01-22 13:17:18.249362: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_0/encdec_attention/multihead_attention/q/kernel/Adam not found in checkpoint
2018-01-22 13:17:18.250523: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_0/encdec_attention/multihead_attention/q/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:18.252598: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_0/encdec_attention/multihead_attention/v/kernel/Adam not found in checkpoint
2018-01-22 13:17:18.255496: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_0/encdec_attention/multihead_attention/v/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:18.257510: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_0/ffn/conv1/bias/Adam not found in checkpoint
2018-01-22 13:17:18.258562: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_0/ffn/conv1/bias/Adam_1 not found in checkpoint
2018-01-22 13:17:18.260398: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/symbol_modality_8044_512/shared/weights_9/Adam_1 not found in checkpoint
2018-01-22 13:17:18.260445: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_0/ffn/conv1/kernel/Adam not found in checkpoint
2018-01-22 13:17:18.261641: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_0/ffn/conv1/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:18.262444: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_0/ffn/conv2/bias/Adam not found in checkpoint
2018-01-22 13:17:18.266065: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/symbol_modality_8044_512/shared/weights_9/Adam not found in checkpoint
2018-01-22 13:17:18.266305: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_0/ffn/conv2/bias/Adam_1 not found in checkpoint
2018-01-22 13:17:18.267049: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_0/ffn/conv2/kernel/Adam not found in checkpoint
2018-01-22 13:17:18.268993: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_0/ffn/conv2/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:18.271535: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_0/ffn/layer_prepostprocess/layer_norm/layer_norm_bias/Adam not found in checkpoint
2018-01-22 13:17:18.271570: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/symbol_modality_8044_512/shared/weights_8/Adam_1 not found in checkpoint
2018-01-22 13:17:18.271582: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_0/ffn/layer_prepostprocess/layer_norm/layer_norm_bias/Adam_1 not found in checkpoint
2018-01-22 13:17:18.275271: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_0/ffn/layer_prepostprocess/layer_norm/layer_norm_scale/Adam not found in checkpoint
2018-01-22 13:17:18.279004: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_0/self_attention/layer_prepostprocess/layer_norm/layer_norm_bias/Adam not found in checkpoint
2018-01-22 13:17:18.279219: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/symbol_modality_8044_512/shared/weights_8/Adam not found in checkpoint
2018-01-22 13:17:18.280011: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_0/ffn/layer_prepostprocess/layer_norm/layer_norm_scale/Adam_1 not found in checkpoint
2018-01-22 13:17:18.281194: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_0/self_attention/layer_prepostprocess/layer_norm/layer_norm_bias/Adam_1 not found in checkpoint
2018-01-22 13:17:18.282779: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_0/self_attention/layer_prepostprocess/layer_norm/layer_norm_scale/Adam not found in checkpoint
2018-01-22 13:17:18.285108: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/symbol_modality_8044_512/shared/weights_7/Adam_1 not found in checkpoint
2018-01-22 13:17:18.286062: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_0/self_attention/layer_prepostprocess/layer_norm/layer_norm_scale/Adam_1 not found in checkpoint
2018-01-22 13:17:18.289651: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_0/self_attention/multihead_attention/k/kernel/Adam not found in checkpoint
2018-01-22 13:17:18.291956: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_0/self_attention/multihead_attention/k/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:18.294987: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/symbol_modality_8044_512/shared/weights_7/Adam not found in checkpoint
2018-01-22 13:17:18.294993: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_0/self_attention/multihead_attention/output_transform/kernel/Adam not found in checkpoint
2018-01-22 13:17:18.299554: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_0/self_attention/multihead_attention/output_transform/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:18.300758: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_0/self_attention/multihead_attention/q/kernel/Adam not found in checkpoint
2018-01-22 13:17:18.302941: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/symbol_modality_8044_512/shared/weights_6/Adam_1 not found in checkpoint
2018-01-22 13:17:18.302944: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_0/self_attention/multihead_attention/q/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:18.308501: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_0/self_attention/multihead_attention/v/kernel/Adam not found in checkpoint
2018-01-22 13:17:18.310261: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_0/self_attention/multihead_attention/v/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:18.310598: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_1/encdec_attention/layer_prepostprocess/layer_norm/layer_norm_bias/Adam not found in checkpoint
2018-01-22 13:17:18.311196: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/symbol_modality_8044_512/shared/weights_6/Adam not found in checkpoint
2018-01-22 13:17:18.314194: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_1/encdec_attention/layer_prepostprocess/layer_norm/layer_norm_bias/Adam_1 not found in checkpoint
2018-01-22 13:17:18.315575: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_1/encdec_attention/layer_prepostprocess/layer_norm/layer_norm_scale/Adam_1 not found in checkpoint
2018-01-22 13:17:18.315982: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_1/encdec_attention/layer_prepostprocess/layer_norm/layer_norm_scale/Adam not found in checkpoint
2018-01-22 13:17:18.319021: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/symbol_modality_8044_512/shared/weights_5/Adam_1 not found in checkpoint
2018-01-22 13:17:18.320861: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_1/encdec_attention/multihead_attention/k/kernel/Adam not found in checkpoint
2018-01-22 13:17:18.321881: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_1/encdec_attention/multihead_attention/k/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:18.324112: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_1/encdec_attention/multihead_attention/output_transform/kernel/Adam not found in checkpoint
2018-01-22 13:17:18.324112: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/symbol_modality_8044_512/shared/weights_5/Adam not found in checkpoint
2018-01-22 13:17:18.328721: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_1/encdec_attention/multihead_attention/output_transform/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:18.329639: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_1/encdec_attention/multihead_attention/q/kernel/Adam not found in checkpoint
2018-01-22 13:17:18.330205: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/symbol_modality_8044_512/shared/weights_4/Adam_1 not found in checkpoint
2018-01-22 13:17:18.330601: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_1/encdec_attention/multihead_attention/q/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:18.334301: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_1/encdec_attention/multihead_attention/v/kernel/Adam not found in checkpoint
2018-01-22 13:17:18.335013: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/symbol_modality_8044_512/shared/weights_4/Adam not found in checkpoint
2018-01-22 13:17:18.338784: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_1/encdec_attention/multihead_attention/v/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:18.339137: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_1/ffn/conv1/bias/Adam not found in checkpoint
2018-01-22 13:17:18.341488: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_1/ffn/conv1/bias/Adam_1 not found in checkpoint
2018-01-22 13:17:18.341542: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/symbol_modality_8044_512/shared/weights_3/Adam_1 not found in checkpoint
2018-01-22 13:17:18.342412: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_1/ffn/conv1/kernel/Adam not found in checkpoint
2018-01-22 13:17:18.347380: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_1/ffn/conv1/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:18.349555: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/symbol_modality_8044_512/shared/weights_3/Adam not found in checkpoint
2018-01-22 13:17:18.349723: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_1/ffn/conv2/bias/Adam_1 not found in checkpoint
2018-01-22 13:17:18.350315: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_1/ffn/conv2/bias/Adam not found in checkpoint
2018-01-22 13:17:18.352014: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_1/ffn/conv2/kernel/Adam not found in checkpoint
2018-01-22 13:17:18.355825: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/symbol_modality_8044_512/shared/weights_2/Adam_1 not found in checkpoint
2018-01-22 13:17:18.355825: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_1/ffn/conv2/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:18.356710: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_1/ffn/layer_prepostprocess/layer_norm/layer_norm_bias/Adam not found in checkpoint
2018-01-22 13:17:18.358779: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_1/ffn/layer_prepostprocess/layer_norm/layer_norm_bias/Adam_1 not found in checkpoint
2018-01-22 13:17:18.362131: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/symbol_modality_8044_512/shared/weights_2/Adam not found in checkpoint
2018-01-22 13:17:18.363363: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_1/ffn/layer_prepostprocess/layer_norm/layer_norm_scale/Adam_1 not found in checkpoint
2018-01-22 13:17:18.364417: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_1/ffn/layer_prepostprocess/layer_norm/layer_norm_scale/Adam not found in checkpoint
2018-01-22 13:17:18.368992: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_1/self_attention/layer_prepostprocess/layer_norm/layer_norm_bias/Adam not found in checkpoint
2018-01-22 13:17:18.370900: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/symbol_modality_8044_512/shared/weights_15/Adam_1 not found in checkpoint
2018-01-22 13:17:18.371253: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_1/self_attention/layer_prepostprocess/layer_norm/layer_norm_scale/Adam not found in checkpoint
2018-01-22 13:17:18.372246: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_1/self_attention/layer_prepostprocess/layer_norm/layer_norm_bias/Adam_1 not found in checkpoint
2018-01-22 13:17:18.374566: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_1/self_attention/layer_prepostprocess/layer_norm/layer_norm_scale/Adam_1 not found in checkpoint
2018-01-22 13:17:18.378576: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_1/self_attention/multihead_attention/k/kernel/Adam not found in checkpoint
2018-01-22 13:17:18.378579: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/symbol_modality_8044_512/shared/weights_15/Adam not found in checkpoint
2018-01-22 13:17:18.380225: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_1/self_attention/multihead_attention/k/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:18.381195: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_1/self_attention/multihead_attention/output_transform/kernel/Adam not found in checkpoint
2018-01-22 13:17:18.384099: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_1/self_attention/multihead_attention/output_transform/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:18.384187: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/symbol_modality_8044_512/shared/weights_14/Adam_1 not found in checkpoint
2018-01-22 13:17:18.389493: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_1/self_attention/multihead_attention/q/kernel/Adam not found in checkpoint
2018-01-22 13:17:18.390672: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_1/self_attention/multihead_attention/q/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:18.392080: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_1/self_attention/multihead_attention/v/kernel/Adam not found in checkpoint
2018-01-22 13:17:18.392986: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/symbol_modality_8044_512/shared/weights_14/Adam not found in checkpoint
2018-01-22 13:17:18.395466: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_1/self_attention/multihead_attention/v/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:18.397760: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_2/encdec_attention/layer_prepostprocess/layer_norm/layer_norm_bias/Adam not found in checkpoint
2018-01-22 13:17:18.399169: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_2/encdec_attention/layer_prepostprocess/layer_norm/layer_norm_bias/Adam_1 not found in checkpoint
2018-01-22 13:17:18.399926: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/symbol_modality_8044_512/shared/weights_13/Adam_1 not found in checkpoint
2018-01-22 13:17:18.401000: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_2/encdec_attention/layer_prepostprocess/layer_norm/layer_norm_scale/Adam not found in checkpoint
2018-01-22 13:17:18.402758: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_2/encdec_attention/layer_prepostprocess/layer_norm/layer_norm_scale/Adam_1 not found in checkpoint
2018-01-22 13:17:18.405682: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_2/encdec_attention/multihead_attention/k/kernel/Adam not found in checkpoint
2018-01-22 13:17:18.407768: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/symbol_modality_8044_512/shared/weights_13/Adam not found in checkpoint
2018-01-22 13:17:18.408609: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_2/encdec_attention/multihead_attention/k/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:18.409439: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_2/encdec_attention/multihead_attention/output_transform/kernel/Adam not found in checkpoint
2018-01-22 13:17:18.411014: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_2/encdec_attention/multihead_attention/output_transform/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:18.412415: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/symbol_modality_8044_512/shared/weights_12/Adam_1 not found in checkpoint
2018-01-22 13:17:18.412656: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_2/encdec_attention/multihead_attention/q/kernel/Adam not found in checkpoint
2018-01-22 13:17:18.418792: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_2/encdec_attention/multihead_attention/q/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:18.419662: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_2/encdec_attention/multihead_attention/v/kernel/Adam not found in checkpoint
2018-01-22 13:17:18.421113: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/symbol_modality_8044_512/shared/weights_12/Adam not found in checkpoint
2018-01-22 13:17:18.421127: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_2/encdec_attention/multihead_attention/v/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:18.424391: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_2/ffn/conv1/bias/Adam not found in checkpoint
2018-01-22 13:17:18.425134: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_2/ffn/conv1/bias/Adam_1 not found in checkpoint
2018-01-22 13:17:18.428854: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/symbol_modality_8044_512/shared/weights_11/Adam_1 not found in checkpoint
2018-01-22 13:17:18.431078: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_2/ffn/conv1/kernel/Adam not found in checkpoint
2018-01-22 13:17:18.433034: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_2/ffn/conv1/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:18.433147: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_2/ffn/conv2/bias/Adam not found in checkpoint
2018-01-22 13:17:18.439181: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/symbol_modality_8044_512/shared/weights_11/Adam not found in checkpoint
2018-01-22 13:17:18.440295: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_2/ffn/conv2/bias/Adam_1 not found in checkpoint
2018-01-22 13:17:18.441708: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_2/ffn/conv2/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:18.442132: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_2/ffn/conv2/kernel/Adam not found in checkpoint
2018-01-22 13:17:18.445227: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/symbol_modality_8044_512/shared/weights_10/Adam_1 not found in checkpoint
2018-01-22 13:17:18.446008: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_2/ffn/layer_prepostprocess/layer_norm/layer_norm_bias/Adam not found in checkpoint
2018-01-22 13:17:18.449370: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_2/ffn/layer_prepostprocess/layer_norm/layer_norm_bias/Adam_1 not found in checkpoint
2018-01-22 13:17:18.450139: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_2/ffn/layer_prepostprocess/layer_norm/layer_norm_scale/Adam not found in checkpoint
2018-01-22 13:17:18.452332: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_3/encdec_attention/multihead_attention/v/kernel/Adam not found in checkpoint
2018-01-22 13:17:18.455035: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_2/ffn/layer_prepostprocess/layer_norm/layer_norm_scale/Adam_1 not found in checkpoint
2018-01-22 13:17:18.455962: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_2/self_attention/layer_prepostprocess/layer_norm/layer_norm_bias/Adam not found in checkpoint
2018-01-22 13:17:18.456449: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_2/self_attention/layer_prepostprocess/layer_norm/layer_norm_bias/Adam_1 not found in checkpoint
2018-01-22 13:17:18.460374: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_3/encdec_attention/multihead_attention/q/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:18.461379: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_2/self_attention/layer_prepostprocess/layer_norm/layer_norm_scale/Adam not found in checkpoint
2018-01-22 13:17:18.462082: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_2/self_attention/layer_prepostprocess/layer_norm/layer_norm_scale/Adam_1 not found in checkpoint
2018-01-22 13:17:18.462687: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_2/self_attention/multihead_attention/k/kernel/Adam not found in checkpoint
2018-01-22 13:17:18.467189: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_2/self_attention/multihead_attention/k/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:18.468006: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_3/encdec_attention/multihead_attention/q/kernel/Adam not found in checkpoint
2018-01-22 13:17:18.468128: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_2/self_attention/multihead_attention/output_transform/kernel/Adam not found in checkpoint
2018-01-22 13:17:18.469620: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_2/self_attention/multihead_attention/output_transform/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:18.471352: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_2/self_attention/multihead_attention/q/kernel/Adam not found in checkpoint
2018-01-22 13:17:18.472990: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_2/self_attention/multihead_attention/q/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:18.472990: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_3/encdec_attention/multihead_attention/output_transform/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:18.478606: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_2/self_attention/multihead_attention/v/kernel/Adam not found in checkpoint
2018-01-22 13:17:18.479917: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_2/self_attention/multihead_attention/v/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:18.480992: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_3/encdec_attention/multihead_attention/output_transform/kernel/Adam not found in checkpoint
2018-01-22 13:17:18.481232: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_3/encdec_attention/layer_prepostprocess/layer_norm/layer_norm_bias/Adam not found in checkpoint
2018-01-22 13:17:18.484182: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_3/encdec_attention/layer_prepostprocess/layer_norm/layer_norm_bias/Adam_1 not found in checkpoint
2018-01-22 13:17:18.488938: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_3/encdec_attention/layer_prepostprocess/layer_norm/layer_norm_scale/Adam not found in checkpoint
2018-01-22 13:17:18.491249: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_3/encdec_attention/multihead_attention/k/kernel/Adam_1 not found in checkpoint
2018-01-22 13:17:18.491249: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_3/encdec_attention/layer_prepostprocess/layer_norm/layer_norm_scale/Adam_1 not found in checkpoint
2018-01-22 13:17:18.492776: W C:\tf_jenkins\workspace\tf-nightly-windows\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Key training/transformer/body/decoder/layer_3/encdec_attention/multihead_attention/k/kernel/Adam not found in checkpoint
Traceback (most recent call last):
  File "C:\Users\Artem\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\client\session.py", line 1350, in _do_call
    return fn(*args)
  File "C:\Users\Artem\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\client\session.py", line 1329, in _run_fn
    status, run_metadata)
  File "C:\Users\Artem\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\framework\errors_impl.py", line 473, in __exit__
    c_api.TF_GetCode(self.status.status))
tensorflow.python.framework.errors_impl.NotFoundError: Key training/transformer/body/decoder/layer_4/ffn/conv2/bias/Adam not found in checkpoint
         [[Node: save/RestoreV2_167 = RestoreV2[dtypes=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_save/Const_0_0, save/RestoreV2_167/tensor_names, save/RestoreV2_167/shape_and_slices)]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\Artem\AppData\Local\Programs\Python\Python36\Lib\site-packages\tensor2tensor\bin\t2t_trainer.py", line 241, in <module>
    tf.app.run()
  File "C:\Users\Artem\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\platform\app.py", line 124, in run
    _sys.exit(main(argv))
  File "C:\Users\Artem\AppData\Local\Programs\Python\Python36\Lib\site-packages\tensor2tensor\bin\t2t_trainer.py", line 237, in main
    execute_schedule(exp)
  File "C:\Users\Artem\AppData\Local\Programs\Python\Python36\Lib\site-packages\tensor2tensor\bin\t2t_trainer.py", line 217, in execute_schedule
    getattr(exp, FLAGS.schedule)()
  File "C:\Users\Artem\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\contrib\framework\python\framework\experimental.py", line 64, in new_func
    return func(*args, **kwargs)
  File "C:\Users\Artem\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\contrib\learn\python\learn\experiment.py", line 770, in continuous_train_and_eval
    saving_listeners=self._saving_listeners)
  File "C:\Users\Artem\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\contrib\learn\python\learn\experiment.py", line 868, in _call_train
    saving_listeners=saving_listeners)
  File "C:\Users\Artem\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\estimator\estimator.py", line 324, in train
    loss = self._train_model(input_fn, hooks, saving_listeners)
  File "C:\Users\Artem\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\estimator\estimator.py", line 849, in _train_model
    log_step_count_steps=self._config.log_step_count_steps) as mon_sess:
  File "C:\Users\Artem\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\training\monitored_session.py", line 381, in MonitoredTrainingSession
    stop_grace_period_secs=stop_grace_period_secs)
  File "C:\Users\Artem\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\training\monitored_session.py", line 788, in __init__
    stop_grace_period_secs=stop_grace_period_secs)
  File "C:\Users\Artem\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\training\monitored_session.py", line 512, in __init__
    self._sess = _RecoverableSession(self._coordinated_creator)
  File "C:\Users\Artem\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\training\monitored_session.py", line 973, in __init__
    _WrappedSession.__init__(self, self._create_session())
  File "C:\Users\Artem\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\training\monitored_session.py", line 978, in _create_session
    return self._sess_creator.create_session()
  File "C:\Users\Artem\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\training\monitored_session.py", line 669, in create_session
    self.tf_sess = self._session_creator.create_session()
  File "C:\Users\Artem\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\training\monitored_session.py", line 441, in create_session
    init_fn=self._scaffold.init_fn)
  File "C:\Users\Artem\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\training\session_manager.py", line 273, in prepare_session
    config=config)
  File "C:\Users\Artem\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\training\session_manager.py", line 205, in _restore_checkpoint
    saver.restore(sess, ckpt.model_checkpoint_path)
  File "C:\Users\Artem\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\training\saver.py", line 1750, in restore
    {self.saver_def.filename_tensor_name: save_path})
  File "C:\Users\Artem\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\client\session.py", line 895, in run
    run_metadata_ptr)
  File "C:\Users\Artem\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\client\session.py", line 1128, in _run
    feed_dict_tensor, options, run_metadata)
  File "C:\Users\Artem\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\client\session.py", line 1344, in _do_run
    options, run_metadata)
  File "C:\Users\Artem\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\client\session.py", line 1363, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.NotFoundError: Key training/transformer/body/decoder/layer_4/ffn/conv2/bias/Adam not found in checkpoint
         [[Node: save/RestoreV2_167 = RestoreV2[dtypes=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_save/Const_0_0, save/RestoreV2_167/tensor_names, save/RestoreV2_167/shape_and_slices)]]

Caused by op 'save/RestoreV2_167', defined at:
  File "C:\Users\Artem\AppData\Local\Programs\Python\Python36\Lib\site-packages\tensor2tensor\bin\t2t_trainer.py", line 241, in <module>
    tf.app.run()
  File "C:\Users\Artem\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\platform\app.py", line 124, in run
    _sys.exit(main(argv))
  File "C:\Users\Artem\AppData\Local\Programs\Python\Python36\Lib\site-packages\tensor2tensor\bin\t2t_trainer.py", line 237, in main
    execute_schedule(exp)
  File "C:\Users\Artem\AppData\Local\Programs\Python\Python36\Lib\site-packages\tensor2tensor\bin\t2t_trainer.py", line 217, in execute_schedule
    getattr(exp, FLAGS.schedule)()
  File "C:\Users\Artem\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\contrib\framework\python\framework\experimental.py", line 64, in new_func
    return func(*args, **kwargs)
  File "C:\Users\Artem\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\contrib\learn\python\learn\experiment.py", line 770, in continuous_train_and_eval
    saving_listeners=self._saving_listeners)
  File "C:\Users\Artem\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\contrib\learn\python\learn\experiment.py", line 868, in _call_train
    saving_listeners=saving_listeners)
  File "C:\Users\Artem\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\estimator\estimator.py", line 324, in train
    loss = self._train_model(input_fn, hooks, saving_listeners)
  File "C:\Users\Artem\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\estimator\estimator.py", line 849, in _train_model
    log_step_count_steps=self._config.log_step_count_steps) as mon_sess:
  File "C:\Users\Artem\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\training\monitored_session.py", line 381, in MonitoredTrainingSession
    stop_grace_period_secs=stop_grace_period_secs)
  File "C:\Users\Artem\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\training\monitored_session.py", line 788, in __init__
    stop_grace_period_secs=stop_grace_period_secs)
  File "C:\Users\Artem\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\training\monitored_session.py", line 512, in __init__
    self._sess = _RecoverableSession(self._coordinated_creator)
  File "C:\Users\Artem\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\training\monitored_session.py", line 973, in __init__
    _WrappedSession.__init__(self, self._create_session())
  File "C:\Users\Artem\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\training\monitored_session.py", line 978, in _create_session
    return self._sess_creator.create_session()
  File "C:\Users\Artem\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\training\monitored_session.py", line 669, in create_session
    self.tf_sess = self._session_creator.create_session()
  File "C:\Users\Artem\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\training\monitored_session.py", line 432, in create_session
    self._scaffold.finalize()
  File "C:\Users\Artem\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\training\monitored_session.py", line 212, in finalize
    self._saver.build()
  File "C:\Users\Artem\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\training\saver.py", line 1297, in build
    self._build(self._filename, build_save=True, build_restore=True)
  File "C:\Users\Artem\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\training\saver.py", line 1334, in _build
    build_save=build_save, build_restore=build_restore)
  File "C:\Users\Artem\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\training\saver.py", line 789, in _build_internal
    restore_sequentially, reshape)
  File "C:\Users\Artem\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\training\saver.py", line 501, in _AddShardedRestoreOps
    name="restore_shard"))
  File "C:\Users\Artem\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\training\saver.py", line 448, in _AddRestoreOps
    restore_sequentially)
  File "C:\Users\Artem\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\training\saver.py", line 269, in bulk_restore
    self.restore_op(filename_tensor, saveable, preferred_shard))
  File "C:\Users\Artem\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\training\saver.py", line 296, in restore_op
    [spec.tensor.dtype])[0])
  File "C:\Users\Artem\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\ops\gen_io_ops.py", line 1111, in restore_v2
    shape_and_slices=shape_and_slices, dtypes=dtypes, name=name)
  File "C:\Users\Artem\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\framework\op_def_library.py", line 787, in _apply_op_helper
    op_def=op_def)
  File "C:\Users\Artem\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\framework\ops.py", line 3172, in create_op
    op_def=op_def)
  File "C:\Users\Artem\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\framework\ops.py", line 1617, in __init__
    self._traceback = self._graph._extract_stack()  # pylint: disable=protected-access

NotFoundError (see above for traceback): Key training/transformer/body/decoder/layer_4/ffn/conv2/bias/Adam not found in checkpoint
         [[Node: save/RestoreV2_167 = RestoreV2[dtypes=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_save/Const_0_0, save/RestoreV2_167/tensor_names, save/RestoreV2_167/shape_and_slices)]]```

Closing comment (2018-01-22T14:14:41Z): It seems that you saved the checkpoint with a pre-v1.3.0 version of Tensor2tensor and now you try to load it with v1.3.0 or later version. This is not possible, see the 1.3.0 release notes.
In general, I don't recommend mixing t2t versions - even if the checkpoint format is compatible, there may be other subtle differences, e.g. different optimal hyperparameters (e.g. as reported in #529).
BTW: If you don't use pastebin for long log output, please use Markdown (three backtics) to mark the code, so it renders with an extra scrollbar.
It is useful also to always report your tensor2tensor (and tensorflow) version.