Issue 1:
Please make sure that this is a bug. As per our GitHub Policy, we only address code/doc bugs, performance issues, feature requests and build/installation issues on GitHub. tag:bug_template

System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow): yes

OS Platform and Distribution (e.g., Linux Ubuntu 16.04): mac

Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device: none

TensorFlow installed from (source or binary): binary

TensorFlow version (use command below): 2.0.alpha0

Python version:

Bazel version (if compiling from source): none

GCC/Compiler version (if compiling from source): none

CUDA/cuDNN version: none

GPU model and memory: none

You can collect some of this information using our environment capture

script

You can also obtain the TensorFlow version with: 1. TF 1.0: python -c "import tensorflow as tf; print(tf.GIT_VERSION, tf.VERSION)" 2. TF 2.0: python -c "import tensorflow as tf; print(tf.version.GIT_VERSION, tf.version.VERSION)"

Describe the current behavior

It works fine when run this in one console

import tensorflow as tf
import numpy as np

m = tf.Module()
m.v = tf.Variable(np.random.normal(size=(200, 200)).astype(np.float64))
m.f = tf.function(lambda x: x + m.v[0, 0])

tf.saved_model.save(m, "save/model", signatures={
"predict": m.f.get_concrete_function(tf.constant(3., dtype=tf.float64))
})

and then the other

import tensorflow as tf
m = tf.saved_model.load("save/model")

If you change 200 to 20000, you will see errors on load

import tensorflow as tf
import numpy as np

m = tf.Module()
m.v = tf.Variable(np.random.normal(size=(20000, 20000)).astype(np.float64))
m.f = tf.function(lambda x: x + m.v[0, 0])

tf.saved_model.save(m, "save/model", signatures={
"predict": m.f.get_concrete_function(tf.constant(3., dtype=tf.float64))
})

import tensorflow as tf
tf.saved_model.load("save/model")
# tensorflow.python.framework.errors_impl.InvalidArgumentError: save/model/variables/variables.data-00000-of-00001; Invalid argument [Op:RestoreV2]

Describe the expected behavior

Should be able to load regardless of the size of variable

Code to reproduce the issue

Provide a reproducible test case that is the bare minimum necessary to generate the problem.

see above

Other info / logs

Include any logs or source code that would be helpful to diagnose the problem. If including tracebacks, please include the full traceback. Large logs and files should be attached.

Issue 2:
I get an InvalidArgumentError with no further information when I try to save and then restore parts of my model later to continue training it (due to needing my laptop for class).

Initialization:

saver = tf.train.Saver({"embeddings": embeddings, "weights": nce_weights, "biases": nce_biases})

Save:

saver.save(sess, model_checkpoint_path)

Load:

saver.restore(sess, model_checkpoint_path)

2018-04-21 22:45:00.143245: W tensorflow/core/framework/op_kernel.cc:1273] OP_REQUIRES failed at save_restore_v2_ops.cc:184 : Invalid argument: /Users/nroth/Documents/****/trained_model/****embeddings.ckpt.data-00000-of-00001; Invalid argument
Traceback (most recent call last):
File "/Users/nroth/tf_python/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1327, in _do_call
return fn(*args)
File "/Users/nroth/tf_python/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1312, in _run_fn
options, feed_dict, fetch_list, target_list, run_metadata)
File "/Users/nroth/tf_python/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1420, in _call_tf_sessionrun
status, run_metadata)
File "/Users/nroth/tf_python/lib/python3.6/site-packages/tensorflow/python/framework/errors_impl.py", line 516, in __exit__
c_api.TF_GetCode(self.status.status))
tensorflow.python.framework.errors_impl.InvalidArgumentError: /Users/nroth/Documents/****/trained_model/****embeddings.ckpt.data-00000-of-00001; Invalid argument
[[Node: save/RestoreV2 = RestoreV2[dtypes=[DT_FLOAT, DT_FLOAT, DT_FLOAT], _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_save/Const_0_0, save/RestoreV2/tensor_names, save/RestoreV2/shape_and_slices)]]

... <contains sensitive info> ...

InvalidArgumentError (see above for traceback): /Users/nroth/Documents/****/trained_model/****embeddings.ckpt.data-00000-of-00001; Invalid argument
[[Node: save/RestoreV2 = RestoreV2[dtypes=[DT_FLOAT, DT_FLOAT, DT_FLOAT], _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_save/Const_0_0, save/RestoreV2/tensor_names, save/RestoreV2/shape_and_slices)]]

Clarification requested by tensorflowbutler

Have I written custom code:

Yes, I modified this code (https://github.com/PacktPublishing/TensorFlow-Machine-Learning-Cookbook/blob/master/Chapter%2007/doc2vec.py) to work with TensorFlow 1.7 and to use the same embeddings variable for documents as for words with average instead of concatenation. I also updated the saved variables to include nce_weights and nce_biases so that training may be resumed.

OS Platform and Distribution

MacOS 10.13.4 (17E199)

TensorFlow installed from

pip on VirtualEnv, according to instructions (https://www.tensorflow.org/install/install_mac)

TensorFlow version

1.7

Bazel version

NA

CUDA/cuDNN version

NA

GPU model and memory

NA

Exact command to reproduce

saver = tf.train.Saver({"embeddings": embeddings, "weights": nce_weights, "biases": nce_biases})

saver.restore(sess, "../trained_model/saved_stuff")