Issue 1:
Description

RMSPropOptimizer does not work with models utilizing embedding layers, which appears to be a consequence of it not implementing _apply_sparse. The other optimizers all implement _apply_sparse, so I assume this is an oversight.

Environment info

Operating System: Ubuntu 14.04 LTS

Installed from source with hash: 03bff43

Steps to reproduce

import numpy as np
import tensorflow as tf

sess = tf.Session()

weights = tf.get_variable('weights', [100, 32], 'float32', trainable=True)
words = tf.constant(np.arange(100, dtype='int32'))
logits = tf.nn.embedding_lookup(weights, words)
labels = tf.constant(np.arange(100, dtype='int64') % 32)
loss = tf.nn.sparse_softmax_cross_entropy_with_logits(logits, labels)
optimizer = tf.train.RMSPropOptimizer(0.001)
step = optimizer.minimize(loss)

Logs

log.txt

Issue 2:
It seems that tf.nce_loss is not compatible with the optimizers RMSProp, ADAGRAD and Momentum. (while SGD, ADAM and FTRL works fine).

When using rmsprop, I get this error:

optimizer = tf.train.RMSPropOptimizer(learning_rate = learning_rate, decay = rms_prop_decay).minimize(nce_loss)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/optimizer.py", line 167, in minimize
name=name)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/optimizer.py", line 256, in apply_gradients
update_ops.append(self._apply_sparse(grad, var))
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/rmsprop.py", line 81, in _apply_sparse
raise NotImplementedError()
NotImplementedError

When using adagrad or momentum, I get this error:

optimizer = tf.train.MomentumOptimizer(learning_rate, learning_momentum).minimize(nce_loss)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/optimizer.py", line 167, in minimize
name=name)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/optimizer.py", line 256, in apply_gradients
update_ops.append(self._apply_sparse(grad, var))
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/momentum.py", line 51, in _apply_sparse
self._momentum_tensor, use_locking=self._use_locking).op
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/gen_training_ops.py", line 237, in sparse_apply_momentum
name=name)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/op_def_library.py", line 633, in apply_op
op_def=op_def)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 1712, in create_op
set_shapes_for_outputs(ret)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 1417, in set_shapes_for_outputs
shapes = shape_func(op)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/training_ops.py", line 111, in _SparseApplyMomentumShape
tensor_shape.TensorShape([None]).concatenate(accum_shape[1:]))
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/tensor_shape.py", line 481, in merge_with
self.assert_same_rank(other)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/tensor_shape.py", line 524, in assert_same_rank
"Shapes %s and %s must have the same rank" % (self, other))
ValueError: Shapes TensorShape([Dimension(128), Dimension(11), Dimension(192)]) and TensorShape([Dimension(None), Dimension(192)]) must have the same rank

Is that expected?

The exact same code works perfectly fine with adam or sgd optimizers, so I do not think I made a mistake when constructing the graph.