Q:

TensorFlow: initializing state for RNN between inferences in Android

We have a working TensorFlow network (graphdef) running on Android and I'm noting that the inference results over time tend to be correlated.   That is if label A is returned there will tend to be a stream of A's that follow in time, even when the input data switches to data that should generate B labels.   Eventually, the results will switch to B but there seems to be a lag and suggests the RNN is stateful between inference calls.   Our network is using RNN/LSTMs. 
cellLSTM    = tf.nn.rnn_cell. BasicLSTMCell(nHidden)
cellsLSTM   = tf.nn.rnn_cell. MultiRNNCell([cellLSTM] * 2)
RNNout, RNNstates = tf.nn.rnn(cellsLSTM, Xin)

I'm wondering if I need to reinitialize the RNN state between inference calls.   I will note that no such method is available in the TensorFlowInferenceInterface.java interface.    I imagine that it would be possible to insert an RNN cell initialization node into the graph that could be activated with a node value (using FillNodeInt or similar). 
So my question is: what is best practise with RNN/LSTMs in Tensorflow.  Does one need to clear state between inferences?   If so, how does one do it? 

A:

While I can't comment on general practice for RNN state initialization, here is how we managed to force the initial state definition.   The problem was that while the batch size was indeed a constant parameter for the training set, it was not for the Test set.   The test set is always 20% of the data corpus and so its size is different with each change to the corpus. 
The solution was to create a new variable for the batchsize:
batch_size_T  = tf.shape(Xin)[0]

where Xin is the input Tensor of size [b x m x n] where b is the batch size and m x n is the size of the training frame.   Xin is fed in from the feed_dict. 
The initial state can then be defined as:
initial_state = lstm_cells.zero_state(batch_size_T, tf.float32) 

Finally, the RNN is defined in terms of the new Dynamic RNN:
outputs, state = tf.nn.dynamic_rnn(cell=lstm_cells, inputs=Xin, dtype=tf.float32, initial_state=initial_state)

