Repository: GPflow/GPflow

Issue:
minibatches with _optimize_tf
Apologies if I'm missing something with usage here, but it seems that minibatches are not actually changing when using a tensorflow optimizer. Is the optimize function intended to be called once per batch? This requires a recompile each time so I suspect not.

When I moved the lines
`feed_dict = {}
self.update_feed_dict(self._feed_dict_keys, feed_dict)
`
(225 of model) into the loop it works fine, however, but otherwise only the first batch is used for all the iterations. 


Discussion:
@hughsalimbeni I will have a look into this.
@hughsalimbeni: How could you tell that the same minibatch was used? I'm pretty sure I have tested this in the past and I found that repeated calls to `model._objective()` yielded noisy results, therefore I concluded that different minibatches were passed. Similar for optimisation.
I noticed this when trying the following sanity check:

```python
from tensorflow.examples.tutorials.mnist import input_data
mnist = input_data.read_data_sets("MNIST_data/", one_hot=True)

X, Y = mnist.train.next_batch(50000)
Xs, Ys = mnist.validation.next_batch(5000)

rand = lambda s: np.reshape(np.random.randn(np.prod(s)), s)

class Linear_model(GPflow.model.Model):
    def __init__(self, X, Y, b):
        self.X = GPflow.svgp.MinibatchData(X, b, np.random.RandomState(0))
        self.Y = GPflow.svgp.MinibatchData(Y, b, np.random.RandomState(0))
        self.W = GPflow.param.Param(rand((X.shape[1], Y.shape[1])))
        self.b = GPflow.param.Param(np.zeros(Y.shape[1]))
        GPflow.model.Model.__init__(self)
            
    def build_likelihood(self):
        Y_preds = self.build_predict(self.X)
        return tf.reduce_mean(tf.reduce_sum(self.Y * tf.log(Y_preds), 1))
        
    def build_predict(self, Xnew):
        return tf.nn.softmax(tf.matmul(Xnew, self.W) + self.b)
        
    @GPflow.param.AutoFlow((tf.float64, [None, None]), (tf.float64, [None, None]))
    def accuracy(self, Xnew, Ynew):
        Y_preds = tf.arg_max(self.build_predict(Xnew), 1)
        Y_true = tf.arg_max(Ynew, 1)
        correct = tf.cast(tf.equal(Y_preds, Y_true), tf.float64)
        return tf.reduce_mean(correct)
    
batch_size = 100
m = Linear_model(X, Y, b=batch_size)
m.optimize(tf.train.AdamOptimizer(), maxiter=5000)
print m.accuracy(Xs, Ys) # should get around 88%, but gets more like 58%

m_100 = Linear_model(X[:batch_size, :], Y[:batch_size, :], b=100)
m_100.optimize(tf.train.AdamOptimizer(), maxiter=5000)
print m_100.accuracy(Xs, Ys) # similar result, but with only 100 points
```

but when I move the dict update inside the loop I get the expected 88% with the full data and 58% with only the first 100 points. 
I think @hughsalimbeni is right. It looks like I might have broken this :(

Could we fix, and get the above sanity check in as a regression test?
@jameshensman I will fix this. 

Resolving patch:
diff --git a/GPflow/model.py b/GPflow/model.py
--- a/GPflow/model.py
+++ b/GPflow/model.py
@@ -223,11 +223,11 @@ def _optimize_tf(self, method, callback, maxiter):
         """
         opt_step = self._compile(optimizer=method)
         feed_dict = {}
-        self.update_feed_dict(self._feed_dict_keys, feed_dict)
 
         try:
             iteration = 0
             while iteration < maxiter:
+                self.update_feed_dict(self._feed_dict_keys, feed_dict)
                 self._session.run(opt_step, feed_dict=feed_dict)
                 if callback is not None:
                     callback(self._session.run(self._free_vars))


Test patch: