Question: What exactly is new about this stochastic gradient descent algorithm?
Evidence: DNNs are usually trained with first-order online methods like stochastic gradient descent (SGD). This method yields a crucial advantage in crisis situations, where retraining the whole model each time a small batch of labeled data arrives is impractical. Algorithm "Online Learning" demonstrates how our CNN model can be trained in a purely online setting. We first initialize the model parameters $\theta _0$ (line 1), which can be a trained model from other disaster events or it can be initialized randomly to start from scratch.
As a new batch of labeled tweets $B_t= \lbrace \mathbf {s}_1 \ldots \mathbf {s}_n \rbrace $ arrives, we first compute the log-loss (cross entropy) in Equation 11 for $B_t$ with respect to the current parameters $\theta _t$ (line 2a). Then, we use backpropagation to compute the gradients $f^{\prime }(\theta _{t})$ of the loss with respect to the current parameters (line 2b). Finally, we update the parameters with the learning rate $\eta _t$ and the mean of the gradients (line 2c). We take the mean of the gradients to deal with minibatches of different sizes. Notice that we take only the current minibatch into account to get an updated model. Choosing a proper learning rate $\eta _t$ can be difficult in practice. Several adaptive methods such as ADADELTA BIBREF6 , ADAM BIBREF7 , etc., have been proposed to overcome this issue. In our model, we use ADADELTA.
Algorithm "Online Learning" demonstrates how our CNN model can be trained in a purely online setting. We first initialize the model parameters $\theta _0$ (line 1), which can be a trained model from other disaster events or it can be initialized randomly to start from scratch.

As a new batch of labeled tweets $B_t= \lbrace \mathbf {s}_1 \ldots \mathbf {s}_n \rbrace $ arrives, we first compute the log-loss (cross entropy) in Equation 11 for $B_t$ with respect to the current parameters $\theta _t$ (line 2a). Then, we use backpropagation to compute the gradients $f^{\prime }(\theta _{t})$ of the loss with respect to the current parameters (line 2b). Finally, we update the parameters with the learning rate $\eta _t$ and the mean of the gradients (line 2c). We take the mean of the gradients to deal with minibatches of different sizes. Notice that we take only the current minibatch into account to get an updated model. 
Answer: The answers are shown as follows:
* CNN model can be trained in a purely online setting. We first initialize the model parameters $\theta _0$ (line 1), which can be a trained model from other disaster events or it can be initialized randomly to start from scratch.

As a new batch of labeled tweets $B_t= \lbrace \mathbf {s}_1 \ldots \mathbf {s}_n \rbrace $ arrives, we first compute the log-loss (cross entropy) in Equation 11 for $B_t$ with respect to the current parameters $\theta _t$ (line 2a). Then, we use backpropagation to compute the gradients $f^{\prime }(\theta _{t})$ of the loss with respect to the current parameters (line 2b). Finally, we update the parameters with the learning rate $\eta _t$ and the mean of the gradients (line 2c). We take the mean of the gradients to deal with minibatches of different sizes. Notice that we take only the current minibatch into account to get an updated model.