We analyse multimodal time-series data corresponding to weight, sleep and steps measurements , derived from a dataset spanning 15000 users, collected across a range of consumer-grade health devices by Nokia Digital Health - Withings . We focus on predicting whether a user will successfully achieve his/her weight objective. For this, we design several deep long short-term memory (LSTM) architectures, including a novel cross-modal LSTM (X-LSTM), and demonstrate their superiority over several baseline approaches. The X-LSTM improves parameter efficiency of the feature extraction by separately processing each modality , while also allowing for information flow between modalities by way of recurrent cross-connections. We derive a general hyperparameter optimisation technique for X-LSTMs, allowing us to significantly improve on the LSTM , as well as on a prior state-of-the-art cross-modal approach, using a comparable number of parameters. Finally, we visualise the X-LSTM classification models, revealing interesting potential implications about latent variables in this task.
We analyse multimodal time-series data corresponding to weight, sleep and steps measurements . We focus on predicting whether a user will successfully achieve his/her weight objective. For this, we design several deep long short-term memory (LSTM) architectures, including a novel cross-modal LSTM (X-LSTM), and demonstrate their superiority over baseline approaches. The X-LSTM improves parameter efficiency by processing each modality separately and allowing for information flow between them by way of recurrent cross-connections. We present a general hyperparameter optimisation technique for X-LSTMs, which allows us to significantly improve on the LSTM and a prior state-of-the-art cross-modal approach, using a comparable number of parameters. Finally, we visualise the model's predictions, revealing implications about latent variables in this task.
D
, derived from a dataset spanning 15000 users, collected across a range of consumer-grade health devices by Nokia Digital Health - Withings
D
several
R
of the feature extraction by separately
by
R
, while also
separately and
R
modalities
them
R
derive
present
R
allowing
which allows
R
, as well as on
and
R
X-LSTM classification models, revealing interesting potential
model's predictions, revealing