  Medical time-series datasets have unique characteristics that make prediction
tasks challenging. Most notably, patient trajectories often contain
longitudinal variations in their input-output relationships, generally referred
to as temporal conditional shift. Designing sequence models capable of adapting
to such time-varying distributions remains a prevailing problem. To address
this we present Model-Attentive Ensemble learning for Sequence modeling (MAES).
MAES is a mixture of time-series experts which leverages an attention-based
gating mechanism to specialize the experts on different sequence dynamics and
adaptively weight their predictions. We demonstrate that MAES significantly
out-performs popular sequence models on datasets subject to temporal shift.
