  The task of sequential sentence classification enables the semantic
structuring of research papers. This can enhance academic search engines to
support researchers in finding and exploring research literature more
effectively. However, previous work has not investigated the potential of
transfer learning with datasets from different scientific domains for this task
yet. We propose a uniform deep learning architecture and multi-task learning to
improve sequential sentence classification in scientific texts across domains
by exploiting training data from multiple domains. Our contributions can be
summarised as follows: (1) We tailor two common transfer learning methods,
sequential transfer learning and multi-task learning, and evaluate their
performance for sequential sentence classification; (2) The presented
multi-task model is able to recognise semantically related classes from
different datasets and thus supports manual comparison and assessment of
different annotation schemes; (3) The unified approach is capable of handling
datasets that contain either only abstracts or full papers without further
feature engineering. We demonstrate that models, which are trained on datasets
from different scientific domains, benefit from one another when using the
proposed multi-task learning architecture. Our approach outperforms the state
of the art on three benchmark datasets.
