  The accurate prediction of biological features from genomic data is paramount
for precision medicine, sustainable agriculture and climate change research.
For decades, neural network models have been widely popular in fields like
computer vision, astrophysics and targeted marketing given their prediction
accuracy and their robust performance under big data settings. Yet neural
network models have not made a successful transition into the medical and
biological world due to the ubiquitous characteristics of biological data such
as modest sample sizes, sparsity, and extreme heterogeneity. Here, we
investigate the robustness, generalization potential and prediction accuracy of
widely used convolutional neural network and natural language processing models
with a variety of heterogeneous genomic datasets. While the perspective of a
robust out-of-the-box neural network model is out of reach, we identify certain
model characteristics that translate well across datasets and could serve as a
baseline model for translational researchers.
