H2O-Danube-1.8B Technical Report
Authors: name: Philipp Singer, name: Pascal Pfeiffer, name: Yauhen Babakhin, name: Maximilian Jeblick, name: Nischay Dhankhar, name: Gabor Fodor, name: Sri Satish Ambati

We present H2O-Danube-1.8B, a 1.8B language model trained on 1T tokens
following the core principles of LLama 2 and Mistral. We leverage and refine
various techniques for pre-training large language models. Although our model
is trained on significantly fewer total tokens compared to reference models of
similar size, it exhibits highly competitive metrics across a multitude of
benchmarks. We additionally release a chat model trained with supervised
fine-tuning followed by direct preference optimization. We make H2O-Danube-1.8B
openly available under Apache 2.0 license further democratizing LLMs to a wider
audience economically.