H 2 O Open Ecosystem for State-of-the-art Large Language Models (2023)

Abstract: Large Language Models (LLMs) represent a revolution in AI. However, they also pose many significant risks, such as the presence of biased, private, copyrighted or harmful text. For this reason we need open, transparent and safe solutions. We introduce a complete open-source ecosystem for developing and testing LLMs. The goal of this project is to boost open alternatives to closed-source approaches. We release h2oGPT, a family of fine-tuned LLMs from 7 to 70 Billion parameters. We also introduce H2O LLM Studio, a framework and no-code GUI designed for efficient fine-tuning, evaluation, and deployment of LLMs using the most recent state-of-the-art techniques. Our code and models are licensed under fully permissive Apache 2.0 licenses. We believe open-source language models help to boost AI development and make it more accessible and trustworthy. Our demo is available at: https://gpt.h2o.ai/

## introduction since the transformer @xcite was introduced in the natural language processing (nlp) community, the advances in this field have increased exponentially @xcite . starting from popular models such as bert @xcite or generative pre-trained transformers (gpt) @xcite -both introduced in 2018-, researchers have been pushing the limits of scaling and learned representations in language models @xcite @xcite . recent advances in large language models (llms) are all over the news; these models represent a revolution in artificial intelligence (ai) due to their real-world applications through natural language processing (nlp), from internet chatbots to virtual assistants and programmers. however, these also pose significant risks and challenges. the most popular models (e.g., chatgpt (openai, 2023)) are proprietary and not truly open-source, either transparent regarding their training data. this fast advance leads to a wide range of practical challenges that must be addressed in order for these models to be widely utilized and explored. the popularity and demand of llms call for systems to train, fine-tune, evaluate, scale, and deploy the models on a variety of platforms. given the training costs (millions of dollars), practitioners increasingly rely on pre-trained general-purpose llms and fine-tune them for specific downstream tasks and datasets. this requires a wide catalogue of open-source pre-trained llms, and sophisticated procedures and tools for efficient fine-tuning. moreover, considering the massive size of these models (usually from 7 to 100 billion parameters), we also need compression techniques to deploy them successfully on different platforms. we believe open-source language models help to boost ai development and make it more accessible and trustworthy. they lower entry hurdles, allowing people to tailor these models to their needs. this openness increases innovation, transparency, and fairness. as part of this effort, we introduce two open-source libraries: h2ogpt and h2o llm studio, for llms development, including multi llm deployment and evaluationwidely adopted in the nlp community (see fig. 1 ). h2o llm studio ( https://github.com/ h2oai/h2o-llmstudio ) complements the previous library, and allows users to efficiently fine-tune any llm using the most recent state-of-the-art techniques such as lora adapters @xcite , reinforcement learning (rlhf), and 4-bit training. after fine-tuning (or training), the models can be easily exported and deployed at the hugging face hub foot_0 . moreover, the library includes a graphic user interface (gui) specially designed for large language models. ## related work large language models (llms) are designed to process and understand vast amounts of natural language data e.g., internet questions, text in documents, financial data, textbook material, etc. as foundation models @xcite , these are trained from broad data at scale @xcite , and can be adapted (ie. fine-tuned) to a wide range of down-stream tasks @xcite . they are built on the transformer neural network architecture @xcite , which allows them to capture complex language patterns and relationships. derived from the transformer, we find bert-like models @xcite @xcite focused on pretraining with bidirectional encoders. we also find the popular generative pre-trained transformers (gpts) @xcite @xcite , focused on generative pretraining. these serve as the engine of chatgpt. since 2022, we experience a new revolution in nlp with the rise of llms (over billion parameters models). these models usually follow a multistage training strategy, starting with a task-agnostic pre-training on large and diverse datasets. some related llms are llama @xcite , gpt-neox @xcite , bloom @xcite , palm @xcite , opt @xcite , and gpt-4 (openai, 2023). we also explore community models such as falcon @xcite , alpaca @xcite , and openassistant @xcite . ## h2o llm studio an open-source framework for efficient fine-tuning llms without coding, using a graphic user interface (gui) specially designed for large language models foot_2 . this is illustrated in figures 2 and 4 . we use the most popular adapters for fast finetuning such as low-rank adaptation (lora) @xcite and qlora @xcite , as well as 8-bit (up to 4-bit) model training with a low memory footprint, and the corresponding quantization. this allows to fine-tune small llms in regular gpus, even using google colab or kaggle. for @xmath0.g., llama-2 7b) can be fine-tuned in a single nvidia-t4 (16gb). we also integrate reinforcement learning from human feedback (rlhf) @xcite . this feature is inspired in trl foot_3 @xcite , with the proximal policy optimisation (ppo) by ( ziegler the users can track several simultaneous experiments, and easily export the logs and results. moreover, the models can be easily exported to the hugging face hub, to be shared with the community or deploy locally and privately. the framework supports any open-source language model, we here highlight the most popular state-of-the-art large models: gpt-neox @xcite , falcon @xcite , llama and llama 2 @xcite , vicuna @xcite , wizardlm @xcite , h2ogpt @xcite , and mpt (mosaicml, 2023). we summarize these models in ## multi llm deployment and evaluation any model produced from llm studio can be easily integrated into huggingface's space & models. we refer to our own space for more information and access to our models 5 . in fig. 3 (top) we show a snapshot of our demo h2ogpt https://gpt.h2o.ai/ . we deploy multiple state-of-the-art llm models including falcon (7/40b), llama 2 (7/13/70b), and gpt-3.5. this allows us to compare different models and setups. the user's prompt is evaluated by the different llms concurrently. we can see the answer generation progress for each model, at the same time. using this software we can identify clear differences between llms easily, for example fast/low inference, hallucinations, common response patterns, bias, memorized data etc. also, we can analyze the effect of prompt engineering on the different models and expose vulnerabilities. the users can deploy the models on a wide variety of inference servers (hf tgi server, vllm, gradio, openai), and evaluate performance using reward models. document analysis h2ogpt also allows to query and summarize documents in many formats (e.g., pdfs, word, code, text, markdown, etc). ## https://huggingface.co/h2oai we implement an efficient use of context using instruct-tuned llms (no need for langchain). note that this ecosystem can be reproduced locally, to analyze the models in a private and safe manner.we also provide a openai-compliant python client api for client-server control. guides & material we provide a short video tutorial (2 mins), and a complete video overview of the ecosystem (16 min, 340k views) on youtube. also a step-by-step tutorial make your own gpt with h2ogpt & h2o llm studio (1hr). we also host all of our models in hf: https: //huggingface.co/h2oai . we refer the reader to our githubs for more demos, and documentation. ## future work our open-source llm ecosystem is in constant development, h2ogpt and llm studio are updated based on the most recent research advances and demands. we plan to integrate new model quantization techniques, distillation and long-context training (context length over 100k tokens). we also plan to support more multi-lingual models, and multi-modal models. ## limitations datasets fine-tuning requires data text pairs of instruction and expected result/answer. biases and offensiveness llms are trained on a diverse range of unfiltered internet text data, which may contain biased, racist, offensive, or otherwise inappropriate content. therefore, the generated content by these models may sometimes exhibit biases or produce content that is offensive or inappropriate. we do not endorse, support, or promote any such content or viewpoints. usage the large language model is an ai-based tool and not a human. it may produce incorrect, offensive, nonsensical, or irrelevant responses. it is the user's responsibility to critically evaluate the generated content and use it at their discretion. carbon footprint training llms is expensive and their use is associated to tons of co 2 emissions @xcite . hallucinations llms are probabilistic, therefore, certain "random" behaviour is natural and expected, especially on complex prompts (e.g., logical paradoxes, reasoning problems, etc) and "unknown content" not present in the training corpus. multiple llm evaluation. this visualization-evaluation allows the user to detect clear differences between the models for example, inference speed and clear hallucinations. and evaluation allows the user to detect clear differences between the models e.g. faster or slower inference, clear hallucinations, common memorized patterns. demo available at https://gpt.h2o.ai/ completely free.