Upload 301 files

.dockerignore

@@ -0,0 +1,4 @@
+.git
+.github
+data
+output

.flake8

@@ -0,0 +1,15 @@
+[flake8]
+exclude=.cache, .local, server.wave, output, data, reports
+max-line-length = 88
+# E203, W503 - black-compatible config
+extend-ignore = E203, W503
+per-file-ignores =
+    */__init__.py: F401
+    train.py: E402
+    prompt.py: E402
+    train_wave.py: E402, I001, I003
+    app.py: E402
+    tests/src/datasets/test_text_dpo_modeling_ds.py: E501
+    tests/src/models/test_dpo_modeling_model.py: E501
+
+inline-quotes = "

.github/ISSUE_TEMPLATE/bug-report.md

@@ -0,0 +1,18 @@
+---
+name: "\U0001F41B Bug Report"
+about: Create a bug report
+title: "[BUG]"
+labels: type/bug
+assignees: ''
+---
+
+### 🐛 Bug
+
+<!-- A clear and concise description of what the bug is. Please include the full error stack trace, if applicable.-->
+
+### To Reproduce
+
+<!-- Steps to reproduce the behavior. Please include the model configuration yaml file if the error is related to model training. -->
+
+### LLM Studio version
+<!-- Please provide the commit hash of the version you are using (running `git rev-parse HEAD`) in your report. If you are pasting the UI error message, the commit hash will also be included in the error message. -->

.github/ISSUE_TEMPLATE/code-improvement.md

@@ -0,0 +1,15 @@
+---
+name: "\U0001F527 Code improvement"
+about: Suggest a code improvement, e.g. refactoring, deprecation, etc.
+title: "[CODE IMPROVEMENT]"
+labels: area/core
+assignees: ''
+---
+
+### 🔧 Proposed code refactoring
+
+<!-- A clear and concise description of the code improvement -->
+
+### Motivation
+
+<!-- Please outline the motivation for the proposal. If this is related to another GitHub issue, please link here too -->

.github/ISSUE_TEMPLATE/doc-request.md

@@ -0,0 +1,15 @@
+---
+name: "\U0001F41B Documentation request"
+about: Create a doc request
+title: "[DOC]"
+labels: type/doc
+assignees: ''
+---
+
+### 📃 Documentation
+
+<!-- A clear and concise description of the documentation/tutorial request. -->
+
+### Motivation
+
+<!-- Please mention the type of documentation request (new tutorial, FAQ, feature documentation, or doc bug) and link any related Github issues and pull requests. -->

.github/ISSUE_TEMPLATE/feature-request.md

@@ -0,0 +1,15 @@
+---
+name: "\U0001F680 Feature Request"
+about: Submit a proposal/request for a new H2O LLM Studio feature
+title: "[FEATURE]"
+labels: type/feature
+assignees: ''
+---
+
+### 🚀 Feature
+
+<!-- A clear and concise description of the feature proposal -->
+
+### Motivation
+
+<!-- Please outline the motivation for the proposal. Is your feature request related to a problem? e.g., I'm always frustrated when [...]. If this is related to another GitHub issue, please link here too -->

.github/workflows/build-and-push-nightly.yml

@@ -0,0 +1,39 @@
+name: Build and Push to Vorvan - Nightly
+
+on:
+  schedule:
+    - cron: "0 4 * * *"
+  workflow_dispatch:
+
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@v3
+      - id: 'auth'
+        uses: google-github-actions/auth@v1
+        with:
+          credentials_json: '${{ secrets.GCP_CRED_JSON }}'
+      - name: Configure Google Cloud SDK
+        uses: google-github-actions/setup-gcloud@v1
+      - name: Configure Docker Client
+        run: |-
+          gcloud auth configure-docker --quiet #authenticate to gcr
+      - name: Clean Docker images
+        run: |-
+          echo "Available storage before cleaning:"
+          df -h
+          docker system prune --all --force
+          echo "Available storage:"
+          df -h
+          echo "Removing dotnet"
+          sudo rm -rf /usr/share/dotnet
+          echo "Available storage:"
+          df -h
+      - name: Docker Build Image
+        run: |-
+          docker build -t gcr.io/$GCLOUD_PROJECT/h2oai/h2o-llmstudio:nightly .
+      - name: Push to Vorvan
+        run: |-
+          docker push gcr.io/$GCLOUD_PROJECT/h2oai/h2o-llmstudio:nightly

.github/workflows/build-and-push-release.yml

@@ -0,0 +1,40 @@
+name: Build and Push to Vorvan - Release
+
+on:
+  push:
+    tags:
+      - '**'
+  workflow_dispatch:
+
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@v3
+      - id: 'auth'
+        uses: google-github-actions/auth@v1
+        with:
+          credentials_json: '${{ secrets.GCP_CRED_JSON }}'
+      - name: Configure Google Cloud SDK
+        uses: google-github-actions/setup-gcloud@v1
+      - name: Configure Docker Client
+        run: |-
+          gcloud auth configure-docker --quiet #authenticate to gcr
+      - name: Clean Docker images
+        run: |-
+          echo "Available storage before cleaning:"
+          df -h
+          docker system prune --all --force
+          echo "Available storage:"
+          df -h
+          echo "Removing dotnet"
+          sudo rm -rf /usr/share/dotnet
+          echo "Available storage:"
+          df -h
+      - name: Docker Build Image
+        run: |-
+          docker build -t gcr.io/$GCLOUD_PROJECT/h2oai/h2o-llmstudio:${{ github.ref_name }} .
+      - name: Push to Vorvan
+        run: |-
+          docker push gcr.io/$GCLOUD_PROJECT/h2oai/h2o-llmstudio:${{ github.ref_name }}

.github/workflows/deploy-to-github-pages.yml

@@ -0,0 +1,32 @@
+name: Deploy documentation to GitHub pages
+
+on:
+  workflow_dispatch:
+
+jobs:
+  deploy:
+    name: Deploy to GitHub Pages
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v3
+      - uses: actions/setup-node@v3
+        with:
+          always-auth: true
+          registry-url: https://npm.pkg.github.com/
+          node-version: 18
+          cache: npm
+          cache-dependency-path: documentation/package-lock.json
+
+      - name: Install dependencies
+        run: cd documentation && npm install --frozen-lockfile
+        env:
+          NODE_AUTH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+      - name: Build docs
+        run: cd documentation && npm run build
+      - name: Deploy to GitHub Pages
+        uses: peaceiris/actions-gh-pages@v3
+        with:
+          github_token: ${{ secrets.GITHUB_TOKEN }}
+          publish_dir: ./documentation/tmp/build
+          user_name: sherenem ##swap username out with the username of someone with admin access to the repo 
+          user_email: sherene.mahanama@h2o.ai ##swap email out with the email of someone with admin access to the repo 

.github/workflows/requirements.yml

@@ -0,0 +1,25 @@
+name: Requirements
+
+on:
+  pull_request:
+
+jobs:
+  requirements:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v3
+      - name: Setup Python
+        uses: actions/setup-python@v4
+        with:
+          python-version: 3.10.11
+      - run: make setup
+
+      - name: Generate requirements.txt
+        run: make export-requirements
+
+      - name: Commit changes
+        uses: stefanzweifel/git-auto-commit-action@v4
+        with:
+          commit_message: "Update requirements.txt"
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

.github/workflows/style.yml

@@ -0,0 +1,18 @@
+name: Style
+
+on:
+  push:
+    branches: [ main ]
+  pull_request:
+
+jobs:
+  style:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v3
+      - name: Setup Python
+        uses: actions/setup-python@v4
+        with:
+          python-version: 3.10.11
+      - run: make setup-dev
+      - run: make style

.github/workflows/test.yml

@@ -0,0 +1,19 @@
+name: Test
+
+on:
+  push:
+    branches: [ main ]
+  pull_request:
+
+jobs:
+  test:
+    runs-on: self-hosted
+    steps:
+      - uses: actions/checkout@v3
+      - name: Setup Python
+        uses: actions/setup-python@v4
+        with:
+          python-version: 3.10.11
+      - run: nvidia-smi
+      - run: make setup-dev
+      - run: make test

.gitignore

@@ -0,0 +1,163 @@
+# Folder
+input/
+notebooks/
+demo_data/
+output/
+output_old/
+tmp/
+data/
+examples/data_oasst2
+examples/output_oasst2
+data_old/
+tests_tmp/
+subs/
+/datasets/
+.idea/
+.local/
+
+output
+
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+*$py.class
+
+.neptune/*
+.vscode/*
+
+# C extensions
+*.so
+*.c
+
+# Distribution / packaging
+.Python
+build/
+develop-eggs/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+pip-wheel-metadata/
+share/python-wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+MANIFEST
+
+# PyInstaller
+#  Usually these files are written by a python script from a template
+#  before PyInstaller builds the exe, so as to inject date/other infos into it.
+*.manifest
+*.spec
+
+# Installer logs
+pip-log.txt
+pip-delete-this-directory.txt
+
+# Unit test / coverage reports
+htmlcov/
+.tox/
+.nox/
+.coverage
+.coverage.*
+.cache
+nosetests.xml
+coverage.xml
+*.cover
+*.py,cover
+.hypothesis/
+.pytest_cache/
+reports/
+
+# Translations
+*.mo
+*.pot
+
+# Django stuff:
+*.log
+local_settings.py
+db.sqlite3
+db.sqlite3-journal
+
+# Flask stuff:
+instance/
+.webassets-cache
+
+# Scrapy stuff:
+.scrapy
+
+# Documentation
+node_modules
+tmp
+.docusaurus 
+.cach-loader
+
+# PyBuilder
+target/
+
+# Jupyter Notebook
+.ipynb_checkpoints
+*.ipynb
+
+# IPython
+profile_default/
+ipython_config.py
+
+# pyenv
+.python-version
+
+# pipenv
+#   According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
+#   However, in case of collaboration, if having platform-specific dependencies or dependencies
+#   having no cross-platform support, pipenv may install dependencies that don't work, or not
+#   install all needed dependencies.
+#Pipfile.lock
+
+# PEP 582; used by e.g. github.com/David-OConnor/pyflow
+__pypackages__/
+
+# Celery stuff
+celerybeat-schedule
+celerybeat.pid
+
+# SageMath parsed files
+*.sage.py
+
+# Environments
+.env
+.venv
+env/
+venv/
+ENV/
+env.bak/
+venv.bak/
+
+# Spyder project settings
+.spyderproject
+.spyproject
+
+# Rope project settings
+.ropeproject
+
+
+
+# mypy
+.mypy_cache/
+.dmypy.json
+dmypy.json
+
+# Pyre type checker
+.pyre/
+h2o_wave.state
+.DS_Store
+
+# IDE
+.vscode
+
+# playwright
+test-results/

CODE_OF_CONDUCT.md

@@ -0,0 +1,133 @@
+
+# Contributor Covenant Code of Conduct
+
+## Our Pledge
+
+We as members, contributors, and leaders pledge to make participation in our
+community a harassment-free experience for everyone, regardless of age, body
+size, visible or invisible disability, ethnicity, sex characteristics, gender
+identity and expression, level of experience, education, socio-economic status,
+nationality, personal appearance, race, caste, color, religion, or sexual identity
+and orientation.
+
+We pledge to act and interact in ways that contribute to an open, welcoming,
+diverse, inclusive, and healthy community.
+
+## Our Standards
+
+Examples of behavior that contributes to a positive environment for our
+community include:
+
+* Demonstrating empathy and kindness toward other people
+* Being respectful of differing opinions, viewpoints, and experiences
+* Giving and gracefully accepting constructive feedback
+* Accepting responsibility and apologizing to those affected by our mistakes,
+  and learning from the experience
+* Focusing on what is best not just for us as individuals, but for the
+  overall community
+
+Examples of unacceptable behavior include:
+
+* The use of sexualized language or imagery, and sexual attention or
+  advances of any kind
+* Trolling, insulting or derogatory comments, and personal or political attacks
+* Public or private harassment
+* Publishing others' private information, such as a physical or email
+  address, without their explicit permission
+* Other conduct which could reasonably be considered inappropriate in a
+  professional setting
+
+## Enforcement Responsibilities
+
+Community leaders are responsible for clarifying and enforcing our standards of
+acceptable behavior and will take appropriate and fair corrective action in
+response to any behavior that they deem inappropriate, threatening, offensive,
+or harmful.
+
+Community leaders have the right and responsibility to remove, edit, or reject
+comments, commits, code, wiki edits, issues, and other contributions that are
+not aligned to this Code of Conduct, and will communicate reasons for moderation
+decisions when appropriate.
+
+## Scope
+
+This Code of Conduct applies within all community spaces, and also applies when
+an individual is officially representing the community in public spaces.
+Examples of representing our community include using an official e-mail address,
+posting via an official social media account, or acting as an appointed
+representative at an online or offline event.
+
+## Enforcement
+
+Instances of abusive, harassing, or otherwise unacceptable behavior may be
+reported to the community leaders responsible for enforcement at
+this repository.
+All complaints will be reviewed and investigated promptly and fairly.
+
+All community leaders are obligated to respect the privacy and security of the
+reporter of any incident.
+
+## Enforcement Guidelines
+
+Community leaders will follow these Community Impact Guidelines in determining
+the consequences for any action they deem in violation of this Code of Conduct:
+
+### 1. Correction
+
+**Community Impact**: Use of inappropriate language or other behavior deemed
+unprofessional or unwelcome in the community.
+
+**Consequence**: A private, written warning from community leaders, providing
+clarity around the nature of the violation and an explanation of why the
+behavior was inappropriate. A public apology may be requested.
+
+### 2. Warning
+
+**Community Impact**: A violation through a single incident or series
+of actions.
+
+**Consequence**: A warning with consequences for continued behavior. No
+interaction with the people involved, including unsolicited interaction with
+those enforcing the Code of Conduct, for a specified period of time. This
+includes avoiding interactions in community spaces as well as external channels
+like social media. Violating these terms may lead to a temporary or
+permanent ban.
+
+### 3. Temporary Ban
+
+**Community Impact**: A serious violation of community standards, including
+sustained inappropriate behavior.
+
+**Consequence**: A temporary ban from any sort of interaction or public
+communication with the community for a specified period of time. No public or
+private interaction with the people involved, including unsolicited interaction
+with those enforcing the Code of Conduct, is allowed during this period.
+Violating these terms may lead to a permanent ban.
+
+### 4. Permanent Ban
+
+**Community Impact**: Demonstrating a pattern of violation of community
+standards, including sustained inappropriate behavior,  harassment of an
+individual, or aggression toward or disparagement of classes of individuals.
+
+**Consequence**: A permanent ban from any sort of public interaction within
+the community.
+
+## Attribution
+
+This Code of Conduct is adapted from the [Contributor Covenant][homepage],
+version 2.0, available at
+[https://www.contributor-covenant.org/version/2/0/code_of_conduct.html][v2.0].
+
+Community Impact Guidelines were inspired by
+[Mozilla's code of conduct enforcement ladder][Mozilla CoC].
+
+For answers to common questions about this code of conduct, see the FAQ at
+[https://www.contributor-covenant.org/faq][FAQ]. Translations are available
+at [https://www.contributor-covenant.org/translations][translations].
+
+[homepage]: https://www.contributor-covenant.org
+[v2.0]: https://www.contributor-covenant.org/version/2/0/code_of_conduct.html
+[Mozilla CoC]: https://github.com/mozilla/diversity
+[FAQ]: https://www.contributor-covenant.org/faq
+[translations]: https://www.contributor-covenant.org/translations

CONTRIBUTING.md

@@ -0,0 +1,67 @@
+# Contributing to H2O LLM STUDIO
+
+H2O LLM Studio is an open source project released under the Apache Software Licence v2. Open Source projects live by
+their user and developer communities. We welcome and encourage your contributions of any kind!
+
+## Bug Reports and Feature Requests
+
+Found a bug or have an idea for a new feature? Your feedback is invaluable! To ensure a smooth and collaborative
+process, please follow these steps:
+
+1. Provide the full error message and stack trace, if applicable.
+2. Attach the model configuration yaml file if the error is related to model training.
+3. Specify the commit hash of the version you are using (running `git rev-parse HEAD`) in your report. If you are
+   pasting the UI error message, the commit hash will also be included in the error message.
+4. If the error is reproducible, kindly include the steps to reproduce it.
+5. If possible, attempt to reproduce the error using the default dataset.
+6. Please mention any other details that might be useful, e.g. if you are using LLM Studio in a Docker container, etc.
+
+## Pull Requests
+
+You can contribute to the project by fixing bugs, adding new features, refactoring code, or enhancing documentation.
+To ensure a smooth and collaborative process for everyone, please follow these guidelines:
+
+1. Check if the issue you plan to address is already [reported](https://github.com/h2oai/h2o-llmstudio/issues). If not,
+   please open a new issue
+   to discuss your proposed changes.
+2. Avoid duplicating work by commenting on the issue you're working on and feel free to seek assistance or ask
+   questions; our team is happy to help.
+3. Fork the repository and create a new branch from `main`. To develop, please follow the setup instructions below.
+4. Implement your changes and commit them to your branch.
+5. When you feel ready, open a pull request with your changes. You can also open the PR as a draft to receive early
+   feedback. To facilitate the review process, we have provided a PR checklist below.
+6. Our team will review your pull request and provide feedback. Once everything looks good, we will proceed to merge
+   your contribution.
+
+## Setting up your development environment
+
+Follow the instructions in [README](https://github.com/h2oai/h2o-llmstudio/blob/main/README.md) to set up your
+environment. Run `make setup-dev` instead of `make setup` to install the development dependencies.
+
+## Running linters and tests
+
+Before submitting your pull request, ensure that your code passes the linters and tests.
+To format your code, run `make format`. You can check for any style issues by running `make style`. To run the tests,
+run `make test`.
+
+## PR checklist
+
+Please make sure your pull request fulfills the following checklist:
+
+☐ The PR title should provide a clear summary of your contribution.  
+☐ Link the related issue (e.g., closes #123) in your PR description.  
+☐ If your contribution is still a work in progress, change the PR to draft mode.  
+☐ Ensure that the existing tests pass by running `make test`.  
+☐ Make sure `make style` passes to maintain consistent code style.  
+
+## Installing custom packages
+
+If you need to install additional Python packages into the environment, you can do so using pip after activating your virtual environment via ```make shell```. For example, to install flash-attention, you would use the following commands:
+
+```bash
+make shell
+pip install flash-attn --no-build-isolation
+pip install git+https://github.com/HazyResearch/flash-attention.git#subdirectory=csrc/rotary
+```
+
+For a PR, update the Pipfile and the Pipfile.lock via ```pipenv install package_name```.

Dockerfile

@@ -0,0 +1,48 @@
+FROM nvidia/cuda:11.8.0-devel-ubuntu20.04
+
+ARG DEBIAN_FRONTEND=noninteractive
+
+RUN apt-get update && apt-get upgrade -y
+
+RUN apt-get update && apt-get install -y \
+    git \
+    curl \
+    software-properties-common \
+    && add-apt-repository ppa:deadsnakes/ppa \
+    && apt install -y python3.10 \
+    && apt install -y python3.10-distutils \
+    && rm -rf /var/lib/apt/lists/*
+
+# Pick an unusual UID for the llmstudio user.
+# In particular, don't pick 1000, which is the default ubuntu user number.
+# Force ourselves to test with UID mismatches in the common case.
+RUN adduser --uid 1999 llmstudio
+USER llmstudio
+
+# Python virtualenv is installed in /home/llmstudio/.local
+# Application code and data lives in /workspace
+#
+# Make all of the files in the llmstudio directory writable so that the
+# application can install other (non-persisted) new packages and other things
+# if it wants to.  This is really not advisable, though, since it's lost when
+# the container exits.
+WORKDIR /workspace
+RUN \
+    curl -sS https://bootstrap.pypa.io/get-pip.py | python3.10 && \
+    chmod -R a+w /home/llmstudio
+COPY Makefile .
+COPY Pipfile .
+COPY Pipfile.lock .
+RUN \
+    make setup && \
+    mkdir -p /home/llmstudio/mount && \
+    chmod -R a+w /home/llmstudio
+COPY . .
+
+ENV HOME=/home/llmstudio
+ENV H2O_WAVE_APP_ADDRESS=http://127.0.0.1:8756
+ENV H2O_WAVE_MAX_REQUEST_SIZE=25MB
+ENV H2O_WAVE_NO_LOG=true
+ENV H2O_WAVE_PRIVATE_DIR="/download/@/workspace/output/download"
+EXPOSE 10101
+ENTRYPOINT [ "python3.10", "-m", "pipenv", "run", "wave", "run", "--no-reload", "app" ]

LICENSE

@@ -0,0 +1,191 @@
+
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   Copyright 2023 H2O.ai, Inc.
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this repository except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

Makefile

@@ -0,0 +1,188 @@
+PYTHON_VERSION ?= 3.10
+PYTHON ?= python$(PYTHON_VERSION)
+PIP ?= $(PYTHON) -m pip
+PIPENV ?= $(PYTHON) -m pipenv
+PIPENV_PYTHON = $(PIPENV) run python
+PIPENV_PIP = $(PIPENV_PYTHON) -m pip
+PWD = $(shell pwd)
+DOCKER_IMAGE ?= gcr.io/vorvan/h2oai/h2o-llmstudio:nightly
+
+ifeq ($(origin H2O_LLM_STUDIO_WORKDIR), environment)
+    WORKDIR := $(H2O_LLM_STUDIO_WORKDIR)
+else
+    WORKDIR := $(shell pwd)
+endif
+
+ifeq ($(LOG_LEVEL), $(filter $(LOG_LEVEL), debug trace))
+    PW_DEBUG = DEBUG=pw:api
+else
+    PW_DEBUG =
+endif
+
+PHONY: pipenv
+pipenv:
+	$(PIP) install pip==24.0
+	$(PIP) install pipenv==2023.12.1
+
+.PHONY: setup
+setup: pipenv
+	$(PIPENV) install --verbose --python $(PYTHON_VERSION)
+	-$(PIPENV_PIP) install flash-attn==2.5.5 --no-build-isolation --upgrade --no-cache-dir
+
+.PHONY: setup-dev
+setup-dev: pipenv
+	$(PIPENV) install --verbose --dev --python $(PYTHON_VERSION)
+	- $(PIPENV_PIP) install flash-attn==2.5.5 --no-build-isolation --upgrade --no-cache-dir
+	$(PIPENV) run playwright install
+
+.PHONY: setup-no-flash
+setup-no-flash: pipenv
+	$(PIPENV) install --verbose --python $(PYTHON_VERSION)
+
+setup-ui: pipenv
+	$(PIPENV) install --verbose --categories=dev-packages --python $(PYTHON_VERSION)
+	$(PIPENV) run playwright install
+
+.PHONY: export-requirements
+export-requirements: pipenv
+	$(PIPENV) requirements > requirements.txt
+
+clean-env:
+	$(PIPENV) --rm
+
+clean-data:
+	rm -rf data
+
+clean-output:
+	rm -rf output
+
+reports:
+	mkdir -p reports
+
+.PHONY: style
+style: reports pipenv
+	@echo -n > reports/flake8_errors.log
+	@echo -n > reports/mypy_errors.log
+	@echo -n > reports/mypy.log
+	@echo
+
+	-$(PIPENV) run flake8 | tee -a reports/flake8_errors.log
+	@if [ -s reports/flake8_errors.log ]; then exit 1; fi
+
+	-$(PIPENV) run mypy . --check-untyped-defs | tee -a reports/mypy.log
+	@if ! grep -Eq "Success: no issues found in [0-9]+ source files" reports/mypy.log ; then exit 1; fi
+
+.PHONY: format
+format: pipenv
+	$(PIPENV) run isort .
+	$(PIPENV) run black .
+
+.PHONY: isort
+isort: pipenv
+	$(PIPENV) run isort .
+
+.PHONY: black
+black: pipenv
+	$(PIPENV) run black .
+
+.PHONY: test
+test: reports
+	@bash -c 'set -o pipefail; export PYTHONPATH=$(PWD); \
+	$(PIPENV) run pytest -v --junitxml=reports/junit.xml \
+	--import-mode importlib \
+	--html=./reports/pytest.html \
+	--cov=llm_studio \
+	--cov-report term \
+	--cov-report html:./reports/coverage.html \
+    -o log_cli=true -o log_level=INFO -o log_file=reports/tests.log \
+    tests/* 2>&1 | tee reports/tests.log'
+
+.PHONY: test-ui
+test-ui: reports setup-ui
+	$(PW_DEBUG) $(PIPENV) run pytest \
+	-v \
+	--junitxml=reports/junit_ui.xml \
+	--html=./reports/pytest_ui.html \
+	-o log_cli=true \
+	-o log_level=$(LOG_LEVEL) \
+	-o log_file=reports/tests_ui.log \
+	tests/ui/test.py 2>&1 | tee reports/tests_ui.log
+
+.PHONY: test-ui-headed
+test-ui-headed: setup-ui
+	$(PW_DEBUG) $(PIPENV) run pytest \
+	-vvs \
+	--headed \
+	--video=on \
+	--screenshot=on \
+	--slowmo=100 \
+	tests/ui/test.py 2>&1 | tee reports/tests.log
+
+.PHONY: wave
+wave:
+	HF_HUB_ENABLE_HF_TRANSFER=True \
+	H2O_WAVE_APP_ADDRESS=http://127.0.0.1:8756 \
+	H2O_WAVE_MAX_REQUEST_SIZE=25MB \
+	H2O_WAVE_NO_LOG=true \
+	H2O_WAVE_PRIVATE_DIR="/download/@$(WORKDIR)/output/download" \
+	$(PIPENV) run wave run app
+
+.PHONY: llmstudio
+llmstudio:
+	H2O_WAVE_APP_ADDRESS=http://127.0.0.1:8756 \
+	H2O_WAVE_MAX_REQUEST_SIZE=25MB \
+	H2O_WAVE_NO_LOG=true \
+	H2O_WAVE_PRIVATE_DIR="/download/@$(WORKDIR)/output/download" \
+	$(PIPENV) run wave run --no-reload app
+
+.PHONY: docker-build-nightly
+docker-build-nightly:
+	docker build -t $(DOCKER_IMAGE) .
+
+.PHONY: docker-run-nightly
+docker-run-nightly:
+ifeq (,$(wildcard ./data))
+	mkdir data
+endif
+ifeq (,$(wildcard ./output))
+	mkdir output
+endif
+	docker run \
+		--runtime=nvidia \
+		--shm-size=64g \
+		--init \
+		--rm \
+		-u `id -u`:`id -g` \
+		-p 10101:10101 \
+		-v `pwd`/data:/workspace/data \
+		-v `pwd`/output:/workspace/output \
+		$(DOCKER_IMAGE)
+
+.PHONY: docker-clean-all
+docker-clean-all:
+	@CONTAINERS=$$(docker ps -a -q --filter ancestor=$(DOCKER_IMAGE)); \
+	if [ -n "$$CONTAINERS" ]; then \
+		docker stop $$CONTAINERS; \
+		docker rm $$CONTAINERS; \
+	fi
+	docker rmi $(DOCKER_IMAGE)
+
+.PHONY: shell
+shell:
+	$(PIPENV) shell
+
+setup-doc:  # Install documentation dependencies
+	cd documentation && npm install 
+
+run-doc:  # Run the doc locally
+	cd documentation && npm start
+
+update-documentation-infrastructure:
+	cd documentation && npm update @h2oai/makersaurus
+	cd documentation && npm ls
+
+build-doc-locally:  # Bundles your website into static files for production 
+	cd documentation && npm run build
+
+serve-doc-locally:  # Serves the built website locally 
+	cd documentation && npm run serve

Pipfile

@@ -0,0 +1,71 @@
+[[source]]
+name = "pypi"
+url = "https://pypi.org/simple"
+verify_ssl = true
+
+[[source]]
+name = "pytorch"
+url = "https://download.pytorch.org/whl/cu118"
+verify_ssl = false
+
+[requires]
+python_version = "3.10"
+
+[packages]
+torch = {index = "pytorch", version = "==2.2.0+cu118"}
+tqdm = ">=4.65.0, <5.0.0"
+transformers = "==4.40.1"
+numpy = ">=1.26.0, <2.0.0"
+pandas = ">=2.2.0, <3.0.0"
+scikit-learn = ">=1.4.0, <2.0.0"
+boto3 = ">=1.20.24, <2.0.0"
+SQLAlchemy = ">=2.0.25, <3.0.0"
+dill = ">=0.3.4, <0.4.0"
+pyarrow = ">=14.0.1"
+kaggle = ">=1.5.12, <2.0.0"
+coolname = ">=2.2.0, <3.0.0"
+bokeh = ">=3.2.1, <4.0.0"
+beautifulsoup4 = ">=4.11.1, <5.0.0"
+sqlitedict = "==1.7.0"
+sentencepiece = ">=0.1.96, <0.2.0"
+sacrebleu = "==2.0.0"
+toml = ">=0.10.2, <0.11.0"
+pyyaml = ">=6.0.0, <7.0.0"
+protobuf = "==3.20.3"
+fastparquet = ">=2023.7.0"
+gputil = ">=1.4.0, <2.0.0"
+huggingface-hub = "==0.21.1"
+bitsandbytes = "==0.42.0"
+accelerate = "==0.27.2"
+openai = ">=1.12.0, <2.0.0"
+einops = "==0.7.0"
+datasets = ">=2.11.0, <3.0.0"
+neptune = "==1.9.1"
+Jinja2 = ">=3.1.3, <4.0.0"
+h2o-wave = "==1.1.2"
+tiktoken = "==0.6.0"
+hf-transfer = "==0.1.5"
+peft = "==0.9.0"
+azure-storage-file-datalake = ">=12.12.0"
+deepspeed = "==0.13.2"
+keyring = "==24.3.1"
+
+[dev-packages]
+black = "==24.3.0"
+coverage = "==7.4.3"
+flake8 = "==7.0.0"
+flake8-black = "==0.3.6"
+flake8-isort = "==6.1.1"
+isort = "==5.13.2"
+mypy = "==1.8.0"
+pytest = "==8.0.0"  # >=8.0.1 is not supported by transformers https://github.com/huggingface/transformers/issues/29155
+pytest-cov = "==4.1.0"
+pytest-dependency = "==0.6.0"
+pytest-html = "4.1.1"
+types-pyyaml = ">=6.0"
+types-requests = ">=2.31"
+types-toml = ">=0.10"
+wheel = "==0.42.0"
+pytest-bdd = "==7.0.1"
+hac-playwright = { file = "http://h2o-public-test-data.s3.amazonaws.com/e2e-testing/hac_playwright-1.38.0-py3-none-any.whl" }
+pytest-base-url = "==2.1.0"

README.md

@@ -1,13 +1,274 @@
----
-title: H2OTest
-emoji: 🐢
-colorFrom: purple
-colorTo: blue
-sdk: gradio
-sdk_version: 4.28.3
-app_file: app.py
-pinned: false
-license: apache-2.0
----
-
-Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
+<p align="center"><img src="llm_studio/app_utils/static/llm-studio-logo-light.png#gh-dark-mode-only"></p>
+<p align="center"><img src="llm_studio/app_utils/static/llm-studio-logo.png#gh-light-mode-only"></p>
+
+<h3 align="center">
+    <p>Welcome to H2O LLM Studio, a framework and no-code GUI designed for<br />
+    fine-tuning state-of-the-art large language models (LLMs).
+</p>
+</h3>
+
+<a href="https://user-images.githubusercontent.com/1069138/233859311-32aa1f8c-4d68-47ac-8cd9-9313171ff9f9.png"><img width="50%" alt="home" src="https://user-images.githubusercontent.com/1069138/233859311-32aa1f8c-4d68-47ac-8cd9-9313171ff9f9.png"></a><a href="https://user-images.githubusercontent.com/1069138/233859315-e6928aa7-28d2-420b-8366-bc7323c368ca.png"><img width="50%" alt="logs" src="https://user-images.githubusercontent.com/1069138/233859315-e6928aa7-28d2-420b-8366-bc7323c368ca.png"></a>
+
+## Jump to
+
+- [With H2O LLM Studio, you can](#with-h2o-llm-studio-you-can)
+- [Quickstart](#quickstart)
+- [What's New](#whats-new)
+- [Setup](#setup)
+  - [Recommended Install](#recommended-install)
+  - [Using requirements.txt](#using-requirementstxt)
+- [Run H2O LLM Studio GUI](#run-h2o-llm-studio-gui)
+- [Run H2O LLM Studio GUI using Docker from a nightly build](#run-h2o-llm-studio-gui-using-docker-from-a-nightly-build)
+- [Run H2O LLM Studio GUI by building your own Docker image](#run-h2o-llm-studio-gui-by-building-your-own-docker-image)
+- [Run H2O LLM Studio with command line interface (CLI)](#run-h2o-llm-studio-with-command-line-interface-cli)
+- [Data format and example data](#data-format-and-example-data)
+- [Training your model](#training-your-model)
+- [Example: Run on OASST data via CLI](#example-run-on-oasst-data-via-cli)
+- [Model checkpoints](#model-checkpoints)
+- [Documentation](#documentation)
+- [Contributing](#contributing)
+- [License](#license)
+
+## With H2O LLM Studio, you can
+
+- easily and effectively fine-tune LLMs **without the need for any coding experience**.
+- use a **graphic user interface (GUI)** specially designed for large language models.
+- finetune any LLM using a large variety of hyperparameters.
+- use recent finetuning techniques such as [Low-Rank Adaptation (LoRA)](https://arxiv.org/abs/2106.09685) and 8-bit model training with a low memory footprint.
+- use Reinforcement Learning (RL) to finetune your model (experimental)
+- use advanced evaluation metrics to judge generated answers by the model.
+- track and compare your model performance visually. In addition, [Neptune](https://neptune.ai/) integration can be used.
+- chat with your model and get instant feedback on your model performance.
+- easily export your model to the [Hugging Face Hub](https://huggingface.co/) and share it with the community.
+
+## Quickstart
+
+For questions, discussing, or just hanging out, come and join our [Discord](https://discord.gg/WKhYMWcVbq)!
+
+We offer several ways of getting started quickly.
+
+Using CLI for fine-tuning LLMs:
+
+[![Kaggle](https://kaggle.com/static/images/open-in-kaggle.svg)](https://www.kaggle.com/code/ilu000/h2o-llm-studio-cli/) [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1soqfJjwDJwjjH-VzZYO_pUeLx5xY4N1K?usp=sharing)
+
+## What's New
+
+- [PR 592](https://github.com/h2oai/h2o-llmstudio/pull/599) Added `KTOPairLoss` for DPO modeling allowing to train models with simple preference data. Data currently needs to be manually prepared by randomly matching positive and negative examples as pairs.
+- [PR 592](https://github.com/h2oai/h2o-llmstudio/pull/592) Starting to deprecate RLHF in favor of DPO/IPO optimization. Training is disabled, but old experiments are still viewable. RLHF will be fully removed in a future release.
+- [PR 530](https://github.com/h2oai/h2o-llmstudio/pull/530) Introduced a new problem type for DPO/IPO optimization. This optimization technique can be used as an alternative to RLHF.
+- [PR 288](https://github.com/h2oai/h2o-llmstudio/pull/288) Introduced Deepspeed for sharded training allowing to train larger models on machines with multiple GPUs. Requires NVLink. This feature replaces FSDP and offers more flexibility. Deepspeed requires a system installation of cudatoolkit and we recommend using version 11.8. See [Recommended Install](#recommended-install).
+- [PR 449](https://github.com/h2oai/h2o-llmstudio/pull/449) New problem type for Causal Classification Modeling allows to train binary and multiclass models using LLMs.
+- [PR 364](https://github.com/h2oai/h2o-llmstudio/pull/364) User secrets are now handled more securely and flexible. Support for handling secrets using the 'keyring' library was added. User settings are tried to be migrated automatically.
+
+Please note that due to current rapid development we cannot guarantee full backwards compatibility of new functionality. We thus recommend to pin the version of the framework to the one you used for your experiments. For resetting, please delete/backup your `data` and `output` folders.
+
+## Setup
+
+H2O LLM Studio requires a machine with Ubuntu 16.04+ and at least one recent Nvidia GPU with Nvidia drivers version >= 470.57.02. For larger models, we recommend at least 24GB of GPU memory.
+
+For more information about installation prerequisites, see the [Set up H2O LLM Studio](https://docs.h2o.ai/h2o-llmstudio/get-started/set-up-llm-studio#prerequisites) guide in the documentation.
+
+For a performance comparison of different GPUs, see the [H2O LLM Studio performance](https://h2oai.github.io/h2o-llmstudio/get-started/llm-studio-performance) guide in the documentation.
+
+### Recommended Install
+
+The recommended way to install H2O LLM Studio is using pipenv with Python 3.10. To install Python 3.10 on Ubuntu 16.04+, execute the following commands:
+
+#### System installs (Python 3.10)
+
+```bash
+sudo add-apt-repository ppa:deadsnakes/ppa
+sudo apt install python3.10
+sudo apt-get install python3.10-distutils
+curl -sS https://bootstrap.pypa.io/get-pip.py | python3.10
+```
+
+#### Installing NVIDIA Drivers (if required)
+
+If deploying on a 'bare metal' machine running Ubuntu, one may need to install the required Nvidia drivers and CUDA. The following commands show how to retrieve the latest drivers for a machine running Ubuntu 20.04 as an example. One can update the following based on their OS.
+
+```bash
+wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/cuda-ubuntu2004.pin
+sudo mv cuda-ubuntu2004.pin /etc/apt/preferences.d/cuda-repository-pin-600
+wget https://developer.download.nvidia.com/compute/cuda/11.8.0/local_installers/cuda-repo-ubuntu2004-11-8-local_11.8.0-520.61.05-1_amd64.deb
+sudo dpkg -i cuda-repo-ubuntu2004-11-8-local_11.8.0-520.61.05-1_amd64.deb
+sudo cp /var/cuda-repo-ubuntu2004-11-8-local/cuda-*-keyring.gpg /usr/share/keyrings/
+sudo apt-get update
+sudo apt-get -y install cuda
+```
+
+alternatively, one can install cudatoolkits in a cuda environment:
+
+```bash
+conda create -n llmstudio python=3.10
+conda activate llmstudio
+conda install -c "nvidia/label/cuda-11.8.0" cuda-toolkit
+```
+
+#### Create virtual environment (pipenv)
+
+The following command will create a virtual environment using pipenv and will install the dependencies using pipenv:
+
+```bash
+make setup
+```
+
+If you are having troubles installing the flash_attn package, consider running
+
+```bash
+make setup-no-flash
+```
+
+instead. This will install the dependencies without the flash_attn package. Note that this will disable the use of Flash Attention 2 and model training will be slower and consume more memory.
+
+### Using requirements.txt
+
+If you wish to use conda or another virtual environment, you can also install the dependencies using the requirements.txt file:
+
+```bash
+pip install -r requirements.txt
+pip install flash-attn==2.5.5 --no-build-isolation  # optional for Flash Attention 2
+```
+
+## Run H2O LLM Studio GUI
+
+You can start H2O LLM Studio using the following command:
+
+```bash
+make llmstudio
+```
+
+This command will start the [H2O wave](https://github.com/h2oai/wave) server and app.
+Navigate to <http://localhost:10101/> (we recommend using Chrome) to access H2O LLM Studio and start fine-tuning your models!
+
+If you are running H2O LLM Studio with a custom environment other than Pipenv, you need to start the app as follows:
+
+```bash
+H2O_WAVE_APP_ADDRESS=http://127.0.0.1:8756 \
+H2O_WAVE_MAX_REQUEST_SIZE=25MB \
+H2O_WAVE_NO_LOG=true \
+H2O_WAVE_PRIVATE_DIR="/download/@output/download" \
+wave run app
+```
+
+## Run H2O LLM Studio GUI using Docker from a nightly build
+
+Install Docker first by following instructions from [NVIDIA Containers](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html#docker). Make sure to have `nvidia-container-toolkit` installed on your machine as outlined in the instructions.
+
+H2O LLM Studio images are stored in the h2oai GCR vorvan container repository.
+
+```bash
+mkdir -p `pwd`/data
+mkdir -p `pwd`/output
+
+# make sure to pull latest image if you still have a prior version cached
+docker pull gcr.io/vorvan/h2oai/h2o-llmstudio:nightly
+
+# run the container
+docker run \
+    --runtime=nvidia \
+    --shm-size=64g \
+    --init \
+    --rm \
+    -u `id -u`:`id -g` \
+    -p 10101:10101 \
+    -v `pwd`/data:/workspace/data \
+    -v `pwd`/output:/workspace/output \
+    -v ~/.cache:/home/llmstudio/.cache \
+    gcr.io/vorvan/h2oai/h2o-llmstudio:nightly
+```
+
+Navigate to <http://localhost:10101/> (we recommend using Chrome) to access H2O LLM Studio and start fine-tuning your models!
+
+(Note other helpful docker commands are `docker ps` and `docker kill`.)
+
+## Run H2O LLM Studio GUI by building your own Docker image
+
+```bash
+docker build -t h2o-llmstudio .
+
+mkdir -p `pwd`/data
+mkdir -p `pwd`/output
+
+docker run \
+    --runtime=nvidia \
+    --shm-size=64g \
+    --init \
+    --rm \
+    -u `id -u`:`id -g` \
+    -p 10101:10101 \
+    -v `pwd`/data:/workspace/data \
+    -v `pwd`/output:/workspace/output \
+    -v ~/.cache:/home/llmstudio/.cache \
+    h2o-llmstudio
+```
+
+Alternatively, you can run H2O LLM Studio GUI by using our self-hosted Docker image available [here](https://console.cloud.google.com/gcr/images/vorvan/global/h2oai/h2o-llmstudio).
+
+## Run H2O LLM Studio with command line interface (CLI)
+
+You can also use H2O LLM Studio with the command line interface (CLI) and specify the configuration .yaml file that contains all the experiment parameters. To finetune using H2O LLM Studio with CLI, activate the pipenv environment by running `make shell`, and then use the following command:
+
+```bash
+python train.py -Y {path_to_config_yaml_file}
+```
+
+To run on multiple GPUs in DDP mode, run the following command:
+
+```bash
+bash distributed_train.sh {NR_OF_GPUS} -Y {path_to_config_yaml_file}
+```
+
+By default, the framework will run on the first `k` GPUs. If you want to specify specific GPUs to run on, use the `CUDA_VISIBLE_DEVICES` environment variable before the command.
+
+To start an interactive chat with your trained model, use the following command:
+
+```bash
+python prompt.py -e {experiment_name}
+```
+
+where `experiment_name` is the output folder of the experiment you want to chat with (see configuration).
+The interactive chat will also work with model that were finetuned using the UI.
+
+To publish the model to Hugging Face, use the following command:
+
+```bash
+make shell 
+
+python publish_to_hugging_face.py -p {path_to_experiment} -d {device} -a {api_key} -u {user_id} -m {model_name} -s {safe_serialization}
+```
+
+`path_to_experiment` is the output folder of the experiment.
+`device` is the target device for running the model, either 'cpu' or 'cuda:0'. Default is 'cuda:0'.
+`api_key` is the Hugging Face API Key. If user logged in, it can be omitted.
+`user_id` is the Hugging Face user ID. If user logged in, it can be omitted.
+`model_name` is the name of the model to be published on Hugging Face. It can be omitted.
+`safe_serialization` is a flag indicating whether safe serialization should be used. Default is True.
+
+## Data format and example data
+
+For details on the data format required when importing your data or example data that you can use to try out H2O LLM Studio, see [Data format](https://docs.h2o.ai/h2o-llmstudio/guide/datasets/data-connectors-format#data-format) in the H2O LLM Studio documentation.
+
+## Training your model
+
+With H2O LLM Studio, training your large language model is easy and intuitive. First, upload your dataset and then start training your model. Start by [creating an experiment](https://docs.h2o.ai/h2o-llmstudio/guide/experiments/create-an-experiment). You can then [monitor and manage your experiment](https://docs.h2o.ai/h2o-llmstudio/guide/experiments/view-an-experiment), [compare experiments](https://docs.h2o.ai/h2o-llmstudio/guide/experiments/compare-experiments), or [push the model to Hugging Face](https://docs.h2o.ai/h2o-llmstudio/guide/experiments/export-trained-model) to share it with the community.
+
+## Example: Run on OASST data via CLI
+
+As an example, you can run an experiment on the OASST data via CLI. For instructions, see [Run an experiment on the OASST data](https://docs.h2o.ai/h2o-llmstudio/guide/experiments/create-an-experiment#run-an-experiment-on-the-oasst-data-via-cli) guide in the H2O LLM Studio documentation.
+
+## Model checkpoints
+
+All open-source datasets and models are posted on [H2O.ai's Hugging Face page](https://huggingface.co/h2oai/) and our [H2OGPT](https://github.com/h2oai/h2ogpt) repository.
+
+## Documentation
+
+Detailed documentation and frequently asked questions (FAQs) for H2O LLM Studio can be found at <https://docs.h2o.ai/h2o-llmstudio/>. If you wish to contribute to the docs, navigate to the `/documentation` folder of this repo and refer to the [README.md](documentation/README.md) for more information.
+
+## Contributing
+
+We are happy to accept contributions to the H2O LLM Studio project. Please refer to the [CONTRIBUTING.md](CONTRIBUTING.md) file for more information.
+
+## License
+
+H2O LLM Studio is licensed under the Apache 2.0 license. Please see the [LICENSE](LICENSE) file for more information.

app.py

@@ -0,0 +1,44 @@
+import logging
+import os
+
+from llm_studio.app_utils.sections.chat_update import is_app_blocked_while_streaming
+from llm_studio.src.utils.logging_utils import initialize_logging
+
+os.environ["MKL_THREADING_LAYER"] = "GNU"
+
+from h2o_wave import Q, app, copy_expando, main, ui  # noqa: F401
+
+from llm_studio.app_utils.handlers import handle
+from llm_studio.app_utils.initializers import initialize_app, initialize_client
+from llm_studio.app_utils.sections.common import heap_redact, interface
+
+logger = logging.getLogger(__name__)
+
+
+def on_startup():
+    initialize_logging()
+    logger.info("STARTING APP")
+
+
+@app("/", on_startup=on_startup)
+async def serve(q: Q):
+    """Serving function."""
+
+    # Chat is still being streamed but user clicks on another button.
+    # Wait until streaming has been completed
+    if await is_app_blocked_while_streaming(q):
+        return
+
+    if not q.app.initialized:
+        await initialize_app(q)
+
+    copy_expando(q.args, q.client)
+
+    await initialize_client(q)
+    await handle(q)
+
+    if not q.args["experiment/display/chat/chatbot"]:
+        await interface(q)
+
+    await heap_redact(q)
+    await q.page.save()

distributed_train.sh

@@ -0,0 +1,4 @@
+#!/bin/bash
+NUM_PROC=$1
+shift
+torchrun --nproc_per_node=$NUM_PROC train.py "$@"

documentation/.gitignore

@@ -0,0 +1,17 @@
+node_modules
+tmp
+
+# Generated files 
+.docusaurus 
+.cach-loader
+
+# Misc
+.DS_Store
+.env.local
+.env.development.local
+.env.test.local
+.env.production.local
+
+npm-debug.log*
+yarn-debug.log*
+yarn-error.log*

documentation/README.md

@@ -0,0 +1,98 @@
+# H2O LLM Studio Documentation
+
+- The LLM Studio documentation is built using [Makersaurus](https://github.com/h2oai/makersaurus/pkgs/npm/makersaurus) which is a very thin wrapper around Facebook's Docusaurus.
+- The documentation is displayed at {{ https://docs.h2o.ai/h2o-llm-studio/ }}
+
+To view, edit, and cut a version of the documentation, the following is required:
+
+- Node.js version 16.14+ (you can check your version by running `node -v`). Use nvm to manage multiple Node versions installed on a single machine.
+
+- To install Node.js and npm with nvm in Mac or Ubuntu, run: `curl -o-
+https://raw.githubusercontent.com/creationix/nvm/v0.33.0/install.sh | bash` and `nvm install node`
+
+- Makersaurus (the H2O themed documentation site) is hosted on H2O's Github npm registry. npm must authenticate to the registry before you can download Makersaurus. Follow the 3 steps below to authenticate the npm package.
+
+ If you have already installed `@h2oai/ui-kit` or any other private `@h2oai`-prefixed npm package you can skip this step.
+
+ **Step 1:** Create a "classic" [personal access token](https://github.com/settings/tokens) (PAT) on Github. Note that you only need to enable the `read:packages` scope for this token. 
+
+ **Step 2:** Add the PAT to your `~/.npmrc` file. Create this file if it doesn't exist yet. 
+ ```
+ @h2oai:registry=https://npm.pkg.github.com/
+ //npm.pkg.github.com/:_authToken=YOUR-GENERATED-TOKEN
+ ```
+ **Step 3:** Verify that it worked by running the following command:
+ ```
+ npm whoami --registry=https://npm.pkg.github.com
+ ```
+ If this command returns your username, you can proceed to the next step. If you get an error, you are not yet authenticated. You might find the [Github registry docs](https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-npm-registry#authenticating-with-a-personal-access-token) helpful for debugging.
+
+### Documentation structure
+
+
+```
+├── documentation
+│ ├── docs
+│ ├── tmp
+│ ├── makersaurus.config.js
+│ ├── sidebars.js
+│ ├── package.json
+│ ├── package-lock.json
+```
+
+- `documentation/docs`: Contains Markdown documentation files to edit the next documentation version.
+Customize the order of the docs sidebar in `sidebars.js`
+- `documentation/tmp`: Temporary files generated by Makersaurus. Do not edit these files.
+- `documentation/makersaurus.config.js`: Makersaurus [config file](https://h2oai.github.io/makersaurus/api/config)
+- `documentation/sidebars.js`: Sidebar configuration file
+- `documentation/package.json`: npm configuration file
+- `documentation/package-lock.json`: Generated by npm. Do not edit this file.
+
+
+### Edit locally
+
+To setup the local `env` to view and edit the next or past documentation versions ([first, ensure you install
+Node.js](#requirements)):
+
+1. Enter the documentation folder
+
+`cd documentation`
+
+2. Install dependencies
+
+`npm install`
+
+3. Start Makersaurus
+
+`npm start`
+
+- **Next documentation version**: To view your edits for the next documentation version, navigate to the provided URL.
+Then, select **Next** on the **Versions** dropdown menu.
+- **Debug**
+- If you don't see anything after clicking **Next**, run the following command and try again:
+`make setup-doc`
+- Ensure that the following variable is set to `true` in the `makersaurus.config.js` file (located at `docs`):
+`includeCurrentVersion`
+- **Past documentation versions**: To view your edits for past documentation versions (located at
+`docs/versioned_docs/`), navigate to the provided URL (for example, `http://localhost:3000/h2o-llm-studio/`). 
+Then, select a *version* (for example, v0.2.0) on the **Versions** dropdown menu.
+
+### Cut a version
+
+To cut a new version after making specific changes at `documentation/docs` to align with the next version of the application, consider the following instructions:
+
+1. Before a new version of the documentation is released, and right before we cut a version (`make version-doc`), change the following variable located in the `makersaurus.config.js` file to `false`: `includeCurrentVersion`
+2. Run: `make version-doc` (for example, `make version-doc DOC_VERSION=v0.3.0`)
+3. After the previous steps are executed and all generated files are pushed to the main branch, trigger the following
+script in GitHub actions: `deploy-to-github-pages.yml`
+4. After publishing the new documentation version, change the following variable located in the
+`makersaurus.config.js` file to `true`: `includeCurrentVersion`
+- This ensures the next doc version to edit will be visible while editing locally
+
+
+## More information
+
+Use the [Makersaurus docs](https://h2oai.github.io/makersaurus/) to learn more about how to edit docs, deploy the site, set up versioning and more.
+
+
+

documentation/docs/concepts.md

@@ -0,0 +1,58 @@
+---
+description: Learn about concepts around H2O LLM Studio.
+---
+# Concepts
+
+H2O LLM Studio is based on a few key concepts and uses several key terms across its documentation. Each, in turn, is explained within the sections below.
+
+## LLM
+
+A Large Language Model (LLM) is a type of AI model that uses deep learning techniques and uses massive datasets to analyze and generate human-like language. For example, many AI chatbots or AI search engines are powered by LLMs.  
+
+Generally speaking, LLMs can be characterized by the following parameters: 
+- size of the training dataset
+- cost of training (computational power)
+- size of the model (parameters)
+- performance after training (or how well the model is able to respond to a particular question)
+
+## Parameters and hyperparameters
+
+In the context of an LLM, parameters and hyperparameters are a crucial part of determinining the model's performance and overall behaviour. 
+
+- **Parameters:** The internal variables of the model that are learned during the training process. In the case of an LLM, parameters typically include the weights and biases associated with the neural network layers. The values of parameters directly influence the model's predictions and the quality of generated text.
+
+- **Hyperparameters:** The configuration choices that are set before training the model and are not learned directly from the data (e.g., number of epochs, batch size etc.). These choices impact the learning process and influence the model's overall behavior. Hyperparameters need to be tuned and optimized to achieve the best performance. H2O LLM Studio GUI shows tooltips next to each hyperparameter to explain what each hyperparameter is for. You can also see the following references for more details about hyperparameters in H2O LLM Studio.
+    - Dataset settings
+    - [Experiment settings](./guide/experiments/experiment-settings)
+
+
+## LLM Backbone
+
+LLM Backbone is a key hyperparamter that determines the model's architecture. This option is the most important setting when it comes to experiment creation, as it sets the pretrained model weights. For more information about LLM Backbone, see [Experiment settings](guide/experiments/experiment-settings.md#llm-backbone).
+
+
+## Generative AI
+
+Generative AI refers to AI models that can generate new content, such as images, videos, or text, that did not exist before. These models learn from large datasets and use this knowledge to create new content that is similar in style or content to the original dataset.
+
+
+## Foundation model
+
+A particular adaptive model that has been trained on a large amount of data and starts to derive relationships between words and concepts. Foundation models are fine-tuned to become more specific and adapt to the related domain more efficiently.
+
+## Fine-tuning
+
+Fine-tuning refers to the process of taking a pre-trained language model and further training it on a specific task or domain to improve its performance on that task. It is an important technique used to adapt LLMs to specific tasks and domains. 
+
+## LoRA (Low-Rank Adaptation)
+
+Low-Rank Adapation (LoRa) involves modifying the pre-trained model by adjusting its weights and biases to better fit the new task. This adaptation is done in a way that preserves the pre-trained weights from the original dataset while also adjusting for the new task's specific requirements. This method of training or fine-turning models consumes less memory. By using low rank adaptation, the pre-trained model can be quickly adapted to new tasks, without requiring a large amount of new training data.
+
+## Quantization
+
+Quantization is a technique used to reduce the size and memory requirements of a large language model without sacrificing its accuracy. This is done by converting the floating-point numbers used to represent the model's parameters to lower-precision numbers, such as half-floats or bfloat16. Quantization can be used to make language models more accessible to users with limited computing resources. 
+
+## 8-bit model training with a low memory footprint
+
+8-bit model training with a low memory footprint refers to a fine-tuning technique that reduces the memory requirements for training neural networks by using 8-bit integers instead of 32-bit floating-point numbers. This approach can significantly reduce the amount of memory needed to store the model's parameters and can make it possible to train larger models on hardware with limited memory capacity.
+

documentation/docs/faqs.md

@@ -0,0 +1,120 @@
+---
+description: Learn about frequently asked questions. 
+---
+import Icon from "@material-ui/core/Icon";
+
+# FAQs
+
+The sections below provide answers to frequently asked questions. If you have additional questions, please send them to [cloud-feedback@h2o.ai](mailto:cloud-feedback@h2o.ai).
+
+---
+
+### How much data is generally required to fine-tune a model?
+
+There is no clear answer. As a rule of thumb, 1000 to 50000 samples of conversational data should be enough. Quality and diversity is very important. Make sure to try training on a subsample of data using the "sample" parameter to see how big the impact of the dataset size is. Recent studies suggest that less data is needed for larger foundation models.
+
+---
+
+### Are there any recommendations for which backbone to use? Are some backbones better for certain types of tasks?
+
+The majority of the LLM backbones are trained on a very similar corpus of data. The main difference is the size of the model and the number of parameters. Usually, the larger the model, the better they are. The larger models also take longer to train. It is recommended to start with the smallest model and then increase the size if the performance is not satisfactory. If you are looking to train for tasks that are not directly question answering in English, it is also a good idea to look for specialized LLM backbones.
+
+---
+
+### What if my data is not in question-and-answer form and I just have documents? How can I fine-tune the LLM model?
+
+To train a chatbot style model, you need to convert your data into a question and answer format.
+
+If you really want to continue pretraining on your own data without teaching a question-answering style, prepare a dataset with all your data in a single column Dataframe. Make sure that the length of the text in each row is not too long. In the experiment setup, remove all additional tokens (e.g. `<|prompt|>`, `<|answer|>`, for Text Prompt Start and Text Answer Start respectively) and disable **Add Eos Token To Prompt** and **Add Eos Token To Answer**. Deselect everything in the Prompt Column.
+
+There are also other enterprise solutions from H2O.ai that may help you convert your data into a Q&A format. For more information, see [H2O.ai's Generative AI page](https://h2o.ai/) and this blogpost about [H2O LLM DataStudio: Streamlining Data Curation and Data Preparation for LLMs related tasks](https://blog.h2o.ai/blog/streamlining-data-preparation-for-fine-tuning-of-large-language-models/).
+
+---
+
+###  I encounter GPU out-of-memory issues. What can I change to be able to train large models?
+
+There are various parameters that can be tuned while keeping a specific LLM backbone fixed. It is advised to choose 4bit/8bit precision as a backbone dtype to be able to train models >=7B on a consumer type GPU. [LORA](concepts#lora-low-rank-adaptation) should be enabled. Besides that there are the usual parameters such as batch size and maximum sequence length that can be decreased to save GPU memory (please ensure that your prompt+answer text is not truncated too much by checking the train data insights).
+
+---
+
+### When does the model stop the fine-tuning process?
+
+The number of epochs are set by the user.
+
+---
+
+### How many records are recommended for fine-tuning?
+
+An order of 100K records is recommended for fine-tuning.
+
+---
+
+### Where does H2O LLM Studio store its data?
+
+By default, H2O LLM Studio stores its data in two folders located in the root directory in the app. The folders are named `data` and `output`. Here is the breakdown of the data storage structure:
+- `data/dbs`: This folder contains the user database used within the app.
+- `data/user`: This folder is where uploaded datasets from the user are stored.
+- `output/user`: All experiments conducted in H2O LLM Studio are stored in this folder. For each experiment, a separate folder is created within the `output/user` directory, which contains all the relevant data associated with that particular experiment.
+- `output/download`: Utility folder that is used to store data the user downloads within the app. 
+
+It is possible to change the default working directory of H2O LLM Studio by setting the `H2O_LLM_STUDIO_WORKDIR` environment variable. By default, the working directory is set to the root directory of the app.
+
+----
+
+### How can I update H2O LLM Studio?
+
+To update H2O LLM Studio, you have two options:
+
+1. Using the latest main branch: Execute the commands `git checkout main` and `git pull` to obtain the latest updates from the main branch.
+2. Using the latest release tag: Execute the commands `git pull` and `git checkout v0.0.3` (replace 'v0.0.3' with the desired version number) to switch to the latest release branch.
+
+The update process does not remove or erase any existing data folders or experiment records. This means that all your old data, including the user database, uploaded datasets, and experiment results, will still be available to you within the updated version of H2O LLM Studio.
+
+Before updating, it is recommended to run the `git rev-parse --short HEAD` command and save the commit hash. 
+This will allow you to revert to your existing version if needed. 
+
+---
+
+### Once I have the [LoRA](guide/experiments/experiment-settings.md#lora), what is the recommended way of utilizing it with the base model?
+
+You can also export the LoRA weights. You may add them to the files to be exported [here](https://github.com/h2oai/h2o-llmstudio/blob/main/llm_studio/app_utils/sections/experiment.py#L1552). Before exporting, the LoRA weights are merged back into the original LLM backbone weights to make downstream tasks easier. You do not need to have PEFT, or anything else for your deployment.
+
+---
+
+### How to use H2O LLM Studio in Windows? 
+
+Use WSL 2 on Windows 
+
+---
+
+### How can I easily fine-tune a large language model (LLM) using the command-line interface (CLI) of H2O LLM Studio when I have limited GPU memory?
+
+If you have limited GPU memory but still want to fine-tune a large language model using H2O LLM Studio's CLI, there are alternative methods you can use to get started quickly.
+
+- [Using Kaggle kernels](https://www.kaggle.com/code/ilu000/h2o-llm-studio-cli/) 
+- [Using Google Colab](https://colab.research.google.com/drive/1soqfJjwDJwjjH-VzZYO_pUeLx5xY4N1K?usp=sharing)
+
+---
+
+### Can I run a validation metric on a model post-training, optionally on a different validation dataset?
+
+Yes.
+
+1. After you have finished creating an experiment, click on the <Icon>more_vert</Icon> Kebab menu of the relevant experiment and select **New Experiment**. 
+
+2. Enable the **Use previous experiments weight** setting found at the top of the screen. 
+   This will now load the previous weights, and you can now change eval dataset, metric, and anything else as you see fit. To only do evaluation without any retraining, set the **Epochs** to 0.
+
+----
+
+### What are the hardware/infrastructure sizing recommendations for H2O LLM Studio?
+
+When it comes to hardware requirements, it is important to note that the primary demand centers around the GPU and its associated VRAM. In terms of CPUs, most modern choices should suffice as NLP tasks typically do not heavily stress CPU performance. As for RAM, it's advisable to have a minimum of 128GB, with a stronger recommendation of 256GB or more, particularly when dealing with substantial model weights that must be accommodated in the CPU RAM.
+
+----
+
+
+
+
+
+

documentation/docs/get-started/core-features.md

@@ -0,0 +1,34 @@
+---
+description: Learn about the core features of LLM Studio.
+---
+# Core features
+
+## No-code fine-tuning
+
+NLP practioners can easily fine-tune models without the need for code expertise. The user interface, which is specifically designed for LLMs, allows users to upload large datasets easily and configure [hyperparameters](../concepts#parameters-and-hyperparameters) to fine-tune the model. 
+
+## Highly customizable (wide range of hyperparameters)
+
+H2O LLM Studio supports a wide variety of hyperparameters that can be used to fine-tune the model and supports the following fine-tuning techniques to enable advanced customization:
+
+- [Low-Rank Adaptation (LoRA)](../concepts#lora-low-rank-adaptation)
+- [8-bit model training with a low memory footprint](../concepts#8-bit-model-training-with-a-low-memory-footprint)
+
+## Advanced evaluation metrics and experiment comparison
+
+Advanced evaluation metrics in H2O LLM Studio can be used to validate the answers generated by the LLM. This helps to make data-driven decisions about the model. It also offers visual tracking and comparison of experiment performance, making it easy to analyze and compare different fine-tuned models.You can also visualize how different parameters affect the model performance, and optionally use the [Neptune](https://neptune.ai/) integraton to track and log your experiments. 
+
+## Instant publishing models
+
+H2O LLM Studio enables easy model sharing with the community by allowing you to export the model to the [Hugging Face Hub](https://huggingface.co/h2oai) with a single click. 
+
+## Instant feedback on model performance
+
+Additionally, H2O LLM Studio lets you chat with the fine-tuned model and recieve instant feedback about model performance.
+
+
+
+
+
+
+

documentation/docs/get-started/llm-studio-flow.md

@@ -0,0 +1,48 @@
+---
+description: The flow of creating and fine-tuning large language models using H2O LLM Studio.
+---
+# Model flow
+
+The flow of creating and fine-tuning large language models using H2O LLM Studio can be summarized in the following sequential steps:
+
+- [Step 1: Import a dataset](#step-1-import-a-dataset)
+- [Step 2: Create an experiment](#step-2-create-an-experiment)
+- [Step 3: Monitor an experiment](#step-3-monitor-an-experiment)
+- [Step 4: Compare experiments](#step-4-compare-experiments)
+- [Step 5: Export a model to Hugging Face Hub](#step-5-export-a-model-to-hugging-face-hub)
+
+In the below sections, each step above, in turn, is summarized.
+
+## Step 1: Import a dataset
+
+As the first step in the experiment flow, prep your data and import your dataset to H2O LLM Studio. 
+
+- To learn about supported data connectors and data format, see [Supported data connectors and format](../guide/datasets/data-connectors-format).
+- To learn about how to import a dataset to H2O LLM Studio, see [Import a dataset](../guide/datasets/import-dataset).
+- To learn about reviewing and editing a dataset, see [View and manage dataset](../guide/datasets/view-dataset.md).
+
+## Step 2: Create an experiment
+
+As the second step in the experiment flow, create an experiment using the imported dataset. H2O LLM Studio offers several hyperparameter settings that you can adjust for your experiment model. To ensure that your training process is effective, you may need to specify the [hyperparameters](../concepts#parameters-and-hyperparameters) like learning rate, batch size, and the number of epochs. H2O LLM Studio provides an overview of all the parameters you’ll need to specify for your experiment.
+
+- To learn about creating a new experiment, see [Create an experiment](../guide/experiments/create-an-experiment.md).
+- To learn about the settings available for creating an experiment, see [Experiment settings](../guide/experiments/experiment-settings.md).
+
+## Step 3: Monitor an experiment
+
+As the third step in the experiment flow, monitor the launched experiment. H2O LLM Studio allows you to inspect your experiment (model) during and after model training. Simple interactive graphs in H2O LLM Studio allow you to understand the impact of selected hyperparameter values during and after model training. You can then adjust the [hyperparameters](../concepts#parameters-and-hyperparameters) to further optimize model performance. 
+
+To learn about viewing and monitoring an experiment, see [View and manage experiments](../guide/experiments/view-an-experiment.md).
+
+## Step 4: Compare experiments
+
+The H2O LLM studio provides a useful feature  that allows comparing various experiments and analyzing how different model parameters affect model performance. This feature is a powerful tool for fine-tuning your machine-learning models and ensuring they meet your desired performance metrics.
+
+To learn about comparing multiple experiments, see [Compare experiments](../guide/experiments/compare-experiments.md).
+
+## Step 5: Export a model to Hugging Face Hub
+
+As the final step in the experiment flow, you can export the fine-tuned model to Hugging Face with a single click.
+
+To learn about exporting a trained model to Hugging Face Hub, see, [Export trained model to Hugging Face](../guide/experiments/export-trained-model.md).
+

documentation/docs/get-started/llm-studio-performance.md

@@ -0,0 +1,170 @@
+---
+description: Setting up and runnning H2O LLM Studio requires the following minimal prerequisites. This page lists out the speed and performance metrics of H2O LLM Studio based on different hardware setups.
+---
+# H2O LLM Studio performance
+
+Setting up and runnning H2O LLM Studio requires the following minimal [prerequisites](set-up-llm-studio.md#prerequisites). This page lists out the speed and performance metrics of H2O LLM Studio based on different hardware setups.
+
+The following metrics were measured. 
+
+- **Hardware setup:** The type and number of computing devices used to train the model.
+- **LLM backbone:** The underlying architecture of the language model. For more information, see [LLM backbone](concepts.md#llm-backbone).
+- **Quantization:** A technique used to reduce the size and memory requirements of the model. For more information, see [Quantization](concepts.md#quantization).
+- **Train**: The amount of time it took to train the model in hours and minutes.
+- **Validation:** The amount of time it took to validate the mode in hours and minutes. 
+
+| Hardware setup | LLM backbone | Quantization | Train (hh:mm:ss)| Validation (hh:mm:ss) |
+|---|---|---|---|---|
+| 8xA10G | h2oai/h2ogpt-4096-llama2-7b | bfloat16 | 11:35 | 3:32 |
+| 4xA10G | h2oai/h2ogpt-4096-llama2-7b | bfloat16 | 21:13 | 06:35 |
+| 2xA10G | h2oai/h2ogpt-4096-llama2-7b | bfloat16 | 37:04 | 12:21 |
+| 1xA10G | h2oai/h2ogpt-4096-llama2-7b | bfloat16 | 1:25:29 | 15:50 |
+| 8xA10G | h2oai/h2ogpt-4096-llama2-7b | nf4 | 14:26 | 06:13 |
+| 4xA10G | h2oai/h2ogpt-4096-llama2-7b | nf4 | 26:55 | 11:59 |
+| 2xA10G | h2oai/h2ogpt-4096-llama2-7b | nf4 | 48:24 | 23:37 |
+| 1xA10G | h2oai/h2ogpt-4096-llama2-7b | nf4 | 1:26:59 | 42:17 |
+| 8xA10G | h2oai/h2ogpt-4096-llama2-13b | bfloat16 | OOM | OOM |
+| 4xA10G | h2oai/h2ogpt-4096-llama2-13b | bfloat16 | OOM | OOM |
+| 2xA10G | h2oai/h2ogpt-4096-llama2-13b | bfloat16 | OOM | OOM |
+| 1xA10G | h2oai/h2ogpt-4096-llama2-13b | bfloat16 | OOM | OOM |
+| 8xA10G | h2oai/h2ogpt-4096-llama2-13b | nf4 | 25:07 | 10:58 |
+| 4xA10G | h2oai/h2ogpt-4096-llama2-13b | nf4 | 48:43 | 21:25 |
+| 2xA10G | h2oai/h2ogpt-4096-llama2-13b | nf4 | 1:30:45 | 42:06 |
+| 1xA10G | h2oai/h2ogpt-4096-llama2-13b | nf4 | 2:44:36 | 1:14:20 |
+| 8xA10G | h2oai/h2ogpt-4096-llama2-70b | nf4 | OOM | OOM |
+| 4xA10G | h2oai/h2ogpt-4096-llama2-70b | nf4 | OOM | OOM |
+| 2xA10G | h2oai/h2ogpt-4096-llama2-70b | nf4 | OOM | OOM |
+| 1xA10G | h2oai/h2ogpt-4096-llama2-70b | nf4 | OOM | OOM |
+|---|---|---|---|---|
+| 4xA100 80GB | h2oai/h2ogpt-4096-llama2-7b | bfloat16 | 7:04 | 3:55 |
+| 2xA100 80GB | h2oai/h2ogpt-4096-llama2-7b | bfloat16 | 13:14 | 7:23 |
+| 1xA100 80GB | h2oai/h2ogpt-4096-llama2-7b | bfloat16 | 23:36 | 13:25 |
+| 4xA100 80GB | h2oai/h2ogpt-4096-llama2-7b | nf4 | 9:44 | 6:30 |
+| 2xA100 80GB | h2oai/h2ogpt-4096-llama2-7b | nf4 | 18:34 | 12:16 |
+| 1xA100 80GB | h2oai/h2ogpt-4096-llama2-7b | nf4 | 34:06 | 21:51 |
+| 4xA100 80GB | h2oai/h2ogpt-4096-llama2-13b | bfloat16 | 11:46 | 5:56 |
+| 2xA100 80GB | h2oai/h2ogpt-4096-llama2-13b | bfloat16 | 21:54 | 11:17 |
+| 1xA100 80GB | h2oai/h2ogpt-4096-llama2-13b | bfloat16 | 39:10 | 18:55 |
+| 4xA100 80GB | h2oai/h2ogpt-4096-llama2-13b | nf4 | 16:51 | 10:35 |
+| 2xA100 80GB | h2oai/h2ogpt-4096-llama2-13b | nf4 | 32:05 | 21:00 |
+| 1xA100 80GB | h2oai/h2ogpt-4096-llama2-13b | nf4 | 59:11 | 36:53 |
+| 4xA100 80GB | h2oai/h2ogpt-4096-llama2-70b | nf4 | 1:13:33 | 46:02 |
+| 2xA100 80GB | h2oai/h2ogpt-4096-llama2-70b | nf4 | 2:20:44 | 1:33:42 |
+| 1xA100 80GB | h2oai/h2ogpt-4096-llama2-70b | nf4 | 4:23:57 | 2:44:51 |
+
+:::info
+The runtimes were gathered using the default parameters. 
+
+<details>
+<summary>Expand to see the default parameters </summary>
+
+```
+architecture:
+    backbone_dtype: int4
+    force_embedding_gradients: false
+    gradient_checkpointing: true
+    intermediate_dropout: 0.0
+    pretrained: true
+    pretrained_weights: ''
+augmentation:
+    random_parent_probability: 0.0
+    skip_parent_probability: 0.0
+    token_mask_probability: 0.0
+dataset:
+    add_eos_token_to_answer: true
+    add_eos_token_to_prompt: true
+    add_eos_token_to_system: true
+    answer_column: output
+    chatbot_author: H2O.ai
+    chatbot_name: h2oGPT
+    data_sample: 1.0
+    data_sample_choice:
+    - Train
+    - Validation
+    limit_chained_samples: false
+    mask_prompt_labels: true
+    parent_id_column: None
+    personalize: false
+    prompt_column:
+    - instruction
+    system_column: None
+    text_answer_separator: <|answer|>
+    text_prompt_start: <|prompt|>
+    text_system_start: <|system|>
+    train_dataframe: /data/user/oasst/train_full.pq
+    validation_dataframe: None
+    validation_size: 0.01
+    validation_strategy: automatic
+environment:
+    compile_model: false
+    find_unused_parameters: false
+    gpus:
+    - '0'
+    - '1'
+    - '2'
+    - '3'
+    - '4'
+    - '5'
+    - '6'
+    - '7'
+    huggingface_branch: main
+    mixed_precision: true
+    number_of_workers: 8
+    seed: -1
+    trust_remote_code: true
+    use_fsdp: false
+experiment_name: default-8-a10g
+llm_backbone: h2oai/h2ogpt-4096-llama2-7b
+logging:
+    logger: None
+    neptune_project: ''
+output_directory: /output/...
+prediction:
+    batch_size_inference: 0
+    do_sample: false
+    max_length_inference: 256
+    metric: BLEU
+    metric_gpt_model: gpt-3.5-turbo-0301
+    metric_gpt_template: general
+    min_length_inference: 2
+    num_beams: 1
+    num_history: 4
+    repetition_penalty: 1.2
+    stop_tokens: ''
+    temperature: 0.3
+    top_k: 0
+    top_p: 1.0
+problem_type: text_causal_language_modeling
+tokenizer:
+    add_prompt_answer_tokens: false
+    max_length: 512
+    max_length_answer: 256
+    max_length_prompt: 256
+    padding_quantile: 1.0
+    use_fast: true
+training:
+    batch_size: 2
+    differential_learning_rate: 1.0e-05
+    differential_learning_rate_layers: []
+    drop_last_batch: true
+    epochs: 1
+    evaluate_before_training: false
+    evaluation_epochs: 1.0
+    grad_accumulation: 1
+    gradient_clip: 0.0
+    learning_rate: 0.0001
+    lora: true
+    lora_alpha: 16
+    lora_dropout: 0.05
+    lora_r: 4
+    lora_target_modules: ''
+    loss_function: TokenAveragedCrossEntropy
+    optimizer: AdamW
+    save_best_checkpoint: false
+    schedule: Cosine
+    train_validation_data: false
+    warmup_epochs: 0.0
+    weight_decay: 0.0
+```
+</details>
+:::

documentation/docs/get-started/set-up-llm-studio.md

@@ -0,0 +1,326 @@
+---
+description: Learn how to set up LLM Studio.
+---
+import Tabs from "@theme/Tabs";
+import TabItem from "@theme/TabItem";
+
+# Set up H2O LLM Studio
+
+## Prerequisites
+
+H2O LLM Studio requires the following minimum requirements:
+
+- A machine with Ubuntu 16.04+ with atleast one recent Nvidia GPU
+- Have at least 128GB+ of system RAM. Larger models and complex tasks may require 256GB+ or more.
+- Nvidia drivers v470.57.02 or a later version
+- Access to the following URLs:
+  - developer.download.nvidia.com
+  - pypi.org
+  - huggingface.co
+  - download.pytorch.org
+  - cdn-lfs.huggingface.co
+
+:::info Notes
+
+- Atleast 24GB of GPU memory is recommended for larger models.
+- For more information on performance benchmarks based on the hardware setup, see [H2O LLM Studio performance](llm-studio-performance.md).
+- The required URLs are accessible by default when you start a GCP instance, however, if you have network rules or custom firewalls in place, it is recommended to confirm that the URLs are accessible before running `make setup`.
+  :::
+
+## Installation
+
+:::note Installation methods
+
+<Tabs className="unique-tabs">
+  <TabItem
+    value="recommended-install"
+    label="Linux/Ubuntu installation (recommended)"
+    default
+  >
+    <p>
+      The recommended way to install H2O LLM Studio is using pipenv with Python
+      3.10. To install Python 3.10 on Ubuntu 16.04+, execute the following
+      commands.
+    </p>
+    <p>
+      <b>System installs (Python 3.10)</b>
+    </p>
+    <pre>
+      <code>
+        sudo add-apt-repository ppa:deadsnakes/ppa <br></br>
+        sudo apt install python3.10 <br></br>
+        sudo apt-get install python3.10-distutils <br></br>
+        curl -sS https://bootstrap.pypa.io/get-pip.py | python3.10
+      </code>
+    </pre>
+    <p>
+      <b>Install NVIDIA drivers (if required)</b>
+      <br></br>
+      If you are deploying on a 'bare metal' machine running Ubuntu, you may need
+      to install the required Nvidia drivers and CUDA. The following commands show
+      how to retrieve the latest drivers for a machine running Ubuntu 20.04 as an
+      example. You can update the following based on your respective operating system.
+    </p>
+    <pre>
+      <code>
+        wget
+        https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/cuda-ubuntu2004.pin{" "}
+        <br></br>
+        sudo mv cuda-ubuntu2004.pin
+        /etc/apt/preferences.d/cuda-repository-pin-600 <br></br>
+        wget
+        https://developer.download.nvidia.com/compute/cuda/11.4.3/local_installers/cuda-repo-ubuntu2004-11-4-local_11.4.3-470.82.01-1_amd64.deb{" "}
+        <br></br>
+        sudo dpkg -i
+        cuda-repo-ubuntu2004-11-4-local_11.4.3-470.82.01-1_amd64.deb <br></br>
+        sudo apt-key add /var/cuda-repo-ubuntu2004-11-4-local/7fa2af80.pub <br></br>
+        sudo apt-get -y update <br></br>
+        sudo apt-get -y install cuda
+      </code>
+    </pre>
+    <p>
+      <b>Create virtual environment (pipenv) </b>
+      <br></br>
+      The following command creates a virtual environment using pipenv and will install
+      the dependencies using pipenv.
+      <pre>
+        <code>make setup</code>
+      </pre>
+    </p>
+  </TabItem>
+  <TabItem value="using-requirements" label="Using requirements.txt">
+    <p>
+      If you wish to use conda or another virtual environment, you can also
+      install the dependencies using the <code>requirements.txt</code>{" "}
+      file.{" "}
+    </p>
+    <pre>
+      <code>pip install -r requirements.txt</code>
+    </pre>
+  </TabItem>
+  <TabItem value="wsl2-install" label="Windows installation" default>
+    <p>
+      Follow the steps below to install H2O LLM Studio on a Windows machine
+      using Windows Subsystem for Linux{" "}
+      <a href="https://learn.microsoft.com/en-us/windows/wsl/">WSL2</a>
+    </p>
+    <p>
+      1. Download the{" "}
+      <a href="https://www.nvidia.com/download/index.aspx">
+        latest nvidia driver
+      </a>{" "}
+      for Windows.{" "}
+    </p>
+    <p>
+      2. Open PowerShell or a Windows Command Prompt window in administrator
+      mode.{" "}
+    </p>
+    <p>
+      3. Run the following command to confirm that the driver is installed
+      properly and see the driver version.
+      <pre>
+        <code>nvidia-smi</code>
+      </pre>
+    </p>
+    <p>
+      4. Run the following command to install WSL2.
+      <pre>
+        <code>wsl --install</code>
+      </pre>
+    </p>
+    <p>5. Launch the WSL2 Ubuntu installation. </p>
+    <p>
+      6. Install the{" "}
+      <a href="https://developer.nvidia.com/cuda-downloads?target_os=Linux&target_arch=x86_64&Distribution=WSL-Ubuntu&target_version=2.0">
+        WSL2 Nvidia Cuda Drivers
+      </a>
+      .
+      <pre>
+        <code>
+          wget
+          https://developer.download.nvidia.com/compute/cuda/repos/wsl-ubuntu/x86_64/cuda-wsl-ubuntu.pin{" "}
+          <br></br>
+          sudo mv cuda-ubuntu2004.pin
+          /etc/apt/preferences.d/cuda-repository-pin-600 <br></br>
+          wget
+          https://developer.download.nvidia.com/compute/cuda/12.2.0/local_installers/cuda-repo-wsl-ubuntu-12-2-local_12.2.0-1_amd64.deb{" "}
+          <br></br>
+          sudo dpkg -i cuda-repo-wsl-ubuntu-12-2-local_12.2.0-1_amd64.deb <br></br>
+          sudo cp /var/cuda-repo-wsl-ubuntu-12-2-local/cuda-*-keyring.gpg
+          /usr/share/keyrings/ <br></br>
+          sudo apt-get update <br></br>
+          sudo apt-get -y install cuda
+        </code>
+      </pre>
+    </p>
+    <p>
+      7. Set up the required python system installs (Python 3.10).
+      <pre>
+        <code>
+          sudo add-apt-repository ppa:deadsnakes/ppa <br></br>
+          sudo apt install python3.10 <br></br>
+          sudo apt-get install python3.10-distutils <br></br>
+          curl -sS https://bootstrap.pypa.io/get-pip.py | python3.10
+        </code>
+      </pre>
+    </p>
+    <p>
+      8. Create the virtual environment.
+      <pre>
+        <code>
+          sudo apt install -y python3.10-venv<br></br>
+          python3 -m venv llmstudio<br></br>
+          source llmstudio/bin/activate<br></br>
+        </code>
+      </pre>
+    </p>
+    <p>
+      9.Clone the H2O LLM Studio repository locally.
+      <pre>
+        <code>
+          git clone https://github.com/h2oai/h2o-llmstudio.git<br></br>
+          cd h2o-llmstudio
+        </code>
+      </pre>
+    </p>
+    <p>
+      10. Install H2O LLM Studio using the `requirements.txt`.
+      <pre>
+        <code>pip install -r requirements.txt</code>
+      </pre>
+    </p>
+    <p>
+      11. Run the H2O LLM Studio application.
+      <pre>
+        <code>
+          H2O_WAVE_MAX_REQUEST_SIZE=25MB \ <br></br>
+          H2O_WAVE_NO_LOG=True \ <br></br>
+          H2O_WAVE_PRIVATE_DIR="/download/@output/download" \ <br></br>
+          wave run app
+        </code>
+      </pre>
+    </p>
+    <p>
+      This will start the H2O Wave server and the H2O LLM Studio app. Navigate
+      to <a>http://localhost:10101/</a> (we recommend using Chrome) to access
+      H2O LLM Studio and start fine-tuning your models.
+    </p>
+  </TabItem>
+</Tabs>
+:::
+
+## Install custom package
+
+If required, you can install additional Python packages into your environment. This can be done using pip after activating your virtual environment via `make shell`. For example, to install flash-attention, you would use the following commands:
+
+```bash
+make shell
+pip install flash-attn --no-build-isolation
+pip install git+https://github.com/HazyResearch/flash-attention.git#subdirectory=csrc/rotary
+```
+
+Alternatively, you can also directly install the custom package by running the following command.
+
+```bash
+pipenv install package_name
+```
+
+## Run H2O LLM Studio
+
+There are several ways to run H2O LLM Studio depending on your requirements.
+
+1. [Run H2O LLM Studio GUI](#run-h2o-llm-studio-gui)
+2. [Run using Docker from a nightly build](#run-using-docker-from-a-nightly-build)
+3. [Run by building your own Docker image](#run-by-building-your-own-docker-image)
+4. [Run with the CLI (command-line interface)](#run-with-command-line-interface-cli)
+
+### Run H2O LLM Studio GUI
+
+Run the following command to start the H2O LLM Studio.
+
+```sh
+make llmstudio
+```
+
+This will start the H2O Wave server and the H2O LLM Studio app. Navigate to [http://localhost:10101/](http://localhost:10101/) (we recommend using Chrome) to access H2O LLM Studio and start fine-tuning your models.
+
+![home-screen](llm-studio-home-screen.png)
+
+If you are running H2O LLM Studio with a custom environment other than Pipenv, start the app as follows:
+
+```sh
+H2O_WAVE_APP_ADDRESS=http://127.0.0.1:8756 \
+H2O_WAVE_MAX_REQUEST_SIZE=25MB \
+H2O_WAVE_NO_LOG=True \
+H2O_WAVE_PRIVATE_DIR="/download/@output/download" \
+wave run app
+```
+
+### Run using Docker from a nightly build
+
+First, install Docker by following the instructions from the [NVIDIA Container Installation Guide](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html#docker). H2O LLM Studio images are stored in the `h2oai GCR vorvan` container repository.
+
+```sh
+mkdir -p `pwd`/data
+mkdir -p `pwd`/output
+docker run \
+    --runtime=nvidia \
+    --shm-size=64g \
+    --init \
+    --rm \
+    -p 10101:10101 \
+    -v `pwd`/data:/workspace/data \
+    -v `pwd`/output:/workspace/output \
+    -v ~/.cache:/home/llmstudio/.cache \
+    gcr.io/vorvan/h2oai/h2o-llmstudio:nightly
+```
+
+Navigate to [http://localhost:10101/](http://localhost:10101/) (we recommend using Chrome) to access H2O LLM Studio and start fine-tuning your models.
+
+:::info
+Other helpful docker commands are `docker ps` and `docker kill`.
+:::
+
+### Run by building your own Docker image
+
+```sh
+docker build -t h2o-llmstudio .
+docker run \
+    --runtime=nvidia \
+    --shm-size=64g \
+    --init \
+    --rm \
+    -p 10101:10101 \
+    -v `pwd`/data:/workspace/data \
+    -v `pwd`/output:/workspace/output \
+    -v ~/.cache:/home/llmstudio/.cache \
+    h2o-llmstudio
+```
+
+### Run with command line interface (CLI)
+
+You can also use H2O LLM Studio with the command line interface (CLI) and specify the configuration .yaml file that contains all the experiment parameters. To finetune using H2O LLM Studio with CLI, activate the pipenv environment by running `make shell`.
+
+To specify the path to the configuration file that contains the experiment parameters, run:
+
+```sh
+python train.py -Y {path_to_config_yaml_file}
+```
+
+To run on multiple GPUs in DDP mode, run:
+
+```sh
+bash distributed_train.sh {NR_OF_GPUS} -Y {path_to_config_yaml_file}
+```
+
+:::info
+By default, the framework will run on the first `k` GPUs. If you want to specify specific GPUs to run on, use the `CUDA_VISIBLE_DEVICES` environment variable before the command.
+:::
+
+To start an interactive chat with your trained model, run:
+
+```sh
+python prompt.py -e {experiment_name}
+```
+
+`experiment_name` is the output folder of the experiment you want to chat with. The interactive chat will also work with models that were fine-tuned using the GUI.

documentation/docs/get-started/videos.md

@@ -0,0 +1,49 @@
+---
+description: Learn from a collection of videos about LLM Studio.
+---
+import ReactPlayer from 'react-player'
+
+
+# Videos
+
+## Discovering the Potential of LLMs
+
+<iframe width="930" height="515" src="https://www.youtube.com/embed/u48QaIAIFw4" title="Discovering the Potential of LLMs: A Journey through H2O.ai's LLM Studio!" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" allowfullscreen></iframe>
+
+
+:::info Note 
+  In this video, Andreea Turcu delves in-depth into the world of language models, showcasing how users can use H2O.ai's LLM Studio to their full advantage.
+:::
+
+---
+
+## The Fine Art of Fine-Tuning Large Language Models
+
+<iframe width="930" height="515" src="https://www.youtube.com/embed/YWAS3QDFg40" title="The Fine Art of Fine-Tuning LLMs" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" allowfullscreen></iframe>
+
+
+:::info Note 
+  In this video, Pascal Pfeiffer, Principal Data Scientist at H2O.ai and Kaggle Grandmaster, announces the release of H2O LLM Studio and talks about fine-tuning LLMs using H2O LLM Studio at H2O World India 2023. 
+:::
+
+---
+
+## Basic introduction to H2O LLM Studio 
+
+<iframe width="930" height="515" src="https://www.youtube.com/embed/aFU3VRGE2gk" title="Introduction to H2O LLM Studio" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" allowfullscreen></iframe>
+
+
+:::info Note 
+  In this video, Avkash Chauhan, founder of Prodramp Inc, gives a basic introduction about H2O LLM Studio. 
+:::
+
+----
+
+## LLM Fine-Tuning, Falcon 40b, and the State of Open-Source
+
+<iframe width="930" height="515" src="https://www.youtube.com/embed/Ur-1PI9SMfw" title="Pascal Pfeiffer - Kaggle, Fine-Tuning, H2O.ai, GPT4, Falcon 40b, Open Source" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" allowfullscreen></iframe>
+
+
+:::info Note 
+  In this video, Pascal Pfeiffer, the Principal Data Scientist at h2o.ai is interviewed about LLM fine-tuning, being a Kaggle Grandmaster, H2O.ai, Falcon 40b, the state of open-source, and more. 
+:::

documentation/docs/get-started/what-is-h2o-llm-studio.md

@@ -0,0 +1,16 @@
+---
+description: H2O LLM Studio is an open-source, no-code LLM graphical user interface (GUI) designed for fine-tuning state-of-the-art large language models. 
+---
+# What is H2O LLM Studio?
+
+H2O LLM Studio is an open-source, no-code [LLM](../concepts#llm) graphical user interface (GUI) designed for fine-tuning state-of-the-art large language models. 
+
+[Fine-tuning](../concepts#fine-tuning) a pretrained language model requires coding expertise and extensive knowledge about the model and its [hyperparameters](../concepts#parameters-and-hyperparameters), however  H2O LLM Studio enables NLP practioners to fine-tune their LLMs easily with no need for coding and better flexibility over customization. 
+
+H2O LLM Studio also lets you chat with the fine-tuned model and recieve instant feedback about model performance.
+
+## Who is H2O LLM Studio for?
+
+H2O LLM Studio is a free and open-source tool that is designed for anyone who wants to fine-tune their own language models. It is designed to be easy to use and accessible to everyone regardless of technical expertise.
+
+NLP practioners and data scientists in particular may find it useful to easily and effectively create and fine-tune large language models. 

documentation/docs/guide/datasets/data-connectors-format.md

@@ -0,0 +1,31 @@
+# Supported data connectors and format
+
+## Data connectors
+
+H2O LLM Studio supports the following data connectors to access or upload external data sources.
+
+- **Upload**: Upload a local dataset from your machine. 
+- **Local**: Specify the file location of the dataset on your machine. 
+- **AWS S3 (Amazon AWS S3)**: Connect to an Amazon AWS S3 data bucket. 
+- **Kaggle**: Connect to a Kaggle dataset. 
+
+## Data format 
+
+- Each data connector requires either a single `.csv` or `.pq` file, or the data to be in a `.zip` file for a successful import. 
+
+- H2O LLM studio requires a `.csv` file with a minimum of two columns, where one contains the instructions and the other has the model’s expected output. You can also include an additional validation dataframe in the same format or allow for an automatic train/validation split to assess the model’s performance.
+
+- Optionally, a **Parent Id** can be used for training nested data prompts that are linked to a parent question. 
+
+- During an experiment you can adapt the data representation with the following settings:
+    - **Prompt Column:** The column in the dataset containing the user prompt.
+    - **Answer Column:** The column in the dataset containing the expected output.
+    - **Parent Id Column:** An optional column specifying the parent id to be used for chained conversations. The value of this column needs to match an additional column with the name `id`. If provided, the prompt will be concatenated after preceeding parent rows.
+
+:::info
+To train a chatbot style model, you need to convert your data into a question and answer format. There are other enterprise solutions by H2O.ai that may help you prep your data. For more information, see [H2O.ai's Generative AI page](https://h2o.ai/) and this blogpost about [H2O LLM DataStudio: Streamlining Data Curation and Data Preparation for LLMs related tasks](https://blog.h2o.ai/blog/streamlining-data-preparation-for-fine-tuning-of-large-language-models/).
+
+## Example data
+
+H2O LLM Studio provides a sample dataset (converted dataset from [OpenAssistant/oasst2](https://huggingface.co/datasets/OpenAssistant/oasst2))
+that can be downloaded [here](https://www.kaggle.com/code/philippsinger/openassistant-conversations-dataset-oasst2?scriptVersionId=160485459). It is recommended to use `train_full.csv` for training. This dataset is also downloaded and prepared by default when first starting the GUI. Multiple dataframes can be uploaded into a single dataset by uploading a `.zip` archive.

documentation/docs/guide/datasets/import-dataset.md

@@ -0,0 +1,148 @@
+---
+description: H2O LLM Studio provides a number of data connectors to support importing data from local or external sources and requires your data to be in a certain format for successful importing of data.
+---
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+import Admonition from '@theme/Admonition';
+import upload_dataset from './upload-dataset.png';
+import upload_local_file from './upload-local-file.png';
+import import_s3_bucket from './import-s3-bucket.png';
+import import_kaggle_dataset from './import-kaggle-dataset.png';
+import TrainDataframeTooltip from '../../tooltips/experiments/_train-dataframe.mdx';
+import ValidationDataframeTooltip from '../../tooltips/experiments/_validation-dataframe.mdx';
+import PromptColumnTooltip from '../../tooltips/experiments/_prompt-column.mdx';
+import AnswerColumnTooltip from '../../tooltips/experiments/_answer-column.mdx';
+import ParentIdColumnTooltip from '../../tooltips/experiments/_parent-id-column.mdx';
+
+# Import a dataset
+
+H2O LLM Studio provides a number of data connectors to support importing data from local or external sources and requires your data to be in a certain format for successful importing of data. 
+
+For more information, see [Supported data connectors and format](data-connectors-format).
+
+## Import data
+
+Follow the relevant steps below to import a dataset to H2O LLM Studio.
+
+1. On the H2O LLM Studio left-navigation pane, click **Import dataset**.
+2. Select the relevant **Source** (data connector) that you want to use from the dropdown list .
+    :::note Data sources
+    <Tabs className="unique-tabs">
+    <TabItem value="upload" label="Upload" default>
+        <ol>
+        <li>
+        Drag and drop the file, or click <b>Browse</b> and select the file you want to upload.
+        </li>
+        <li>
+        Click <b>Upload</b>.
+        <img src={upload_dataset} alt="upload-dataset" />
+        </li>
+        </ol>
+    </TabItem>
+    <TabItem value="local" label="Local">
+        <ol>
+        <li>
+        Enter the file path as the <b>File Location</b> or select the relevant local directory that the dataset is located in. 
+        </li>
+        <li>
+        Click <b>Continue</b>.
+        <img src={upload_local_file} alt="upload-local-file" />
+        </li>
+        </ol>
+    </TabItem>
+    <TabItem value="aws" label="AWS S3">
+        <ol>
+        <li>
+        Enter values for the following fields:
+            <ul>
+            <li>
+            <b>S3 bucket name: </b> <br></br>
+            The name of the S3 bucket including the reletive file paths.
+            </li>
+            <li>
+            <b>AWS access key: </b><br></br>
+            The access key associated with your S3 bucket. This field is optional. If the S3 bucket is public, you can leave this empty for anonymous access. 
+            </li>
+            <li>
+            <b>AWS access secret: </b><br></br>
+            The access secret associated with your S3 bucket. This field is optional. If the S3 bucket is public, you can leave this empty for anonymous access.
+            </li>
+            <li>
+            <b>File name: </b><br></br>
+            Enter the file name of the dataset that you want to import. 
+            </li>
+            </ul>
+            <div>
+            <Admonition type="info" title="Note">
+                <p>For more information, see <a href="https://docs.aws.amazon.com/IAM/latest/UserGuide/security-creds.html#access-keys-and-secret-access-keys">AWS credentials</a> and <a href="https://docs.aws.amazon.com/AmazonS3/latest/userguide/access-bucket-intro.html">Methods for accessing a bucket</a> in the AWS Documentation.</p>
+            </Admonition>
+            </div>
+        </li>
+        <li>
+        Click <b>Continue</b>.
+        <img src={import_s3_bucket} alt="import-s3-bucket" />
+        </li>
+        </ol>
+    </TabItem>
+    <TabItem value="kaggle" label="Kaggle">
+        <ol>
+        <li>
+        Enter values for the following fields:
+            <ul>
+            <li>
+            <b>Kaggle API command: </b><br></br>
+            Enter the Kaggle API command that you want to execute.
+            </li>
+            <li>
+            <b>Kaggle username: </b><br></br>
+            Your Kaggle username for API authentication
+            </li>
+            <li>
+            <b>Kaggle secret key: </b><br></br>
+            Your Kaggle secret key for API authentication.
+            </li>
+            </ul>
+        </li>
+        <li>
+        Click <b>Continue</b>.
+        <img src={import_kaggle_dataset} alt="import-kaggle-dataset" />
+        </li>
+        </ol>
+    </TabItem>
+    </Tabs>
+    :::
+ 
+## Configure dataset
+
+Once you have successfully uploaded or imported your dataset, you can configure the dataset settings.
+
+:::info Tip
+You can upload a `.zip` file with both training and validation sets to avoid having to separately upload files.
+:::
+
+- **Dataset name:** <br/>
+    A suitable name for the whole dataset which includes both the train dataframe and validation dataframe.
+
+- **Train Dataframe:** <TrainDataframeTooltip />
+
+- **Validation Dataframe:** <ValidationDataframeTooltip />
+
+- **Prompt Column:** <PromptColumnTooltip />
+
+- **Answer Column:** <AnswerColumnTooltip />
+
+- **Parent Id Column:** <ParentIdColumnTooltip />
+
+![configure-dataset](configure-dataset.png)
+
+## Data validity check
+
+H2O LLM Studio will provide a preview of the dataset input (sample questions) and output (sample answers) according to the content of the imported dataset. Review the text to ensure that the input and output is as intended, and then click **Continue**. 
+
+## View dataset
+
+You will now be redirected to the **View datasets** screen. You should be able to see the dataset you just imported listed on the screen. 
+
+![view-dataset](view-imported-dataset.png)
+
+For more information about viewing dataset summary and statistics, see [View and manage datasets](view-dataset)

documentation/docs/guide/datasets/merge-datasets.md

@@ -0,0 +1,34 @@
+---
+description: H2O LLM Studio enables you to merge imported datasets into one main dataset. This functionality can be used to merge training and validation data together into one dataset or extend your existing dataset with more data and increase your dataset size. 
+---
+import Icon from "@material-ui/core/Icon";
+
+# Merge datasets
+
+H2O LLM Studio enables you to merge imported datasets into one main dataset. This functionality can be used to merge training and validation data together into one dataset or extend your existing dataset with more data and increase your dataset size. 
+
+:::info
+H2O LLM Studio does not merge dataset files in the sense that rows are combined, and duplicate rows are removed. "Merge", in this case, refers to bringing the dataset files a dataset might have to a single dataset (another dataset), continuing other dataset files already.
+:::
+
+Generally, you might want to merge datasets in H2O LLM Studio to have both the training data .csv and validation data .csv in one final dataset. 
+
+1. On the H2O LLM Studio left-navigation pane, click **View datasets**.  
+2. Click the <Icon>more_vert</Icon> Kebab menu of the dataset you want to merge with. 
+3. Click **Edit dataset**. 
+4. Click **Merge with existing dataset**.
+5. Select the dataset you want that you want to merge with. 
+    ![merge-datasets](merge-datasets.png)
+6. Click **Merge**.
+7. Adjust the dataset configuration if needed. For more information about the configurations, see [Configure dataset](./import-dataset#configure-dataset). 
+8. Click **Continue**.
+9. Review the text to ensure that the input and output is as intended, and then click **Continue**.
+
+Your datasets are now merged. 
+
+:::info
+Alternatively, you can also merge datasets at the point of [importing a dataset](./import-dataset) or combine both datasets (.csv files) into a `.zip` file before uploading it as a whole dataset. 
+:::
+
+
+

documentation/docs/guide/datasets/view-dataset.md

@@ -0,0 +1,74 @@
+---
+description: You can view, review, edit, or delete your datasets once you have imported them. You can also start a new experiment using a dataset you have imported.
+---
+import Icon from "@material-ui/core/Icon";
+
+# View and manage dataset
+
+You can view, review, edit, or delete your datasets once you have imported them. You can also start a new experiment using a dataset you have imported. 
+
+## View a dataset
+
+To view an imported dataset:
+
+1. On the H2O LLM Studio left-navigation pane, click **View datasets**.
+
+2. You will see the datasets table with a list of all the datasets you have imported so far. Click the name of the dataset that you want to view.
+
+    ![view-datasets](view-imported-dataset.png)
+
+    :::info
+    For more information about the dataset details you see on the table above, see [dataset configurations](import-dataset.md#configure-a-dataset).
+    :::
+
+## Dataset tabs
+
+You will see the following tabs that provide details and different aspects of your dataset.
+
+- **Sample train data** : This tab contains sample training data from the imported dataset.
+
+- **Sample train visualization:** This tab visualizes a few sample training data from the imported dataset in a question-answer format; simulating the way the chatbot would answer questions based on the training data. 
+
+- **Train data statistics:** This tab contains metrics about the training data (e.g., unique values) from the imported dataset.
+
+- **Summary:** This tab contains the following details about the dataset. 
+
+    | Name      | Description                          |
+    | ----------- | ------------------------------------ |
+    | **Name**        | Name of the dataset.  |
+    | **Problem type**        | Problem type of the dataset. |
+    | **Train dataframe**   | Name of the training dataframe in the imported dataset. An imported dataset can contain train, test, and validation dataframes.  |
+    | **Train rows**       | The number of rows the train dataframe contains.  |
+    | **Validation dataframe**       | Name of the validation dataframe in the imported dataset. An imported dataset can contain train, test, and validation dataframes.  |
+    | **Validation rows**         | The number of rows the validation dataframe contains. |
+    | **Labels**       | The labels the imported dataset contains.  |
+
+
+## Edit a dataset
+
+To edit an imported dataset,
+
+1. On the H2O LLM Studio left-navigation pane, click **View datasets**. You will see the datasets table with a list of all the datasets you have imported so far.
+2. Locate the row of the dataset you want to edit and click the <Icon>more_vert</Icon> Kebab menu.
+3. Select **Edit dataset**.
+4. Make the desired changes to the dataset configuration. You can also [merge the dataset with an existing dataset](merge-datasets) at this point.
+5. Click **Continue** and review the dataset with your changes. 
+
+<!-- 
+## Start a new experiment
+
+
+link to start a new experiment page in the experiments sub page.  -->
+
+## Delete a dataset
+
+When a dataset is no longer needed, you can delete it. Deleted datasets are permanently removed from the H2O LLM Studio instance.
+
+:::caution
+You can only delete datasets that are not linked to any experiments. If you wish to delete a dataset that is linked to an experiment, first [delete the experiment](../experiments/view-an-experiment#delete-an-experiment), and then delete the dataset. 
+:::
+
+1. On the H2O LLM Studio left-navigation pane, click **View datasets**.
+2. Click **Delete datasets**.
+3. Select the dataset(s) that you want to delete.
+4. Click **Delete** to confirm deletion.

documentation/docs/guide/experiments/compare-experiments.md

@@ -0,0 +1,21 @@
+---
+description: The H2O LLM studio provides a useful feature to compare experiments which allow comparing multiple experiments and analyzing how different model parameters affect model performance. 
+---
+# Compare experiments
+
+The H2O LLM studio provides a useful feature to compare experiments which allow comparing multiple experiments and analyzing how different model parameters affect model performance. 
+
+Follow the relevant steps below to compare experiments in H2O LLM Studio.
+
+1. On the H2O LLM Studio left-navigation pane, click **View experiments**.
+2. Click **Compare experiments**.
+3. Select the experiments you want to compare.
+4. Click **Compare experiments**.
+
+    ![compare experiments](compare-experiments.png)
+
+    The **Charts** tab visually represents the comparison of train/validation loss, metrics, and learning rate of selected experiments. The **Config** tab compares the configuration settings of selected experiments.  
+
+:::info note
+In addition, H2O LLM Studio also integrates with [Neptune](https://neptune.ai/), a powerful experiment tracking platform. By enabling Neptune logging when starting an experiment, you can easily track and visualize all aspects of your experiment in real time. This includes model performance, hyperparameter tuning, and other relevant metrics.
+:::