ChatDEIP

ChatDEIP is a conversational AI front end portal for Edge Infrastructure as a Service platform whose software architecture comprises of three main components, namely: Platform manager and certain telemetry components that runs on the cluster, Intel Backend and the Platform Director. This interactive interface helps you to navigate the platform and provides you with answers to questions about the platform.

We utilized a pre-trained LLaMa model which is a 7B-parameter model and Finetuned it using the Stanford Alpaca Self Instruct Methodology. The model was fine-tuned on our domain specific dataset obtained from the IaaS plaform for 3 tasks:

• Causal LM
• Function Calling
• Flux Query Generation

Training procedure

The following bitsandbytes quantization config was used during training:

• load_in_8bit: False
• load_in_4bit: True
• llm_int8_threshold: 6.0
• llm_int8_skip_modules: None
• llm_int8_enable_fp32_cpu_offload: False
• llm_int8_has_fp16_weight: False
• bnb_4bit_quant_type: nf4
• bnb_4bit_use_double_quant: False
• bnb_4bit_compute_dtype: float16

Framework versions

• PEFT 0.4.0

ChatDEIP is a Natural language interface for EdgeIaaS infrastructure. this was built on a methodology similar to Alpaca Lora's self instruct paper, first we collated a dataset of flux Queries Generated with Heuristics then used GPT3.5 to generate a matching user conversation structure to match the Generated Queries. we repeated the same process for the function calling Dataset and then trained a LLAMA v1 using the low-rank adaptation (LoRA), which freezes the pretrained model weights and injects trainable rank decomposition matrices into each layer of the Transformer architecture, greatly reducing the number of trainable parameters for downstream tasks. For fine-tuning, the Hugging Face's PEFT and the Tim Dettmers' bitsandbytes was used.

Documentation

• Quickstart
• User Guide
• API Reference
• Examples

Pre-requisite

• Operating System

  • This set up was done using Linux Ubuntu v22.04

• Programming Language

  • Python version >=3.9

• Hardware used

  • Single A100 50GB GPU with 83GB of CPU RAM

• Documentation: https://iaas-platform-conversational-ai.readthedocs.io.

Quick Install

We recommend to first setup a clean Python environment for your project with Python 3.7+ using your favorite tool (conda, venv, virtualenv with or without virtualenvwrapper).

Once your environment is set up you can move to set up:

How To Set up

Clone the repository accordingly from your terminal whilst in your home directory. From your command line:

# Clone this repository
$ git clone https://github.com/VoidZeroe/ChatDEIP.git

# Change directory 
$ cd ChatDEIP

# Install dependencies
$ pip3 install -r requirements.txt
 
# To run the app demo, on your current terminal:
$ python3 main.py

If bitsandbytes doesn't work, install it from source.

*Note This might take a couple of minutes to run depending on your hardware (CPU or GPU). Once complete, the app should be up and running on your browser on the default port at: http://localhost:7861/ or http://localhost:7860/. Feel free to specify or change to your port of preference.

Directory Structure

├── LICENSE
├── Makefile           <- Makefile with commands like `make data` or `make train`
├── README.md          <- The top-level README for developers using this project.
├── data
│   ├── external       <- Data from third party sources.
│   ├── interim        <- Intermediate data that has been transformed.
│   ├── processed      <- The final, canonical data sets for modeling.
│   └── raw            <- The original, immutable data dump.
│
├── docs               <- A default Sphinx project; see sphinx-doc.org for details
│
├── models             <- Trained and serialized models, model predictions, or model summaries
│
├── notebooks          <- Jupyter notebooks. Naming convention is a number (for ordering),
│                         the creator's initials, and a short `-` delimited description, e.g.
│                         `1.0-jqp-initial-data-exploration`.
|
├── scripts           <-  Scripts used for data extraction and conversion.
│
├── references         <- Data dictionaries, manuals, and all other explanatory materials.
│
├── reports            <- Generated analysis as HTML, PDF, LaTeX, etc.
│   └── figures        <- Generated graphics and figures to be used in reporting
│
├── requirements.txt   <- The requirements file for reproducing the analysis environment, e.g.
│                         generated with `pip freeze > requirements.txt`
│
├── setup.py           <- makes project pip installable (pip install -e .) so src can be imported
├── src                <- Source code for use in this project.
│   ├── __init__.py    <- Makes src a Python module
│   │
│   ├── data           <- Scripts to download or generate data
│   │   └── make_dataset.py
│   │
│   ├── features       <- Scripts to turn raw data into features for modeling
│   │   └── build_features.py
│   │
│   ├── models         <- Scripts to train models and then use trained models to make
│   │   │                 predictions
│   │   ├── predict_model.py
│   │   └── train_model.py
│   │
│   └── visualization  <- Scripts to create exploratory and results oriented visualizations
│       └── visualize.py
│
└── tox.ini            <- tox file with settings for running tox; see tox.readthedocs.io

---