--- license: mit datasets: - keivalya/MedQuad-MedicalQnADataset language: - en library_name: peft tags: - medical pipeline_tag: question-answering --- # Model Card for GaiaMiniMed This is a medical fine tuned model from the [Falcon-7b-Instruction](https://huggingface.co/tiiuae/falcon-7b-instruct) Base using 500 steps & 6 epochs with [MedAware](https://huggingface.co/datasets/keivalya/MedQuad-MedicalQnADataset) Dataset from [keivalya](https://huggingface.co/datasets/keivalya) Check out a cool demo with chat memory here : [pseudolab/GaiaFalconChat](https://huggingface.co/spaces/pseudolab/GaiaMiniMed_ChatWithFalcon) ## Model Details ### Model Description - **Developed by:** [Tonic](https://www.huggingface.co/tonic) - **Shared by :** [Tonic](https://www.huggingface.co/tonic) - **Model type:** Medical Fine-Tuned Conversational Falcon 7b (Instruct) - **Language(s) (NLP):** English - **License:** MIT - **Finetuned from model:**[tiiuae/falcon-7b-instruct](https://huggingface.co/tiiuae/falcon-7b-instruct) - ### Model Sources - **Repository:** [Github](https://github.com/Josephrp/AI-challenge-hackathon/blob/master/falcon_7b_instruct_GaiaMiniMed_dataset.ipynb) - **Demo :** [pseudolab/gaiafalconchat](https://huggingface.co/spaces/pseudolab/GaiaMiniMed_ChatWithFalcon)[pseudolab/gaiaminimed](https://huggingface.co/spaces/pseudolab/gaiaminimed) & [tonic/gaiaminimed](https://huggingface.com/spaces/tonic/gaiaminimed) ## Uses Use this model like you would use Falcon Instruct Models ### Direct Use This model is intended for educational purposes only , always consult a doctor for the best advice. This model should perform better at medical QnA tasks in a conversational manner. It is our hope that it will help improve patient outcomes and public health. ### Downstream Use Use this model next to others and have group conversations to produce diagnoses , public health advisory , and personal hygene improvements. ### Out-of-Scope Use This model is not meant as a decision support system in the wild, only for educational use. ## Bias, Risks, and Limitations {{ bias_risks_limitations | default("[More Information Needed]", true)}} ## How to Get Started with the Model - Try it here : [Pseudolab/GaiaMiniMed](https://huggingface.co/spaces/pseudolab/GaiaMiniMed) - See the [author's demo](https://huggingface.co/spaces/tonic/gaiaminimed) - Use the code below to get started with the model. ```python # Gaia MiniMed ⚕️🦅 Quick Start from transformers import AutoConfig, AutoTokenizer, AutoModelForSeq2SeqLM, AutoModelForCausalLM, MistralForCausalLM from peft import PeftModel, PeftConfig import torch import gradio as gr import random from textwrap import wrap def wrap_text(text, width=90): lines = text.split('\n') wrapped_lines = [textwrap.fill(line, width=width) for line in lines] wrapped_text = '\n'.join(wrapped_lines) return wrapped_text def multimodal_prompt(user_input, system_prompt): formatted_input = f"{{{{ {system_prompt} }}}}\nUser: {user_input}\nFalcon:" encodeds = tokenizer(formatted_input, return_tensors="pt", add_special_tokens=False) model_inputs = encodeds.to(device) output = peft_model.generate( **model_inputs, max_length=500, use_cache=True, early_stopping=False, bos_token_id=peft_model.config.bos_token_id, eos_token_id=peft_model.config.eos_token_id, pad_token_id=peft_model.config.eos_token_id, temperature=0.4, do_sample=True ) response_text = tokenizer.decode(output[0], skip_special_tokens=True) return response_text device = "cuda" if torch.cuda.is_available() else "cpu" base_model_id = "tiiuae/falcon-7b-instruct" model_directory = "Tonic/GaiaMiniMed" tokenizer = AutoTokenizer.from_pretrained(base_model_id, trust_remote_code=True, padding_side="left") model_config = AutoConfig.from_pretrained(base_model_id) peft_model = AutoModelForCausalLM.from_pretrained(model_directory, config=model_config) peft_model = PeftModel.from_pretrained(peft_model, model_directory) class ChatBot: def __init__(self, system_prompt="You are an expert medical analyst:"): self.system_prompt = system_prompt self.history = [] def predict(self, user_input, system_prompt): formatted_input = f"{{{{ {self.system_prompt} }}}}\nUser: {user_input}\nFalcon:" input_ids = tokenizer.encode(formatted_input, return_tensors="pt", add_special_tokens=False) response = peft_model.generate(input_ids=input_ids, max_length=900, use_cache=False, early_stopping=False, bos_token_id=peft_model.config.bos_token_id, eos_token_id=peft_model.config.eos_token_id, pad_token_id=peft_model.config.eos_token_id, temperature=0.4, do_sample=True) response_text = tokenizer.decode(response[0], skip_special_tokens=True) self.history.append(formatted_input) self.history.append(response_text) return response_text bot = ChatBot() title = "👋🏻Welcome to Tonic's GaiaMiniMed Chat🚀" description = "You can use this Space to test out the current model [(Tonic/GaiaMiniMed)](https://huggingface.co/Tonic/GaiaMiniMed) or duplicate this Space and use it locally or on 🤗HuggingFace. [Join me on Discord to build together](https://discord.gg/VqTxc76K3u)." examples = [["What is the proper treatment for buccal herpes?", "You are a medicine and public health expert, you will receive a question, answer the question, and provide a complete answer"]] iface = gr.Interface( fn=bot.predict, title=title, description=description, examples=examples, inputs=["text", "text"], outputs="text", theme="ParityError/Anime" ) iface.launch() ``` - See the code below for more advanced deployment , including a naive memory store and user controllable parameters: ```Python # Gaia MiniMed⚕️🦅Falcon Chat from transformers import AutoConfig, AutoTokenizer, AutoModelForCausalLM from peft import PeftModel, PeftConfig import torch import gradio as gr import json import os import shutil import requests # Define the device device = "cuda" if torch.cuda.is_available() else "cpu" #Define variables temperature=0.4 max_new_tokens=240 top_p=0.92 repetition_penalty=1.7 max_length=2048 # Use model IDs as variables base_model_id = "tiiuae/falcon-7b-instruct" model_directory = "Tonic/GaiaMiniMed" # Instantiate the Tokenizer tokenizer = AutoTokenizer.from_pretrained(base_model_id, trust_remote_code=True, padding_side="left") tokenizer.pad_token = tokenizer.eos_token tokenizer.padding_side = 'left' # Load the GaiaMiniMed model with the specified configuration # Load the Peft model with a specific configuration # Specify the configuration class for the model model_config = AutoConfig.from_pretrained(base_model_id) # Load the PEFT model with the specified configuration peft_model = AutoModelForCausalLM.from_pretrained(model_directory, config=model_config) peft_model = PeftModel.from_pretrained(peft_model, model_directory) # Class to encapsulate the Falcon chatbot class FalconChatBot: def __init__(self, system_prompt="You are an expert medical analyst:"): self.system_prompt = system_prompt def process_history(self, history): if history is None: return [] # Ensure that history is a list of dictionaries if not isinstance(history, list): return [] # Filter out special commands from the history filtered_history = [] for message in history: if isinstance(message, dict): user_message = message.get("user", "") assistant_message = message.get("assistant", "") # Check if the user_message is not a special command if not user_message.startswith("Falcon:"): filtered_history.append({"user": user_message, "assistant": assistant_message}) return filtered_history def predict(self, user_message, assistant_message, history, temperature=0.4, max_new_tokens=700, top_p=0.99, repetition_penalty=1.9): # Process the history to remove special commands processed_history = self.process_history(history) # Combine the user and assistant messages into a conversation conversation = f"{self.system_prompt}\nFalcon: {assistant_message if assistant_message else ''} User: {user_message}\nFalcon:\n" # Encode the conversation using the tokenizer input_ids = tokenizer.encode(conversation, return_tensors="pt", add_special_tokens=False) # Generate a response using the Falcon model response = peft_model.generate(input_ids=input_ids, max_length=max_length, use_cache=False, early_stopping=False, bos_token_id=peft_model.config.bos_token_id, eos_token_id=peft_model.config.eos_token_id, pad_token_id=peft_model.config.eos_token_id, temperature=0.4, do_sample=True) # Decode the generated response to text response_text = tokenizer.decode(response[0], skip_special_tokens=True) # Append the Falcon-like conversation to the history self.history.append(conversation) self.history.append(response_text) return response_text # Create the Falcon chatbot instance falcon_bot = FalconChatBot() # Define the Gradio interface title = "👋🏻Welcome to Tonic's 🦅Falcon's Medical👨🏻‍⚕️Expert Chat🚀" description = "You can use this Space to test out the GaiaMiniMed model [(Tonic/GaiaMiniMed)](https://huggingface.co/Tonic/GaiaMiniMed) or duplicate this Space and use it locally or on 🤗HuggingFace. [Join me on Discord to build together](https://discord.gg/VqTxc76K3u). Please be patient as we " history = [ {"user": "hi there how can you help me?", "assistant": "Hello, my name is Gaia, i'm created by Tonic, i can answer questions about medicine and public health!"}, # Add more user and assistant messages as needed ] examples = [ [ { "user_message": "What is the proper treatment for buccal herpes?", "assistant_message": "My name is Gaia, I'm a health and sanitation expert ready to answer your medical questions.", "history": [], "temperature": 0.4, "max_new_tokens": 700, "top_p": 0.90, "repetition_penalty": 1.9, } ] ] additional_inputs=[ gr.Textbox("", label="Optional system prompt"), gr.Slider( label="Temperature", value=0.9, minimum=0.0, maximum=1.0, step=0.05, interactive=True, info="Higher values produce more diverse outputs", ), gr.Slider( label="Max new tokens", value=256, minimum=0, maximum=3000, step=64, interactive=True, info="The maximum numbers of new tokens", ), gr.Slider( label="Top-p (nucleus sampling)", value=0.90, minimum=0.01, maximum=0.99, step=0.05, interactive=True, info="Higher values sample more low-probability tokens", ), gr.Slider( label="Repetition penalty", value=1.2, minimum=1.0, maximum=2.0, step=0.05, interactive=True, info="Penalize repeated tokens", ) ] iface = gr.Interface( fn=falcon_bot.predict, title=title, description=description, examples=examples, inputs=[ gr.inputs.Textbox(label="Input Parameters", type="text", lines=5), ] + additional_inputs, outputs="text", theme="ParityError/Anime" ) # Launch the Gradio interface for the Falcon model iface.launch() ``` ## Training Details ### Results ![image/png](https://cdn-uploads.huggingface.co/production/uploads/62a3bb1cd0d8c2c2169f0b88/F8GfMSJcAaH7pXvpUK_r3.png) ```json TrainOutput(global_step=6150, training_loss=1.0597990553941183, {'epoch': 6.0}) ``` ### Training Data ```json DatasetDict({ train: Dataset({ features: ['qtype', 'Question', 'Answer'], num_rows: 16407 }) }) ``` ### Training Procedure #### Preprocessing [optional] ``` trainable params: 4718592 || all params: 3613463424 || trainables%: 0.13058363808693696 ``` #### Training Hyperparameters - **Training regime:** {{ training_regime | default("[More Information Needed]", true)}} #### Speeds, Sizes, Times [optional] ```json metrics={'train_runtime': 30766.4612, 'train_samples_per_second': 3.2, 'train_steps_per_second': 0.2, 'total_flos': 1.1252790565109983e+18, 'train_loss': 1.0597990553941183,", true)}} ``` ## Environmental Impact Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700). - **Hardware Type:** {{ hardware | default("[More Information Needed]", true)}} - **Hours used:** {{ hours_used | default("[More Information Needed]", true)}} - **Cloud Provider:** {{ cloud_provider | default("[More Information Needed]", true)}} - **Compute Region:** {{ cloud_region | default("[More Information Needed]", true)}} - **Carbon Emitted:** {{ co2_emitted | default("[More Information Needed]", true)}} ## Technical Specifications ### Model Architecture and Objective ```json PeftModelForCausalLM( (base_model): LoraModel( (model): FalconForCausalLM( (transformer): FalconModel( (word_embeddings): Embedding(65024, 4544) (h): ModuleList( (0-31): 32 x FalconDecoderLayer( (self_attention): FalconAttention( (maybe_rotary): FalconRotaryEmbedding() (query_key_value): Linear4bit( in_features=4544, out_features=4672, bias=False (lora_dropout): ModuleDict( (default): Dropout(p=0.05, inplace=False) ) (lora_A): ModuleDict( (default): Linear(in_features=4544, out_features=16, bias=False) ) (lora_B): ModuleDict( (default): Linear(in_features=16, out_features=4672, bias=False) ) (lora_embedding_A): ParameterDict() (lora_embedding_B): ParameterDict() ) (dense): Linear4bit(in_features=4544, out_features=4544, bias=False) (attention_dropout): Dropout(p=0.0, inplace=False) ) (mlp): FalconMLP( (dense_h_to_4h): Linear4bit(in_features=4544, out_features=18176, bias=False) (act): GELU(approximate='none') (dense_4h_to_h): Linear4bit(in_features=18176, out_features=4544, bias=False) ) (input_layernorm): LayerNorm((4544,), eps=1e-05, elementwise_affine=True) ) ) (ln_f): LayerNorm((4544,), eps=1e-05, elementwise_affine=True) ) (lm_head): Linear(in_features=4544, out_features=65024, bias=False) ) ) ) ``` ### Compute Infrastructure Google Collaboratory #### Hardware A100 ## Model Card Authors [Tonic](https://huggingface.co/tonic) ## Model Card Contact "[Tonic](https://huggingface.co/tonic)