Radiantloom
/

radiantloom-llama-70b-instruct

@@ -1,201 +1,151 @@
 ---
 library_name: transformers
-tags: []
 ---
-# Model Card for Model ID
-<!-- Provide a quick summary of what the model is/does. -->
-## Model Details
-### Model Description
-<!-- Provide a longer summary of what this model is. -->
-This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-### Model Sources [optional]
-<!-- Provide the basic links for the model. -->
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-## Uses
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-### Direct Use
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-[More Information Needed]
-### Downstream Use [optional]
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-[More Information Needed]
-### Out-of-Scope Use
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-[More Information Needed]
-## Bias, Risks, and Limitations
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-[More Information Needed]
-### Recommendations
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-## How to Get Started with the Model
-Use the code below to get started with the model.
-[More Information Needed]
-## Training Details
-### Training Data
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-[More Information Needed]
-### Training Procedure
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-#### Preprocessing [optional]
-[More Information Needed]
-#### Training Hyperparameters
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-#### Speeds, Sizes, Times [optional]
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-[More Information Needed]
-## Evaluation
-<!-- This section describes the evaluation protocols and provides the results. -->
-### Testing Data, Factors & Metrics
-#### Testing Data
-<!-- This should link to a Dataset Card if possible. -->
-[More Information Needed]
-#### Factors
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-[More Information Needed]
-#### Metrics
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-[More Information Needed]
-### Results
-[More Information Needed]
-#### Summary
-## Model Examination [optional]
-<!-- Relevant interpretability work for the model goes here -->
-[More Information Needed]
-## Environmental Impact
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-## Technical Specifications [optional]
-### Model Architecture and Objective
-[More Information Needed]
-### Compute Infrastructure
-[More Information Needed]
-#### Hardware
-[More Information Needed]
-#### Software
-[More Information Needed]
-## Citation [optional]
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-**BibTeX:**
-[More Information Needed]
-**APA:**
-[More Information Needed]
-## Glossary [optional]
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-[More Information Needed]
-## More Information [optional]
-[More Information Needed]
-## Model Card Authors [optional]
-[More Information Needed]
-## Model Card Contact
-[More Information Needed]

 ---
 library_name: transformers
+tags:
+- SFT
+- Llama
+- Llama 70B Chat
+license: llama2
 ---
+<img src="https://huggingface.co/Radiantloom/radiantloom-llama-70b-instruct/resolve/main/radiantloom-llama-70b-instruct.jpeg" alt="Radiantloom Llama 70B Instruct" width="800" style="margin-left:'auto' margin-right:'auto' display:'block'"/>
+## Radiantloom Llama 70B Instruct
+The Radiantloom Llama 70B Instruct, a large language model (LLM) developed by Radiantloom AI, features approximately 70 billion parameters. From vibes-check evaluations, the Radiantloom Llama 70B Instruct demonstrates great performance in various applications like creative writing, multi-turn conversations, in-context learning through Retrieval Augmented Generation (RAG), and coding tasks. Its out-of-the-box performance already delivers impressive results, particularly in writing, coding, and reasoning tasks. This model produces longer form content and provides detailed explanations of its actions. To maximize its potential, consider implementing instruction tuning and Reinforcement Learning with Human Feedback (RLHF) techniques for further refinement. Alternatively, you can utilize it in its current form.
+## Intended Uses & Limitations
+The Radiantloom Llama 70B Instruct is versatile and can be utilized for various text generation tasks such as summarization, chat, coding, question answering, retrieval augmented generation (RAG), role play, and content generation.
+It can be considered a state-of-the-art generative language model, it demonstrates competitive performance in general tasks when compared to other open and closed-source large language models such as GPT 3.5 Turbo, and Mixtral 8x7B Instruct.
+## Model Usage
+You can deploy it on a Runpod GPU instance using our [one-click Runpod template](https://www.runpod.io/console/gpu-secure-cloud?ref=80eh3891&template=tbkdia6qdv) (Our Referral Link. Please consider Supporting). This template provides you with an OpenAI-compatible API endpoint that you can integrate into your existing codebase designed for OpenAI APIs.
+## Prompt Template
+We have fine-tuned this model using the ChatML format, and you can achieve optimal performance by utilizing the ChatML format.
+```
+<s>[INST] <<SYS>>{system_message}<</SYS>> {user_message}[/INST]
+```
+## Inference Code
+```python
+## install dependencies
+## !pip install -q -U git+https://github.com/huggingface/transformers.git
+## !pip install -q -U git+https://github.com/huggingface/accelerate.git
+## !pip install -q -U bitsandbytes sentencepiece
+## load model
+import torch
+from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig, TextStreamer
+model_name = "Radiantloom/radiantloom-llama-70b-instruct"
+bnb_config = BitsAndBytesConfig(
+    load_in_4bit=True,
+    bnb_4bit_use_double_quant=True,
+    bnb_4bit_compute_dtype=torch.bfloat16
+)
+# load the model and tokenizer
+model = AutoModelForCausalLM.from_pretrained(
+    model_name,
+    quantization_config=bnb_config,
+    device_map="auto",
+)
+tokenizer = AutoTokenizer.from_pretrained(
+    model_name,
+    trust_remote_code=True
+)
+tokenizer.pad_token = tokenizer.eos_token
+## inference
+prompt = """
+<s>[INST]<<SYS>>You are a helpful AI assistant developed by Radiantloom AI.<</SYS>>Take a deep breath and think step-by-step. Write a tale about a time-traveling historian who's determined to witness the most significant events in human history.[/INST]
+"""
+# tokenize the input string
+inputs = tokenizer(
+    prompt,
+    return_tensors="pt",
+    return_attention_mask=False
+)
+# generate text using the model
+streamer = TextStreamer(tokenizer, skip_prompt=True, skip_special_tokens=True)
+outputs = model.generate(**inputs, max_length=2048, streamer=streamer, do_sample=True, repetition_penalty=1.1"))
+# decode and print the output
+text = tokenizer.batch_decode(outputs)[0]
+```
+Here are a few example use cases. if we query the model with
+```
+<s>[INST]<<SYS>>You are a helpful AI assistant developed by Radiantloom AI.<</SYS>>Take a deep breath and think step-by-step. Write a tale about a time-traveling historian who's determined to witness the most significant events in human history.[/INST]
+```
+the model will return
+```
+Story: The Time Chronicles of Professor Amelia Harcourt
+Professor Amelia Harcourt was an exceptional historian, renowned for her remarkable contributions to the field of historical studies. However, her passion for understanding the past extended far beyond academics. She had always dreamt of witnessing firsthand the events that shaped the world as we know it. When she stumbled upon an ancient manuscript filled with cryptic symbols while working on a research project, she wasn't aware that it would change her life forever.
+After translating the mysterious text, Amelia discovered that it contained blueprints for constructing an extraordinary device - a time machine that could transport its user across different eras. Fueled by an all-consuming fervor for exploring pivotal moments in history, Amelia spent her remaining savings on procuring the materials necessary to build the intricate apparatus. The process required tireless dedication and rigorous labor, but through a mixture of intelligence, skill, and unrelenting determination, she finally constructed a functioning time machine three years later.
+With anticipation surging through her veins, Amelia embarked on a transcendent expedition to experience the most significant occurrences in human history. Her very first stop took place more than two million years ago: the East African Rift Valley, where homo habilis, our early ancestors, were believed to have emerged. In this remote corner of prehistory, Amelia studied their behavior closely and marveled at their resourcefulness. Their innovative use of tools and rudimentary social systems helped shape complex human societies of the future, making the encounter a moving lesson on the importance of adaptability.
+The second destination brought Amelia even further back in time, landing directly in Earth's arctic region during the late Cretaceous period. This journey held both personal ambition and scientific curiosity; Amelia wished to better understand why dinosaurs went extinct. Upon arrival, the historian felt a wave of unease as she navigated through the treacherous environment filled with towering dinosaur species, including the voracious Tyrannosaurus rex. With immense caution, she conducted her analysis and came to the shocking conclusion that asteroid impacts, not volcanic activity, were primarily responsible for wiping out the majority of dinosaur species. As these creatures faced catastrophic conditions, mammals thrived and eventually evolved into humans.
+Amelia continued her chronological pilgrimage with an extended stay at Stonehenge, one of the most mystifying historical sites she'd ever visited. After learning from several sources during her academic career, she hypothesized that the magnificent monument was built as part of a larger complex dedicated to worshipping gods tied to celestial bodies and fertility. Upon arriving in Neolithic Britain, Amelia watched in awe as people worked together to erect massive bluestones using ropes made from animal hides, wooden rollers, and levers carved from antlers. Witnessing this collaborative effort revealed a newfound appreciation for the technological ingenuity and social organization present in early human civilizations.
+Throughout subsequent adventures, Amelia became captivated by the enlightening teachings of Socrates in Ancient Greece and traveled through the bustling metropolis of imperial Rome. These excursions showcased the timeless themes of wisdom, governance, justice, art, and culture. They illuminated various aspects of human nature and the inherent values shared across every epoch.
+Overwhelmed by curiosity and a newfound sense of wonderment, Amelia decided to explore other transformative eras such as the discovery of the New World and the Industrial Revolution. Eventually, she found herself amidst the burgeoning civil rights movement in America during the 1960s, watching in admiration as advocates for racial equality peacefully rallied against systemic oppression and fought for equal rights.
+As Professor Amelia Harcourt immersed herself in the diverse expanse of human history, her encounters revealed a profound understanding of progress. Each event she observed had contributed something unique and essential to the development, growth, and evolution of mankind. This revelation reminded her of an old saying: "Only those who look backward can truly see forward." Through her time travels, Amelia began recognizing the patterns and interconnections between different eras, gaining immense insight into the human spirit's resilience and capacity for change.
+Inspired by her once-in-a-lifetime odyssey, Amelia Harcourt set forth creating The Time Chronicles-an extensive body of work featuring detailed accounts from her journeys through time. This collection transformed academic views regarding the past and offered generations to come priceless knowledge and fascinating perspectives vital to understanding our shared human story.
+```
+and if we query the model with
+```
+<s>[INST]<<SYS>>You are an AI assistant. Take a deep breath, think step-by-step and respond to following instructions from the user. <</SYS>>The cafeteria had 23 apples. If they used 20 for lunch and bought 6 more, how many apples do they have?[/INST]
+```
+the model will return
+```
+Let's break this down:
+1. The cafeteria initially had 23 apples.
+2. They used 20 for lunch. To find out how many apples were left after lunch, subtract the number of apples used for lunch (20) from the initial number of apples (23). This gives us 23 - 20 = 3 apples left after lunch.
+3. Then they purchased 6 more apples. To find out the total number of apples now, add the number of apples left after lunch (3) to the number of apples bought recently (6). So we have 3 + 6 = 9 apples in total.
+```
+## Evaluations
+We are encouraged by the initial assessments conducted using the [LLM-as-a-Judge method](https://arxiv.org/abs/2306.05685), particularly with the utilization of GPT-4. Our ongoing efforts involve a more comprehensive evaluation of the model. We have also entered this model into the HuggingFace Open Leaderboard for assessment. As we progress, we will continue to provide updates on our findings and share the results on this platform.
+## Ethical Considerations and Limitations
+Radiantloom Llama 70B Instruct, a powerful AI language model, can produce factually incorrect output and content not suitable for work (NSFW). It should not be relied upon to provide factually accurate information and should be used with caution. Due to the limitations of its pre-trained model and the finetuning datasets, it may generate lewd, biased, or otherwise offensive content. Consequently, developers should conduct thorough safety testing prior to implementing any applications of this model.
+## About Radiantloom AI
+Radiantloom AI trains open-source large language models tailored for specific business tasks such as copilots, email assistance, customer support, and database operations.
+Learn more about Radiantloom by visiting our [website](https://radiantloom.com). Follow us on Twitter at [Radiantloom](https://twitter.com/radiantloom) to gain early access to upcoming Radiantloom AI large language models.
+## Want to Support Our Work?
+We are passionate about contributing open-source AI models to the broader AI community to accelerate AI development and deployment. If you find our work valuable and would like to contribute to its continued success, consider buying us a few coffees!
+Your support goes a long way in helping us cover training costs, fund development tools, and keep the project thriving. Every coffee you buy is a small but meaningful contribution that fuels our dedication to providing a top-notch experience for users like you.
+[Buy us a coffee](https://www.buymeacoffee.com/aigeek_) and be a part of our journey towards making this project even more awesome. We appreciate your support!