QuantFactory/Llama-3-8B-Instruct-Finance-RAG-GGUF

This is quantized version of curiousily/Llama-3-8B-Instruct-Finance-RAG created using llama.cpp

Original Model Card

Llama 3 8B Instruct (Financial RAG)

This model is a fine-tuned version of the original Llama 3 8B Instruct model on 4000 examples from the virattt/financial-qa-10K dataset.

The model is fine-tuned using a LoRA adapter for RAG use cases. It is optimized to answer a question based on a context:

Answer the question:
{question}

Using the information:
{context}

Usage

Load the model:

MODEL_NAME = "curiousily/Llama-3-8B-Instruct-Finance-RAG"
tokenizer = AutoTokenizer.from_pretrained(MODEL_NAME, use_fast=True)
model = AutoModelForCausalLM.from_pretrained(
    MODEL_NAME,
    device_map="auto"
)

pipe = pipeline(
    task="text-generation",
    model=model,
    tokenizer=tokenizer,
    max_new_tokens=128,
    return_full_text=False,
)

Format the prompt (uses the original Instruct prompt format):

prompt = """
<|begin_of_text|><|start_header_id|>system<|end_header_id|>

Use only the information to answer the question<|eot_id|><|start_header_id|>user<|end_header_id|>

How much did the company's net earnings amount to in fiscal 2022?

Information:

```
Net earnings were $17.1 billion in fiscal 2022.
```<|eot_id|><|start_header_id|>assistant<|end_header_id|>
"""

And make a prediction:

print(outputs[0]["generated_text"])

$17.1 billion

Here's a helper function to build your prompts:

def create_test_prompt(data_row):
    prompt = dedent(f"""
    {data_row["question"]}

    Information:

    ```
    {data_row["context"]}
    ```
    """)
    messages = [
        {"role": "system", "content": "Use only the information to answer the question"},
        {"role": "user", "content": prompt},
    ]
    return tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)

Where data_row must be a dict:

data_row = {
  "question": "...",
  "context": "..."
}

Sample Predictions

Here's a sample of the predictions from trained and untrained models

Example 1

answer: Delta Air Lines' agreements with its regional carriers typically last at least ten years with options for extensions. Delta controls operational aspects like scheduling and pricing.

trained: Delta Connection agreements typically last at least ten years with options for extensions.

untrained: According to the information, the terms of Delta Air Lines' agreements with its regional carriers through Delta Connection are:

1. The agreements typically last at least ten years.
2. There are options for extensions.
3. Delta controls major operational aspects like scheduling and pricing.
4. The regional carriers supply the services.

Example 2

answer: The company evaluates acquisition-related intangibles for impairment by comparing the asset's carrying amount to undiscounted future net cash flows expected from the asset. An impairment loss is recognized if the carrying amount exceeds the asset's recoverable amount.

trained: The company evaluates acquisition-related intangible and other long-lived assets for impairment whenever events or changes in circumstances indicate that the carrying amount may not be recoverable. This involves measuring the recoverability of the asset by comparing its carrying amount to the future undiscounted net cash flows expected to be generated by the asset group.

untrained: According to the information, the company uses the following criteria to determine whether an impairment loss should be recognized on acquisition-related intangible assets:

* Events or changes in circumstances that indicate the carrying amount of an asset may not be recoverable.
* A comparison of the carrying amount of an asset to future undiscounted net cash flows expected to be generated by the asset group.

In other words, the company evaluates whether the asset's carrying amount is recoverable by comparing it to the expected future cash flows, which involves assumptions about future prospects and computations of estimated future cash flows.

Example 3

answer: In the United States, the approval process for biosimilars is governed by the Public Health Service Act (PHSA) and the regulations implementing these statutes, specifically including provisions made under federal health care reform legislation enacted in March 2010.

trained: The Federal Food, Drug, and Cosmetic Act (the FFDCA) and the Public Health Service Act (PHSA)

untrained: The legal framework that governs the approval process for biosimilars in the United States is the Federal Food, Drug, and Cosmetic Act (FFDCA) and the Public Health Service Act (PHSA), as well as the regulations implementing these statutes.

Example 4

answer: Timothy S. Teter holds a B.S. degree in Mechanical Engineering from the University of California at Davis and a J.D. degree from Stanford Law School.

trained: B.S. in Mechanical Engineering from the University of California at Davis and a J.D. from Stanford Law School

untrained: According to the information, Timothy S. Teter holds:

1. A B.S. degree in Mechanical Engineering from the University of California at Davis.
2. A J.D. degree from Stanford Law School.

Example 5

answer: Beginning in fiscal year 2024, the company plans to exclude paused Connected Fitness subscriptions from its new 'Ending Paid Connected Fitness Subscriptions' metric and will treat a pause action as a churn event in its 'Average Net Monthly Paid Connected Fitness Subscription Churn' metric.

trained: Starting in fiscal year 2024, the company will no longer include paused Connected Fitness subscriptions in their Ending Paid Connected Fitness Subscriptions metric and will treat a pause action as a churn event in their Average Net Monthly Paid Connected Fitness Subscription Churn.

untrained: Starting in fiscal year 2024, the company will:

* No longer include paused Connected Fitness subscriptions in the Ending Paid Connected Fitness Subscriptions metric
* Treat a pause action as a churn event in the Average Net Monthly Paid Connected Fitness Subscription Churn

License

Uses the original Llama 3 License. A custom commercial license is available at: https://llama.meta.com/llama3/license