---
library_name: transformers
tags:
- unsloth
datasets:
- Ayansk11/Mental_health_data_conversational
language:
- en
base_model:
- meta-llama/Llama-3.2-1B
Quantized:
- unsloth/Llama-3.2-1B-bnb-4bit
---

# Model Card for Model ID

<!-- Provide a quick summary of what the model is/does. -->



## Model Details

### Model Description

<!-- Provide a longer summary of what this model is. -->

This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.

- **Developed by:** Ayan Javeed Shaikh and Srushti Sonavane 
- **Finetuned from model:** unsloth/Llama-3.2-1B-bnb-4bit

### Model Sources [optional]



<div class="alert alert-block alert-success">  
<h2><center><strong>Mental Health Llama 3.2 - 1B ConversationalBot</strong></center></h2>
        
</div>

## Inference

```python
from unsloth import FastLanguageModel
model, tokenizer = FastLanguageModel.from_pretrained(
model_name = "Ayansk11/Mental_health_Llama3.2-1B_conversationalBot",
max_seq_length = 5020,
dtype = None,
load_in_4bit = True)
```


## Using this text to feed into model for getting the response 

text="I'm going through some things with my feelings and myself. I barely sleep and I do nothing but think about how I'm worthless and how I shouldn't be here. I've never tried or contemplated suicide. I've always wanted to fix my issues, but I never get around to it. How can I change my feeling of being worthless to everyone?"

<div style="background-color: #f2f2f2; border-left: 5px solid #4CAF50; padding: 15px; margin: 20px 0;">
    <strong>Note:</strong> Lets use the fine-tuned model for inference in order to generate responses based on mental health-related prompts !
</div>



<h3 style="color: #388e3c; font-family: Arial, sans-serif;">Key Points to Note:</h3>

<ol style="margin-left: 20px;">
    <li>
        <p>The <code>model = FastLanguageModel.for_inference(model)</code> command prepares the model specifically for inference, ensuring it is optimized for generating responses efficiently.</p>
    </li>
    <li>
        <p>The input text is processed using the <code>tokenizer</code>, which converts it into a format suitable for the model. The <code>data_prompt</code> is used to structure the input text, leaving a placeholder for the model's response. Additionally, the <code>return_tensors = "pt"</code> argument ensures the output is in PyTorch tensor format, which is then transferred to the GPU using <code>.to("cuda")</code> for faster processing.</p>
    </li>
    <li>
        <p>The <code>model.generate</code> function generates responses based on the tokenized input. Parameters like <code>max_new_tokens = 5020</code> and <code>use_cache = True</code> enable the model to produce lengthy, coherent outputs efficiently by leveraging cached computations from prior layers.</p>
    </li>
</ol>


```python
model = FastLanguageModel.for_inference(model)
inputs = tokenizer(
[
    data_prompt.format(
        #instructions
        text,
        #answer
        "",
    )
], return_tensors = "pt").to("cuda")

outputs = model.generate(**inputs, max_new_tokens = 5020, use_cache = True)
answer=tokenizer.batch_decode(outputs)
answer = answer[0].split("### Response:")[-1]
print("Answer of the question is:", answer)
```