---
license: apache-2.0
language:
- fr
- en
pipeline_tag: text-generation
library_name: transformers
tags:
- LLM
inference: false
---

<p align="center" width="100%">
<img src="https://huggingface.co/bofenghuang/vigogne-falcon-7b-chat/resolve/main/vigogne_logo.png" alt="Vigogne" style="width: 40%; min-width: 300px; display: block; margin: auto;">
</p>

# Vigogne-Falcon-7B-Chat: A French Chat Falcon Model

Vigogne-Falcon-7B-Chat is a [Falcon-7B](https://huggingface.co/tiiuae/falcon-7b) model fine-tuned to conduct multi-turn dialogues in 🇫🇷 French between human user and AI assistant.

For more information, please visit the Github repo: https://github.com/bofenghuang/vigogne

## Usage

```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, GenerationConfig

device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")

model_name_or_path = "bofenghuang/vigogne-falcon-7b-chat"

tokenizer = AutoTokenizer.from_pretrained(model_name_or_path, padding_side="right", use_fast=False)
tokenizer.pad_token = tokenizer.eos_token

model = AutoModelForCausalLM.from_pretrained(
    model_name_or_path,
    load_in_8bit=False,
    torch_dtype=torch.float16,
    device_map="auto",
    trust_remote_code=True,
)

prompt_template = """Below is a conversation between a user and an AI assistant named Vigogne.
Vigogne is an open-source AI assistant created by Zaion (https://zaion.ai/).
Vigogne is polite, emotionally aware, humble-but-knowledgeable, always providing helpful and detailed answers.
Vigogne is skilled in responding proficiently in the languages its users use and can perform a wide range of tasks such as text editing, translation, question answering, logical reasoning, coding, and many others.
Vigogne cannot receive or generate audio or visual content and cannot access the internet.
Vigogne strictly avoids discussing sensitive, offensive, illegal, ethical, or political topics and caveats when unsure of the answer.

<|USER|>: Salut, assistant !
<|ASSISTANT|>: Bonjour, que puis-je pour vous ?
<|USER|>: {user_query}
<|ASSISTANT|>: """

user_query = "Expliquez la différence entre DoS et phishing."

prompt = prompt_template.format(user_query=user_query)

input_ids = tokenizer(prompt, return_tensors="pt")["input_ids"].to(device)
input_length = input_ids.shape[1]

generated_output = model.generate(
    input_ids=input_ids,
    generation_config=GenerationConfig(
        max_new_tokens=512,
        temperature=0.1,
        do_sample=True,
    ),
    pad_token_id=tokenizer.eos_token_id,
    eos_token_id=tokenizer.eos_token_id,
)
generated_text = tokenizer.decode(generated_output[0, input_length:], skip_special_tokens=True)
print(generated_text)
```

<!-- You can infer this model by using the following Google Colab Notebook.

<a href="https://colab.research.google.com/github/bofenghuang/vigogne/blob/main/notebooks/infer_instruct.ipynb" target="_blank"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/></a> -->

## Limitations

Vigogne is still under development, and there are many limitations that have to be addressed. Please note that it is possible that the model generates harmful or biased content, incorrect information or generally unhelpful answers.