Create README.md

README.md

@@ -0,0 +1,93 @@
+---
+license: apache-2.0
+inference: false
+tags:
+- generated_from_trainer
+- text-generation-inference
+model-index:
+- name: Mistral-7B-Banking-v2
+  results: []
+model_type: mistral
+pipeline_tag: text-generation
+widget:
+- messages:
+  - role: user
+    content: I want to close an online account
+---
+
+# Mistral-7B-Banking-v2
+
+## Model Description
+
+"Mistral-7B-Banking-v2" is a specialized adaptation of the [mistralai/Mistral-7B-Instruct-v0.2](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2), designed to provide precise answers related to general banking queries. This model is fine-tuned to help users with common banking tasks and inquiries.
+
+## Intended Use
+
+- **Recommended applications**: Ideal for deployment in digital banking platforms where quick and accurate customer service responses are needed. It can be integrated into banking chatbots to assist users with transactions, account information, and other banking services.
+- **Out-of-scope**: This model is not suited for non-banking related questions and should not be used for providing health, legal, or critical safety advice.
+
+## Usage Example
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+model = AutoModelForCausalLM.from_pretrained("bitext-llm/Mistral-7B-Banking-v2")
+tokenizer = AutoTokenizer.from_pretrained("bitext-llm/Mistral-7B-Banking-v2")
+
+inputs = tokenizer("<s>[INST] How can I transfer money to another account?[/INST]", return_tensors="pt")
+outputs = model.generate(inputs['input_ids'], max_length=50)
+print(tokenizer.decode(outputs[0], skip_special_tokens=True))
+```
+
+## Model Architecture
+
+This model utilizes the `MistralForCausalLM` architecture with a `LlamaTokenizer`, ensuring it retains the foundational capabilities of the base model while being specifically enhanced for banking-related interactions.
+
+## Training Data
+
+The model was fine-tuned on a dataset comprising various banking-related intents, including transactions like balance checks, money transfers, loan applications, and more, totaling 89 intents each represented by approximately 1000 examples. This comprehensive training helps the model address a broad spectrum of banking-related questions effectively. The dataset follows the same structured approach as our dataset published on Hugging Face as [bitext/Bitext-customer-support-llm-chatbot-training-dataset](https://huggingface.co/datasets/bitext/Bitext-customer-support-llm-chatbot-training-dataset), but with a focus on banking.
+
+## Training Procedure
+
+### Hyperparameters
+
+- **Optimizer**: AdamW
+- **Learning Rate**: 0.0002 with a cosine learning rate scheduler
+- **Epochs**: 4
+- **Batch Size**: 10
+- **Gradient Accumulation Steps**: 8
+- **Maximum Sequence Length**: 8192 tokens
+
+### Environment
+
+- **Transformers Version**: 4.40.0.dev0
+- **Framework**: PyTorch 2.2.1+cu121
+- **Tokenizers**: Tokenizers 0.15.0
+
+## Limitations and Bias
+
+- The model is trained for banking-specific contexts but may underperform in unrelated areas.
+- Potential biases in the training data could affect the neutrality of the responses; users are encouraged to evaluate responses critically.
+
+## Ethical Considerations
+
+It is important to use this technology thoughtfully, ensuring it does not substitute for human judgment where necessary, especially in sensitive financial situations.
+
+## Acknowledgments
+
+This model was developed and trained by Bitext using proprietary data and technology.
+
+## License
+
+This model, "Mistral-7B-Banking-v2", is licensed under the Apache License 2.0 by Bitext Innovations International, Inc. This open-source license allows for free use, modification, and distribution of the model but requires that proper credit be given to Bitext.
+
+### Key Points of the Apache 2.0 License
+
+- **Permissibility**: Users are allowed to use, modify, and distribute this software freely.
+- **Attribution**: You must provide proper credit to Bitext Innovations International, Inc. when using this model, in accordance with the original copyright notices and the license.
+- **Patent Grant**: The license includes a grant of patent rights from the contributors of the model.
+- **No Warranty**: The model is provided "as is" without warranties of any kind.
+
+You may view the full license text at [Apache License 2.0](http://www.apache.org/licenses/LICENSE-2.0).
+
+This licensing ensures the model can be used widely and freely while respecting the intellectual contributions of Bitext. For more detailed information or specific legal questions about using this license, please refer to the official license documentation linked above.