license: bigscience-bloom-rail-1.0
language:
- ar
- en
Noon - a 7-billion parameter Arabic Large Language Model
We present the 7-billion parameter variant of Noon, an Arabic Large Language model based on BLOOM, a foundation model released by the bigscience workshop.
Noon was trained with the main focus of having a model that responds to various types of instructions and questions (text generation, code generation, mathematical problems, closed/open-book questions, etc.)
We trained the model using the ColossalAI framework which fully supports the HuggingFace library models, and implements different optimization and quantization techniques for billion-scale LLMs.
The training data is a combination of Arabic datasets covering multiple tasks, more details are provided in the dataset section.
Usage
The usage of our model only requires the Transformers library, and can be loaded as follows:
from transformers import BloomTokenizerFast, BloomForCausalLM, pipeline
text="اكتب مقالا من عدة أسطر عن الذكاء الصناعي وتطوراته"
prompt = f'Instruction:\n{text}\n\nResponse:'
model = BloomForCausalLM.from_pretrained('Naseej/noon-7b')
# We use the pretrained BLOOM tokenizer as it provides full support for the Arabic vocabulary
tokenizer = BloomTokenizerFast.from_pretrained('bigscience/bloomz-7b1-mt')
generation_pipeline = pipeline("text-generation", model=model, tokenizer=tokenizer)
# We recommend the provided hyperparameters for generation
# But encourage you to try different values
response = generation_pipeline(prompt,
pad_token_id=tokenizer.eos_token_id,
num_beams=4,
max_length=500,
top_p=0.9,
repetition_penalty = 3.0,
no_repeat_ngram_size = 3)[0]['generated_text']
print(response)
Training's computational requirements
Dataset
Disclaimer
The generated responses from this AI model are purely algorithmic and should be interpreted with caution. The model's outputs may occasionally exhibit bias, offensive language, or potentially harmful content. It is important to note that these responses do not reflect the personal preferences or viewpoints of the authors or the organization of Naseej.
While every effort is made to mitigate the harmfulness of the model's outputs, it is impossible to guarantee complete elimination of biases or offensive content. The model learns from vast amounts of data and may inadvertently replicate or amplify existing societal biases present in the training data.
Users are advised to critically evaluate and verify the information provided by the model. Exercise discretion when utilizing the model's responses, particularly in sensitive or controversial topics.
We are committed to ongoing research and development to improve the model's performance, minimize biases, and reduce harmful outputs. Your feedback and insights are valuable in helping us achieve these goals.