Ali-C137 (Ali Elfilali)

posted an update 10 days ago

Post

2699

Is it just me or is it real that whenever APPLE releases an open model, they accompany it with a library !? First was MLX, about a month ago AXLEARN and now CORENET ! Could it be just coincidences or does Apple playing some game ? if yes then what is it ... ? What do you think ? maybe i'm just hallucinating now 😅

replied to their post 26 days ago

Not really 😕

posted an update 27 days ago

Post

2106

Honestly i don't understand how come we as the open source community haven't surpassed GPT-4 yet ? Like for me it looks like everything is out there just need to be exploited! Clearly specialized small models outperforms gpt4 on downstream tasks ! So why haven't we just trained a 1B-2B really strong general model and then continue pertained and/or finetuned it on datasets for downstream tasks like math, code...well structured as Textbooks format or other datasets formats that have been proven to be really efficient and good! Ounce you have 100 finetuned model, just wrap them all into a FrankenMoE and Voila ✨
And that's just what a NOOB like myself had in mind, I'm sure there is better, more efficient ways to do it ! So the question again, why we haven't yet ? I feel I'm missing something... Right?

5 replies

·

replied to their post about 1 month ago

Almsot 24 Hours after the release of the Arabic cohort of DIBT-MPEP, we are at 100 prompts translated/corrected !
Shout out to the hero @seyf1elislam for contributing more than 60 prompts 🔥

posted an update about 1 month ago

Post

2190

Today we launch our space in colab with @dvilasuero & @davanstrien so you can help us translate/correct our curated prompt dataset, that will be used to evaluate the performance of Arabic LLMs later and help our community to identify how open models perform on Arabic.

How to Get Involved?

1. Visit our Argilla Space and start reviewing prompts.
https://2a2i-prompt-translation-for-arabic.hf.space/

2. Join our Discord channel in the HuggingFace's discord server to connect with the community and share your insights.
https://discord.com/channels/879548962464493619/1217179730869096448

3 replies

·

replied to smangrul's post 2 months ago

Hi @smangrul , apparently i can't push the merged adapter to the hub ???
Cuz when i do so it create num_of_adapters_to_merge + 1 (the merged adapter) and when i want to load the the merged adapter with model = PeftModel.from_pretrained(model, adapter) i got the error in image 2 !

image 1:

image 2:

Your help is much appreciated, tnx 🤗

posted an update 2 months ago

Post

Super excited to share with you all our latest contribution from 2A2I.

Today we announce : 2A2I/Arabic-OpenHermes-2.5

Arabic-OpenHermes-2.5 Is simply the translation of the original dataset released by @teknium couple months ago !
In fact it looks as a simple task ! In reality it was quite a laborious job !
But thanks to @maghwa & @medmac01 this dataset managed to see the light today and help creating better / more aligned arabic LLMs in the near future.

If you are interested to join us and/or help us, please leave a comment below or visit our HuggingFace Org Card for more details about How/What you can do.

More datasets to come and more models are in the way 🔥

replied to clefourrier's post 2 months ago

Thanks to your support 🤗

replied to davidberenstein1957's post 3 months ago

Got some input from @ybelkada about not needing a ref_model because we can just swap out the LoRa adapters during training.

About this part 😄

replied to their post 3 months ago

@Ali-C137 it should be fixed now. Thank you for your feedback!

Thank you so much 🤗

replied to macadeliccc's post 3 months ago

Hi !
I think for NEFTune it should be supported out of the box as you just need to pass the correct argument neftune_noise_alpha in TrainingArguments right?

Yes indeed (AFAIK) but i asked if Unsloth support it as well by incorporating it in their code base (i assume they are based on PEFT & TRL as well !?)

posted an update 3 months ago

Post

The Aya project ( CohereForAI/aya_dataset, CohereForAI/aya_collection and CohereForAI/aya_evaluation_suite) by CohereForAI got released yesterday ! And today I'am excited to introduce Arabic Aya (2A) 🌟

Arabic Aya is a carefully curated dataset, derived from the vast Aya collection by CohereForAI, tailored specifically for Arabic language processing. It consolidates texts across Modern Standard Arabic (MSA) and other dialects, simplifying access to high-quality data for researchers, developers, and linguists.

🔍 Why Arabic Aya?
- Time-saving : Jump straight into your projects with pre-filtered Arabic texts.
- Diverse applications : Perfect for language modeling, sentiment analysis, dialect identification, and more.
- Community-driven : Your contributions and feedback can help enrich this resource further.

📚 Utilize Arabic Aya for your next NLP/LLM projects and be part of advancing Arabic language technologies. Let’s collaborate to make Arabic AI research more accessible and robust!

Check it out here: 2A2I/Arabic_Aya

replied to their post 3 months ago

Hi @julien-c , always about the viewer, this white view of sections within dark mode been really annoying, do you think you guys can do something about it ?

PS : I have been using this viewer for almost 6 hours now 🔥🤗

replied to macadeliccc's post 3 months ago

Is all-linear (most recent update of PEFT) supported in the target_modules arg ? Also what about NEFTune ?

posted an update 3 months ago

Post

I love the new Viewer and i didn't knew how much i needed it until now
@sylvain , @lhoestq and team, GREAT JOB 🔥 and THANK YOU 🤗

4 replies

·

replied to euclaise's post 3 months ago

Amazing work 🤩
I wish if we had a save button for posts here

I'm also interested to know more about this :
"To prevent catastrophic forgetting, I used weight averaging between iterations."
Can you please elaborate !? Tnx 🤗

replied to akhaliq's post 3 months ago

Can't wait for the release soon 🔥

replied to merve's post 3 months ago

The idea itself was not that revolutionary tho, cuz practically chess moves are just sequences and better they are expressed with letters and numbers that are familiar to LLMs. I remember back in July i had a discussion about the very same idea with some folks during a summer school

replied to their post 3 months ago

I don't even wanna think about my email inbox 🤦🏻‍♂️😂

posted an update 3 months ago

Post

3 hours between the two pictures 🔥
Finally that paranoid inside me got some rest 😂

2 replies

·

posted an update 3 months ago

Post

🎉🥳🎉
Today, we are thrilled to officially launch the "2A2I" Arabic Artificial Intelligence Initiative. This is a community-driven initiative founded on the philosophy of "Small team, Big work" Our goal is to elevate Arabic AI (LLMs, Diffusion Models, ASR, etc.) to the same level as English (and also Chinese 🐉).

Naturally, our focus today is primarily on datasets. We aim to provide high-quality datasets, especially for LLMs this month, to support our future efforts. In line with this, we're excited to introduce the Arabic version of H4-no_robots, find here : 2A2I/H4_no_robots (and yes, we know it's not "no_robots" anymore 😄). Stay tuned for more exciting, high-quality datasets in the next couple of weeks (+ 4 million rows🔥)

In parallel, we're also developing a model 🐪 that we hope will set new high standards for Arabic LLMs. 🔥 This model is planned for release in the coming months.

For more information, please visit our Organization card here : https://huggingface.co/2A2I

If you're interested in Arabic AI and want to help pushing the wheel as well, fill out this form, and let us know your motivation and your exciting ideas 🔥

The form link : https://forms.gle/kZLVuynWFU2FyTm57

If you have any questions, feel free to reach out to us at the email address below.

Additionally, if you believe as we do in this mission and would like to help this community and contribute some compute resources 😉 or any other form of help you might think about, please contact us at the same email address below or reach out to me through LinkedIn 🔥

2A2I Contact Email : arabic.ai.initiative@gmail.com
My LinkedIn : https://www.linkedin.com/in/alielfilali01/

replied to victor's post 3 months ago

This comment has been hidden

replied to davidberenstein1957's post 3 months ago

Can you elaborate more plz ?

posted an update 3 months ago

Post

Hi friends, i'am happy to share with you all a tool that i built a week ago or so, i'am talking here about the "LLM Training Cost Calculator" - a handy tool now available on Hugging Face Spaces! This interactive Gradio app provides an easy-to-use interface for estimating the training costs of large language models (LLMs).

(I've been asked to provide a report about the cost of finetuning each model etc... so i decided to do the lazy job and build a tool for it, Prof later can choose whatever config he likes 😆)

🔍 But Why this is important?
As LLMs continue to grow in size and complexity, understanding the computational and financial requirements is crucial for planning and managing AI projects. I believe this tool simplifies this process, giving you insights into potential expenses based on the number of parameters and tokens in your dataset.

🌟 Features:
- Input the number of parameters (in billions) and tokens (in trillions).
- Adjust for GPU utilization rates and overhead costs.
- Get an instant estimate of your training costs.
+ Choose your GPU (A100 80GB PCle, A100 80GB SXM, V100, H100 SXM, H100 PCle)

📈 Coming Soon:
Plans are in place to expand the calculator's capabilities to include fine-tuning costs for models using LoRA or QLoRA. You'll be able to input a model ID from the Hugging Face Hub, select your fine-tuning strategy, and specify quantization details if using QLoRA.

I believe this tool will be a valuable asset to the AI community, helping to plan and allocate resources more effectively 🤗.

Should you have any suggestions or feedback, please don't hesitate to contribute your thoughts in the comments below. Together, we can refine and enhance this resource for all.

🔗 Try it here : Ali-C137/LLM-Training-Cost-Calculator

PS : All thanks to Gradio, Hugging Face and the community ofc 🔥 😉

replied to akhaliq's post 3 months ago

It would be super helpful if they released their dataset 🔥

replied to their post 3 months ago

Done ✅
But the text is not full displayed ! so a preview when a user hover on the text would be nice 🤗

Thank you guys for making all this fun

replied to victor's post 3 months ago

Just created mine 🔥
I've been willing to create HuggingAssist for soo long and you guys just made it a lot easier 🔥 tnx 🤗

posted an update 3 months ago

Post

First post alert 🚨

Super excited to share with you my first Chat assistant :

HuggingAssist, meant to offer guidance with the large HuggingFace ecosystem

Chat with it from here : https://hf.co/chat/assistant/65bd0adc08560e58be454d86

It would be more helpful when the RAG / WEB features are available !
Looking forward to it 🔥

ps : tnx @Chunte for the cool Huggies

4 replies

·

replied to soldni's post 3 months ago

I don't know why i always thought it would be multilingual 🤦🏻‍♂️
Great job 🔥 the paper is a masterpiece 👏🏻 tnx for it

Ali Elfilali

AI & ML interests

Organizations

Ali-C137's activity