How can I pretrain the BLOOM?
I want to pretrain the BLOOM use my corpus, how can I do it? Now I use the code from https://github.com/huggingface/transformers/tree/main/examples/pytorch/language-modeling. Can it be used to pretrain the BLOOM?
Hi, I have been doing exactly what you are talking about with my open source mobile text editor called Maker+ Ci. I have built a AI chat interface on top of Maker+ Ci called chatLink which uses files stored in M+ to set context/pre-prompts for the bloom model which is the LLM that chatLink is using. So in chatLink the user can easily select the file that they want to set as context, which tells bloom how to behave when a prompt is submitted in chatLink. I have created lots of cool prompts for bloom, including using it as a terminal, mental health chatbot, Ai personal assistant, chatbots with different personalities such as the intj personality and the infp personality and the most recent one which is a qna model built from prompts and no fine tuning. You can jump between all these different contexts in chatLink easily. I’m happy to share all these different prompts including teaching how to use Maker+ Ci and chatLink which are both open source. If you are interested please join my discord server and feel free to come and ask any questions you like about how to work with bloom in your own projects or how work with bloom in Maker+ Ci and chatLink there.
Hi, I have been doing exactly what you are talking about with my open source mobile text editor called Maker+ Ci. I have built a AI chat interface on top of Maker+ Ci called chatLink which uses files stored in M+ to set context/pre-prompts for the bloom model which is the LLM that chatLink is using. So in chatLink the user can easily select the file that they want to set as context, which tells bloom how to behave when a prompt is submitted in chatLink. I have created lots of cool prompts for bloom, including using it as a terminal, mental health chatbot, Ai personal assistant, chatbots with different personalities such as the intj personality and the infp personality and the most recent one which is a qna model built from prompts and no fine tuning. You can jump between all these different contexts in chatLink easily. I’m happy to share all these different prompts including teaching how to use Maker+ Ci and chatLink which are both open source. If you are interested please join my discord server https://discord.gg/47pXk7CY and feel free to come and ask any questions you like about how to work with bloom in your own projects or how work with bloom in Maker+ Ci and chatLink there.
Hi! Is your invitation still valid? I tried to join, but the link you have posted is dead unfortunately.
Thank you!
@tmighty I have updated the link it should work now.
Do you have a Github on how you are doing the pre-training?
@vlsalsa No I don't have a github on it, come in the discord I occasionally drop examples in there.
Invite expired :(
@TheHarshAvinash here's a working one https://discord.gg/EtRcMRTh3G
Thanks mate!