Train a Llama model from scratch
•
17
Noa Roggendorff
nroggendorff
AI & ML interests
Large language models, Illusion Diffusion, Stable Diffusion, Stable Video Diffusion
Articles
Organizations
nroggendorff's activity
posted
an
update
about 10 hours ago
Post
183
What could have been
"{{bos_token}}{% for message in messages %}{% if (message['role'] == 'user') != (loop.index0 % 2 == 0) %}{{ raise_exception('Conversation roles must alternate user/assistant/user/assistant/...') }}{% endif %}{% if message['role'] == 'user' %}{{ '<|user|>\n' + message['content'] + '<|end|>\n' }}{% elif message['role'] == 'assistant' %}{{ '<|bot|>\n' + message['content'] }}{% if loop.last %}{{ '<|end|>' + eos_token }}{% else %}{{ '<|end|>\n' }}{% endif %}{% else %}{{ raise_exception('Only user and assistant roles are supported!') }}{% endif %}{% endfor %}"
"{{bos_token}}{% for message in messages %}{% if (message['role'] == 'user') != (loop.index0 % 2 == 0) %}{{ raise_exception('Conversation roles must alternate user/assistant/user/assistant/...') }}{% endif %}{% if message['role'] == 'user' %}{{ '<|user|>\n' + message['content'] + '<|end|>\n' }}{% elif message['role'] == 'assistant' %}{{ '<|bot|>\n' + message['content'] }}{% if loop.last %}{{ '<|end|>' + eos_token }}{% else %}{{ '<|end|>\n' }}{% endif %}{% else %}{{ raise_exception('Only user and assistant roles are supported!') }}{% endif %}{% endfor %}"
- 1 reply
we know, stop bragging
not that im jealous or anything
jk congrats
replied to
their
post
1 day ago
Im pretty sure you can add additional tokens and special tokens, so I suppose
but that doesnt really have anything to do with this article, thats just a transformers feature
replied to
their
post
1 day ago
Im pretty sure you can add additional tokens and special tokens, so I suppose
replied to
their
post
3 days ago
posted
an
update
3 days ago
Post
1928
Updated https://huggingface.co/blog/nroggendorff/train-with-llama-architecture so you can "train" your own tokenizer from your dataset.
-
- 6 replies
replied to
their
post
10 days ago
posted
an
update
10 days ago
Post
550
I updated
nroggendorff/llava to be even worse. The point of this space is to point out how bad the base model for Llava is, and how without an image it struggles quite a bit. In the new update, from my testing, even if you ask the model about the image, it won't be able to tell you. I experimented with quite a few things, including an image where all the values are 0 (shoutout np.zeros), an image with the most generic portrait photo I could think of (black hair, brown eyes, plain white shirt, etc..) (generated with
nroggendorff/epicrealismxl), an image that has text that reads "The image you are looking for is unavailable", and much.. much more.. *sigh*. The image I'm ending up with seems to work best. I don't think it would work with other multi-modal LLMs, but I'm not sure. If you find a problem, feel free to message me on the [huggingface discord](http://hf.co/join/discord), or open a pull request.
-
-
- 2 replies
posted
an
update
14 days ago
Post
706
I changed my mind about kaggle nb versions. They're goated
replied to
their
post
22 days ago
laughs in overpowered
replied to
their
post
22 days ago
indeed
replied to
their
post
26 days ago
I mean like replacing the binary data with decimals, so I could store a message
for example,
The letter "J" would be 74
replied to
their
post
26 days ago
You don't either?
@nroggendorff wth, i do on my end D:
not on your profile
replied to
their
post
26 days ago
hmm
replied to
their
post
26 days ago
we seek that fucking delete form of any subscribing to your platorm ; it is not ficking possible make it so ; because CEO I'll fuck you up personaly big time if you can't fucking get real
What is wrong with you
replied to
their
post
26 days ago
where is the delete account button ? we d\wish not to read hiilbillies
what??
replied to
their
post
26 days ago
You don't either?
replied to
their
post
26 days ago
ratio
posted
an
update
26 days ago
Post
2466
I'm about to start storing files in my TFLOPS count..
-
- 38 replies
replied to
their
post
about 1 month ago
Thanks
posted
an
update
about 1 month ago
Post
3308
I wish I had a cheaper and more appreciated by the masses hobby.
- 7 replies
replied to
their
post
about 2 months ago
It's not very good, but it's funny.