11 1

Hrishbh Dalal

HriDal

https://hrishbh.com/

HrishbhDalal

AI & ML interests

Large Language Models, Vision Language Models, Synthetic Data Generation and Reinforcement Learning

Recent Activity

updated a model 21 days ago

HriDal/agent-2048-game-qwen-7b-8k-ds

published a model 21 days ago

HriDal/agent-2048-game-qwen-7b-8k-ds

updated a model 21 days ago

HriDal/agent-2048-game-qwen-7b-2k-ds

View all activity

Organizations

HriDal's activity

updated a model 21 days ago

HriDal/agent-2048-game-qwen-7b-8k-ds

Reinforcement Learning • Updated 21 days ago • 2

published a model 21 days ago

HriDal/agent-2048-game-qwen-7b-8k-ds

Reinforcement Learning • Updated 21 days ago • 2

updated a model 21 days ago

HriDal/agent-2048-game-qwen-7b-2k-ds

Reinforcement Learning • Updated 21 days ago • 32

published a model 21 days ago

HriDal/agent-2048-game-qwen-7b-2k-ds

Reinforcement Learning • Updated 21 days ago • 32

New activity in meta-llama/Llama-3.1-8B-Instruct 9 months ago

Garbage output ?

#30 opened 9 months ago by

danielus

New activity in OpenGVLab/InternVL2-Llama3-76B 9 months ago

Diminishing returns over size

#2 opened 9 months ago by

HriDal

Error: Expected all tensors to be on the same device, but found at least two devices, cuda:2 and cuda:0!

#1 opened 9 months ago by

HriDal

New activity in OpenGVLab/InternVL2-40B 9 months ago

Multiple GPUs for inference error

#2 opened 10 months ago by

Mostudy

New activity in internlm/internlm-xcomposer2d5-7b 9 months ago

Run this model on multiple 24GB cards?

#9 opened 10 months ago by

LemonBranny

New activity in m-a-p/OpenCodeInterpreter-DS-33B 10 months ago

Can someone point to the chat template used for this model? Would be awesome.

#7 opened about 1 year ago by

HriDal

replied to clem's post 11 months ago

Hopefully something open source. They have a long history of open sourcing stuff in the past. Hopefully they open source Gato. That would be a legendary model to be released in the public. Maybe hugginface should train their own Gato? I could contribute to code and data... oss community is awesome.

posted an update 11 months ago

Post

774

I just saw that openai is using an updated tokenizer and it greatly increases the speed of the model and maybe even performance as if we increase the size of vocabulary, it could predict a single token which might be equivalent to two or three tokens in current tokenizers with 50-60k tokens or even 100k. I was thinking of scaling this to a million vocabulary size and then pre training llama3 8b using lora. I know that the model might go to shit, but we can increase the speed of the tokens generation greatly imo. And as one of meta papers said that predicting multiple tokens at the same time can actually increase the performance of a model, so I can imagine increasing the vocabulary in this way means multiple token generation in a way. Yann Lecunn also says that we don’t think in tokens but more like representations or abstractions of situations or problem to be solved. Can scaling to a million vocab size or even 10 million vocab size lead to better and more robust models? Please give me your thoughts on what can go wrong, what can go right etc…