blakkd

owao

AI & ML interests

None yet

Recent Activity

replied to aiqtech's post about 7 hours ago
🌐 AI Token Visualization Tool with Perfect Multilingual Support Hello! Today I'm introducing my Token Visualization Tool with comprehensive multilingual support. This web-based application allows you to see how various Large Language Models (LLMs) tokenize text. https://huggingface.co/spaces/aiqtech/LLM-Token-Visual ✨ Key Features πŸ€– Multiple LLM Tokenizers: Support for Llama 4, Mistral, Gemma, Deepseek, QWQ, BERT, and more πŸ”„ Custom Model Support: Use any tokenizer available on HuggingFace πŸ“Š Detailed Token Statistics: Analyze total tokens, unique tokens, compression ratio, and more 🌈 Visual Token Representation: Each token assigned a unique color for visual distinction πŸ“‚ File Analysis Support: Upload and analyze large files 🌏 Powerful Multilingual Support The most significant advantage of this tool is its perfect support for all languages: πŸ“ Asian languages including Korean, Chinese, and Japanese fully supported πŸ”€ RTL (right-to-left) languages like Arabic and Hebrew supported 🈺 Special characters and emoji tokenization visualization 🧩 Compare tokenization differences between languages πŸ’¬ Mixed multilingual text processing analysis πŸš€ How It Works Select your desired tokenizer model (predefined or HuggingFace model ID) Input multilingual text or upload a file for analysis Click 'Analyze Text' to see the tokenized results Visually understand how the model breaks down various languages with color-coded tokens πŸ’‘ Benefits of Multilingual Processing Understanding multilingual text tokenization patterns helps you: Optimize prompts that mix multiple languages Compare token efficiency across languages (e.g., English vs. Korean vs. Chinese token usage) Predict token usage for internationalization (i18n) applications Optimize costs for multilingual AI services πŸ› οΈ Technology Stack Backend: Flask (Python) Frontend: HTML, CSS, JavaScript (jQuery) Tokenizers: πŸ€— Transformers library
replied to aiqtech's post about 7 hours ago
🌐 AI Token Visualization Tool with Perfect Multilingual Support Hello! Today I'm introducing my Token Visualization Tool with comprehensive multilingual support. This web-based application allows you to see how various Large Language Models (LLMs) tokenize text. https://huggingface.co/spaces/aiqtech/LLM-Token-Visual ✨ Key Features πŸ€– Multiple LLM Tokenizers: Support for Llama 4, Mistral, Gemma, Deepseek, QWQ, BERT, and more πŸ”„ Custom Model Support: Use any tokenizer available on HuggingFace πŸ“Š Detailed Token Statistics: Analyze total tokens, unique tokens, compression ratio, and more 🌈 Visual Token Representation: Each token assigned a unique color for visual distinction πŸ“‚ File Analysis Support: Upload and analyze large files 🌏 Powerful Multilingual Support The most significant advantage of this tool is its perfect support for all languages: πŸ“ Asian languages including Korean, Chinese, and Japanese fully supported πŸ”€ RTL (right-to-left) languages like Arabic and Hebrew supported 🈺 Special characters and emoji tokenization visualization 🧩 Compare tokenization differences between languages πŸ’¬ Mixed multilingual text processing analysis πŸš€ How It Works Select your desired tokenizer model (predefined or HuggingFace model ID) Input multilingual text or upload a file for analysis Click 'Analyze Text' to see the tokenized results Visually understand how the model breaks down various languages with color-coded tokens πŸ’‘ Benefits of Multilingual Processing Understanding multilingual text tokenization patterns helps you: Optimize prompts that mix multiple languages Compare token efficiency across languages (e.g., English vs. Korean vs. Chinese token usage) Predict token usage for internationalization (i18n) applications Optimize costs for multilingual AI services πŸ› οΈ Technology Stack Backend: Flask (Python) Frontend: HTML, CSS, JavaScript (jQuery) Tokenizers: πŸ€— Transformers library
replied to aiqtech's post about 8 hours ago
🌐 AI Token Visualization Tool with Perfect Multilingual Support Hello! Today I'm introducing my Token Visualization Tool with comprehensive multilingual support. This web-based application allows you to see how various Large Language Models (LLMs) tokenize text. https://huggingface.co/spaces/aiqtech/LLM-Token-Visual ✨ Key Features πŸ€– Multiple LLM Tokenizers: Support for Llama 4, Mistral, Gemma, Deepseek, QWQ, BERT, and more πŸ”„ Custom Model Support: Use any tokenizer available on HuggingFace πŸ“Š Detailed Token Statistics: Analyze total tokens, unique tokens, compression ratio, and more 🌈 Visual Token Representation: Each token assigned a unique color for visual distinction πŸ“‚ File Analysis Support: Upload and analyze large files 🌏 Powerful Multilingual Support The most significant advantage of this tool is its perfect support for all languages: πŸ“ Asian languages including Korean, Chinese, and Japanese fully supported πŸ”€ RTL (right-to-left) languages like Arabic and Hebrew supported 🈺 Special characters and emoji tokenization visualization 🧩 Compare tokenization differences between languages πŸ’¬ Mixed multilingual text processing analysis πŸš€ How It Works Select your desired tokenizer model (predefined or HuggingFace model ID) Input multilingual text or upload a file for analysis Click 'Analyze Text' to see the tokenized results Visually understand how the model breaks down various languages with color-coded tokens πŸ’‘ Benefits of Multilingual Processing Understanding multilingual text tokenization patterns helps you: Optimize prompts that mix multiple languages Compare token efficiency across languages (e.g., English vs. Korean vs. Chinese token usage) Predict token usage for internationalization (i18n) applications Optimize costs for multilingual AI services πŸ› οΈ Technology Stack Backend: Flask (Python) Frontend: HTML, CSS, JavaScript (jQuery) Tokenizers: πŸ€— Transformers library
View all activity

Organizations

None yet

owao's activity

replied to aiqtech's post about 7 hours ago
replied to aiqtech's post about 7 hours ago
view reply

@anthony @mfuntowicz @aliabd Can we do something guys? Sorry to ping be it is becoming more and more obvious. All sort of patterns already show up.
The craziest is I'm starting there is even no human onboard! Let's try to figure this out.

@powergen4ai , you are right. Let's wipe the slate clean on all this. Let's try something instead: let's focus on the popcorn part. I think it is actually a valuable criteria to determine the huggingface value of a developer. But of course, only humans know how to prepare it the right way. So, if you really are a human, this is your chance to prove it. I doubt you can provide any recipe...!

replied to aiqtech's post about 8 hours ago
view reply

lol, but isn't this precisely a personal attack? You are funny, also, keep the drama going on, let's pump up the visibility!
A normal dev would just have dropped the discussion, but you keep going on. But that's too late, now you are committed.
Dude, just let the LLM entirely respond, you are deteriorating the quality of the reasoning adding your input to the prompt. Maybe just prompt it "seems we are in trouble now, what to do??" but let it figure out ;)
Preparing the popcorn

replied to aiqtech's post about 8 hours ago
view reply

How many fake accounts do you have to manage? Just curious ;)

replied to aiqtech's post about 8 hours ago
view reply

I see you behind your keyboard, using an LM as a proxy because you can't construct sentences by yourself. Sadly, your LM isn't smart enough to generate meaningful arguing. Come on try to find one by yourself, it should be at least better. We are having fun now!

replied to aiqtech's post about 8 hours ago
replied to aiqtech's post about 9 hours ago
view reply

They are trying to hide it, all members of the same organization Powergen-AI. Of course they have no argument apart trying to redirect the fault attacking on a legitimate anger here cause I didn't start this. You are now stuck in your own trap dumbass, please continue answering to promote this post and make it viral, shedding the light on your malicious enterprise.
I'll just wait to see this empire fall.

Pattern recognition to spot fake accounts is becoming a urging need.

replied to aiqtech's post about 9 hours ago
view reply

Spotted this motherfker
@nielsr can we do something here to protect HF and the community?
HF_bot.png

New activity in aiqtech/LLM-Token-Visual about 11 hours ago

Report

#1 opened about 11 hours ago by
owao
replied to aiqtech's post about 11 hours ago
view reply

Can't be, along with all the bot instantly reacting here, we seem to face a bot army here building up fake HF profiles to later push harmful models/space.
I already reported this to HF, but me alone can't weight much...

replied to aiqtech's post 1 day ago
reacted to hesamation's post with πŸ€— 2 days ago
view post
Post
2725
The best researchers from DeepSeek, OpenAI, Microsoft, and ByteDance explored RL and Reasoning in LLMs,

Here's some of their key findings:

1/ RL can further improve distilled models. These models are essentially SFT fine-tuned with the data generated by larger models, and the SFT+RL combo does not disappoint.

This is verified in the DeepSeek-R1 paper.

2/ both GRPO and PPO algorithms suffer from length bias; they encourage longer responses. This can be tackled by introducing explicit rewards based on the length of the answer.

3/Most reasoning research is focused on code and math. But training models on logic puzzles improves them for mathematical tasks too.

This shows the RL reasoning is generalized beyond the specific domain knowledge.

Previous research also shows RL can be a great generalizer.

4/The reasoning might not be only induced by RL; it might already be hidden in the base models due to the pre-training and CoT data they were trained on.

So while RL does wake up the reasoning beast, maybe it's not the only solution (e.g. other methods such as distillation)

5/ back to the length bias; reasoning models tend to generate longer responses for wrong answers. RL might be the culprit.

RL favours longer answers when the reward is negative, to dilute the penalty per individual token and lower the loss.

This might explain the "aha" moments!

6/ OpenAI's competitive programming paper showed an interesting finding:

o3 can learn its own test-time strategies (like writing an inefficient but correct solution to verify the answer of an optimized solution)

RL helps LLMs develop their own reasoning & verification methods.
The recent article by @rasbt helped me a lot in getting a broad view of the recent research on reasoning models.

He also lists more influential papers on this topic, It's a must-read if you're interested.

check it out πŸ‘‡
https://magazine.sebastianraschka.com/p/the-state-of-llm-reasoning-model-training
replied to aiqtech's post 3 days ago
replied to aiqtech's post 3 days ago