Deploying Language Models on Azure Kubernetes: A Complete Beginner's Guide
•
1
Prasanna Kumar V
vpkprasanna
AI & ML interests
None yet
Recent Activity
upvoted
an
article
10 days ago
New activity
17 days ago
meta-llama/Llama-3.2-11B-Vision
updated
a model
about 1 month ago
vpkprasanna/rera_lora_model_qwen2_merged
Articles
Organizations
Posts
1
Post
496
# Excited to Share: New LLM Tokenization - Convert Text to tokens and vice versa! 🚀
I've just developed a powerful tool for anyone working with Language Models (LLMs) or diving into Natural Language Processing (NLP).
🔍 Introducing the LLM Tokenization - Convert Text to tokens and vice versa!!
Key Features:
- Convert text to tokens and token IDs
- Reverse engineer: convert token IDs back to text
- Support for popular models: LLama3 (Will add more models iteratively)
- User-friendly Gradio interface for easy interaction
Whether you're debugging your NLP pipeline, exploring how different models tokenize text, or just curious about the inner workings of LLMs, this tool is for you!
👩💻 Tech Stack:
- Python
- Gradio for the web interface
- Hugging Face Transformers for tokenization
The application is deployed in Hugging Face spaces as Gradio application
🔗 Try it out: https://lnkd.in/g6R5z9k2
#NLP #MachineLearning #AI #PythonDevelopment #OpenSource
I've just developed a powerful tool for anyone working with Language Models (LLMs) or diving into Natural Language Processing (NLP).
🔍 Introducing the LLM Tokenization - Convert Text to tokens and vice versa!!
Key Features:
- Convert text to tokens and token IDs
- Reverse engineer: convert token IDs back to text
- Support for popular models: LLama3 (Will add more models iteratively)
- User-friendly Gradio interface for easy interaction
Whether you're debugging your NLP pipeline, exploring how different models tokenize text, or just curious about the inner workings of LLMs, this tool is for you!
👩💻 Tech Stack:
- Python
- Gradio for the web interface
- Hugging Face Transformers for tokenization
The application is deployed in Hugging Face spaces as Gradio application
🔗 Try it out: https://lnkd.in/g6R5z9k2
#NLP #MachineLearning #AI #PythonDevelopment #OpenSource
spaces
1
models
3
datasets
None public yet