12.5k
Open LLM Leaderboard
🏆
Track, rank and evaluate open LLMs and chatbots
Shashi Kumar
sasikr2
·
AI & ML interests
None yet
Recent Activity
reacted
to
singhsidhukuldeep's
post
with 👍
6 days ago
Fascinating deep dive into Swiggy's Hermes - their in-house Text-to-SQL solution that's revolutionizing data accessibility!
Hermes enables natural language querying within Slack, generating and executing SQL queries with an impressive <2 minute turnaround time. The system architecture is particularly intriguing:
Technical Implementation:
- Built on GPT-4 with a Knowledge Base + RAG approach for Swiggy-specific context
- AWS Lambda middleware handles communication between Slack UI and the Gen AI model
- Databricks jobs orchestrate query generation and execution
Under the Hood:
The pipeline employs a sophisticated multi-stage approach:
1. Metrics retrieval using embedding-based vector lookup
2. Table/column identification through metadata descriptions
3. Few-shot SQL retrieval with vector-based search
4. Structured prompt creation with data snapshots
5. Query validation with automated error correction
Architecture Highlights:
- Compartmentalized by business units (charters) for better context management
- Snowflake integration with seamless authentication
- Automated metadata onboarding with QA validation
- Real-time feedback collection via Slack
What's particularly impressive is how they've solved the data context challenge through charter-specific implementations, significantly improving query accuracy for well-defined metadata sets.
Kudos to the Swiggy team for democratizing data access across their organization. This is a brilliant example of practical AI implementation solving real business challenges.
liked
a Space
about 2 months ago
open-llm-leaderboard/open_llm_leaderboard
liked
a Space
about 2 months ago
bigcode/bigcode-models-leaderboard
Organizations
sasikr2's activity
liked
3
Spaces
about 2 months ago
liked
a
model
5 months ago