21 10 32

Bhadresh Savani

bhadresh-savani

https://www.linkedin.com/in/bhadreshsavani/

AI & ML interests

NLP, Deep Learning, ML

Recent Activity

Reacted to lin-tan's post with 🔥 11 days ago

Can language models replace developers? #RepoCod says “Not Yet”, because GPT-4o and other LLMs have <30% accuracy/pass@1 on real-world code generation tasks. - Leaderboard https://lt-asset.github.io/REPOCOD/ - Dataset: https://huggingface.co/datasets/lt-asset/REPOCOD @jiang719 @shanchao @Yiran-Hu1007 Compared to #SWEBench, RepoCod tasks are - General code generation tasks, while SWE-Bench tasks resolve pull requests from GitHub issues. - With 2.6X more tests per task (313.5 compared to SWE-Bench’s 120.8). Compared to #HumanEval, #MBPP, #CoderEval, and #ClassEval, RepoCod has 980 instances from 11 Python projects, with - Whole function generation - Repository-level context - Validation with test cases, and - Real-world complex tasks: longest average canonical solution length (331.6 tokens) and the highest average cyclomatic complexity (9.00) Introducing hashtag #RepoCod-Lite 🐟 for faster evaluations: 200 of the toughest tasks from RepoCod with: - 67 repository-level, 67 file-level, and 66 self-contains tasks - Detailed problem descriptions (967 tokens) and long canonical solutions (918 tokens) - GPT-4o and other LLMs have < 10% accuracy/pass@1 on RepoCod-Lite tasks. - Dataset: https://huggingface.co/datasets/lt-asset/REPOCOD_Lite #LLM4code #LLM #CodeGeneration #Security

View all activity

Organizations

bhadresh-savani's activity

New activity in bhadresh-savani/distilbert-base-uncased-emotion 4 months ago

Adding `safetensors` variant of this model

#6 opened over 1 year ago by

SFconvertbot

New activity in bhadresh-savani/photo-to-cartoon 4 months ago

Create README.md

#3 opened 4 months ago by

Rikz

New activity in bhadresh-savani/electra-base-squad2 over 1 year ago

Adding `safetensors` variant of this model

#1 opened over 1 year ago by

SFconvertbot

New activity in bhadresh-savani/roberta-base-emotion over 1 year ago

Add verifyToken field to verify evaluation results are produced by Hugging Face's automatic model evaluator

#3 opened almost 2 years ago by

autoevaluator

Adding `safetensors` variant of this model

#4 opened over 1 year ago by

SFconvertbot

New activity in bhadresh-savani/distilbert-base-uncased-emotion over 1 year ago

Add verifyToken field to verify evaluation results are produced by Hugging Face's automatic model evaluator

#3 opened almost 2 years ago by

autoevaluator

New activity in bhadresh-savani/bert-base-uncased-emotion over 1 year ago

Adding `safetensors` variant of this model

#4 opened over 1 year ago by

SFconvertbot

Add verifyToken field to verify evaluation results are produced by Hugging Face's automatic model evaluator

#2 opened almost 2 years ago by

autoevaluator

New activity in bhadresh-savani/electra-base-emotion over 1 year ago

Add verifyToken field to verify evaluation results are produced by Hugging Face's automatic model evaluator

#2 opened almost 2 years ago by

autoevaluator

New activity in bhadresh-savani/bertweet-base-finetuned-emotion over 1 year ago

Add verifyToken field to verify evaluation results are produced by Hugging Face's automatic model evaluator

#3 opened almost 2 years ago by

autoevaluator

New activity in bhadresh-savani/electra-base-discriminator-finetuned-conll03-english over 1 year ago

Add evaluation results on the conll2003 config and test split of conll2003

#1 opened about 2 years ago by

autoevaluator

New activity in bhadresh-savani/bert-base-go-emotion about 2 years ago

Trouble Importing DistilBertForMultilabelSequenceClassification

#1 opened about 2 years ago by

Jamesyyjj

New activity in bhadresh-savani/electra-base-emotion over 2 years ago

Add evaluation results on the default config of emotion

#1 opened over 2 years ago by

autoevaluator

New activity in bhadresh-savani/bertweet-base-finetuned-emotion over 2 years ago

Add evaluation results on the default config of emotion

#2 opened over 2 years ago by

autoevaluator

New activity in bhadresh-savani/distilbert-base-uncased-finetuned-emotion over 2 years ago

Add evaluation results on the default config of emotion

#1 opened over 2 years ago by

autoevaluator

New activity in bhadresh-savani/bertweet-base-finetuned-emotion over 2 years ago

Align label mapping with emotion dataset

#1 opened over 2 years ago by

lewtun

New activity in bhadresh-savani/bert-base-uncased-emotion over 2 years ago

Add evaluation results on emotion dataset

#1 opened over 2 years ago by

autoevaluator

New activity in bhadresh-savani/roberta-base-emotion over 2 years ago

Add evaluation results on emotion dataset

#1 opened over 2 years ago by

autoevaluator

New activity in bhadresh-savani/distilbert-base-uncased-emotion over 2 years ago

Add evaluation results on emotion dataset

#2 opened over 2 years ago by

autoevaluator

License Question

#1 opened over 2 years ago by

mattan