Hugging Face's logo

BigScience Workshop

non-profit
https://bigscience.huggingface.co
bigscienceW
bigscience-workshop
Activity Feed

AI & ML interests

A one-year long research workshop on large language models: the Summer of Language Models 21 🌸

Recent Activity

Skylion007  authored a paper 4 days ago
The GAN is dead; long live the GAN! A Modern GAN Baseline
christopher  new activity 24 days ago
bigscience/bloom-1b1-intermediate:Adding `safetensors` variant of this model
christopher  new activity 24 days ago
bigscience/bloom-7b1-intermediate:Adding `safetensors` variant of this model
View all activity

bigscience's activity

yjernite 
posted an update about 2 hours ago
view post
Post
82
🤗👤 💻 Speaking of AI agents ...
...Is easier with the right words ;)

My colleagues @meg @evijit @sasha and @giadap just published a wonderful blog post outlining some of the main relevant notions with their signature blend of value-informed and risk-benefits contrasting approach. Go have a read!

https://huggingface.co/blog/ethics-soc-7
lewtun 
posted an update 8 days ago
view post
Post
3201
I was initially pretty sceptical about Meta's Coconut paper [1] because the largest perf gains were reported on toy linguistic problems. However, these results on machine translation are pretty impressive!

https://x.com/casper_hansen_/status/1875872309996855343

Together with the recent PRIME method [2] for scaling RL, reasoning for open models is looking pretty exciting for 2025!

[1] Training Large Language Models to Reason in a Continuous Latent Space (2412.06769)
[2] https://huggingface.co/blog/ganqu/prime
lewtun 
posted an update 15 days ago
view post
Post
2085
This paper ( HuatuoGPT-o1, Towards Medical Complex Reasoning with LLMs (2412.18925)) has a really interesting recipe for inducing o1-like behaviour in Llama models:

* Iteratively sample CoTs from the model, using a mix of different search strategies. This gives you something like Stream of Search via prompting.
* Verify correctness of each CoT using GPT-4o (needed because exact match doesn't work well in medicine where there are lots of aliases)
* Use GPT-4o to reformat the concatenated CoTs into a single stream that includes smooth transitions like "hmm, wait" etc that one sees in o1
* Use the resulting data for SFT & RL
* Use sparse rewards from GPT-4o to guide RL training. They find RL gives an average ~3 point boost across medical benchmarks and SFT on this data already gives a strong improvement.

Applying this strategy to other domains could be quite promising, provided the training data can be formulated with verifiable problems!
  • 1 reply
·
christopher 
in bigscience/bloom-1b1-intermediate 24 days ago

Adding `safetensors` variant of this model
#2 opened 24 days ago by
SFconvertbot
christopher 
in bigscience/bloom-7b1-intermediate 24 days ago

Adding `safetensors` variant of this model
#4 opened 25 days ago by
SFconvertbot
NohTow 
authored 3 papers 25 days ago
mgalle 
authored a paper 26 days ago
lewtun 
posted an update 28 days ago
view post
Post
6717
We outperform Llama 70B with Llama 3B on hard math by scaling test-time compute 🔥

How? By combining step-wise reward models with tree search algorithms :)

We show that smol models can match or exceed the performance of their much larger siblings when given enough "time to think"

We're open sourcing the full recipe and sharing a detailed blog post.

In our blog post we cover:

📈 Compute-optimal scaling: How we implemented DeepMind's recipe to boost the mathematical capabilities of open models at test-time.

🎄 Diverse Verifier Tree Search (DVTS): An unpublished extension we developed to the verifier-guided tree search technique. This simple yet effective method improves diversity and delivers better performance, particularly at large test-time compute budgets.

🧭 Search and Learn: A lightweight toolkit for implementing search strategies with LLMs and built for speed with vLLM

Here's the links:

- Blog post: HuggingFaceH4/blogpost-scaling-test-time-compute

- Code: https://github.com/huggingface/search-and-learn

Enjoy!
  • 2 replies
·
gentaiscool 
authored 10 papers 29 days ago