Hugging Face's logo
Harrison Kinsley's picture
100 22

Harrison Kinsley PRO

Sentdex
MarcorpAI's profile picture wodole's profile picture romitjain's profile picture
·
https://youtube.com/Sentdex
Sentdex
Sentdex

AI & ML interests

None yet

Organizations

HydraLM's profile picture SkunkworksAI's profile picture AdapterOcean's profile picture ZeroGPU Explorers's profile picture Social Post Explorers's profile picture ruliad's profile picture ruliad's profile picture

Posts 2

view post
Post
Working through the Reddit dataset, one thing that occurs to me is we pretty much always train LLMs to be a conversation between 2 parties like Bot/Human or Instruction/Response.

It seems far more common with internet data that we have multi-speaker/group discussions with a dynamic number of speakers. This also seems to be more realistic to the real world too and requires a bit more understanding to model.

Is there some research into this? I have some ideas of how I'd like to implement it, but I wonder if some work has already been done here?
view post
Post
Hi, welcome to my first post here!

I am slowly wrangling about 5 years of reddit comments (2015-2020). It's a total of billions samples that can be filtered as comment-reply pairs, chains of discussion, filtered by subreddit, up/down votes, controversy, sentiment, and more.

Any requests or ideas for curated datasets from here? I'll also tinker with uploading the entire dataset potentially in chunks or something, but it's quite a few terabytes in total, so I'll need to break it up still. I have some ideas for datasets I personally want too, but curious if anyone has something they'd really like to see that sounds interesting too.