Article
Welcome, Gradio 5
•
125
Sam Witteveen
samwit
AI & ML interests
conversational AI, NLU
Recent Activity
upvoted
an
article
4 months ago
Welcome, Gradio 5
liked
a model
4 months ago
Mozilla/Llama-3.2-3B-Instruct-llamafile
updated
a model
5 months ago
samwit/cot_gf_plat_2k_phi35_full
Organizations
samwit's activity
upvoted
an
article
4 months ago
liked
a
model
4 months ago
updated
a
model
5 months ago
Is there a ver of Transformers that we can use to run this yet?
3
#1 opened 8 months ago
by
samwit
liked
a
dataset
8 months ago
updated
a
model
9 months ago
liked
a
dataset
9 months ago
liked
a
dataset
10 months ago
liked
a
dataset
12 months ago
reacted to
akhaliq's
post with 👍
12 months ago
Post
In Search of Needles in a 10M Haystack
Recurrent Memory Finds What LLMs Miss
paper addresses the challenge of processing long documents using generative transformer models. To evaluate different approaches, we introduce BABILong, a new benchmark designed to assess model capabilities in extracting and processing distributed facts within extensive texts. Our evaluation, which includes benchmarks for GPT-4 and RAG, reveals that common methods are effective only for sequences up to 10^4 elements. In contrast, fine-tuning GPT-2 with recurrent memory augmentations enables it to handle tasks involving up to 10^7 elements. This achievement marks a substantial leap, as it is by far the longest input processed by any open neural network model to date, demonstrating a significant improvement in the processing capabilities for long sequences
Recurrent Memory Finds What LLMs Miss
paper addresses the challenge of processing long documents using generative transformer models. To evaluate different approaches, we introduce BABILong, a new benchmark designed to assess model capabilities in extracting and processing distributed facts within extensive texts. Our evaluation, which includes benchmarks for GPT-4 and RAG, reveals that common methods are effective only for sequences up to 10^4 elements. In contrast, fine-tuning GPT-2 with recurrent memory augmentations enables it to handle tasks involving up to 10^7 elements. This achievement marks a substantial leap, as it is by far the longest input processed by any open neural network model to date, demonstrating a significant improvement in the processing capabilities for long sequences
liked
a
dataset
about 1 year ago
updated
a
dataset
about 1 year ago
liked
a
model
about 1 year ago
liked
a
dataset
over 1 year ago
authored
a
paper
over 1 year ago
liked
a
dataset
over 1 year ago
updated
4
models
over 1 year ago