Randomized Positional Encodings Boost Length Generalization of Transformers
Paper
•
2305.16843
•
Published
•
2
Robert Csordas
robertcsordas
AI & ML interests
Systematic generalization, algorithmic reasoning
Recent Activity
authored
a paper
about 1 month ago
Randomized Positional Encodings Boost Length Generalization of
Transformers
authored
a paper
about 1 month ago
Mindstorms in Natural Language-Based Societies of Mind
authored
a paper
about 1 month ago
MoEUT: Mixture-of-Experts Universal Transformers
Organizations
None yet
robertcsordas's activity
authored
5
papers
about 1 month ago
Mindstorms in Natural Language-Based Societies of Mind
Paper
•
2305.17066
•
Published
•
3
MoEUT: Mixture-of-Experts Universal Transformers
Paper
•
2405.16039
•
Published
•
1
MrT5: Dynamic Token Merging for Efficient Byte-level Language Models
Paper
•
2410.20771
•
Published
•
3
A Modern Self-Referential Weight Matrix That Learns to Modify Itself
Paper
•
2202.05780
•
Published
updated
a
model
8 months ago