A Controlled Study on Long Context Extension and Generalization in LLMs Paper • 2409.12181 • Published 7 days ago • 40
Gated Slot Attention for Efficient Linear-Time Sequence Modeling Paper • 2409.07146 • Published 14 days ago • 19
Parallelizing Linear Transformers with the Delta Rule over Sequence Length Paper • 2406.06484 • Published Jun 10 • 3
based Collection These language model checkpoints are trained at the 360M and 1.3Bn parameter scales for up to 50Bn tokens on the Pile corpus, for research purposes. • 14 items • Updated May 14 • 8