-
Attention Zoo: Summary of SSMs and Transformers
An interactive guide to modern sequence models, explore architectures and recurrences across the linear-softmax landscape.
-
Raven (Part-2)
Architecture and Results
-
Raven (Part-1)
Memory as a set of Slots
-
On the Legacy of Linear Transformers in Positional Embeddings 📍
Duality of Forget Gates and Position Embeddings in Sequence Modeling
-
LION 🦁 Part IV - Results
Comprehensive results on Vision, MLM and more LION variants
