From Transformers to Mamba: Is Attention All We Need?

Learning Sequential Inputs

Recurrent Neural Network (RNN)

Long Short-Term Memory (LSTM)

Transformers: Attention is All You Need

Low-Rank Adaptation (LoRA)

Vision Transformers (ViT)

State Space Model (SSM): Maybe Attention isn’t All You Need