Attention Mechanism
A computation that lets a model dynamically weigh how relevant other positions are when processing one position.
In plain language
Attention uses queries, keys and values: each token compares its query with all keys, turns the similarities into weights and aggregates the values. It solves the sequential bottleneck and long-range decay of recurrent networks. Multi-head attention lets the model capture relationships in several representation subspaces at once. The cost grows quadratically with sequence length, which is why long context relies on sparse attention, sliding windows or KV caching.
Example
When translating “the bank by the river,” attention helps the model distinguish the riverbank from a financial bank instead of translating word by word.
Recommended tools
Related terms
KV Cache
Caching keys and values of processed tokens during inference so history is not recomputed for every new token.
Positional Encoding
A mechanism that injects word-order information into a Transformer, since self-attention alone is order-agnostic.
Transformer
A neural network architecture built around self-attention and parallel training, the foundation of modern large models.
Agent Memory
A mechanism for an agent to store and retrieve user preferences, past tasks and key facts across sessions.