Next-Token Prediction
The training objective of predicting the next token given the preceding text.
In plain language
LLMs appear to answer questions, but the underlying objective is maximizing the probability of the next token given context. This simple, scalable objective forces models to learn grammar, factual associations and some reasoning patterns. It also explains limitations: the model optimizes for continuing like the training data, not for truth or logical necessity. Hallucination, overconfidence and sycophancy partly follow from this objective and the data distribution.
Example
Given “the capital of France is,” the model assigns the highest probability to “Paris,” but it does not know the statement is true in the world.
Recommended tools
Related terms
Hallucination
When a model produces plausible but factually wrong or unsupported content.
Large Language Model
A model trained on massive text to understand and generate natural language.
Token
The smallest unit of text a model processes, often a word, subword or character group.
Agent Memory
A mechanism for an agent to store and retrieve user preferences, past tasks and key facts across sessions.