FundamentalsUpdated 2026-10-24
Inference
The computation a trained model performs to produce output from an input.
#基础#成本#性能
In plain language
Inference differs from training: training updates parameters, while inference only runs forward computation. Cost depends on model size, input and output length, and hardware. APIs usually charge per token; local deployments are measured by GPU memory and latency.
Example
Every API question and answer is one inference run.
Recommended tools
Related terms
Fundamentals
Latency
The time from sending a request to receiving the model response.
3 related terms
Models & Architecture
Quantization
Representing model weights at lower precision to reduce memory and inference cost.
3 related terms
FundamentalsFeatured
Token
The smallest unit of text a model processes, often a word, subword or character group.
3 related terms
Applications
Agentic Workflow
Breaking a task into steps planned by AI, executed by tools and reviewed by humans.
3 related terms