Models & ArchitectureUpdated 2026-10-24
Quantization
Representing model weights at lower precision to reduce memory and inference cost.
#量化#部署#显存
In plain language
Common precisions are FP16, INT8 and INT4. Quantization dramatically lowers memory needs, enabling larger models on consumer GPUs or edge devices, but may cost quality. Compare on your own tasks.
Example
A 7B model needs about 14GB in FP16 and can drop to 4-6GB at INT4.
Recommended tools
Related terms
Fundamentals
Inference
The computation a trained model performs to produce output from an input.
3 related terms
FundamentalsFeatured
Large Language Model
A model trained on massive text to understand and generate natural language.
3 related terms
Models & ArchitectureFeatured
Open-Weight Model
A model whose weights can be downloaded, deployed locally and modified.
3 related terms
Engineering
Agent Memory
A mechanism for an agent to store and retrieve user preferences, past tasks and key facts across sessions.
3 related terms