Models & ArchitectureUpdated 2026-10-24
Quantization
Representing model weights at lower precision to reduce memory and inference cost.
#量化#部署#显存
In plain language
Common precisions are FP16, INT8 and INT4. Quantization dramatically lowers memory needs, enabling larger models on consumer GPUs or edge devices, but may cost quality. Compare on your own tasks.
Example
A 7B model needs about 14GB in FP16 and can drop to 4-6GB at INT4.
Recommended tools
Related terms
Fundamentals
Inference
The computation a trained model performs to produce output from an input.
3 related terms
FundamentalsFeatured
Large Language Model
A model trained on massive text to understand and generate natural language.
3 related terms
Models & ArchitectureFeatured
Open-Weight Model
A model whose weights can be downloaded, deployed locally and modified.
3 related terms
Applications
Agentic Workflow
Breaking a task into steps planned by AI, executed by tools and reviewed by humans.
3 related terms