Models & ArchitectureUpdated 2026-10-24
Model Distillation
Training a smaller student model on a larger teacher model's outputs to compress capability.
#蒸馏#小模型#成本
In plain language
Distillation lowers serving cost and improves small models on specific tasks. It does not copy the teacher's full capability; the student remains capacity-limited. Data quality and teacher output distribution matter.
Example
Label 100k support conversations with a large model and distill a locally deployable small model.
Recommended tools
Related terms
Engineering
Fine-tuning
Continue training a model on task data to better match a style or task.
3 related terms
FundamentalsFeatured
Large Language Model
A model trained on massive text to understand and generate natural language.
3 related terms
Models & Architecture
Quantization
Representing model weights at lower precision to reduce memory and inference cost.
3 related terms
Applications
Agentic Workflow
Breaking a task into steps planned by AI, executed by tools and reviewed by humans.
3 related terms