Transfer Learning & Fine-Tuning
AI · Intermediate
Definition
Transfer learning reuses a model trained on a big dataset and fine-tunes it on a smaller task.
Gets strong results with little data and compute.
Diagram
Flowchart
Example: Trainable parameter share
frozen, head = 25_000_000, 50_000
print(round(head / (frozen + head) * 100, 2), '%')Output: 0.2 %
Key points
- LoRA fine-tunes LLMs cheaply
- Use a small learning rate
Interview questions
Why freeze early layers?
They hold general features (edges, grammar) that transfer well.