Dropout & Batch Normalization
AI · Advanced
Definition
Dropout randomly disables neurons during training; batch norm normalizes layer inputs per mini-batch.
Both improve generalization and training stability.
Diagram
Flowchart
Example: Normalize a batch
x = [2, 4, 6]
m = sum(x) / 3
s = (sum((v - m) ** 2 for v in x) / 3) ** 0.5
print([round((v - m) / s, 2) for v in x])Output: [-1.22, 0.0, 1.22]
Key points
- Dropout is off at inference
- Typical dropout rate 0.2–0.5
Interview questions
Why is dropout off during testing?
Predictions should be deterministic and use the full network.