Hyperparameter Tuning
Machine Learning · Intermediate
Definition
Hyperparameters are settings chosen before training (learning rate, depth, k); tuning searches for the best values.
Well-tuned models can be much more accurate.
Diagram
Flowchart
Example: Grid search
scores = {(0.1, 3): 0.81, (0.1, 5): 0.86, (0.01, 5): 0.84}
print(max(scores, key=scores.get))Output: (0.1, 5)
Complexity
Grid search cost grows exponentially with parameters
Key points
- Random search is often more efficient
- Bayesian optimization is smarter
Interview questions
Parameter vs hyperparameter?
Parameters are learned (weights); hyperparameters are set before training.