Overfitting, Bias & Variance

    Machine Learning · Intermediate

    Definition

    Overfitting is when a model memorises training data and fails on new data; the bias-variance trade-off explains why.

    Understanding it is key to building models that generalise.

    Diagram

    High bias = underfitBalanced = good fitHigh variance = ove…

    Flowchart

    Split train/validation/testTrain modelCompare train vs validation errorDiagnose bias or varianceRegularise or add data

    Example: Train/test split

    from sklearn.model_selection import train_test_split
X_train, X_test = train_test_split(list(range(10)), test_size=0.2, random_state=0)
print(len(X_train), len(X_test))

    Output: 8 2

    Key points

    • Cross-validation gives reliable estimates
    • Regularisation: L1, L2, dropout
    • More data reduces variance

    Interview questions

    Signs of overfitting?

    Very low training error but high validation error.

    More Machine Learning topics

    What is Machine LearningLinear RegressionLogistic RegressionDecision Trees & Random Forest