Underfitting Explained: When Machine Learning Models Are Too Simple | Day 24

Yesterday, in Day 23: Overfitting Explained, we looked at what happens when a machine learning model works too hard. An overfitted model memorizes every tiny detail, noise, and quirk in the training data, only to fail miserably when faced with fresh, unseen data.

Today, we explore the exact opposite failure mode: Underfitting.

If overfitting is like a student who memorizes every exact practice problem without understanding the underlying math, underfitting is like a student who skims one page of the textbook 5 minutes before the exam and tries to guess every answer. The model is simply too basic to grasp the pattern in the data.

👋 Coach Krish’s Welcome

Welcome to Day 24! In practical Machine Learning engineering, building a great model is a balancing act between two extremes. You already know what happens when a model is overly complex and memorizes noise. Today, you’ll learn how to spot when a model is failing because it’s too simple, why that happens, and how to fix it step by step.

What You Will Learn Today

  • The core definition: What underfitting actually means in Machine Learning.
  • The visual intuitive model: How underfitting looks when mapping inputs to outputs.
  • The metric signature: How training loss and validation loss behave under an underfitted model.
  • Root causes: Why models underfit (model capacity, poor features, over-regularization).
  • Practical remedies: Clear techniques to increase model capacity and fix underfitting.
  • The bridge to Day 25: How underfitting directly leads to the fundamental concept of High Bias.

1. What Is Underfitting?

Underfitting occurs when a machine learning model is unable to capture the underlying trend or pattern of the dataset. It happens when the model lacks the mathematical complexity (capacity) needed to represent the relationship between input features and target outputs.

As a result, an underfitted model performs poorly on both the training data and new, unseen validation data.

Key Distinction: An overfitted model performs brilliantly on training data but poorly on validation data. An underfitted model performs poorly on both.

2. The Mental Model: Straight Line vs. Curved Reality

Imagine you have data points plotted on a graph that clearly form a smooth curve (like a parabola representing house prices based on square footage and age). The relationship is naturally non-linear.

If you force a basic Linear Regression model (a straight line) to fit this curved dataset, it cannot bend. No matter how much data you feed it or how long you train it, a straight line will always miss the curve.

Real-World Scenario: Predicting House Prices

Suppose house prices grow exponentially as size increases in a popular neighborhood. If your model uses only a simple linear formula without accounting for non-linear relationships or key features like location quality and room count, it will underpredict expensive homes and overpredict cheap homes. It has underfitted the problem space.

Where the analogy breaks down: In real-world AI engineering, models operate in high-dimensional space with dozens or hundreds of features. Underfitting isn’t just about drawing a line on a 2D graph—it means the model’s hypothesis space lacks the capacity to capture interactions between complex variables.

3. The Diagnostic Signature: Loss Curve Behavior

How do machine learning engineers know their model is underfitting? They look at the loss metrics during training.

Scenario Training Loss / Error Validation Loss / Error Diagnosis
Underfitting High High Model is too simple to learn the data pattern.
Overfitting Extremely Low High Model memorized training noise; fails to generalize.
Good Fit Low Low (Close to Training Loss) Model learned true underlying patterns.

When tracking performance over training time (epochs), an underfitting model will show a flat or high training loss curve that plateaus early at an unacceptably high error value.

4. Why Do Models Underfit?

Underfitting generally stems from four primary architectural and workflow issues:

  1. Insufficient Model Complexity: Using a model with too few parameters (e.g., a 1-layer neural network for complex image classification, or a shallow decision tree for non-linear data).
  2. Poor Feature Selection: The input features provided to the model do not contain enough signal to predict the target outcome accurately.
  3. Over-Regularization: Applying overly strict L1/L2 penalty weights or excessive Dropout, which forces model weights down so heavily that the model cannot learn true patterns.
  4. Under-Training: Stopping the training process prematurely (too few epochs or learning rate set too high/low) before the optimizer has converged.

5. Technical Example: Observing Underfitting in Python

Let me show you a quick Python example using scikit-learn that demonstrates underfitting versus an appropriate polynomial fit.

import numpy as np
from sklearn.linear_model import LinearRegression
from sklearn.preprocessing import PolynomialFeatures
from sklearn.pipeline import make_pipeline
from sklearn.metrics import mean_squared_error

# 1. Generate non-linear synthetic data (a sine wave with small noise)
np.random.seed(42)
X = np.sort(np.random.rand(40, 1) * 6, axis=0)
y = np.sin(X).ravel() + np.random.normal(0, 0.1, X.shape[0])

# 2. Fit a Simple Linear Model (Degree 1) -> UNDERFITTING
underfitted_model = LinearRegression()
underfitted_model.fit(X, y)
y_pred_underfit = underfitted_model.predict(X)

# 3. Fit a Polynomial Model (Degree 3) -> GOOD FIT
good_model = make_pipeline(PolynomialFeatures(degree=3), LinearRegression())
good_model.fit(X, y)
y_pred_good = good_model.predict(X)

# Compute Mean Squared Error (Higher MSE = Worse Performance)
mse_underfit = mean_squared_error(y, y_pred_underfit)
mse_good = mean_squared_error(y, y_pred_good)

print(f"Underfitted Model (Linear) Training MSE: {mse_underfit:.4f}")  # High error!
print(f"Appropriate Model (Degree 3) Training MSE: {mse_good:.4f}")    # Significantly lower error!

Notice how the simple linear model returns a high training error because a single line cannot capture the curve of a sine wave. That high training error is the classic red flag of underfitting.

6. How to Fix Underfitting: The ML Action Plan

When your diagnostic metrics indicate underfitting, apply these remedies systematically:

🛠️ Actionable Solutions for Underfitting

  • Increase Model Capacity: Switch to a more expressive algorithm (e.g., add layers/neurons to a neural network, increase max tree depth in XGBoost/Random Forest).
  • Perform Feature Engineering: Create richer features (e.g., polynomial features, domain-specific ratios, interaction terms) to give the model better input signals.
  • Reduce Regularization: Lower L1/L2 weight decay penalties, reduce dropout percentage, or loosen constraints on tree depth.
  • Train Longer / Better: Increase the number of training epochs or adjust your optimization algorithm (e.g., fine-tune learning rate schedule).
  • Clean / Reduce Noise in Features: Ensure your input data isn’t corrupted with misleading or irrelevant noisy values.

7. Common Misconceptions About Underfitting

Misconception 1: “Deep Neural Networks can never underfit.”
Fact: While deep neural networks have massive capacity, they can underfit easily if the learning rate is misconfigured, if gradient updates vanish, or if training is stopped after just 1 or 2 epochs.

Misconception 2: “Any non-zero training error means the model is underfitting.”
Fact: Real-world data contains natural noise (irreducible error). Expecting zero training error is unrealistic and dangerous, as aiming for zero error leads straight into overfitting.

💡 Note from Coach Krish

Think of model training like tuning a guitar string. If the string is too loose, you get a dull, flat noise (Underfitting). If you tighten it too aggressively, the string snaps (Overfitting). Your job as an AI engineer is to find the sweet spot where the tension is just right—where the model learns real structural patterns without getting distracted by random noise.

8. Quick Recall Checkpoint

Test your understanding before moving forward:

Question 1: What distinguishes underfitting from overfitting when examining training error?

Answer: An underfitted model displays high training error (it performs poorly even on training data), whereas an overfitted model has extremely low training error but high validation error.

Question 2: If increasing regularization (L2 penalty) reduces overfitting, what happens if you apply too much regularization?

Answer: Excessive regularization constrains the model’s parameters too strictly, restricting its ability to learn real patterns and causing the model to underfit.

Question 3: Name two concrete ways to fix an underfitted model.

Answer: 1) Increase model complexity (e.g., deeper trees, more neural network layers). 2) Engineer better features or reduce regularization strength.

Key Takeaways

  • Underfitting occurs when a model is too simple to capture the underlying structure of the data.
  • The main diagnostic signal of underfitting is high error on both training and validation sets.
  • Underfitting is caused by low model capacity, missing features, excessive regularization, or insufficient training time.
  • Fix underfitting by making the model more flexible, adding features, or loosening regularization parameters.

What Comes Next?

You now know both ends of the modeling error spectrum:

  • Day 23: Overfitting (Model is too complex / high variance)
  • Day 24: Underfitting (Model is too simple / high bias)

Tomorrow, in Day 25: Bias vs Variance, we will unite these two concepts under machine learning’s most famous unified framework: The Bias-Variance Tradeoff.

📚 Prefer learning in Hinglish? Check out the 365 Days of AI Hinglish Edition.

Leave a Comment

Your email address will not be published. Required fields are marked *

Scroll to Top