How a Machine Learning Model Learns: The Complete Training Loop | Day 30

Welcome to Day 30 of 365 Days of AI! If you have followed along over the past few days, you have already encountered several individual building blocks of machine learning: features, model parameters, loss functions, and yesterday, Gradient Descent (Day 29).

Today, we bring all of these pieces together. We are going to answer one fundamental question: How does a machine learning model actually learn?

When people say an AI model is “learning,” it sounds almost magical—as if a digital brain is reading a textbook and gaining wisdom. But under the hood, machine learning follows a precise, mechanical feedback loop. By the end of this lesson, you will understand the complete training loop that powers almost every model in modern artificial intelligence, from simple regression models to massive Large Language Models (LLMs).

Coach’s Note: Learning in machine learning is not memorization. It is an iterative numerical adjustment process. Once you see the full loop, the mystery disappears and clarity takes its place.

What You Will Learn Today

  • The exact definition of “learning” in a machine learning context.
  • The 5 distinct steps of the machine learning training loop.
  • How parameters (weights and biases) transform inputs into outputs.
  • How loss functions and gradient descent work together as a feedback mechanism.
  • A clean Python code representation of the complete learning cycle.
  • Common misconceptions about training versus inference.

What Does “Learning” Mean in Machine Learning?

In human terms, learning means acquiring knowledge, understanding concepts, and applying intuition to new situations. For a computer, learning is far more mathematical.

In machine learning, learning is the process of automatically adjusting internal numerical values (called parameters or weights) to minimize error on a given task.

Think of an un-trained model as a brand-new audio mixing board with hundreds of uncalibrated dials and sliders. At first, all dials are set randomly. When you feed data into the system, the music sounds like harsh noise. Learning is the process of repeatedly listening to the sound, measuring how far off it is from harmony, and slightly turning each dial until the output is clear.

The 5-Step Learning Loop (Mental Model)

Keep this sequence in your mind whenever you think about model training:

  1. Initialize: Start with initial guesses for model parameters (weights & bias).
  2. Predict (Forward Pass): Pass input data through the model to get a prediction.
  3. Calculate Loss: Compare the prediction to the real target to measure error.
  4. Compute Gradients (Backward Step): Determine how each parameter contributed to the error.
  5. Update Parameters: Adjust the weights using Gradient Descent and repeat.

Step-by-Step: Inside the Training Loop

Let’s walk through each stage of the loop in detail so you can see how data flows through a model during training.

1. Initialization

Before a model can learn, its parameters must start somewhere. In most cases, weights are initialized with small random numbers, and biases are set to zero. At this stage, the model knows nothing about the world.

2. The Forward Pass (Prediction)

The model takes input features ($X$) and passes them through its current mathematical formula to compute an output prediction ($\hat{y}$). Because the weights were randomly chosen at the start, this initial prediction is usually completely wrong.

Formula View: $\hat{y} = X \cdot W + b$
Where $X$ is input data, $W$ represents weights, $b$ is the bias, and $\hat{y}$ (y-hat) is the model’s prediction.

3. Calculating the Loss (Error Measurement)

Next, we pass both the prediction ($\hat{y}$) and the true target value ($y$) into a Loss Function. The loss function outputs a single numerical score that quantifies how wrong the model is.

  • If the loss is high, the model performed poorly.
  • If the loss is zero (or close to zero), the model’s prediction was highly accurate.

4. Computing Gradients (Backpropagation / Derivative Step)

Measuring the error isn’t enough; we need to know which specific weights caused the error and in which direction they should move. Using calculus (specifically partial derivatives), the system computes the gradient of the loss with respect to each parameter.

As we saw on Day 29, the gradient tells us the slope of the loss landscape at our current position.

5. Parameter Update (Gradient Descent Step)

Finally, we update the model’s parameters using the update rule:

# Parameter update rule
new_weight = current_weight - (learning_rate * gradient)

By subtracting a small fraction (controlled by the learning rate) of the gradient, we nudge the weights in the exact direction that lowers the loss. We then repeat steps 2 through 5 over thousands of batches of data until the loss stops decreasing—a state known as convergence.


The Training Loop in Code

Here is a minimalist Python example showing how these 5 steps work in practice using basic operations. This illustrates the exact structure used in frameworks like PyTorch and TensorFlow.

# Minimal Python illustration of a single training epoch

# 1. Inputs (X) and True Targets (y)
X = [1.0, 2.0, 3.0, 4.0]
y_true = [2.0, 4.0, 6.0, 8.0]  # The true pattern is y = 2 * x

# Initialize weight randomly and bias to 0
weight = 0.5
bias = 0.0
learning_rate = 0.05

# Run training loop for multiple iterations (epochs)
for epoch in range(100):
    total_loss = 0.0
    weight_gradient = 0.0
    bias_gradient = 0.0
    N = len(X)
    
    # Process each sample in the dataset
    for i in range(N):
        # Step 2: Forward Pass (Predict)
        y_pred = weight * X[i] + bias
        
        # Step 3: Calculate Loss (Mean Squared Error per sample)
        error = y_pred - y_true[i]
        total_loss += error ** 2
        
        # Step 4: Compute Gradients (Partial Derivatives)
        weight_gradient += (2 / N) * error * X[i]
        bias_gradient += (2 / N) * error
    
    # Step 5: Update Parameters
    weight -= learning_rate * weight_gradient
    bias -= learning_rate * bias_gradient

print(f"Trained Weight: {weight:.2f}")  # Target value is ~2.00
print(f"Trained Bias: {bias:.2f}")      # Target value is ~0.00

Training vs. Inference: An Important Distinction

Beginners often confuse training a model with using a model. Let’s make this distinction explicit:

Feature Training Phase Inference Phase (Deployment)
Goal Learn optimal parameters ($W, b$) Generate predictions for new data
Data Required Labeled training dataset ($X$ and $y$) New input data ($X$ only)
Weights State Updated continuously in a loop Fixed / Frozen
Computation Requires forward pass, loss calculation, & backpropagation Requires forward pass only

Common Misconceptions

Misconception 1: “AI models learn continuously while answering user prompts.”

Fact: Standard AI models (like ChatGPT or standard vision classifiers) do not learn during normal inference. Their parameters are frozen after training. When you ask ChatGPT a question, it runs a forward pass to produce text, but its internal weights remain unchanged.

Misconception 2: “Learning means storing all training data in memory.”

Fact: Machine learning models compress patterns into mathematical weights, not raw data files. Once training is complete, the original training data is usually discarded; only the learned weights are stored in the final model file.


Mini Exercise: Trace the Feedback Loop

To lock this concept into your memory, answer this quick scenario in your head:

Imagine a house price prediction model. It currently predicts a home’s value is $300,000, but the actual sale price was $400,000.

  1. Is the loss positive, zero, or negative?
  2. Did the model underestimate or overestimate?
  3. In the next step, will the gradient descent update push the weight connected to square footage higher or lower?
Click to reveal the answers

Answers:

  1. The loss is positive (e.g., Error = $300k – $400k = -$100k$, but squared loss yields a positive scalar value).
  2. The model underestimated the true price by $100,000.
  3. The update rule will increase the weights associated with positive features (like square footage) so future predictions move closer to $400,000.

Quick Recall Checkpoint

Test your understanding of today’s core ideas:

1. What are the 5 stages of the standard ML training loop?

Initialization, Forward Pass (Prediction), Loss Calculation, Gradient Computation, and Parameter Update (Optimization).

2. What happens to a model’s weights during the inference phase?

During inference, the weights are frozen (fixed). No learning or gradient computation takes place.

3. Why is the gradient necessary during training?

The gradient tells the optimizer the slope of the loss landscape, indicating which direction and by how much each weight should be adjusted to decrease prediction error.


Key Takeaways

  • Machine learning is an iterative optimization loop that adjusts weights to minimize error.
  • The forward pass computes predictions; the backward step calculates how to fix mistakes.
  • Loss functions quantify error, while gradient descent updates parameters step by step.
  • Training modifies weights; inference uses fixed weights to predict new unseen data.

What Comes Next?

Now that you understand the universal learning loop that drives all machine learning models, it is time to put this loop into action on our very first complete algorithm.

Tomorrow, on Day 31, we will explore Linear Regression Explained—where we apply today’s 5-step learning loop to predict real numeric targets from scratch!

Prefer learning in Hinglish? Check out the Hinglish version of 365 Days of AI.

Leave a Comment

Your email address will not be published. Required fields are marked *

Scroll to Top