Semi-Supervised Learning Explained | Day 15 of 365 Days of AI

What Is Semi-Supervised Learning?

Semi-supervised learning is a machine learning paradigm that combines a small amount of labeled data with a large amount of unlabeled data during training.

In the real world, unlabeled data is cheap and abundant (think raw text on the internet, audio recordings, security camera feeds, or uncurated photos). Labeled data, on the other hand, is expensive, time-consuming, and requires human expertise. Semi-supervised learning lives in the high-leverage sweet spot where you use a small, expensive seed dataset to guide the model, while allowing a massive, cheap dataset to teach the model the underlying shape and structure of the problem space.

The Mental Model: The Tutor and the Workbook

How Semi-Supervised Learning Works: Self-Training & Pseudo-Labeling

One of the simplest and most widely used techniques in semi-supervised learning is Self-Training (or Pseudo-Labeling). Here is the step-by-step workflow:

1. Initial Training: Train a baseline model on the small set of labeled data ($D_{labeled}$).
2. Prediction: Use this initial model to predict labels for the large unlabeled dataset ($D_{unlabeled}$). These predictions are called pseudo-labels.
3. Confidence Filtering: Select only the pseudo-labeled examples where the model’s confidence exceeds a high threshold (e.g., probability $> 0.95$).
4. Dataset Expansion: Add these high-confidence pseudo-labeled samples into the training set.
5. Retraining: Re-train the model on the combined dataset. Repeat the process iteratively.

Technical Example: Semi-Supervised Learning in Python

Here is a concise example using Python’s scikit-learn library with a SelfTrainingClassifier. Unlabeled samples are designated by convention with a label of -1.

import numpy as np
from sklearn.datasets import make_classification
from sklearn.semi_supervised import SelfTrainingClassifier
from sklearn.ensemble import RandomForestClassifier

# 1. Generate a synthetic dataset with 1,000 samples
X, y_true = make_classification(n_samples=1000, n_features=10, random_state=42)

# 2. Simulate scarce labels: keep only 50 labels (5%), mark the rest as -1 (unlabeled)
y_semi = np.copy(y_true)
rng = np.random.RandomState(42)
unlabeled_mask = rng.rand(len(y_true)) > 0.05
y_semi[unlabeled_mask] = -1  # -1 signals unlabeled data to scikit-learn

# 3. Create a base estimator and wrap it in a SelfTrainingClassifier
base_model = RandomForestClassifier(n_estimators=50, random_state=42)
semi_model = SelfTrainingClassifier(base_model, threshold=0.8)

# 4. Train using both labeled samples and unlabeled samples (-1)
semi_model.fit(X, y_semi)

print(f"Total labeled iterations completed: {semi_model.n_iter_}")
# The model iteratively pseudo-labels high-confidence unlabelled points and retrains!

Real-World Applications

• Medical Diagnostics: Training disease detection classifiers using thousands of unlabelled MRI or X-ray scans paired with a small set of expert-annotated images.
• Speech Recognition: Raw audio feeds are endless, but transcribing audio frame-by-frame is tedious. Semi-supervised approaches use transcribed audio snippets to label large unlabelled audio archives.
• Content Moderation & Spam Detection: Platforms collect billions of user comments daily. Human moderators label a tiny fraction, while semi-supervised pipelines help propagate labels across similar comment clusters.

Common Misconceptions

Quick Recall Checkpoint