In 2014, Ian Goodfellow introduced the Generative Adversarial Network (GAN), fundamentally shifting how AI generates data.

Instead of trying to calculate complex probability distributions directly, GANs use a brilliant architecture consisting of two distinct neural networks that compete against each other in a zero-sum game.

Let's look at the exact architecture and mathematical principles that make this possible.

1. The Core Principle: Adversarial Training

A GAN is built on game theory—specifically a Minimax Game.

The Discriminator () acts as a judge. It wants to accurately classify real images as Real (1) and fake images as Fake (0).
The Generator () acts as a counterfeiter. It wants to generate fake images so realistic that the Discriminator classifies them as Real (1).

Mathematically, they are fighting over a single Value function . The Discriminator is trying to maximize the score (by catching the fakes), while the Generator is trying to minimize the score (by tricking the Discriminator).

2. Inside the Generator Network ()

The Generator is a neural network designed to take random numbers and map them into a structured data format, like an image.

A. The Input: Latent Space ()

The Generator does not take an image as input. It takes a vector of purely random numbers (called noise), usually drawn from a Gaussian distribution. This random vector is referred to as Latent Vector .

Think of as the "DNA" of the image. By slightly changing the numbers in , the Generator will produce a slightly different output image.

B. The Architecture: Transposed Convolutions

In a standard CNN, we use convolutions and pooling to downsample a large image into a small vector. The Generator does the exact opposite: it must upsample.

It takes the small noise vector and passes it through a Dense layer to reshape it into a small grid.
It then uses Transposed Convolutions (sometimes incorrectly called Deconvolutions). A Transposed Convolution takes a small grid, applies weights, and expands it into a larger spatial grid.
It continues upsampling through multiple layers until it reaches the desired image size (e.g., for an RGB image).

C. The Output: Fake Data

The final layer usually uses a Tanh activation function to output the final pixel values, producing the artificially generated fake image, denoted as .

3. Inside the Discriminator Network ()

The Discriminator is much simpler to understand because it is essentially just a standard Convolutional Neural Network (CNN) used for binary classification.

A. The Input: Real and Fake

During training, the Discriminator is fed two batches of data:

True images straight from the real dataset, denoted as .
Fake images freshly produced by the Generator, denoted as .

B. The Architecture: Downsampling CNN

The Discriminator takes these images and passes them through standard Convolutional Layers and Leaky ReLU activation functions. It detects edges, shapes, and features, progressively downsampling the image.

C. The Output: The Probability

The final layer is flattened into a single node with a Sigmoid Activation Function.

It outputs a probability between and .
If is close to , the Discriminator believes the image is Real.
If is close to , the Discriminator believes the image is Fake.

4. The Training Loop (How they learn together)

Training a GAN requires a delicate balancing act. You cannot train them both at the same time. You must alternate:

Step 1: Train the Discriminator.
- We lock the Generator's weights. We feed the Discriminator real images and label them as . We feed it fake images and label them as . We update the Discriminator's weights using Back-Propagation so it gets better at spotting flaws in the fakes.
Step 2: Train the Generator.
- We lock the Discriminator's weights. We have the Generator make a new batch of fake images and pass them to the Discriminator. But this time, we tell the loss function that the target label is (Real).
- Because the Discriminator will likely output a low score (e.g., ), the network calculates a massive error. The error is back-propagated through the locked Discriminator directly into the Generator. The Generator uses this gradient to adjust its pixels to look slightly more realistic next time.

The DCGAN Breakthrough: Early GANs were notoriously unstable and almost impossible to train. In 2015, researchers introduced DCGANs (Deep Convolutional GANs), which standardized the architecture. They proved that replacing Pooling layers with strided convolutions, and applying Batch Normalization heavily in both the Generator and Discriminator, stabilized the training and allowed for high-resolution image generation!

Training RNNs: BPTT & The Vanishing Gradient Problem

1. Unrolling the Network

Unlock Full Lesson