Welcome to the fifth section of our Deep Learning module, where we will explore Generative Adversarial Networks, or GANs. GANs are a class of machine learning frameworks designed by Ian Goodfellow and his colleagues in 2014. They consist of two neural networks, a generator and a discriminator, which are trained simultaneously through adversarial processes. The generator aims to create realistic data samples, while the discriminator tries to distinguish between real and generated samples.

Generator Network

The generator network in a GAN takes random noise as input and generates data samples that resemble the training data. It is typically composed of layers of transposed convolutions, which help in upsampling the input to the desired output dimensions.

Example:

Let’s implement a simple generator network using TensorFlow:

import tensorflow as tf
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Reshape, UpSampling2D, Conv2D, BatchNormalization, ReLU

# Define the generator model
def build_generator():
    model = Sequential([
        Dense(7*7*128, input_dim=100),
        BatchNormalization(),
        ReLU(),
        Reshape((7, 7, 128)),
        UpSampling2D(),
        Conv2D(128, (3, 3), padding='same'),
        BatchNormalization(),
        ReLU(),
        UpSampling2D(),
        Conv2D(64, (3, 3), padding='same'),
        BatchNormalization(),
        ReLU(),
        Conv2D(1, (3, 3), padding='same', activation='tanh')
    ])
    return model

generator = build_generator()
generator.summary()

This code snippet defines a simple generator network using the Keras API in TensorFlow.

Discriminator Network

The discriminator network in a GAN takes an image as input and outputs a single scalar value, representing the probability that the input image is real. It is typically composed of convolutional layers that downsample the input and extract features for classification.

Example:

Let’s implement a simple discriminator network using TensorFlow:

from tensorflow.keras.layers import LeakyReLU, Flatten

# Define the discriminator model
def build_discriminator():
    model = Sequential([
        Conv2D(64, (3, 3), padding='same', input_shape=(28, 28, 1)),
        LeakyReLU(alpha=0.2),
        Conv2D(128, (3, 3), strides=(2, 2)),
        LeakyReLU(alpha=0.2),
        Flatten(),
        Dense(1, activation='sigmoid')
    ])
    return model

discriminator = build_discriminator()
discriminator.summary()

This code snippet defines a simple discriminator network using the Keras API in TensorFlow.

Image Generation

One of the most popular applications of GANs is in image generation. GANs can be used to generate realistic images from random noise, which has applications in art, design, and entertainment. GANs have also been used to create high-resolution images from low-resolution inputs, known as super-resolution.

Example:

Let’s build and train a simple GAN to generate images using the MNIST dataset:

import numpy as np
from tensorflow.keras.datasets import mnist
from tensorflow.keras.optimizers import Adam
import matplotlib.pyplot as plt

# Load and preprocess the MNIST dataset
(X_train, _), (_, _) = mnist.load_data()
X_train = (X_train - 127.5) / 127.5
X_train = np.expand_dims(X_train, axis=-1)

# Define the GAN
def build_gan(generator, discriminator):
    discriminator.compile(optimizer=Adam(0.0002, 0.5), loss='binary_crossentropy', metrics=['accuracy'])
    discriminator.trainable = False
    gan = Sequential([generator, discriminator])
    gan.compile(optimizer=Adam(0.0002, 0.5), loss='binary_crossentropy')
    return gan

gan = build_gan(generator, discriminator)

# Training the GAN
def train_gan(gan, generator, discriminator, epochs=10000, batch_size=128):
    for epoch in range(epochs):
        # Train the discriminator
        idx = np.random.randint(0, X_train.shape[0], batch_size)
        real_images = X_train[idx]
        noise = np.random.normal(0, 1, (batch_size, 100))
        fake_images = generator.predict(noise)
        real_labels = np.ones((batch_size, 1))
        fake_labels = np.zeros((batch_size, 1))
        d_loss_real = discriminator.train_on_batch(real_images, real_labels)
        d_loss_fake = discriminator.train_on_batch(fake_images, fake_labels)
        d_loss = 0.5 * np.add(d_loss_real, d_loss_fake)
        
        # Train the generator
        noise = np.random.normal(0, 1, (batch_size, 100))
        valid_labels = np.ones((batch_size, 1))
        g_loss = gan.train_on_batch(noise, valid_labels)
        
        # Print the progress
        if epoch % 1000 == 0:
            print(f"Epoch {epoch} [D loss: {d_loss[0]}, acc.: {100 * d_loss[1]}] [G loss: {g_loss}]")
            sample_images(generator, epoch)

# Sample and save generated images
def sample_images(generator, epoch, n=10):
    noise = np.random.normal(0, 1, (n * n, 100))
    generated_images = generator.predict(noise)
    generated_images = 0.5 * generated_images + 0.5
    fig, axs = plt.subplots(n, n, figsize=(10, 10))
    cnt = 0
    for i in range(n):
        for j in range(n):
            axs[i, j].imshow(generated_images[cnt, :, :, 0], cmap='gray')
            axs[i, j].axis('off')
            cnt += 1
    plt.show()

train_gan(gan, generator, discriminator)

This code snippet defines and trains a simple GAN for image generation using the MNIST dataset.

Style Transfer

Another fascinating application of GANs is style transfer, where the style of one image is applied to the content of another. This technique is widely used in artistic and creative fields, enabling the transformation of photos into artworks in the style of famous painters.

Example:

While implementing a full style transfer algorithm is complex and beyond the scope of this introduction, you can explore existing pre-trained models like Neural Style Transfer (NST) using frameworks such as TensorFlow or PyTorch.

In this section, we've covered the basics of Generative Adversarial Networks, including the architecture of the generator and discriminator networks. We also explored practical examples of building and training GANs for image generation. These concepts and techniques are fundamental to understanding and working with GANs in various generative modeling applications.

Below, you have a link to the Jupyter/Colab notebook where you can practice the theory from this section: