Autoencoders & GANs by Hand: Encoders, Bottlenecks, Reconstruction Error, Anomaly Scores, Latent Space, Variational Sampling, Generator vs Discriminator Losses & Mode Collapse

What you'll build

What you will be able to do

Each lab ends with a concrete result that you build yourself and the checker confirms, cell by cell.

• Encode six four-pixel toy images into two-value codes with given weights and count how many parameters an encoder needs compared with the size of its input.
• Rebuild four toy images from their two-value codes with a given decoder, squash the outputs with a sigmoid, and measure how far each reconstruction is from the original.
• Take one gradient-descent step on the decoder weights for one toy image and show that the reconstruction loss really drops.
• Use reconstruction error as an anomaly score, set a threshold from normal samples with PERCENTILE.INC, flag new samples and measure precision and recall on a labelled toy set.
• Plot the codes of the training images, walk in a straight line from one code to another and decode the mixes, and measure how much a denoising autoencoder cleans up a noisy input.
• Turn the encoder outputs μ and log-variance into sampled codes with a given ε table, calculate the KL term per sample and combine it with the reconstruction loss for three β values.
• Score real and generated toy images with a given discriminator, calculate both GAN losses and follow them through a short training log.
• Read a 12-step GAN training log, count the steps near the balance point and check the generated samples for mode collapse with STDEV.S and a COUNTIFS diversity table, and chart the scores over the run.