Winning Kaggle the Reproducible Way

Chapter 42

Image Augmentation

An augmentation is an assumption about which changes keep the label; test it before trusting it.

Shifts, mirror flips and half-turns are all easy to add and only some are valid for a given task. One demonstration trains a classifier on real digit images with each augmentation, on digits as scanned and on a constructed variant where half the images are mirrored, and measures the accuracy change against no augmentation.

Every example is constructed for teaching. Its numbers come from running the chapter activity's code, not from a competition result.

Demonstration 1 of 1

Which augmentations help? Digits as scanned and digits in random mirror orientation

When does adding transformed copies of the training images raise held-out accuracy, and when does it lower it?

A flip, shift or turn only adds information when the transformed picture is a legitimate example of the same class that the test images could contain. For scanned digits none of the three is: a flip or a half-turn produces many pictures that never occur at test time (a half-turned 6 is a 9), and even a one-pixel shift is large on an 8 x 8 grid. When the test images are themselves randomly mirrored, the flip is exactly the missing invariance and helps most with little data.

Equation: Delta equals the held-out accuracy with augmentation minus the held-out accuracy with the original images only

Scroll sideways for the whole equation

Delta is the change in held-out accuracy from adding a transformed copy of each training image, compared with training on the originals alone; positive means the augmentation helped.

Predict first. With 100 training images, horizontal-flip augmentation changes held-out accuracy by what, on digits as scanned and on digits in random mirror orientation?

Your prediction

Choose an example

Scroll sideways for the whole figure

Figure: Which augmentations help? Digits as scanned and digits in random mirror orientation. Two bar charts of accuracy change against no augmentation with 100 training images. Digits as scanned: shift -0.032, flip -0.042, half-turn -0.096. Digits in random mirror orientation: shift -0.043, flip +0.091, half-turn -0.111.
Training images (before augmentation): 100
Real data: scikit-learn's bundled digits (8 x 8 scans). Constructed: the randomly mirrored variant, built from the same images. Measured by the chapter activity.

Calculated values

No augmentation, scanned
0.882
No augmentation, mirrored
0.749
Flip, scanned
0.840
Flip, mirrored
0.840
Half-turn, scanned
0.786
6/9 swapped by half-turn
20%

With 100 training images, a left-right flip changes accuracy by 0.840 - 0.882 = -0.042 on digits as scanned and by 0.840 - 0.749 = +0.091 on digits in random mirror orientation. A one-pixel shift changes it by -0.032 and -0.043, and a half-turn by -0.096 and -0.111. The half-turn makes 20% of true 6s and 9s be predicted as the other digit, against 0% without augmentation. An unchanged duplicate of every image changes accuracy by +0.003.

Worked steps

  1. Flip on scanned digits: 0.840 - 0.882 = -0.042.
  2. Flip on mirrored digits: 0.840 - 0.749 = +0.091.
  3. Half-turn on scanned digits: 0.786 - 0.882 = -0.096.
  4. Unchanged duplicate on scanned digits: 0.885 - 0.882 = +0.003.

Use the idea

Write down what must stay invariant, check each transform against it on a few images, and compare every candidate with the unaugmented baseline (and an unchanged-duplicate control) on untouched assessment images.

Where the conclusion applies

Real digits at 8 x 8 resolution and a linear classifier, so conclusions about shifts do not carry over to larger images or convolutional networks, where small shifts are often valid. The mirrored variant is constructed to make the flip valid. One transformed copy per image; no test-time augmentation.

Check your understanding: With scanned digits, a half-turn keeps the label. Which classes does it damage, and how can you tell the damage comes from the transform rather than from doubling the training rows?
A half-turn turns a 6 into a 9 while labeling it 6, so the model confuses the two: with 100 training images 20% of true 6s and 9s are predicted as the other digit, against 0% with no augmentation, and accuracy falls by 0.096. An unchanged duplicate of every image changes accuracy by only +0.003, so the loss comes from the transform, not from the extra rows.

Chapter 42 source: section "Decide Which Changes Preserve the Target".