Training CNNs: Dropout, Normalization, and Data Augmentation

Because Convolutional Neural Networks have millions of trainable weights, they are incredibly powerful. However, this power comes with a massive drawback: Overfitting (which we discussed in Chapter 1).

If you train a large CNN on a small dataset, the network will simply memorize the exact pixels of the training images. When you test it on a brand new image, it will fail completely.

To prevent this, we use regularization and optimization techniques. Here are the three most important techniques you will use in TensorFlow to train robust CNNs.