When and how to normalize your Data

I would like to discuss the needs for normalizing your data — with a specific focus on neural networks. In order to understand the concepts discussed, it’s important to have an understanding of gradient descent.

As a quick refresher, when training neural networks we will feed in observations and compare the expected output to the true output of the network. We’ll then use gradient descent to update the parameters of the model in the direction which will minimize the difference between our expected outcome and the true outcome. In other words, we are attempting to…