It’s shocking how a number of the primary topics in machine studying are nonetheless unknown by researchers and regardless of being elementary and customary to make use of, appear to be mysterious. It’s a enjoyable factor about machine studying that we construct issues that work after which determine why they work in any respect!
Right here, I goal to research the unknown territory in some machine studying ideas to be able to present whereas these concepts can appear primary, in actuality, they’re constructed by layers upon layers of abstraction. This helps us to observe questioning the depth of our data.
In this text, we discover a number of key phenomena in deep studying that problem our conventional understanding of neural networks.
- We begin with Batch Normalization and its underlying mechanisms that stay not absolutely understood.
- We study the counterintuitive remark that overparameterized fashions usually generalize higher, contradicting the classical machine studying theories.
- We discover the implicit regularization results of gradient descent, which appear to naturally bias neural networks in direction of easier, extra…