What is the change from ordinary neural network to convolutional neural network?
ConvNet architectures make the explicit assumption that the inputs are images, which allows us to encode certain properties into the architecture. These then make the forward function more efficient to implement and vastly reduce the amount of parameters in the network.
Which layers have parameters?
CONV/FC do, RELU/POOL don’t.Which layers have additional hyperparameters?
CONV/FC/POOL do, RELU doesn’t.Why sometimes zero-padding?
The nice feature of zero padding is that it will allow us to control the spatial size of the output volumes (most commonly we will use it to exactly preserve the spatial size of the input volume so the input and output width and height are the same).How to compute the output volume as a function of the input volume size (W), the receptive field size of the Conv Layer neurons (F), the stride with which they are applied (S), and the amount of zero padding used (P) on the border.
The formula is: $$\frac{W-F+2P}{S} + 1$$What hyperparameters the convolutional layer has?
Number of filters, their spatial extent (kernel size), the stride, the amount of zero padding.
Final challenge: can you code a convolutional layer?