Deep Learning: Exploring Convolution Neural Networks (CNNs) | by Naveed Ul Mustafa | Oct, 2023

Neural Networks embody a chain of interconnected artificial neurons, crafting a pathway for signals to traverse in a predefined direction across multiple layers. Consisting of an input layer, several hidden layers, and an output layer, these networks form the fundamental architecture that drives machine learning.

In a Feedforward Neural Network, signals traverse from the input layer through the hidden layers to the output layer without looping back at any point. When such a network features multiple layers, it’s dubbed a Multi-Layer Perceptron (MLP). Despite its simplicity, MLPs encounter a significant hurdle when grappling with image data due to its high dimensionality.

Consider a 256×256 RGB image, which embodies 196,608 input features. Creating an MLP to process this data demands immense computational power and memory, especially if each feature connects to numerous neurons in a hidden layer.

This is where Convolutional Neural Networks (CNNs) come into the spotlight.

Designed to efficiently handle image data, CNNs, much like MLPs, consist of layered stacks that process input images and predict output labels. However, they differentiate themselves through their unique neuron connectivity and processing of 2D spatial data, ensuring they’re computationally more efficient.

Unlike an MLP, the neurons in CNN layer are not connected to all the neurons in the previous layer. Instead, each neuron in CNN is connected only to a small region of the layer before it, and are applied to whole input layer in a sliding window fashion. This arrangement allows the CNN to have fewer parameters, and thus be more efficient to train than an equivalent MLP.

Convolutions stand as the cornerstone of CNNs, enabling them to discern patterns in input data while maintaining the spatial relationships between pixels.

In essence, convolution involves:

• Input Matrix: The original image data.
• Kernel or Filter: A smaller matrix that slides across the input, identifying patterns.
• Output Matrix: A resultant matrix (also known as a feature map) showcasing detected patterns.

Parameters that influence convolution include:

1. Depth: Dictated by the number of filters used.
2. Stride: The pixel steps taken by the filter during convolution.
3. Zero Padding: Additional zero pixels on the input border, ensuring the filter fits aptly.

The kernel values, or weights, influence the features detected during convolution, enabling the identification of various low-level features like edges and blurs.

Convolution is the first layer to extract features from an input image. Convolution preserves the spatial relationship between pixels by learning image features using small squares of input data. It is a mathematical operation that takes two inputs such as image matrix and a filter or kernel. The filter is smaller than the input image. It slides over the input image, performing element wise multiplication of the filter and the input and then summing them up. This result is a single value in the output image called the activation map or feature map. The filter is then slid over all the locations, performing the same operation and creating the output image. The output image is smaller than the input image because of the reduction caused by sliding the filter over.

A standard CNN features several key layers:

1. Convolution Layer

In the Convolution Layer, filters (small 3×3 or 5×5 matrices) slide across the input image, isolating various features and forming a collection of feature maps.

2. Activation Layer (Typically ReLU)

ReLU (Rectified Linear Unit), a popular activation function, introduces non-linearity to the network by converting all negative pixel values to zero, ensuring the network can solve complex problems.

3. Pooling Layer

Pooling, or Sub-Sampling, reduces dimensionality (typically via max pooling), simplifying the extracted features and reducing computational demands while retaining crucial information.

4. Fully Connected (Classification) Layer

The Fully Connected Layer utilizes the high-level features derived from previous layers to classify the input image into categories, employing a softmax activation function in its output layer to predict class probabilities.

5. Batch Normalization Layer

Batch Normalization stabilizes and accelerates training by normalizing layer inputs, providing benefits like faster convergence and reduced sensitivity to weight initialization.

In the realm of image classification, object detection, and even more nuanced domains like medical image analysis, CNNs have demonstrated not only their utility but their supremacy. The ability to adaptively learn from data and discern patterns renders them indispensable in various applications across diverse sectors, from autonomous vehicles deciphering their surroundings to healthcare models diagnosing ailments from medical images.

Yet, the journey of CNNs is not devoid of challenges. The pursuit of optimizing their architecture, enhancing computational efficiency, and ensuring robustness in diverse scenarios remains a vibrant field of exploration and research. Questions pertaining to interpretability, generalization in diverse contexts, and ethical considerations in deployment continue to propel inquiries and innovations in the domain.

I learned this theoretical knowledge in my quest to achieve Deep Learning certification from OpenCV. If you like this content, you can follow me on X & Github, and your feedback is the most valuable to me.