Building a Convolutional Neural Network (CNN) from Scratch

August 25, 2025

🧱 Building a Convolutional Neural Network (CNN) from Scratch

If you've ever wondered how machines can recognize faces, detect objects, or classify handwritten digits — the answer often lies in Convolutional Neural Networks (CNNs).

In this post, we’ll walk through how to build a CNN from scratch, understand its key components, and train it on a real dataset — using Python and TensorFlow/Keras.

Let’s get started.

👓 What is a CNN?

A Convolutional Neural Network (CNN) is a type of deep learning model designed to process grid-like data — most commonly images. Instead of looking at each pixel individually, CNNs learn to detect patterns, shapes, and structures through layers of convolution operations.

🧠 CNN Architecture — The Building Blocks

A typical CNN has the following layers:

Input Layer: Takes in the raw image (e.g., 28x28 pixels, 3 channels).

Convolutional Layers: Apply filters (kernels) to extract features like edges, curves, textures.

Activation Function: Usually ReLU to introduce non-linearity.

Pooling Layers: Downsample the image, reducing size and computation.

Fully Connected Layers: Perform the final classification.

Output Layer: Gives prediction probabilities (e.g., softmax).

📦 Let's Build a CNN in Keras (TensorFlow)

We’ll use the MNIST dataset (handwritten digits: 0–9) as our example.

✅ Step 1: Import Libraries

import tensorflow as tf

from tensorflow.keras import layers, models

from tensorflow.keras.datasets import mnist

import matplotlib.pyplot as plt

✅ Step 2: Load and Preprocess Data

(x_train, y_train), (x_test, y_test) = mnist.load_data()

# Normalize and reshape

x_train = x_train.reshape(-1, 28, 28, 1).astype("float32") / 255.0

x_test = x_test.reshape(-1, 28, 28, 1).astype("float32") / 255.0

✅ Step 3: Build the CNN Model

model = models.Sequential([

layers.Conv2D(32, (3, 3), activation='relu', input_shape=(28, 28, 1)),

layers.MaxPooling2D((2, 2)),

layers.Conv2D(64, (3, 3), activation='relu'),

layers.MaxPooling2D((2, 2)),

layers.Flatten(),

layers.Dense(64, activation='relu'),

layers.Dense(10, activation='softmax') # 10 classes for digits 0-9

])

✅ Step 4: Compile the Model

model.compile(optimizer='adam',

loss='sparse_categorical_crossentropy',

metrics=['accuracy'])

✅ Step 5: Train the Model

history = model.fit(x_train, y_train, epochs=5, validation_split=0.1)

✅ Step 6: Evaluate the Model

test_loss, test_acc = model.evaluate(x_test, y_test)

print(f"Test accuracy: {test_acc:.2f}")

📈 Visualize Accuracy and Loss

plt.plot(history.history['accuracy'], label='Train Acc')

plt.plot(history.history['val_accuracy'], label='Val Acc')

plt.xlabel('Epoch')

plt.ylabel('Accuracy')

plt.legend()

plt.title('Training vs Validation Accuracy')

plt.show()

🧪 What Just Happened?

Our model learned to detect patterns in handwritten digits.

The convolutional layers automatically extracted features.

Pooling layers reduced dimensionality.

Fully connected layers made the classification.

Even with just a few lines of code, you’ve created a powerful neural network that can recognize digits with high accuracy.

🛠️ Try This Next

Add Dropout layers to reduce overfitting.

Experiment with more convolutional layers or different filter sizes.

Try training on a more complex dataset like CIFAR-10 or Fashion-MNIST.

🧠 Final Thoughts

Building a CNN from scratch may sound complex, but thanks to libraries like TensorFlow/Keras, it’s easier than ever. Understanding the components — like convolutions, filters, and pooling — is crucial if you want to go beyond plug-and-play models.

With practice, you’ll move from recognizing digits to solving more complex image recognition tasks, from medical imaging to autonomous driving.