Convolution is a mathematical operation that combines two functions to produce a third function. It is widely used in image processing, particularly in the context of applying filters to images. I'm interested in convolution because it's used widely in the eponymously named convolutional neural networks (CNNs).
And its matrix form is:
Where is the input image, is the filter (or kernel), and are the coordinates of the output pixel. In a neural network the convolution kernel is learned from the data, but here I will use some common filters to build intuition in the concept.
We draw the function
(shown in blue) and convolve it with a Gaussian kernel:
and the convolved result is shown in red.
The true curve is , the noisy curve is the true curve plus some random noise. and we convolve it with a normalised Gaussian kernel
to smooth it out. The convolved result is shown in red.
This image is taking from the Fashion-MNIST dataset. It is a 28x28 pixel grayscale image of a handwritten digit. The image is represented as a 2D array of pixel values, where each pixel value is an integer between 0 and 1, representing the intensity of the pixel (0 = white, 1 = black).
Mild low-pass filter that smooths fine-grained noise without overly smearing edges.
Computes the horizontal gradient, lighting up vertical edges where brightness changes left-to-right. Sign of the response encodes edge direction.
Like shining a light on the image from the left, revealing vertical features.
Computes the vertical gradient, highlighting horizontal edges where brightness changes top-to-bottom. Like shining a light on the image from above, revealing horizontal features.
Isotropic second-derivative operator that fires on any abrupt intensity change, regardless of direction. Excellent for edge maps but very sensitive to noise, so a prior blur is common.
Boosts high-frequency detail while keeping overall brightness. Enhances texture and clarity but can exaggerate noise and ringing.
Offsets a first-derivative kernel to accentuate one lighting direction, making foreground features appear raised. Produces a grayish bas-relief look useful for stylistic effects.