Enhancement & smoothing filters

Image enhancement makes the information already in a picture easier to see. This page covers the two spatial-domain families from the lecture. The first is point processing: negative, log, gamma, contrast stretching, slicing, bit planes and histogram equalisation. The second is linear neighbourhood filters: correlation, convolution, box and Gaussian smoothing. The page works out every numerical.

What is image enhancement?

Image enhancement improves the visual quality of an image. The image is then easier to interpret, analyse and use for decisions. Enhancement highlights edges, textures, objects or regions of interest. It follows image correction (noise and distortion removal). A person can view the result, or a later analysis step can use it.

The slide asks: does enhancement create new information, or make existing information more visible? It makes existing information more visible. A dark photo already holds the detail. Enhancement stretches the detail into a range that your eye (or an algorithm) can use. Enhancement adds nothing, and it can lose some information (clipping, blur).

Classification of enhancement methods

BasisCategoriesExamples
Type of information being changedContrast / intensity vs. sharpnessContrast stretching, histogram equalisation, smoothing, sharpening
Image domainSpatial domain vs. frequency domainSpatial: work directly on pixels. Frequency: low-pass, high-pass, homomorphic filtering

This lecture covers the spatial domain. The operations work directly on pixel values and change the intensities. They are cheap to compute and easy to implement. There are two kinds of operation:

$$ g(x,y) = T\big[f(x,y)\big] $$

$f$ is the input image, $g$ the output image, $T$ the operator. For point processing, $T$ acts on one pixel and is written $s = T(r)$. Here $r$ is the input intensity and $s$ the output intensity. On this page, $L$ is the number of intensity levels (256 for 8-bit, so the maximum value is $L-1 = 255$).

Point processing: intensity transformations

Image negative

The negative flips the intensity scale. Dark becomes bright and bright becomes dark, like a photographic negative.

$$ s = L - 1 - r \qquad\text{(8-bit: } s = 255 - r\text{)} $$

$s$ = output, $r$ = input, $L-1$ = highest intensity level.

Use: the negative suits white or grey detail in dark regions, especially when the black areas are large (for example tissue in a mammogram).

Example. 8-bit patch, apply $s = 255 - r$.

050100
150200255

Result:

255205155
105550

Log transformation

The log transformation spreads out dark pixels and compresses bright pixels. Use it when the dynamic range is huge and you cannot see the small values. An example is a Fourier spectrum where one value is in the millions and most values are tiny.

$$ s = c\,\log(1+r) $$

$c$ is a scaling constant (for 8-bit output, $c = 255/\log(256)$ so that $r=255$ maps to 255). The "+1" is necessary because $\log 0$ is undefined. The transform maps a narrow range of low input values to a wide range of output levels. It does the opposite for high inputs. So it expands dark pixel values, compresses higher values and compresses the dynamic range.

Example (extra practice). 8-bit, natural log, $c = 255/\ln 256 = 255/5.545 \approx 46$. Find $s$ for $r = 15$ and $r = 255$.

$r=15$: $s = 46 \ln 16 = 46 \times 2.773 \approx 127.5$. An input of only 15 (very dark) becomes about 128 (mid-grey).
$r=255$: $s = 46 \ln 256 = 46 \times 5.545 \approx 255$.
Inputs 0 to 15 (6 percent of the range) now fill half of the output range. This is what "expands dark values" means.

The inverse log does the opposite. It expands bright values and compresses dark values.

Power-law (gamma) transformation

$$ s = c\, r^{\gamma} $$

$c$ and $\gamma$ are positive constants (with $r$ normalised to $[0,1]$ the usual choice is $c=1$). $\gamma = c = 1$ is the identity.

Many devices (cameras, printers, monitors) respond with a power law. Gamma correction compensates for this. If a monitor displays $s = r^{2.5}$, pre-process the image with $r^{1/2.5} = r^{0.4}$. The end result then looks correct.

Example (extra practice). 8-bit pixel $r = 100$, normalise: $100/255 = 0.392$.
$\gamma = 0.5$: $0.392^{0.5} = 0.626 \to 0.626\times255 \approx 160$ (brighter).
$\gamma = 2$: $0.392^{2} = 0.154 \to 0.154\times255 \approx 39$ (darker).

r (input) s (output) 0 L-1 L-1 red: negative green: log blue: gamma 0.4 gamma 3 identity
Transfer curves. Log and gamma below 1 lift the dark end steeply. Gamma above 1 does the reverse. The negative is a falling line.

Contrast stretching

A low-contrast image uses only a narrow band of grey levels (say 50 to 150). Contrast stretching expands that band to the full display range. It is a piecewise-linear transform. Two points $(r_1,s_1)$ and $(r_2,s_2)$ define it.

$$ (r_1,s_1) = (r_{min},\,0),\qquad (r_2,s_2) = (r_{max},\,L-1) $$ $$ s = \frac{r - r_{min}}{r_{max}-r_{min}}\,(L-1) $$

$r_{min}$, $r_{max}$ are the smallest and largest intensities present in the image. Special case: if $r_1=r_2$, the transform is a binary threshold. If $r_1=s_1$ and $r_2=s_2$, it is the identity.

Example (extra practice). Image has $r_{min}=50$, $r_{max}=150$, $L=256$. Where does $r=100$ go?
$s = \dfrac{100-50}{150-50}\times 255 = 0.5\times255 = 127.5 \approx 128$. Also $r=50\to0$ and $r=150\to255$.

Intensity-level slicing

This method highlights a specific range of intensities $[A,B]$ (for example a particular tissue or water body). There are two ways:

The range is $[100,150]$. The pixels are $80, 120, 200$. Way 1 (binary): $0, 255, 0$. Way 2 (brighten range to 255, rest unchanged): $80, 255, 200$.

Bit-plane slicing

An 8-bit image is eight 1-bit images stacked. Plane 1 holds the lowest-order bit of every pixel. Plane 8 holds the highest-order bit. This method does not pick intensity ranges. It shows how much each bit adds to the overall appearance.

Example (extra practice). Pixel 150. In binary $150 = 128+16+4+2 = 10010110_2$. The planes are: plane 8 (MSB) = 1, plane 7 = 0, plane 6 = 0, plane 5 = 1, plane 4 = 0, plane 3 = 1, plane 2 = 1, plane 1 = 0.
Reconstruct from planes 8 and 5 only: $128 + 16 = 144$, close to 150. The top bits alone keep the pixel roughly correct.

Histogram and histogram equalisation

An image histogram plots the number of pixels at each tonal value. It shows how the intensities are distributed. A dark image has its bump on the left. A bright image has its bump on the right. A low-contrast image has a narrow bump somewhere in the middle. A well-contrasted image uses the whole range.

Imagine a crowd in a few seats of a stadium. Equalisation moves people so that every section has about the same number. The image then uses the intensity range more fully, and details separate out.

Histogram equalisation automatically finds a transformation that redistributes the intensities. The histogram becomes approximately uniform and spans the whole range $[0, L-1]$. The result is higher contrast and better visibility.

$$ s_k = T(r_k) = (L-1)\sum_{j=0}^{k} p_r(r_j),\qquad p_r(r_j)=\frac{n_j}{MN} $$

$r_k$ = $k$-th input level. $s_k$ = its output level. $n_j$ = number of pixels with level $r_j$. $MN$ = total pixels ($M\times N$ image). $p_r(r_j)$ = normalised histogram (probability of level $j$). The running sum is the CDF (cumulative distribution function).

Steps

  1. Compute the histogram $h(r_k)=n_k$.
  2. Normalise: $p(r_k)=n_k/MN$.
  3. Compute the CDF: $C_k=\sum_{j=0}^{k}p(r_j)$.
  4. Calculate the output intensity: $s_k=(L-1)\,C_k$.
  5. Round to the nearest integer. Map each pixel $r_k \to s_k$.

Slide example. 3-bit image ($L=8$), $MN=4096$ (the slide writes M=N=6, which is a typo. The counts sum to $4096 = 64\times64$). $L-1=7$.

$r_k$$n_k$$p_r(r_k)=n_k/4096$CDF$s_k=7\times$CDFRounded $s_k$
07900.190.191.331
110230.250.443.083
28500.210.654.555
36560.160.815.676
43290.080.896.236
52450.060.956.657
61220.030.986.867
7810.021.007.007

Worked lines as on the slide: $s_0 = 7\times0.19 = 1.33$
$s_1 = 7(0.19+0.25) = 7\times0.44 = 3.08$
$s_2 = 7\times0.65 = 4.55$
$s_3 = 7\times0.81 = 5.67$
$s_4 = 7\times0.89 = 6.23$
$s_5 = 6.65$
$s_6 = 6.86$
$s_7 = 7.00$.

Merging: input levels 3 and 4 both go to 6. Levels 5, 6, 7 all go to 7. New pixel counts: level 1 = 790, level 3 = 1023, level 5 = 850, level 6 = 656+329 = 985, level 7 = 245+122+81 = 448 (total 4096). Levels 0, 2, 4 are now empty. The new histogram is spread out but not perfectly flat, because there are only 8 discrete levels.

Slide practice problem. 8 levels (0 to 7), 64 pixels. Histogram: $n = 8,10,10,2,12,16,4,2$. Equalise it.

$r_k$$n_k$$p=n_k/64$Cumulative $n$CDF$s=7\times$CDFRounded
080.12580.1250.8751
1100.156180.2811.9692
2100.156280.43753.0633
320.031300.4693.2813
4120.1875420.6564.5945
5160.25580.9066.3446
640.0625620.9696.7817
720.031641.0007.0007

Mapping: $0\to1,\ 1\to2,\ 2\to3,\ 3\to3,\ 4\to5,\ 5\to6,\ 6\to7,\ 7\to7$.
New histogram: level 1 = 8; level 2 = 10; level 3 = 10+2 = 12; level 5 = 12; level 6 = 16; level 7 = 4+2 = 6. Check: $8+10+12+12+16+6 = 64$.

Exam shortcut: do the cumulative count column first. Then compute $s_k = (L-1)\times\text{cum count}/MN$ directly. This avoids rounding errors from rounded probabilities. The last level always maps to $L-1$, and $s_k$ is non-decreasing.

Equalisation does not give a perfectly flat histogram for a digital image. Levels merge but are never split, so the result is roughly flat and spiky. A second application changes almost nothing (it is approximately idempotent).

Histogram specification (matching)

Equalisation automatically picks the transform that makes the histogram uniform. Sometimes you need a particular shape instead (for example a darker-biased result). Histogram matching or specification makes an image whose histogram matches a specified one. The standard method has three steps: equalise the input ($r\to s$), equalise the target histogram ($z\to v$), then map each $s$ to the $z$ whose $v$ is closest.

EqualisationSpecification
Output histogramApproximately uniform (fixed)User-chosen shape
User inputNone, automaticDesired histogram
Special case-Specifying a uniform histogram gives equalisation

Spatial filtering basics

A spatial filter has two parts: a neighbourhood and a predefined operation. The neighbourhood is a small rectangle (for example 3x3) around a pixel. The operation works on the pixels inside it. The neighbourhood is called a kernel (also called spatial mask, template or window). The filter writes the result to a new image at the centre pixel of the neighbourhood. Filtering slides the window over the whole image.

$$ g(x,y)=\sum_{s=-a}^{a}\sum_{t=-b}^{b} w(s,t)\,f(x+s,\,y+t) $$

$f$ = input image, $g$ = output, $w$ = the $m\times n$ kernel with $m=2a+1$, $n=2b+1$ (odd sizes so a centre exists), $(s,t)$ = offsets from the centre. For a 3x3 kernel this is $w(-1,-1)f(x-1,y-1)+\dots+w(1,1)f(x+1,y+1)$.

Linear filters

A filter is linear if its output is a linear function of the inputs. The response to a weighted sum of inputs equals the weighted sum of the responses to each input. The lecture lists these properties: superposition, homogeneity, convolution-based, analysable in the frequency domain and predictable behaviour.

$$ R(f+g)=R(f)+R(g)\ \ \text{(superposition)},\qquad R(kf)=k\,R(f)\ \ \text{(scaling / homogeneity)} $$

$R$ is the filter (system), $f,g$ are inputs, $k$ a constant. A device with both properties is linear.

Shift invariance: the response to a translated input is the translated response. A filter that is linear and shift invariant is a shift-invariant linear system. Convolution with the impulse response (the kernel) describes such a system completely.

Linear examples: averaging, Gaussian, Sobel. Non-linear: median filter, max/min filter, thresholding. Exams often ask "is the median filter linear?" The answer is no, because the median of a sum is not the sum of the medians.

Correlation and convolution

Correlation measures how closely two things match. To do it by hand:

  1. Put the kernel on the image.
  2. Multiply the overlapping values and add them.
  3. Write the result at the centre pixel.
  4. Slide the kernel one pixel to the right and repeat.
  5. At the right edge, move down one row and start again from the left.

Convolution is the same process, except the filter is first rotated by 180 degrees (flipped both horizontally and vertically).

$$ \text{Correlation: } (w\star f)(x,y)=\sum_{s}\sum_{t} w(s,t)\,f(x+s,y+t) $$ $$ \text{Convolution: } (w * f)(x,y)=\sum_{s}\sum_{t} w(s,t)\,f(x-s,y-t) $$

The minus signs in convolution give the 180 degree rotation. For symmetric kernels (box, Gaussian, Laplacian) the two give identical results.

Slide example (1D). Image row $f = [0\ 0\ 0\ 1\ 0\ 0\ 0\ 0]$ (a single impulse), kernel $w = [1\ 2\ 3\ 2\ 8]$.

Correlation: slide $w$ unchanged across $f$ (with zero padding). With an impulse at one position, each window position picks out one kernel element, in reverse order. So the output is the kernel reversed. The full result is $0\,0\,0\,8\,2\,3\,2\,1\,0\,0\,0\,0$, and the result cropped to the original size is $0\ 8\ 2\ 3\ 2\ 1\ 0\ 0$.

Convolution: first rotate $w$ by 180 degrees to $[8\ 2\ 3\ 2\ 1]$, then slide. The result is the kernel copied as it is. The full result is $0\,0\,0\,1\,2\,3\,2\,8\,0\,0\,0\,0$, cropped $0\ 1\ 2\ 3\ 2\ 8\ 0\ 0$.

Moral: convolution with an impulse reproduces the kernel, so the kernel is the "impulse response". Correlation gives the kernel flipped.

Extra practice (2D). Patch and Sobel-type kernel:

123
456
789
10-1
20-2
10-1

Correlation at the centre: $1(1)+0+3(-1)+4(2)+0+6(-2)+7(1)+0+9(-1) = 1-3+8-12+7-9 = -8$.
Convolution: rotate the kernel by 180 degrees. Here it becomes the negative of itself, so the answer is $+8$. The magnitude is the same and the sign is opposite.

Boundary handling

When the kernel goes over the image border, pixels are missing. Choose an extension mode:

ModeWhat outside pixels become
zeroAll pixels outside set to 0 (causes dark border)
constant (border colour)All outside pixels set to a specified value
clamp (replicate)Edge pixels repeated indefinitely
wrap (cyclic, tile)Image loops around like a torus
mirrorPixels reflected across the image edge
extendSignal extended by subtracting the mirrored signal from the edge pixel value

Output sizes for an $N\times N$ image and $m\times m$ kernel: "full" $=N+m-1$, "same" $=N$ (needs padding $p=(m-1)/2$ on each side), "valid" $=N-m+1$ (no padding).

Smoothing (low-pass) filters

Smoothing replaces each pixel with the average of the intensities in its neighbourhood. It is an averaging, mean or low-pass filter. It keeps slowly varying content and removes rapid changes.

$$ w=\frac{1}{9}\begin{bmatrix}1&1&1\\1&1&1\\1&1&1\end{bmatrix},\qquad g(x,y)=\frac{1}{mn}\sum_{(s,t)} f(x+s,y+t) $$

The coefficients sum to 1, so the overall brightness stays the same. An $m\times n$ box has all entries $1/(mn)$.

Worked example. A noisy pixel (90) among 10s, 3x3 box filter:

101010
109010
101010

Sum $=8\times10+90=170$, so output $=170/9\approx18.9\approx19$. The spike drops from 90 to 19 but stays (spread out). A median filter gives exactly 10.

Edge blur example (1D, 3-point average). Row $10\ 10\ 10\ 80\ 80\ 80$. The outputs at the inner pixels are $(10+10+10)/3=10$, $(10+10+80)/3=33.3$, $(10+80+80)/3=56.7$ and $(80+80+80)/3=80$. The sharp step $10\to80$ becomes a ramp $10, 33, 57, 80$. The edge is blurred.

Effect of kernel size. The slide shows a 500x500 test image smoothed with square kernels of $m=3,5,9,15,35$. A larger $m$ gives more blur. Small objects and fine detail vanish because their intensity blends into the background. Edges become wide. At $m=35$ the image is very blurry, and zero padding makes a dark border. Use a bigger kernel to remove more noise or to blend small objects into the background. The cost is the loss of detail.

Weighted-average smoothing

This filter multiplies the pixels by different coefficients instead of equal weights. The centre gets the largest weight, and the weight falls with distance. It blurs less than the box filter and keeps edges a little better. The slide's strategy: the intensity of smaller objects blends with the background.

$$ \frac{1}{16}\begin{bmatrix}1&2&1\\2&4&2\\1&2&1\end{bmatrix} $$

Weights sum to 16, so dividing by 16 normalises it.

Gaussian smoothing

The Gaussian filter is a 2D convolution. It blurs images and removes detail and noise. Its kernel has a bell shape. The centre has the highest value, and the values fall smoothly with distance.

$$ G(x)=\frac{1}{\sqrt{2\pi}\,\sigma}\,e^{-\frac{x^{2}}{2\sigma^{2}}}\qquad G(x,y)=\frac{1}{2\pi\sigma^{2}}\,e^{-\frac{x^{2}+y^{2}}{2\sigma^{2}}} $$

$x,y$ = distance from the kernel centre (in pixels). $\sigma$ = standard deviation, which controls the amount of blur (bigger $\sigma$ = wider bell = more smoothing). The factor in front makes the continuous curve integrate to 1. For a discrete kernel, divide by the sum of the entries.

The slide shows these kernels:

121
242
121

Multiply the 3x3 kernel by $1/16$ ($\sigma\approx1$). Multiply the 5x5 kernel by $1/273$:

14741
41626164
72641267
41626164
14741

Check the normalisers: $1+2+1+2+4+2+1+2+1=16$. The 5x5 entries sum to 273.

Extra practice: box vs. Gaussian on the same patch. Same noisy patch as above (centre 90, rest 10).

Gaussian 3x3: corners $4\times(1\times10)=40$; edges $4\times(2\times10)=80$; centre $4\times90=360$. Sum $=480$, output $=480/16=30$.
The box filter gave 19. The Gaussian keeps more of the centre pixel (weight 4/16), so it blurs less. The box filter gives the noisy pixel a weight of only 1/9.

Computing a kernel value. For $\sigma=1$: centre $G(0,0)=1/(2\pi)=0.159$; edge neighbour $G(1,0)=0.159e^{-0.5}=0.0965$; corner $G(1,1)=0.159e^{-1}=0.0586$. The ratio centre : edge : corner is $\approx 1 : 0.61 : 0.37 \approx 4:2.4:1.5$. This is close to the integer approximation $4:2:1$, which is the binomial approximation.

Gaussian is separable: $G(x,y)=G(x)\,G(y)$, so a 2D blur equals a 1D blur along the rows, then along the columns. The cost drops from $m^2$ to $2m$ multiplications per pixel. Example: $[1\,2\,1]^T[1\,2\,1]$ gives the 3x3 kernel above. A rule of thumb: the kernel size is about $6\sigma+1$.

Why smooth before edge detection?

The slides show a road scene. Edge detection on a noisy image gives many false edges, because noise looks like many tiny edges. A Gaussian blur first removes most of the noise. The edge map of the blurred image is clean and shows only the real structures. Gaussian smoothing is therefore the first step of pipelines like Canny and LoG (see the next page).

Box vs. Gaussian vs. median

PropertyBox (mean)GaussianMedian (extra)
Linear?YesYesNo
WeightsAll equalBell-shaped, centre highestNone (rank order)
Blur lookBlocky, can ringSmooth, naturalPreserves edges
Best forQuick average, general noiseGaussian noise, pre-edge detectionSalt-and-pepper noise
ParameterSize $m$$\sigma$ (and size)Size

Median (extra). For the patch above, the sorted values are $10,10,10,10,10,10,10,10,90$ and the median (5th) is 10. The spike is removed completely. Linear filters can only dilute it.

Correlation vs. convolution

CorrelationConvolution
Kernel orientationUsed as isRotated 180 degrees first
Formula$f(x+s,y+t)$$f(x-s,y-t)$
Commutative?NoYes: $f*w=w*f$
Impulse input givesFlipped kernelKernel itself
Same for symmetric kernels?Yes, identical results

Quick self-test

1. Does enhancement create new information? Name the two spatial-domain families.

No, it only makes existing information more visible. The families are point processing (single-pixel intensity transformation) and spatial (neighbourhood) filtering.

2. Apply the negative to an 8-bit pixel of 70, and the log transform ($c=46$, natural log) to $r=7$.

Negative: $255-70=185$. Log: $46\ln(8)=46\times2.079\approx95.7\approx96$. The very dark input 7 becomes about 96.

3. When is $s=c\,r^\gamma$ used with $\gamma<1$ versus $\gamma>1$? What is gamma correction?

$\gamma<1$ expands dark values and brightens; $\gamma>1$ expands bright values and darkens. Gamma correction pre-compensates the power-law response of a display or capture device (for example apply $r^{1/2.5}$ for a display with $\gamma=2.5$).

4. Equalise a 3-bit, 100-pixel image with $n=[20,30,10,10,10,10,5,5]$.

Cumulative counts: 20, 50, 60, 70, 80, 90, 95, 100. $s=7\times\text{cum}/100$: 1.4, 3.5, 4.2, 4.9, 5.6, 6.3, 6.65, 7. Rounded: 1, 4 (3.5 rounds up), 4, 5, 6, 6, 7, 7. Levels 1 and 2 merge into 4 (40 pixels). Levels 4 and 5 merge into 6 (20 pixels). Levels 6 and 7 merge into 7 (10 pixels).

5. Compute the correlation and convolution of $f=[0\,0\,1\,0\,0]$ with $w=[3\,1\,2]$ (same size, zero padding).

Correlation: the output is the reversed kernel centred on the impulse: $[0\ 2\ 1\ 3\ 0]$. Convolution: the output is the kernel itself: $[0\ 3\ 1\ 2\ 0]$.

6. A 3x3 box filter is applied to a patch with centre 50 and all eight neighbours 20. Output? Repeat using the 1-2-1 Gaussian.

Box: $(8\times20+50)/9=210/9=23.3$. Gaussian: corners $4\times1\times20=80$; edges $4\times2\times20=160$; centre $4\times50=200$; total $440/16=27.5$.

7. State three differences between convolution and correlation, and when they coincide.

Convolution rotates the kernel by 180 degrees. Convolution is commutative and associative, but correlation is not commutative. An impulse gives the kernel itself in convolution and a flipped kernel in correlation. They coincide for symmetric kernels such as box and Gaussian.

8. Why does smoothing blur edges, and how does kernel size affect the result?

Averaging reduces any sharp intensity transition, and edges are sharp transitions. A larger kernel averages over more pixels. It removes more noise but blurs more, and small objects blend into the background.

9. Define linearity and shift invariance. Is a median filter linear?

Linearity: $R(f+g)=R(f)+R(g)$ and $R(kf)=kR(f)$. Shift invariance: a translated input gives a translated output. The median filter is not linear (median of a sum is not the sum of medians).

10. Contrast-stretch an image with $r_{min}=40$, $r_{max}=120$ (8-bit). Where does 80 map?

$s=(80-40)/(120-40)\times255=0.5\times255\approx128$.