Local descriptors & texture
The last lecture found where the interesting points are. This lecture describes what they look like, so that a program can match the same point in another photo (SIFT and similar methods). It also describes the feel of a whole surface (LBP, Gabor, GLCM). After this page you can build a scale space and do the four SIFT steps. You can also compute an LBP code and fill in a GLCM with its Haralick features.
Local feature descriptors
A local feature descriptor is a numerical representation (a feature vector) of the appearance of a small region around a detected keypoint. It must let you recognise the same feature in another image after changes in scale, rotation, illumination or viewpoint.
Take a corner of a building seen in two photos. The detector says "there is a corner at this location". The descriptor writes a fingerprint of the patch around it. To match two photos, compare the fingerprints, not the pixels.
| Detector | Descriptor |
|---|---|
| Finds the location (and often scale) of a keypoint | Builds a feature vector for the patch around that keypoint |
| Harris, Hessian, DoG, FAST | SIFT, SURF, BRIEF/ORB, HOG |
Scale space
Objects appear at different sizes depending on camera distance. A fixed-size detector can find a feature at one scale and miss it at another. Scale-space theory solves this: it analyses the image at many scales.
$I$ is the image, $G$ is a Gaussian of standard deviation $\sigma$, $*$ is convolution, $L$ is the blurred image at scale $\sigma$.
As $\sigma$ increases, fine details disappear and larger structures stay. The slides show an owl blurred with $\sigma=1,5,10,30$ pixels. At 30 only a brown smudge remains. The sequence of more and more blurred images is the Gaussian scale-space pyramid.
Gaussian pyramid: octaves and intervals
- Octave: a set of images where the scale doubles ($\sigma\to2\sigma$).
- After one octave the image is downsampled by 2 (this is cheaper, because the next octave uses a smaller image).
- Intervals: the several blurred images generated inside an octave.
So a Gaussian pyramid increases the blur and downsamples at regular steps. The slide picture shows five levels: Level 0 (original), Level 1 ($1/2$ resolution), Level 2 ($1/4$), Level 3 ($1/8$) and Level 4 ($1/16$). "Blur and subsample" makes each level.
Practice numerical. A $640\times480$ image has 4 octaves. The octave sizes are $640\times480$, $320\times240$, $160\times120$ and $80\times60$. Take $s=3$ intervals per octave, $k=2^{1/s}=1.26$ and a first scale $\sigma_0=1.6$. The blur levels inside octave 1 are $1.6,\ 2.02,\ 2.54,\ 3.2,\ 4.03,\ 5.08$. That is $s+3=6$ images. The levels go past $2\sigma_0=3.2$.
DoG pyramid
- Build the Gaussian pyramid.
- Subtract each pair of adjacent Gaussian images.
- Result: DoG images, $\mathrm{DoG}_i = G_{i+1}-G_i$.
Reason: $G(k\sigma)-G(\sigma)\approx(k-1)\sigma^2\nabla^2G$. This is a cheap approximation to the scale-normalised Laplacian of Gaussian (the blob detector from the last lecture). With $n$ blurred images per octave, you get $n-1$ DoG images.
SIFT: Scale-Invariant Feature Transform
SIFT (David G. Lowe) detects and describes keypoints. The keypoints are invariant to scale and rotation. They are partially invariant to changes in illumination and viewpoint. The slides show supermarket-aisle photos that SIFT matches although brightness, zoom and tilt change.
Step 1: scale-space extrema detection
- Construct the Gaussian pyramid, then the DoG pyramid.
- Take every pixel of the DoG images. Compare it with its 8 neighbours in the current scale, 9 in the scale above and 9 in the scale below.
- Check the centre against all 26 neighbours. If it is larger than all of them or smaller than all of them, it is a candidate keypoint.
The slide figure shows the work for each octave. First, subtract the Gaussian images (made by convolution with a Gaussian) to make the DoG. Then find the extrema in the 3-D $(x,y,\sigma)$ space. Last, downsample for the next octave.
Step 2: keypoint localisation
Many candidates are poor. Two filters remove them:
- Low-contrast rejection: discard points with a weak DoG response. First refine the extremum position to sub-pixel accuracy with a quadratic fit. Lowe uses $|D|<0.03$ for images scaled to $[0,1]$.
- Edge-response elimination: an edge has a large curvature in one direction and a small curvature in the perpendicular direction. Its position is unstable along the edge, so remove it. Use the $2\times2$ Hessian of the DoG image. Take the eigenvalue ratio $r=\lambda_1/\lambda_2$. Keep a point only if $\dfrac{\mathrm{tr}(H)^2}{\det(H)}<\dfrac{(r+1)^2}{r}$. Lowe uses $r=10$, so the limit is $12.1$.
This is the same eigenvalue idea as Harris: corners have two large eigenvalues of similar size, edges have one large and one small.
Step 3: orientation assignment (rotation invariance)
For each keypoint, take the pixels around it in the Gaussian-blurred image $L$ at the keypoint's scale. Compute the gradient:
$m$ is gradient magnitude, $\theta$ is gradient orientation. $L_x,L_y$ are the horizontal and vertical derivatives of the blurred image (in Lowe's paper, pixel differences like $L(x+1,y)-L(x-1,y)$).
Build an orientation histogram with 36 bins, 10 degrees per bin ($36\times10^\circ=360^\circ$). Each pixel votes into the bin for its $\theta$ with weight $m$ (Lowe also uses a Gaussian weight centred on the keypoint). The highest peak is the dominant orientation. (Standard addition: each other peak above 80% of the maximum makes an extra keypoint at the same place.) Later, the method rotates the patch to this orientation. This makes the descriptor rotation invariant.
Practice numerical: one gradient vote. $L_x=3,\ L_y=4$. $m=\sqrt{9+16}=5$. $\theta=\tan^{-1}(4/3)=53.13^\circ$. Bin index $=\lfloor53.13/10\rfloor=5$, covering $50^\circ$ to $60^\circ$. Add $5$ to bin 5.
Dominant orientation. The bin totals are (bin 3: 12, bin 4: 30, bin 5: 55, bin 6: 22, others small). The peak is bin 5, so the dominant orientation $\approx55^\circ$ (bin centre). Bin 4 is $30/55=55\%$ of peak, below 80%, so no extra keypoint.
Step 4: keypoint descriptor generation
Rotate the neighbourhood to the dominant orientation. Then do these steps:
- Take a $16\times16$ neighbourhood around the keypoint. Divide it into $4\times4$ sub-regions (16 sub-regions, each $4\times4$ pixels).
- In each sub-region, compute an 8-bin orientation histogram (45 degrees per bin).
Descriptor length: $4\times4\times8=\mathbf{128}$. SIFT gives a 128-dimensional feature vector.
Normalisation. Before matching, SIFT normalises the vector to unit length. This makes it robust to illumination (contrast) changes:
$D=(d_1,\dots,d_{128})$ is the raw descriptor; $D'$ has unit L2 norm. (Standard addition from Lowe: clip the entries of $D'$ at $0.2$ to reduce large gradients from non-linear lighting. Then normalise again.)
Practice numerical: normalisation. A tiny descriptor is $D=(3,4,0,12)$. $\|D\|=\sqrt{9+16+0+144}=\sqrt{169}=13$. $D'=(3/13,\,4/13,\,0,\,12/13)\approx(0.231,\,0.308,\,0,\,0.923)$. Check: $0.0533+0.0948+0+0.852=1.0$.
Illumination check. If the brightness doubles, the gradients double: $D=(6,8,0,24)$, norm $26$, $D'$ is identical. Thus normalisation removes contrast changes.
Matching: compare the descriptors with the Euclidean distance. Standard practice is Lowe's ratio test. Accept a match if (distance to nearest neighbour) / (distance to second nearest) $<0.8$.
SIFT in one view
| Step | What | Gives invariance to |
|---|---|---|
| 1. Scale-space extrema | DoG pyramid, 26-neighbour max/min | Scale |
| 2. Keypoint localisation | Reject low contrast and edge points | Stability, noise |
| 3. Orientation assignment | 36-bin histogram, dominant peak | Rotation |
| 4. Descriptor | $4\times4\times8=128$-D, unit-normalised | Illumination, small viewpoint change |
Descriptor comparison
| Descriptor | Detector used | Dimension | Scale invariant | Rotation invariant | Main idea |
|---|---|---|---|---|---|
| SIFT | DoG | 128 | Yes | Yes | Gradient histograms |
| PCA-SIFT | DoG | ~36 | Yes | Yes | PCA-compressed SIFT |
| ORB | FAST | 256 bit | Partial | Yes | FAST keypoints + rotated BRIEF binary descriptor |
| GLOH | DoG | 128 (after PCA) | Yes | Yes | Log-polar histogram |
| SURF | Hessian | 64 | Yes | Yes | Haar-wavelet responses |
| HOG | Dense grid | Variable | No | Limited | Gradient orientation histograms |
| PHOG | Dense grid | Large | No | Limited | Multi-level (pyramidal) HOG |
| PHOW | Dense SIFT | Large | Yes | Yes | Dense SIFT + spatial pyramid of visual words |
Extra detail on the three most likely exam topics
- SURF: speeds up SIFT. It approximates Gaussian second derivatives with box filters on an integral image (so the cost does not depend on the filter size). It detects keypoints with the determinant of the Hessian. It describes them with Haar-wavelet responses: $4\times4$ sub-regions, each storing $(\sum dx,\ \sum|dx|,\ \sum dy,\ \sum|dy|)$. This gives $4\times4\times4=64$ numbers.
- ORB: oriented FAST corners (the orientation comes from the intensity centroid) plus rotated BRIEF. BRIEF compares pairs of pixel intensities. Each pair gives one bit, so 256 pairs give a 256-bit descriptor. Matching of binary descriptors uses the Hamming distance (the count of differing bits), which is very fast. ORB has no patent and is made for real time.
- HOG: dense, not at keypoints. Divide the image into small cells (e.g. $8\times8$ pixels). In each cell, build a 9-bin histogram of gradient orientations weighted by magnitude (unsigned gradient, $0^\circ$ to $180^\circ$, $20^\circ$ per bin). Normalise over overlapping $2\times2$-cell blocks. For a $64\times128$ window: $7\times15$ blocks $\times$ 36 values $=3780$ numbers. HOG is a classic method for pedestrian detection.
Practice numerical: HOG cell votes. Bin centres at $10^\circ,30^\circ,\dots,170^\circ$. A pixel has gradient magnitude $8$ and angle $25^\circ$. It lies between the centres $10^\circ$ and $30^\circ$ (distance 15 and 5). Split the vote linearly: the $30^\circ$ bin gets $8\times\frac{25-10}{20}=6$ and the $10^\circ$ bin gets $8\times\frac{30-25}{20}=2$. A pixel at exactly $30^\circ$ with magnitude $10$ gives all $10$ to the $30^\circ$ bin.
Block normalisation: the block vector $v=(3,4)$ becomes $v/\sqrt{9+16}=(0.6,0.8)$.
| Point | SIFT | SURF | HOG | ORB |
|---|---|---|---|---|
| Keypoints | DoG extrema | Hessian determinant | None (dense grid) | FAST corners |
| Descriptor | 128 floats | 64 floats | Long histogram | 256 bits |
| Distance | Euclidean (L2) | Euclidean | Euclidean / SVM | Hamming |
| Speed | Slow | Faster | Fast | Fastest |
| Scale / rotation | Yes / yes | Yes / yes | No / limited | Partial / yes |
Texture descriptors
Texture is the spatial arrangement and variation of pixel intensities. It describes surface properties such as smoothness, roughness, regularity, coarseness and directionality (grass vs brick wall in the slides). There are three standard problems:
- Texture segmentation: break an image into regions with constant texture.
- Texture synthesis: build large texture regions from small example images.
- Shape from texture: recover the surface orientation or shape from the distortion of the texture in the image.
Local Binary Patterns (LBP)
LBP is a very cheap micro-texture descriptor. For each pixel, compare the neighbours with it. A neighbour at least as bright gives 1. A darker neighbour gives 0. Read the 8 bits as a binary number.
$g_c$ is the centre pixel value. $g_p$ are the $P$ sampling points evenly spaced on a circle of radius $R$. $s$ is the step function. $2^p$ is the weight of neighbour $p$. For the usual $3\times3$ case $P=8,\ R=1$ and the code is in $0..255$.
Slide example. $3\times3$ neighbourhood, centre $g_c=54$:
| 45 | 90 | 80 |
| 32 | 54 | 12 |
| 78 | 65 | 10 |
Threshold against 54 (neighbour $\ge54$ gives 1):
| 0 | 1 | 1 |
| 0 | - | 0 |
| 1 | 1 | 0 |
Binary weights $2^p$ used on the slide:
| 1 | 2 | 4 |
| 128 | - | 8 |
| 64 | 32 | 16 |
$\mathrm{LBP}=0\cdot1+1\cdot2+1\cdot4+0\cdot8+0\cdot16+1\cdot32+1\cdot64+0\cdot128=2+4+32+64=\mathbf{102}$.
Compute the LBP code for every pixel. Then build a histogram of LBP values (256 bins). The histogram is the feature vector. Usually you split the image into a grid of cells and build one histogram per cell. Then join the cell histograms. This is how face recognition uses LBP.
Practice numerical. Centre $6$, same weights as above:
| 6 | 5 | 2 |
| 7 | 6 | 1 |
| 9 | 8 | 7 |
Top-left $6\ge6$ gives 1 (equal counts as 1). Top 5 gives 0, top-right 2 gives 0. Right 1 gives 0. Bottom-right 7 gives 1, bottom 8 gives 1, bottom-left 9 gives 1, left 7 gives 1. Code $=1\cdot1+0+0+0+1\cdot16+1\cdot32+1\cdot64+1\cdot128=1+16+32+64+128=\mathbf{241}$.
Use the weights in the layout that the question gives. Different books start at a different neighbour or go in a different direction, so the same patch gives different numbers. Always state the order that you use. The slide goes clockwise from the top-left: $1,2,4$ on the top row, $8$ right, $16,32,64$ along the bottom (right to left), $128$ left.
Properties
- Illumination invariance: LBP uses only the sign of $g_p-g_c$, so it is invariant to any monotonic gray-scale transform. Check: add $50$ to every pixel in the slide patch, or multiply every pixel by $2$. The code stays $102$.
- Uniform patterns: a pattern is uniform if it has at most two $0\to1$ or $1\to0$ transitions (circular bit string). Most uniform patterns are spots, flat areas, edges and corners. For $P=8$ there are 58 uniform patterns. Give each its own bin and put all non-uniform patterns in one bin. This gives a 59-bin histogram instead of 256.
Is $11110000$ uniform? Around the circle, there is one $1\to0$ transition and one $0\to1$ transition (wrap-around): 2 in total, so it is uniform. $10101010$ has 8 transitions: not uniform. The slide code $102=01100110$ has 4 transitions: not uniform.
Gabor filter
A Gabor filter is a frequency- and orientation-sensitive filter used to extract texture. It is a sinusoid (a stripe pattern of a chosen wavelength and direction) with a Gaussian window. It is similar to the response of cells in the human visual cortex.
with rotated coordinates $x'=x\cos\theta+y\sin\theta,\ \ y'=-x\sin\theta+y\cos\theta$. Parameters: $\lambda$ = wavelength of the stripes, $\theta$ = orientation, $\sigma$ = Gaussian width, $\gamma$ = aspect ratio, $\phi$ = phase offset.
The filter gives a strong response where the image has stripes with about that wavelength and direction.
Gabor filter bank for texture features:
- Apply a bank of Gabor filters with different orientations and frequencies (typically $0^\circ,45^\circ,90^\circ,135^\circ$).
- Each filter captures texture in one direction.
- For each filtered image, compute the statistics: mean, energy, variance, standard deviation.
- Join all values into the feature vector, e.g. $F=[E_1,E_2,E_3,E_4]$ or $F=[\mathrm{Mean}_1,\mathrm{Var}_1,\dots]$.
With 4 orientations and 3 frequencies, that is 12 filters. Using mean and standard deviation per filtered image gives a $12\times2=24$-dimensional texture vector.
| Advantages | Applications |
|---|---|
| Captures orientation information Multi-scale analysis Biologically inspired Excellent for texture segmentation | Fingerprint recognition Iris recognition Fabric inspection Remote sensing |
Gray Level Co-occurrence Matrix (GLCM)
GLCM is a second-order statistical texture descriptor. First-order statistics (the histogram) show how many pixels have each gray level. GLCM shows how often pairs of gray levels occur with a given spatial relationship: distance $d$ and angle $\theta$ (displacement $(\Delta x,\Delta y)$).
In a smooth image, neighbours have the same level (big numbers on the diagonal of the GLCM). In a noisy or striped image, neighbours have very different levels (mass far from the diagonal).
Construction
- For $N$ distinct gray levels the matrix is $N\times N$.
- $P_{d,\theta}(i,j)$ counts how often a pixel of level $i$ has a neighbour of level $j$ at distance $d$ and angle $\theta$. (The slide words it the other way round. For $\theta=0^\circ$, $d=1$ the pair is (pixel, pixel immediately to its right). The first index is the reference pixel.)
- Common angles: $\theta\in\{0^\circ,45^\circ,90^\circ,135^\circ\}$.
- Normalise so that all entries sum to 1: $P(i,j)=C(i,j)/\sum C$. The counts become joint probabilities.
Slide example. $6\times6$ image, $N=4$ levels $\{0,1,2,3\}$, $d=1$, $\theta=0^\circ$ (neighbour immediately to the right, offset $[0,1]$).
| 0 | 0 | 1 | 1 | 2 | 2 |
| 0 | 0 | 1 | 1 | 2 | 2 |
| 1 | 1 | 2 | 2 | 3 | 3 |
| 1 | 1 | 2 | 2 | 3 | 3 |
| 0 | 1 | 1 | 2 | 2 | 3 |
| 0 | 1 | 1 | 2 | 2 | 3 |
Step 1: pixel pairs (left, right) per row. Each row of 6 gives 5 pairs. The image has $6\times5=30$ pairs.
- Row 0 and Row 1: $(0,0),(0,1),(1,1),(1,2),(2,2)$
- Row 2 and Row 3: $(1,1),(1,2),(2,2),(2,3),(3,3)$
- Row 4 and Row 5: $(0,1),(1,1),(1,2),(2,2),(2,3)$
Step 2: count each pair.
| Pair $(i,j)$ | Found in rows | Count |
|---|---|---|
| (0,0) | 0, 1 | 2 |
| (0,1) | 0, 1, 4, 5 | 4 |
| (1,1) | 0, 1, 2, 3, 4, 5 | 6 |
| (1,2) | 0, 1, 2, 3, 4, 5 | 6 |
| (2,2) | 0, 1, 2, 3, 4, 5 | 6 |
| (2,3) | 2, 3, 4, 5 | 4 |
| (3,3) | 2, 3 | 2 |
Total $=2+4+6+6+6+4+2=30$. Directional count matrix (rows = $i$, columns = $j$, levels 0..3):
| 2 | 4 | 0 | 0 |
| 0 | 6 | 6 | 0 |
| 0 | 0 | 6 | 4 |
| 0 | 0 | 0 | 2 |
Step 3: normalise by dividing by 30:
| 0.067 | 0.133 | 0 | 0 |
| 0 | 0.200 | 0.200 | 0 |
| 0 | 0 | 0.200 | 0.133 |
| 0 | 0 | 0 | 0.067 |
(Exact: $2/30,\ 4/30,\ 6/30,\ 4/30,\ 2/30$.) This matrix is not symmetric, because it counts only the rightward direction. A symmetric GLCM counts both $0^\circ$ and $180^\circ$. It is $C+C^T$ normalised by 60.
Haralick features
The scalar features come from the normalised $P(i,j)$:
$\mu_x=\sum_i i\sum_jP(i,j)$, $\mu_y=\sum_j j\sum_iP(i,j)$ are the means of the row and column marginals, and $\sigma_x,\sigma_y$ their standard deviations.
Features for the slide GLCM (use the counts $C$ and divide by 30 at the end):
Contrast. Only the pairs with $|i-j|=1$ add to the sum: $(0,1)=4,\ (1,2)=6,\ (2,3)=4$. The total is 14 counts, each with weight $1^2$. Contrast $=14/30=0.467$.
Energy. $\sum C^2=4+16+36+36+36+16+4=148$. Energy $=148/900=0.164$.
Homogeneity. Diagonal counts $2+6+6+2=16$ (weight 1). The off-diagonal counts are $14$, each with $|i-j|=1$, so the weight $1/2$ gives $7$. Homogeneity $=(16+7)/30=0.767$.
Correlation. Row sums of $C$: $6,12,10,2$. Column sums: $2,10,12,6$. $\mu_x=(0\cdot6+12+20+6)/30=38/30=1.267$. $\mu_y=(0+10+24+18)/30=52/30=1.733$. $E[i^2]=70/30=2.333$, so $\sigma_x^2=2.333-1.604=0.729$, $\sigma_x=0.854$. $E[j^2]=112/30=3.733$, so $\sigma_y^2=3.733-3.004=0.729$, $\sigma_y=0.854$. $E[ij]=(6\cdot1+6\cdot2+6\cdot4+4\cdot6+2\cdot9)/30=84/30=2.8$. Covariance $=2.8-1.267\cdot1.733=0.604$. Correlation $=0.604/0.729\approx0.83$. This is strongly positive: neighbouring gray levels rise together.
Quick check: high contrast means that the pairs are far apart in level. High homogeneity and energy mean a smooth, uniform texture with mass near the diagonal. Contrast and homogeneity change in opposite directions.
Practice numerical. $3\times3$ image with 2 levels: rows $0\,0\,1$ / $0\,1\,1$ / $1\,1\,1$, $d=1$, $\theta=0^\circ$. Pairs: row 0: $(0,0),(0,1)$; row 1: $(0,1),(1,1)$; row 2: $(1,1),(1,1)$. Counts: $(0,0)=1,\ (0,1)=2,\ (1,1)=3$, total 6. $P=\begin{bmatrix}1/6&2/6\\0&3/6\end{bmatrix}$. Contrast $=(0-1)^2\cdot2/6=0.333$. Energy $=(1+4+9)/36=0.389$. Homogeneity $=1/6+3/6+(2/6)/2=0.833$.
LBP vs Gabor vs GLCM
| Feature / method | LBP | Gabor filter | GLCM |
|---|---|---|---|
| Category | Micro-structural / local non-parametric | Transform / frequency-domain filter | Second-order statistical |
| Computational speed | Extremely fast (real time) | Slow (convolution / FFT intensive) | Moderate (depends on quantisation) |
| Scale sensitivity | Sensitive at a fixed radius $R$ | Handled by multi-scale bank | Depends on distance $d$ |
| Illumination robustness | Very high (monotonic invariant) | Moderate | Low (requires normalisation) |
| Primary strengths | Edge/corner micro-patterns, low memory | Multi-orientation analysis, biological plausibility | Spatial dependency statistics, classic benchmark |
| Common uses | Face recognition, texture classification | Fingerprint analysis, seismic imaging | Medical imaging (CT/MRI classification), remote sensing |
(The scale-sensitivity row is hard to read in the slide image. The entries above are the standard textbook description.)
Quick self-test
1. What is the difference between a feature detector and a descriptor? Give examples.
A detector finds the location (and scale) of an interesting point: Harris, DoG, FAST. A descriptor encodes the local patch around the point as a vector for matching: SIFT, SURF, BRIEF. SIFT does both (DoG detector plus gradient-histogram descriptor).
2. What are octaves and intervals? How many images result from a Gaussian pyramid with 4 octaves and 6 images per octave?
An octave is a set of images where the blur doubles ($\sigma\to2\sigma$). After each octave, the image size is halved. Intervals are the blurred images inside an octave. $4\times6=24$ Gaussian images, and $4\times5=20$ DoG images.
3. In SIFT step 1, how many comparisons does each DoG pixel undergo and why?
26: 8 neighbours in its own scale plus 9 in the scale above plus 9 in the scale below. A maximum or minimum over all 26 means a stable blob-like extremum in $(x,y,\sigma)$. This is how SIFT chooses the scale of the keypoint.
4. Why does SIFT remove edge responses and low-contrast keypoints?
Low-contrast points are sensitive to noise and are not repeatable. Edge points have one large and one small principal curvature, so their position moves along the edge (unstable). Keep points where $\mathrm{tr}(H)^2/\det(H)<(r+1)^2/r$ with $r=10$.
5. A keypoint has gradient $L_x=-6,\ L_y=8$. Give $m$, $\theta$ and the orientation bin (36 bins).
$m=\sqrt{36+64}=10$. $\theta$ is in the second quadrant: $\mathrm{atan2}(8,-6)=180^\circ-53.13^\circ=126.87^\circ$. The plain $\tan^{-1}(8/-6)=-53.13^\circ$ is in the wrong quadrant, so add $180^\circ$. Bin $=\lfloor126.87/10\rfloor=12$, covering $120^\circ$ to $130^\circ$. The vote is $10$.
6. Why is the SIFT descriptor 128-dimensional, and how is it made illumination invariant?
$16\times16$ patch, split into $4\times4=16$ sub-regions, each with an 8-bin orientation histogram: $16\times8=128$. The L2 norm divides the vector, so a uniform change in contrast (which scales all gradients) cancels. Gradients also ignore additive brightness shifts.
7. Compute the LBP code of $\begin{bmatrix}10&20&30\\40&25&15\\5&35&25\end{bmatrix}$ with the slide's weight layout.
Centre 25. Neighbours $\ge25$: top-right 30 (weight 4), left 40 (weight 128), bottom 35 (weight 32), bottom-right 25 (weight 16, equal counts as 1). Others (10, 20, 15, 5) give 0. Code $=4+16+32+128=180$.
8. What is a uniform LBP pattern and why use it?
A uniform pattern is a circular binary pattern with at most two bit transitions (e.g. $00011100$). It shows the main micro-structures (edges, corners, spots). If you put all non-uniform patterns in one bin, the histogram shrinks from 256 to 59 bins and has less noise.
9. Explain the Gabor filter parameters and how a Gabor feature vector is built.
$\lambda$ wavelength, $\theta$ orientation, $\sigma$ Gaussian width, $\gamma$ aspect ratio, $\phi$ phase. Apply a bank at several orientations ($0^\circ,45^\circ,90^\circ,135^\circ$) and frequencies. For each filtered image, compute the mean, energy, variance and standard deviation. Join all values.
10. Build the GLCM ($d=1$, $\theta=0^\circ$) for the image $\begin{bmatrix}0&0&1\\1&2&2\\0&1&2\end{bmatrix}$ and give its contrast.
Pairs: row 0: $(0,0),(0,1)$; row 1: $(1,2),(2,2)$; row 2: $(0,1),(1,2)$. Counts: $(0,0)=1,\ (0,1)=2,\ (1,2)=2,\ (2,2)=1$; total 6. Contrast $=\big[(0-1)^2\cdot2+(1-2)^2\cdot2\big]/6=4/6=0.667$.
11. Name four Haralick features from GLCM and what each measures.
Contrast (local variation), energy/ASM (uniformity), homogeneity/IDM (similarity of neighbours), correlation (linear dependence of gray levels).
- A descriptor is a feature vector of the patch around a keypoint. A detector finds the point. A descriptor fingerprints it.
- Scale space: $L=G(\sigma)*I$; octave = $\sigma$ doubles then downsample by 2; intervals = blurred images within an octave; $\mathrm{DoG}_i=G_{i+1}-G_i$.
- SIFT: (1) DoG extrema vs 26 neighbours, (2) reject low contrast and edge points, (3) 36-bin ($10^\circ$) orientation histogram; the highest peak is the dominant orientation, (4) $4\times4\times8=128$-D descriptor, L2-normalised.
- $m=\sqrt{L_x^2+L_y^2}$, $\theta=\tan^{-1}(L_y/L_x)$.
- Descriptor table: SIFT 128 (DoG), SURF 64 (Hessian, Haar), ORB 256-bit (FAST + rotated BRIEF, Hamming), HOG dense grid, not scale invariant.
- Texture = spatial arrangement of intensities; tasks: segmentation, synthesis, shape from texture.
- LBP: $\sum s(g_p-g_c)2^p$, slide example gives 102; histogram is the feature; monotonic-illumination invariant; uniform = at most 2 transitions.
- Gabor: orientation and frequency selective Gaussian-windowed sinusoid; bank at $0,45,90,135$ degrees; features are mean, energy, variance, std.
- GLCM: second-order statistic, counts gray-level pairs at $(d,\theta)$, normalise to probabilities; slide example total 30; features contrast, energy, homogeneity, correlation.