IX

Morphological operations uses Set Theory to manipulate matrices. Since we now know that images are just matrices of values with layers of channels, it is appropriate that certain algorithms can be devised to do morphological operations with pictures.

This activity deals primarily with some basic shape alteration and recognition. For basic operations, binary "flattened" versions of multi-chanelled images are used. To start off, let's examine how these 2 operations work.

Note: Since the images are in their binary form, zeros are considered as background while ones are the object. Background is ignored by the operations, and is thus useful since the operations would only work if the matrices compared have the same dimensions.

First, matrix A (one containing the original pattern) is scanned with another matrix B (one containing the mask). Then, a new zero matrix C (transformed image) with the same dimensions as matrix A & B are mapped depending on the operation used:

• erosion: All coordinates of the anchor point for when the mask is entirely enclosed by the object is set to one in matrix C. 

Figure 1. Erosion. The anchor point is the center of matrix B.

• dilation: All coordinates of the anchor point for when at least one element of the mask intersects with the object is set to one.

Figure 2. Dilation. The anchor point is the center of matrix B.

Thus, the anchor element of the mask in matrix B determines how the transformed image will be shifted in matrix C with respect to matrix A.

Using 4 original patterns and 5 masks, erosion and dilation are employed and their effects were observed. The anchor points for the masks are as follows:

1. 2x2 square: Top-left pixel
2. 2x1: Top pixel
3. 1x3: Left pixel
4. 3x3 cross: Top-most pixel 

                          Figure 3. Original Patterns: 5x5 square, 3x4 right triangle, 10x10 square annulus 2 pixels thick, 5x5 cross 

                                          Mask: 2x2 square, 2x1 , 1x2, 3x3 cross and a 2 pixel long diagonal.

erode()

Figure 4. My hand drawn predictions for erosion.

Figure 5. scilab's erode() operator results.

My prediction would've been perfect if not for my careless error on the diagonal mask of the square annulus. However, I now fully understand how erosion works. Since erosion "trims" by fitting the mask entirely, it is possible to have completely blank images. We can see this with the cross mask for the 3x4 triangle and the 2x2 mask for the 5x5 cross patterns.

dilate()

Figure 6. My hand drawn predictions for erosion.

Figure 7. scilab's dilate() operator results.

Again, like with my erosion predictions, this would've been perfect if not for the 1x2 mask of the square annulus.

thin() & skel()

Lastly, I examine the thin() and skel() operators of scilab. These are more complex than the erode() and dilate() operators. From the help file, the implementation of thin() on an image of text produces:

Figure 8. Above: Original image. Below: thin() results

thin() seems to trace lines and curves by "thinning" them until they are only one pixel wide. The deviations from straight lines came as a consequence of not using a perfectly binary image. The above image was just converted using im2bw(), in which the conversion led to some unstraightened lines when thin() was used.

As we can see, this may be problematic when we have lots of line nodes in our image. For this, we use the more complex skel(). 

Figure 9. L-R: Original image, result of skel() superimposed with the original and the distance transform.

skel() successfully traced a quite rounded and thick image. These characteristics would have resulted to a poor trace, had we used thin() for the image. skel()  seems to average the whole network of lines and deduce the "skeletal" frame of the image. As such, it also has a distance mapping output. This seems to be a pixel population distribution map of the image with respect to skel()'s traced path.

This was a good introductory activity for morphological transforms of images.

  Self-Assessment: 9/10   

VIII

We can enhance or filter out unwanted frequencies of an image by removing them in the Fourier map and then reapplying the transform to obtain the final image.

Note: The reconstructed images are180 degrees rotated (with respect to the original) due to the FT.

8A. Convolution Theorem

Figure 1. L-R Two-dot binary image and it's FT.

Fig. 1 is an example of how the FT of a FT of an image would revert back to the original image. If we reverse the image labels (i.e. if we take the right image as the original one), one can remember that the 2 dots are representative of the quantitative value of the frequency on the original image.

Figure 2. R-L: Dots were replaced with circles of increasing radii. Top-Bottom: Original image and its FT.

Fig. 2 shows that as the circles were increased, the overall size of its FT diminishes. Why? This is because of the now 2D nature of our image. As the circles increase in size, this is interpreted in a Fourier-sense that a more constant and non-repetitive image is being generated. We can note that the black lines in the FT are remnants of the 1D layout (the center of the circles are still on the x-axis) and the concentric light bands are the 2D components of the circles.

Figure 3. R-L: Dots were replaced with squares of increasing areas. Top-Bottom: Original image and its FT.

Fig. 3 shows a similar behavior as Fig. 2. The shape of the squares are reflected in its FT.

Figure 4. R-L: FT of circles with Gaussian intensity distribution (increasing variance)

Just as with Fig. 2, as the size of the Gaussian circles increase, the radius of the resulting FT pattern decreases. However, due to the the distribution of the intensity, the resulting FT also has a less distinct concentricity.

Figure 5. Convolution of 10 randomly placed dots and a random 3x3 matrix

Fig. 5 uses convolution. This image is not very different from the original image. The only difference is that the dots became broader. The 3x3 matrix appeared to have been transposed to the dot locations, as noticeable from the convolution of a function f(x) with a dirac-delta. 

 (1)

Eq. 1 shows that as the convolution causes f(t) to appear on the previous location of the dirac-delta. Fig. 5 has white dots (1 pixel in size) that are considered dirac-deltas. So the result of imconv() reflects the 3x3 mask on the location of the 10 random dots.

Figure 6. L-R: FT of equally spaced white pixels. (5, 10, 50, 100 & 200 pixel separation on both x and y axis, respectively)

Fig. 6 further cements our first-hand experience that the FT is in frequency space. As the "wavelength" is increased, the frequency decreases, so the magnitude of the separation of the dots in the corresponding FT decreases, too.

8B. Ridge Enhancement

This time, I will do a more practical application of Fourier map knowledge. Since fingerprints have a repetitive structure, filtering in the Fourier map may enhance their images.

Figure 7. L-R: My own fingerprint and it's FT

From Fig. 7, we can see that there are a lot of noise on the radial extreme of the FT. A prominent halo can be distinguished in the middle (with the DC term on the origin). I tried to blacken out the noisy parts and retain the middle parts, and this is what I got:

 

Figure 7. L-R: Filtered FT (top) and their respective reconstructed images (bottom)

Fig. 7 shows how the filtering of signal in the Fourier map affects the quality of the reconstructed image. When the mask covers the "halo" signal, the reconstructed image suffers a poor quality; the ridges become indistinguishable. The leftmost filter works well, however, the clarity could have been better had I removed the DC component (middle spot)

8C. Line Removal

Now, let's try some more basic filtering: line removal.

Figure 7. Top: FTs of original image (left) and it's filtered  form (right) Bottom: Corresponding reconstructed images

Fig. 7 shows the results of line removal. The lines ultimately became less accented. The position is based my previous work with FTs. If the pattern persists in a certain dimension, it would also be in the Fourier map. As such, the lines were along the image's x-axis, as such the FT had reflections on its x-axis, too.

8D. Weave Removal

Finally, let's do masking for 2D signals.

Figure 8. Top: FTs of original image (left) and it's filtered  form (right) Bottom: Corresponding reconstructed images

Similar to Fig. 7, filtering removed the blotch patterns of the original image. Note that the presence of the peaks on the FT that has x & y components signify some angled patterns. These can be thought of the collection of the individual x & y patterns viewed at a certain angle. Comparing this with Fig. 7, we see that since the weave pattern has is 2D repetitive, there are bright spots on the x & y axis of the FT image. Thus, blocking these signals and those that has has both x & y components improves our reconstructed image.

To further illustrate that these are indeed the FT components of the weave pattern, I have reconstructed the mask using FT:

Figure 9. Top-Bottom: Mask and its FT

Fig. 9 shows the inverted mask and its FT. The mask was inverted because the original was meant to block out the signals. Inverting it would approximate a FT that has peaks at the once "masking" areas. The FT clearly reveals that it is indeed the weave pattern as seen in Fig. 8.

This activity has again increased my insight on the nature of FTs and their possible applications.

Self-Assessment: 10/10 

VII

2D Fourier Transforms (2D FT) require a deeper understanding of the underlying mathematical arrangements that happen in order to appreciate its capabilities. More imagination and knowledge are needed to properly visualize the transform's effects on images.

Note: Most of the Fourier Transforms shown here are zoomed and not it their scale. This was done to enhance the clarity of the discussion.

7A. 2D FT Familiarization

Figure 1. L-R Various patterns: Square, annulus, square annulus, 2 slits & 2 dots (top row) 

and their respective Fourier Transforms (bottom row)

Fig. 1 exemplifies the quite complicated nature of visualizing FTs of 2D patterns if one doesn't have a firm grasp of its concept. From the square pattern, one realizes that the focused FT in the middle demonstrates that the frequency is focused in the middle, with the trailing  frequencies demonstrate the x and y dependence, it being a square. The annulus also demonstrates this, however, the 2D FT is also circular. We get the sense that these arise when there are sharp changes of value (edges) in the image. Speaking in terms of frequency, this is a sharp change. From the square annulus, we also observe the same pattern but with more pronounced black segments, due to the interruption in the middle.

The slits and dots are quite straightforward. The line can be treated as a sinusoid with a frequency that can be observed is projected on its FT. The dots show their spherical nature in its FT.

7B. Anamorphic Property of Fourier Transforms

Figure 2. L-R Fringes of varying frequency: f = 4, 8, 16 & 32 (top row) 

and their respective Fourier Transforms (bottom row)

Fig. 2 shows a more simpler approach to understand the Fourier space. We see an immediate trend: as the frequency is increased, the spacing between the values becomes greater. This is now an exemplary example that the scales of the Fourier space is frequency. The presence of the 2 dots is an artifact of the dimension of our image: it is symmetric. Also worth noticing is that the symmetry is confined in one axis, since the corrugated roofs are in one dimension.

Figure 3. L-R: Fringe pattern with f =16 with a bias constant bias of 2 and its FT.

Fig. 3 is the result of adding a constant bias to the function that generates our image. We see that this resulted to a point between the 2 spots in its FT. This indicates the shift of the function from the origin, and thus the symmetry was broken. 

To find the actual frequencies in an interferogram image, it is then immediate that FT is the solution. For non-constant biases, we can find the frequencies by noting that what we will see should be symmetric on a diagonal.

Figure 4. L-R: Fringe pattern with f =16, shift of 30 degrees and its FT 

Fig. 4 illustrates that a shift in the actual image also reflects as a shift of the bright spots in inverse space.

Figure 5. L-R: Fringe pattern with f =16 on x & y axis and its FT 

Fig. 5 further cements the idea of the symmetry of the FT of straightforward sinusoidal functions. The FT can be interpreted by looking at its quadrant 1. Treating the center as the origin, the bright spot is then easily viewed as having x & y components. Then with that, we see that it is also symmetric in those axes, thus producing the 4 spot FT image.

Lastly, I add 3 rotated sinusoids with (same f, theta=30,45,50) to Fig. 5. Before I see the FT, I tried to predict it. The  four spots on the FT of Fig. 5 should remain. The addition of the 3 sinusoids should add a total of 9 spots: each per quadrant (symmetry) with an angle equal to theta measured from the origin.

Figure 5. FT of a 2D corrugated roof added with 3 rotated sinusoids

Disappointment. Apparently, addition doesn't yield a symmetry in the Fourier space since it only adds, it doesn't "convolve" functions such as multiplication.

  Self-Assessment: 9/10   

VI

Activity 6 formally introduces us to the Fourier Transform model of imaging.

 (1)

Eq. 1 is the 2D Fourier transform of a function f(x,y). For image processing implementations in this activity, we will be using Cooley and Tukey's Fast Fourier Transform algorithm. Before I discuss the results, I'd like to note the importance of some Scilab functions to achieve the proper results.

• gray_imread() : As we can see, we can't have a multi-channeled image (hypermatrix). This function basically flattens our subject matrix, else fft2() won't work properly.
• abs() : Let us not forget that employing a FT generates a complex function. This is thus necessary in appropriately projecting our results as an image (matrix of real values).
• fftshift() : This allows us to flip a matrix along its diagonal. Useful, since fft2() has its quadrants interchanged along the diagonal.

6A. Discrete Fourier Transform

Figure 1. Left to Right: Original image, fft2() result, fftshift() result and the twice fft2()'d image.

Fig. 1 shows the result of a series of FTs on an image of a circle (r=0.9 with respect to 128). The fft2() resulted an almost black image with some white-ish pixels uniformly distributed on its vertices. Upon inspection of the numerical results, most of the values are well beyond 100, and thus appeared as black pixels. Applying fftshift() creates a speckled pattern of black pixels on a white background. This function apparently has an effect to the numerical values of the FFT. The right-most image shows that Fourier Transforms are transitive, that is, employing it 2 times results to the original function.

Figure 2. Left to Right: Original "A" image, fft2() result, fftshift() result and the twice fft2()'d image.

Fig. 2 shows the result of an "A" image that underwent the same processes as that in Fig. 1. It is noticeable that the frequency distribution (3rd image from the left) also has the same triangular features as that of the original image spread radially, possible remnants of the 2D nature of the FT used. The final image is a good example of how fftshift() corrects the orientation of the transformed image.

6B. Convolution

Figure 3. Convoluted images of "VIP" and a simulated aperture of various radii. Top to Bottom: r=0.3, 0.4, 0.5, 0.6, 0.7, 0.8 and 0.9

Fig. 3 successfully shows that the convolution is indeed effective in simulating an object-lens relationship. As the aperture is reduced in size, diffraction artifacts became more and more noticeable. These are the result of "smearing" brought by the convolution of the 2 image matrices. Let's investigate the mathematical description for Convolution:

 (2)

The difference in the g function ultimately generates the "smearing" that we observe.

6C. Template Matching using Correlation

Figure 4. Top: L-R Image and pattern filter, "A". Bottom: L-R Output image of correlation, rotated image (180 deg. counter-clockwise)

Fig. 4 exemplifies the capabilities of template matching using correlation. White dots on the bottom-right are visible to all the "A" in our base image. Let us note that the Correlation is given by:

 (3)

The sum in the g function enables proper detection of similar patterns for 2 image matrices. The function conj() is essential to have the inverse signs for the elements of our initial matrix to facilitate with Eq. 3.

6D. Edge detection using the convolution integral

Finally, we examine the capabilities of convolution for identifying patterns, such as linear ones for edge detection.

 

Fig. 5 Results of imcorrcoef(). Patterns used from L-R:  [-1 -1 -1; 2 2 2; -1 -1 -1], 

[-1 2 -1; -1 2 -1; -1 2 -1] and  [-1 -1 -1; -1 8 -1; -1 -1 -1]

The first image shows our image with most of its vertical lines lost. The second one is visibly without its horizontal lines. The third image that used a spot pattern has its vertical and horizontal lines intact, indicating that the algorithm "identifies" closed loops.


This activity has really improved my knowledge on FT and its applications for visual analysis and pattern recognition.

Self-Assessment: 10/10