I open at the close: Morphological Operations

Morphological operations is one of the subjects I really like in 186. I think it should be taught earlier next to image segmentation. During my BS3 days, I struggled on processing a video of a pile of grains with only google skills and no image processing knowledge. At first, I was to track the spaces between the grains. This is quite easy for me since I can use region props. However, when tracking the actual grains, all the grains is in one huge connected component, where region props is pretty useless. It eventually reached the limitation of my google skills and I made no progress on it since then. But lo and behold, morphological operations!!

From the word morphology, referring to shape or structure, a structuring element expands of reduces a shape using dilation and erosion. Combination of dilation and erosion creates new useful morphological operations, which are the closing and the opening operator. The closing operator dilates then erodes and the opening erodes, then dilates. The available tophat function obtains the blobs removed by the opening operator. The closing operator in general ‘closes’ an object or removes the holes inside a shape. The opening is used for cleaning the scattered noise in an image. With this in mind, one can think of using morphological operators to ‘clean’ an image after binarization. I could now go back to A6 to clean the edge detection technique I did. 

There are two main morphological operation, which is the dilation and erosion. Other morphological operations are combinations of these basic operation. Dilation of a shape with a structuring element consists of the intersection of the shape and the translation of the structuring element with the shape. Translation here referes to the area covered when the point of reference of the structuring element traverses the whole shape. Dilation makes the shape expand. Erosion in the other hand can be thought of as the area reached by the point of reference in the structuring element while the structuring element moves inside the shape. This operation reduces the shape. 

For the following shapes: a 5x5 square, a triangle with a base of 4 and a height of 3, a hollow 10x10 square of 2 boxes thick, and a plus sign of 1 thickness and 5 pixels along each line, a dilation and erosion of the following structuring elements: 2x2 ones, 2x1 ones, 1x2 ones, and a cross of 3 pixels long and one pixel thick was done by hand and was plotted using Matlab. The results are shown in figures 1, 2, 3, and 4. The rendered images in matlab was inverted to show the shaped form using black pixels instead of white. Most of the resulting shape manually made are similar to those rendered. For the dilation of the triangle with the cross however, there is a slight difference. This could be due to the limitation of the rendering, where the program tends to smoothen the images when there is an edge. 

Figure 1. The dilation and erosion of the shapes 5x5 square, a 4x3 triangle, a 10x10 hollow square and a 5x5 plus sign with a 2x2 structuring element. 

Figure 2. The dilation and erosion of the shapes with a 2x1 structuring element 

Figure 3. The dilation and erosion of the shapes with a 1x2 structuring element

Figure 4. The dilation and erosion of the shapes with a 3x3 cross structuring element

For the application, we now use the morphological operations in an image of circles. In this, we pretend that the circles are normal sized cells. The area of the normal sized cells should be known in order to locate the abnormally sized cells which could possibly be a cancer cell. So, we binarize the image first using a threshold from the histogram of the image. The peak at the histogram shows the pixels of the background, which is the most abundant pixels. I chose the upper limit of the curve as the threshold to perfectly eliminate the background. In this part, I would like to acknowledge Sir Mario since he saw the bug in my code regarding histogram, in which I was stuck with for days.

Figure 5. The section 7 of the image a) the section 7 b) the binarized image based on the threshold obtained from the PDF c) The PDF of the intensity values of the image. The threshold used was 205, which is at the upper limit determining the pixels of the background.

Figure 6 refers to the sections in the segmented image. Some sections have overlapping areas. For the binarization, shown in figure 7, a different threshold was used for each subsection depending on the histogram of the image. The threshold obtained are just estimated and the range is small, which is from 200 to 218. Using a constant threshold value for all would seem okay. With this method however, some blobs get distorted and gets eliminated after morphological operations. Some blobs also gets merged and turned into a very big blob which is lost data since we need statistics of the area of the cells. To solve this, two methods could be done: 1) personalize the morphological operation used for each section or 2) personalize the threshold values. The result of personalizing the threshold value for each subsection is better since after the morphological operation, fewer blobs got merged. The best result was shown in figure 7, and will be used to the next processes.

After binarization with the best threshold values for each subsection, the images are still not it its ideal look. In figure 7, there are still observable residue from the binarization made. This is more visible in the small white noise-like dots in the middle subsection (section 5) and the lower left subsection (section 7). This is now where the morphological operations are used, which is already repeatedly mentioned since the start. The best result is shown in figure 8, where opening was done first to remove the small dots, then closing, to remove holes in the blobs. The structuring element used is a disk with a radius of 2-5 pixels for both operation.  The structuring element was made to be smallest as possible since at a bigger radius, the blobs became deformed and most of the blobs gets merged, which is lost data. Care should be taken however since at a very small radius, the small white dots does not get removed after opening. Also opening first rather than closing gives out a cleaner result since if closing was done first, the small white dots expands with a big structuring element. When closing with a small structuring element, the holes inside some of the blobs does not get removed. Another problem is that some of the noise in the background grows bigger as seen in the upper left image (section 1).

Figure 6. The image divided into 9 256x256 sections

Figure 7. The image binarized in each subsection. The threshold use for each subsection varies depending on the obtained histogram of the image. For left to right, top to bottom subsections, the thresholds are 210, 213, 220, 200, 200, 218, 205, 205, 220. 

Figure 8. The image applied with morphological operations. First, it was applied with opening, to remove the small dots, then  closing, to remove a few holes inside the blobs. 

It is a sad thing not to be able to separate the the merged blobs. There is another morphological application that can do this. It is the watershed segmentation, which was mentioned in my previous post. I will try to do this, if there are still time. For now, the available data from the isolated cells would suffice.

Each blob was labelled and separated using connected components. The area and radius of each blob was obtained using properties from regionprops. From the labelling done, one blob can be individually displayed as shown in figure 9. The histogram of the areas of the blobs is shown in figure 10, here, the abnormally sized cells were already removed. The threshold value for removal was estimated from the histogram of the original data. 

Figure 9. The blobs in section 7. a) The connected components was separated by looking at the connected white pixels bounded by black pixels as one component. b) each connected component was labelled as shown by the different colors when endered with imshow. c) One blob can be separately shown using the label. 

Figure 10. The PDF of the areas of the cells. The abnormal sizes were already removed. The mean area of the blob is 505.900 px and the standard deviation is 2.9230 px. The mean radius estimate of the included blobs was computed to 14.9869 px with a standard deviation of 2.9230px.

Now, to remove isolate the cancer cells in an image, the opening operator was applied to the image using a structuring element of a disk with a radius of the mean radius of the normal cells. With this operation, the structuring element buries the shapes of the same size during erosion, removing it completely. A bigger structuring element will remove the cancer cells while a smaller one will not completely remove the normal sized cells. Figure 11 shows this step. In figure 11 e) only the cancer cells remained. Using manual checking, 5 abnormally sized cells can be detected in the original image. This shows that the method is good. 

Figure 11. The image with cancer cells. a) The binarized image. b) Image after opening. c) image after closing. d) The different labels of the blobs. e) opening with a disk of radius 15, which is the mean radius of normal cells.

For the activity, I give myself 10/10 since I have completely done all the expected output and the images looks good. Aside from all the many people who helped me finish this activity, I would specifically thank Kuya Franco for mentioning the function strel in matlab. Without it, I was manually creating matrices for my structuring element.

The worst prison would be a closed heart – John Paul II