A14 - Color Image Segmentation

In today's era, a wide selection of "high-technology" imaging tools have come to life. One can choose from the sleek point-and-shoot cameras, to the high resolution DSLR's and the compact built-in high megapixel cellphone cameras. With all these complicated and expensive tools, we only have one simple ultimate goal. It is just to capture a scene or an important moment in our lives. A pretty simple idea right?

The images we capture can always be separated into two parts, the scene or object of our interest and the background. Sometimes the background is not as interesting as the object/scene but we have no choice but to take them also. That's where image segmentation enters. We can use this method to separate the object from the not-so-interesting background. If you have been following my blog, separating a region of interest (ROI) from the background is not a new idea. And it was always done using grayscale images. But many objects have intrinsic colors similar to the background, thus doing segmentation in grayscale world will definitely fail. In this blog post, I'll show you two techniques, the parametric and non-parametric segmentation, of separating a colored image from a background.



The first technique is called Parametric Segmentation. The basic idea is to separate the ROI from the background using a probability distribution that provides the likelihood of a color's occurrence. The probability distribution is assumed to be a joint Gaussian probability distribution of the red and green colors. The Non-parametric Segmentation method, on the other hand, uses the histogram backprojection method to separate the ROI. The concepts will get clear as we progress through this blog post.

Both methods start by obtaining a digital image of a 3D object with a single color

 

Figure 1. Image of a red mug.

(taken from http://thriftwhore.livejournal.com/4218876.html)

We crop a monochromatic portion of the red mug as shown

 

Figure 2. A small patch of figure 1. 

(not same scale as figure 1)

We transform both images into normalized chromaticity coordinates (NCC). It is representing the color space into another one having separate chromaticity and brightness information. It can be done using the formula

 

Equation 1. Normalized chromaticity coordinates

where R, G, B are the red, green and blue layers of the image and r, g, b are the corresponding normalized chromaticity coordinates. 

Parametric segmentation 

We assume independent Gaussian probability distributions for r and g of figure 1 using

 

Equation 2 and 3. Gaussian probability distributions for red and green.

where r and g are the normalized chromaticity coordinates of red and green, sigma is the standard deviation of r or g layer of the patch in figure 2, mu is the mean value of the r or g layer of the patch.

We can then segment the red mug from figure 1 using a joint probability distribution derived by just multiplying equations 2 and 3. The resulting segmentation is

Figure 3. ROI separation using parametric segmentation.

Non-parametric segmentation

This method starts by taking the two-dimensional histogram of the normalized chromaticity coordinates r and g for the patch in figure 2 shown in figure 4a. For verification purposes, we compare it with the normalized chromaticity diagram in figure 4b.

 

Figure 4. (a) 2D histogram (b) normalized chromaticity diagram.

The x- and y-axis of figure 4a and 4b is r and g, respectively. Figure 4a shows that the color of the patch being observed (figure 2) is within the red region when compared with figure 4b. This is definitely true as the region of interest of figure 1 is the mug colored red.

Using the histogram in figure 4a, we can do backprojection (same as in my previous blog post) to segment the red mug in figure 1.

 

Figure 5. ROI separation using non-parametric segmentation.

Comparing the results obtained using both methods (figure 3 and figure 5), we can see that the parametric segmentation produced ROI separation with smoother shade. This can be explained by the fact that the parametric segmentation uses Gaussian distribution as the basis probability distribution thus producing continuous values. The non-parametric segmentation, on the other hand, produced a brighter ROI separation and larger area coverage (look at the mug handle). Overall, both techniques gave impressive and positive ROI separation results.

We can check whether the techniques really are effective in ROI separation by implementing them on another sample shown below

 

Figure 6. Image of a red car (ROI).

(taken from http://69wallpaper.blogspot.com/2011/01/red-car-wallpaper.html)

The patches cropped from figure 6 and the corresponding segmentation results using parametric and non-parametric are shown below.

 

Figure 7. Left column contains the cropped patches. Middle column contains 

the segmentation using parametric method. Right column has results using

non-parametric method.

Results in figure 7 highly suggest that the choice of patch of the region of interest (ROI) greatly affects the resulting segmentation using parametric and non-parametric method.

I think this activity will be useful for future purposes and both the parametric and non-parametric segmentation methods are straightforward to implement. Overall, I would give myself a grade of 10.0 for being able to use parametric and non-parametric segmentation to separate the region of interest from the background.

References:

[1] ' Color Image Segmentation', 2010 Applied Physics 186 manual by Dr. Maricor Soriano