The connected components algorithm
The connected component algorithm is a very common algorithm that's used to segment and identify parts in binary images. The connected component is an iterative algorithm with the purpose of labeling an image using eight or four connectivity pixels. Two pixels are connected if they have the same value and are neighbors. In an image, each pixel has eight neighbor pixels:
Four-connectivity means that only the 2, 4, 5, and 7 neighbors can be connected to the center if they have the same value as the center pixel. With eight-connectivity, the 1, 2, 3, 4, 5, 6, 7, and 8 neighbors can be connected if they have the same value as the center pixel. We can see the differences in the following example from a four- and eight-connectivity algorithm. We are going to apply each algorithm to the next binarized image. We have used a small 9 x 9 image and zoomed in to show how connected components work and the differences between four- and eight-connectivity:
The four-connectivity algorithm detects two objects; we can see this in the left image. The eight-connectivity algorithm detects only one object (the right image) because two diagonal pixels are connected. Eight-connectivity takes care of diagonal connectivity, which is the main difference compared with four-connectivity, since this where only vertical and horizontal pixels are considered. We can see the result in the following image, where each object has a different gray color value:
OpenCV brings us the connected components algorithm with two different functions:
- connectedComponents (image, labels, connectivity= 8, type= CV_32S)
- connectedComponentsWithStats (image, labels, stats, centroids, connectivity= 8, type= CV_32S)
Both functions return an integer with the number of detected labels, where label 0 represents the background. The difference between these two functions is basically the information that is returned. Let's check the parameters of each one. The connectedComponents function gives us the following parameters:
- Image: The input image to be labeled.
- Labels: An output mat that's the same size as the input image, where each pixel has the value of its label, where all OS represents the background, pixels with 1 value represent the first connected component object, and so on.
- Connectivity: Two possible values, 8 or 4, that represent the connectivity we want to use.
- Type: The type of label image we want to use. Only two types are allowed: CV32_S and CV16_U. By default, this is CV32_S.
- The connectedComponentsWithStats function has two more parameters defined. These are stats and centroids:
- Stats: This is an output parameter that gives us the following statistical values for each label (background inclusive):
- CC_STAT_LEFT: The leftmost x coordinate of the connected component object
- CC_STAT_TOP: The topmost y coordinate of the connected component object
- CC_STAT_WIDTH: The width of the connected component object defined by its bounding box
- CC_STAT_HEIGHT: The height of the connected component object defined by its bounding box
- CC_STAT_AREA: The number of pixels (area) of the connected component object
- Centroids: The centroid points to the float type for each label, inclusive of the background that's considered for another connected component.
- Stats: This is an output parameter that gives us the following statistical values for each label (background inclusive):
In our example application, we are going to create two functions so that we can apply these two OpenCV algorithms. We will then show the user the obtained result in a new image with colored objects in the basic connected component algorithm. If we select the connected component with the stats method, we are going to draw the respective calculated area that returns this function over each object.
Let's define the basic drawing for the connected component function:
void ConnectedComponents(Mat img)
{
// Use connected components to divide our image in multiple connected component objects
Mat labels;
auto num_objects= connectedComponents(img, labels);
// Check the number of objects detected
if(num_objects < 2 ){
cout << "No objects detected" << endl;
return;
}else{
cout << "Number of objects detected: " << num_objects - 1 << endl;
}
// Create output image coloring the objects
Mat output= Mat::zeros(img.rows,img.cols, CV_8UC3);
RNG rng(0xFFFFFFFF);
for(auto i=1; i<num_objects; i++){
Mat mask= labels==i;
output.setTo(randomColor(rng), mask);
}
imshow("Result", output);
}
First of all, we call the OpenCV connectedComponents function, which returns the number of objects detected. If the number of objects is less than two, this means that only the background object is detected, and then we don't need to draw anything and we can finish. If the algorithm detects more than one object, we show the number of objects that have been detected on the console:
Mat labels;
auto num_objects= connectedComponents(img, labels);
// Check the number of objects detected
if(num_objects < 2){
cout << "No objects detected" << endl;
return;
}else{
cout << "Number of objects detected: " << num_objects - 1 << endl;
Now, we are going to draw all detected objects in a new image with different colors. After this, we need to create a new black image with the same input size and three channels:
Mat output= Mat::zeros(img.rows,img.cols, CV_8UC3);
We will loop over each label, except for the 0 value, because this is the background:
for(int i=1; i<num_objects; i++){
To extract each object from the label image, we can create a mask for each i label using a comparison and save this in a new image:
Mat mask= labels==i;
Finally, we set a pseudo-random color to the output image using the mask:
output.setTo(randomColor(rng), mask); }
After looping all of the images, we have all of the detected objects with different colors in our output and we only have to show the output image in a window:
imshow("Result", output);
This is the result in which each object is painted with different colors or a gray value:
Now, we are going to explain how to use the connected components with the stats OpenCV algorithm and show some more information in the resultant image. The following function implements this functionality:
void ConnectedComponentsStats(Mat img)
{
// Use connected components with stats
Mat labels, stats, centroids;
auto num_objects= connectedComponentsWithStats(img, labels, stats, centroids);
// Check the number of objects detected
if(num_objects < 2 ){
cout << "No objects detected" << endl;
return;
}else{
cout << "Number of objects detected: " << num_objects - 1 << endl;
}
// Create output image coloring the objects and show area
Mat output= Mat::zeros(img.rows,img.cols, CV_8UC3);
RNG rng( 0xFFFFFFFF );
for(auto i=1; i<num_objects; i++){
cout << "Object "<< i << " with pos: " << centroids.at<Point2d>(i) << " with area " << stats.at<int>(i, CC_STAT_AREA) << endl;
Mat mask= labels==i;
output.setTo(randomColor(rng), mask);
// draw text with area
stringstream ss;
ss << "area: " << stats.at<int>(i, CC_STAT_AREA);
putText(output,
ss.str(),
centroids.at<Point2d>(i),
FONT_HERSHEY_SIMPLEX,
0.4,
Scalar(255,255,255));
}
imshow("Result", output);
}
Let's understand this code. As we did in the non-stats function, we call the connected components algorithm, but here, we do this using the stats function, checking whether we detected more than one object:
Mat labels, stats, centroids;
auto num_objects= connectedComponentsWithStats(img, labels, stats, centroids);
// Check the number of objects detected
if(num_objects < 2){
cout << "No objects detected" << endl;
return;
}else{
cout << "Number of objects detected: " << num_objects - 1 << endl;
}
Now, we have two more output results: the stats and centroid variables. Then, for each detected label, we are going to show the centroid and area through the command line:
for(auto i=1; i<num_objects; i++){
cout << "Object "<< i << " with pos: " << centroids.at<Point2d>(i) << " with area " << stats.at<int>(i, CC_STAT_AREA) << endl;
You can check the call to the stats variable to extract the area using the column constant stats.at<int>(I, CC_STAT_AREA). Now, like before, we paint the object labeled with i over the output image:
Mat mask= labels==i; output.setTo(randomColor(rng), mask);
Finally, in the centroid position of each segmented object, we want to draw some information (such as the area) on the resultant image. To do this, we use the stats and centroid variables using the putText function. First, we have to create a stringstream so that we can add the stats area information:
// draw text with area stringstream ss; ss << "area: " << stats.at<int>(i, CC_STAT_AREA);
Then, we need to use putText, using the centroid as the text position:
putText(output, ss.str(), centroids.at<Point2d>(i), FONT_HERSHEY_SIMPLEX, 0.4, Scalar(255,255,255));
The result for this function is as follows:
