Images can be visualized to be functions of the form f(x,y) where f(x,y) represents the intensity at the pixel position ,y. However images can be grayscale, color or four channels and each channel may consist of integers or floating point numbers. However the changes in the values can be viewed as a continuous function. Here is a nice representation of an image as a continuous function (courtesy: Prof Darrell’s lecture at Berkeley on Filters)

Given that the image can be viewed as a continuous function in 2 or 3 axes we have derivatives that can be taken of the images. The derivates determine the maximum and minimum of this changing function. The key derivatives in image processing are the Sobel, the Scharr and the Laplacian filters. These provide the 1^{st} order or 2^{nd} order derivative and hence can be used for determining edges of an image.

I was keen on playing around with derivatives and also understanding how the histograms look like.

Here is the original image and its histogram. Clearly there is a nice spread of the values.

**Sobel filter and its histogram**

The output of the Sobel filter on the original image is shown. The edges with Sobel’s derivative somehow are not too pronounced. The Sobel derivate can be used for obtaining the gradient of the image. The corresponding histogram of the Sobel’s gradient is shown.

The code snippet ( The complete code is given below)

…

…

IplImage* out_sobel = cvCreateImage( cvSize(img->width, img->height), IPL_DEPTH_16S, 1); cvSobel(in_gray, out_sobel, 1,1,7); cvShowImage("Sobel", out_sobel); //create an image to hold the histogram IplImage* histImage_Sobel = cvCreateImage(cvSize(300,400), 8, 1); //create a histogram to store the information from the image CvHistogram* histSobel = cvCreateHist(1, &hist_size, CV_HIST_ARRAY, ranges, 1); //calculate the histogram and apply to hist cvCalcHist( &histImage_Sobel, histSobel, 0, NULL ); //grab the min and max values and their indeces cvGetMinMaxHistValue( histSobel, &min_value, &max_value, &min_idx, &max_idx); //scale the bin values so that they will fit in the image representation cvScale( histSobel->bins, histSobel->bins, ((double)histImage_Sobel->height)/max_value, 0 ); //set all histogram values to 255 cvSet( histImage_Sobel, cvScalarAll(255), 0 ); //create a factor for scaling along the width bin_w = cvRound((double)histImage_Sobel->width/hist_size); for( i = 0; i < hist_size; i++ ) { //draw the histogram data onto the histogram image cvRectangle( histImage_Sobel, cvPoint(i*bin_w, histImage_Sobel->height), cvPoint((i+1)*bin_w, histImage_Sobel->height - cvRound(cvGetReal1D(histSobel->bins,i))), cvScalarAll(0), -1, 8, 0 ); //get the value at the current histogram bucket float* bins = cvGetHistValue_1D(histSobel,i); //increment the mean value mean += bins[0]; } cvShowImage("Hist Sobel",histImage_Sobel); ... ... (Please see Gavin S. Page's tutorial(vast.uccs.edu/~tboult/CS330/NOTES/OpenCVTutorial_II.ppt) on histograms)

**Laplacian and its histogram**

The Laplacian provides the 2^{nd} order derivative and hence can be used to determine local maxima and local minima. The Laplacian provides for much more pronounced edges and can be used to extract features of an object of interest. Its corresponding histogram is also included.

**Canny filter and Contours**

The third filter is cvCanny which is most suitable for obtaining clear edges in an image. The canny is usually used along with cvFindContours to determine the general shape of an object. I used the canny filter which I passed to a contour detecting function. However the contour detecting function identified more than 228 contours most of which were useless except for 1 which had included the complete contour of the hand as shown.

However when I increased the max_depth to 1 I found that it was immediately able to get the complete contour of the hand besides a lot of extraneous contours.

I guess the challenge with the contour function is being able to programmatically reject all those contours which of lesser importance (possibly a future post).

**Code for Sobel, Laplacian and Histograms**

#include "cv.h" #include "highgui.h" #include "stdio.h" int main(int argc, char** argv) { IplImage* img = cvLoadImage("gazelle.jpg",1); IplImage* dst; IplImage* in_gray; int hist_size=30; float gray_ranges[] = { 0, 255 }; float* ranges[] = { gray_ranges}; int min_idx,max_idx; float min_value,max_value; int bin_w; int i; float mean,variance; cvNamedWindow("Original",CV_WINDOW_AUTOSIZE); cvNamedWindow("histogram",CV_WINDOW_AUTOSIZE); cvNamedWindow("Sobel",CV_WINDOW_AUTOSIZE); cvNamedWindow("Hist Sobel",CV_WINDOW_AUTOSIZE); cvNamedWindow("Laplacian",CV_WINDOW_AUTOSIZE); cvNamedWindow("Hist Laplace",CV_WINDOW_AUTOSIZE); in_gray = cvCreateImage(cvSize(img->width, img->height), IPL_DEPTH_8U, 1); cvCvtColor(img, in_gray, CV_BGR2GRAY); cvShowImage("Original", in_gray); //create a rectangular area to evaluate CvRect rect = cvRect(0, 0, 300, 400 ); //apply the rectangle to the image and establish a region of interest cvSetImageROI(in_gray, rect); //create an image to hold the histogram IplImage* histImage = cvCreateImage(cvSize(300,400), 8, 1); //create a histogram to store the information from the image CvHistogram* hist = cvCreateHist(1, &hist_size, CV_HIST_ARRAY, ranges, 1); //calculate the histogram and apply to hist cvCalcHist( &in_gray, hist, 0, NULL ); //grab the min and max values and their indeces cvGetMinMaxHistValue( hist, &min_value, &max_value, &min_idx, &max_idx); //scale the bin values so that they will fit in the image representation cvScale( hist->bins, hist->bins, ((double)histImage->height)/max_value, 0 ); //set all histogram values to 255 cvSet( histImage, cvScalarAll(255), 0 ); //create a factor for scaling along the width bin_w = cvRound((double)histImage->width/hist_size); for( i = 0; i < hist_size; i++ ) { //draw the histogram data onto the histogram image cvRectangle( histImage, cvPoint(i*bin_w, histImage->height), cvPoint((i+1)*bin_w, histImage->height - cvRound(cvGetReal1D(hist->bins,i))), cvScalarAll(0), -1, 8, 0 ); //get the value at the current histogram bucket float* bins = cvGetHistValue_1D(hist,i); //increment the mean value mean += bins[0]; } //finish mean calculation mean /= hist_size; //go back through now that mean has been calculated in order to calculate variance for( i = 0; i < hist_size; i++ ) { float* bins = cvGetHistValue_1D(hist,i); variance += pow((bins[0] - mean),2); } //finish variance calculation variance /= hist_size; cvShowImage("histogram",histImage); IplImage* out_sobel = cvCreateImage( cvSize(img->width, img->height), IPL_DEPTH_16S, 1); cvSobel(in_gray, out_sobel, 1,1,7); cvShowImage("Sobel", out_sobel); //create an image to hold the histogram IplImage* histImage_Sobel = cvCreateImage(cvSize(300,400), 8, 1); //create a histogram to store the information from the image CvHistogram* histSobel = cvCreateHist(1, &hist_size, CV_HIST_ARRAY, ranges, 1); //calculate the histogram and apply to hist cvCalcHist( &histImage_Sobel, histSobel, 0, NULL ); //grab the min and max values and their indeces cvGetMinMaxHistValue( histSobel, &min_value, &max_value, &min_idx, &max_idx); //scale the bin values so that they will fit in the image representation cvScale( histSobel->bins, histSobel->bins, ((double)histImage_Sobel->height)/max_value, 0 ); //set all histogram values to 255 cvSet( histImage_Sobel, cvScalarAll(255), 0 ); //create a factor for scaling along the width bin_w = cvRound((double)histImage_Sobel->width/hist_size); for( i = 0; i < hist_size; i++ ) { //draw the histogram data onto the histogram image cvRectangle( histImage_Sobel, cvPoint(i*bin_w, histImage_Sobel->height), cvPoint((i+1)*bin_w, histImage_Sobel->height - cvRound(cvGetReal1D(histSobel->bins,i))), cvScalarAll(0), -1, 8, 0 ); //get the value at the current histogram bucket float* bins = cvGetHistValue_1D(histSobel,i); //increment the mean value mean += bins[0]; } cvShowImage("Hist Sobel",histImage_Sobel); // Create Laplacian and the histogram for it IplImage *output=cvCreateImage( cvSize(img->width, img->height), IPL_DEPTH_16S, 1); cvLaplace(in_gray, output, 7); cvShowImage("Laplacian", output); //create an image to hold the histogram IplImage* histImage_Laplace = cvCreateImage(cvSize(300,400), 8, 1); //create a histogram to store the information from the image CvHistogram* histLaplace = cvCreateHist(1, &hist_size, CV_HIST_ARRAY, ranges, 1); //calculate the histogram and apply to hist cvCalcHist( &histImage_Laplace, histLaplace, 0, NULL ); //grab the min and max values and their indeces cvGetMinMaxHistValue( histLaplace, &min_value, &max_value, &min_idx, &max_idx); //scale the bin values so that they will fit in the image representation cvScale( histLaplace->bins, histLaplace->bins, ((double)histImage_Laplace->height)/max_value, 0 ); //set all histogram values to 255 cvSet( histImage_Laplace, cvScalarAll(255), 0 ); //create a factor for scaling along the width bin_w = cvRound((double)histImage_Laplace->width/hist_size); for( i = 0; i < hist_size; i++ ) { //draw the histogram data onto the histogram image cvRectangle( histImage_Laplace, cvPoint(i*bin_w, histImage_Laplace->height), cvPoint((i+1)*bin_w, histImage_Laplace->height - cvRound(cvGetReal1D(histLaplace->bins,i))), cvScalarAll(0), -1, 8, 0 ); //get the value at the current histogram bucket float* bins = cvGetHistValue_1D(histLaplace,i); //increment the mean value mean += bins[0]; } cvShowImage("Hist Laplace",histImage_Laplace); cvWaitKey(0); printf("Mean= %f\n",mean); printf("variance=%f\n",variance); //clean up images cvReleaseImage(&histImage_Laplace); cvReleaseImage(&histImage_Sobel); cvReleaseImage(&histImage); cvReleaseImage(&in_gray); cvReleaseImage(&img); //remove windows cvDestroyWindow("Original"); cvDestroyWindow("histogram"); }Code for Canny & Contours#include "cv.h" #include "highgui.h" #define CVX_RED CV_RGB(0xff,0x00,0x00) #define CVX_GREEN CV_RGB(0x00,0xff,0x00) #define CVX_BLUE CV_RGB(0x00,0x00,0xff) int main(int argc, char* argv[]) { CvSeq* c; int i; cvNamedWindow("Original", 1 ); cvNamedWindow("Canny_Edge", 1 ); cvNamedWindow("Contours", 1 ); IplImage* img_8uc1 = cvLoadImage( argv[1], CV_LOAD_IMAGE_GRAYSCALE ); IplImage* img_edge = cvCreateImage( cvGetSize(img_8uc1), 8, 1 ); IplImage* img_8uc3 = cvCreateImage( cvGetSize(img_8uc1), 8, 3 ); cvThreshold( img_8uc1, img_edge, 128, 255, CV_THRESH_BINARY ); CvMemStorage* storage =cvCreateMemStorage(0); CvSeq* first_contour = NULL; int Nc; int n=0; cvShowImage("Original", img_8uc1); IplImage *out_canny=cvCreateImage( cvSize(img_8uc1->width, img_8uc1->height), IPL_DEPTH_8U, 1); cvCanny(img_8uc1, out_canny, 50.0 ,100.0, 3); cvShowImage("Canny_Edge", out_canny); /* Nc = cvFindContours( img_edge, storage, &first_contour, sizeof(CvContour), CV_RETR_LIST, CV_CHAIN_APPROX_SIMPLE, cvPoint(0,0)// Try all four values and see what happens );*/ Nc = cvFindContours( out_canny, storage, &first_contour, sizeof(CvContour), CV_RETR_TREE, CV_CHAIN_APPROX_SIMPLE, cvPoint(0,0)// Try all four values and see what happens ); printf("Total contours detected: %d\n",Nc); for(c=first_contour; c!=NULL; c=c->h_next ) { cvCvtColor( img_8uc1, img_8uc3, CV_GRAY2BGR ); cvDrawContours( img_8uc3, c, CVX_RED, CVX_BLUE, 1, // Try different values of max_level, and see what happens 2, 8, cvPoint(0,0)); printf("Contour #%d\n",n); cvShowImage("Contours", img_8uc3 ); printf(" %d elements: \n",c->total); for(i=0; i<c->total; ++i ) { CvPoint* p = CV_GET_SEQ_ELEM( CvPoint, c, i ); printf("(%d,%d)\n",p->x,p->y); } cvWaitKey(0); n++; } printf("Finished all contours\n"); cvCvtColor( img_8uc1, img_8uc3, CV_GRAY2BGR ); cvShowImage( argv[0], img_8uc3 ); cvWaitKey(0); cvDestroyWindow( argv[0] ); cvReleaseImage( &img_8uc1 ); cvReleaseImage( &img_8uc3 ); cvReleaseImage( &img_edge ); return 0; }

Hello,

I have been working with opencv for a little over a year (this blog has often been very helpful) and have run, far too often, into the problem you mention of being able to programatically reject (with a good enough accuracy, say >95%) the useless countours or lines (when using houghlines) I have a couple of implementations to help me achieve such a goal but most hover at the 85% accuracy mark.

If you are serious about coming up with a reliable mehod or algorithm and would like some help do let me know, you can contact me at: jaime_apATyahooDOTcom.

Pingback: My 3 video presentations on “Essential R” | Giga thoughts …

Pingback: Deep Learning from first principles in Python, R and Octave – Part 6 | Giga thoughts …

Pingback: Deep Learning from first principles in Python, R and Octave – Part 8 | Giga thoughts …