Images can be visualized to be functions of the form f(x,y) where f(x,y) represents the intensity at the pixel position ,y. However images can be grayscale, color or four channels and each channel may consist of integers or floating point numbers. However the changes in the values can be viewed as a continuous function. Here is a nice representation of an image as a continuous function (courtesy: Prof Darrell’s lecture at Berkeley on Filters)
Given that the image can be viewed as a continuous function in 2 or 3 axes we have derivatives that can be taken of the images. The derivates determine the maximum and minimum of this changing function. The key derivatives in image processing are the Sobel, the Scharr and the Laplacian filters. These provide the 1st order or 2nd order derivative and hence can be used for determining edges of an image.
I was keen on playing around with derivatives and also understanding how the histograms look like.
Here is the original image and its histogram. Clearly there is a nice spread of the values.
Sobel filter and its histogram
The output of the Sobel filter on the original image is shown. The edges with Sobel’s derivative somehow are not too pronounced. The Sobel derivate can be used for obtaining the gradient of the image. The corresponding histogram of the Sobel’s gradient is shown.
The code snippet ( The complete code is given below)
…
…
IplImage* out_sobel = cvCreateImage( cvSize(img->width, img->height), IPL_DEPTH_16S, 1);
cvSobel(in_gray, out_sobel, 1,1,7);
cvShowImage("Sobel", out_sobel);
//create an image to hold the histogram
IplImage* histImage_Sobel = cvCreateImage(cvSize(300,400), 8, 1);
//create a histogram to store the information from the image
CvHistogram* histSobel = cvCreateHist(1, &hist_size, CV_HIST_ARRAY, ranges, 1);
//calculate the histogram and apply to hist
cvCalcHist( &histImage_Sobel, histSobel, 0, NULL );
//grab the min and max values and their indeces
cvGetMinMaxHistValue( histSobel, &min_value, &max_value, &min_idx, &max_idx);
//scale the bin values so that they will fit in the image representation
cvScale( histSobel->bins, histSobel->bins, ((double)histImage_Sobel->height)/max_value, 0 );
//set all histogram values to 255
cvSet( histImage_Sobel, cvScalarAll(255), 0 );
//create a factor for scaling along the width
bin_w = cvRound((double)histImage_Sobel->width/hist_size);
for( i = 0; i < hist_size; i++ ) {
//draw the histogram data onto the histogram image
cvRectangle( histImage_Sobel, cvPoint(i*bin_w, histImage_Sobel->height),
cvPoint((i+1)*bin_w,
histImage_Sobel->height - cvRound(cvGetReal1D(histSobel->bins,i))),
cvScalarAll(0), -1, 8, 0 );
//get the value at the current histogram bucket
float* bins = cvGetHistValue_1D(histSobel,i);
//increment the mean value
mean += bins[0];
}
cvShowImage("Hist Sobel",histImage_Sobel);
...
...
(Please see Gavin S. Page's tutorial(vast.uccs.edu/~tboult/CS330/NOTES/OpenCVTutorial_II.ppt) on histograms)
Laplacian and its histogram
The Laplacian provides the 2nd order derivative and hence can be used to determine local maxima and local minima. The Laplacian provides for much more pronounced edges and can be used to extract features of an object of interest. Its corresponding histogram is also included.
Canny filter and Contours
The third filter is cvCanny which is most suitable for obtaining clear edges in an image. The canny is usually used along with cvFindContours to determine the general shape of an object. I used the canny filter which I passed to a contour detecting function. However the contour detecting function identified more than 228 contours most of which were useless except for 1 which had included the complete contour of the hand as shown.
However when I increased the max_depth to 1 I found that it was immediately able to get the complete contour of the hand besides a lot of extraneous contours.
I guess the challenge with the contour function is being able to programmatically reject all those contours which of lesser importance (possibly a future post).
Code for Sobel, Laplacian and Histograms
#include "cv.h"
#include "highgui.h"
#include "stdio.h"
int main(int argc, char** argv)
{
IplImage* img = cvLoadImage("gazelle.jpg",1);
IplImage* dst;
IplImage* in_gray;
int hist_size=30;
float gray_ranges[] = { 0, 255 };
float* ranges[] = { gray_ranges};
int min_idx,max_idx;
float min_value,max_value;
int bin_w;
int i;
float mean,variance;
cvNamedWindow("Original",CV_WINDOW_AUTOSIZE);
cvNamedWindow("histogram",CV_WINDOW_AUTOSIZE);
cvNamedWindow("Sobel",CV_WINDOW_AUTOSIZE);
cvNamedWindow("Hist Sobel",CV_WINDOW_AUTOSIZE);
cvNamedWindow("Laplacian",CV_WINDOW_AUTOSIZE);
cvNamedWindow("Hist Laplace",CV_WINDOW_AUTOSIZE);
in_gray = cvCreateImage(cvSize(img->width, img->height), IPL_DEPTH_8U, 1);
cvCvtColor(img, in_gray, CV_BGR2GRAY);
cvShowImage("Original", in_gray);
//create a rectangular area to evaluate
CvRect rect = cvRect(0, 0, 300, 400 );
//apply the rectangle to the image and establish a region of interest
cvSetImageROI(in_gray, rect);
//create an image to hold the histogram
IplImage* histImage = cvCreateImage(cvSize(300,400), 8, 1);
//create a histogram to store the information from the image
CvHistogram* hist = cvCreateHist(1, &hist_size, CV_HIST_ARRAY, ranges, 1);
//calculate the histogram and apply to hist
cvCalcHist( &in_gray, hist, 0, NULL );
//grab the min and max values and their indeces
cvGetMinMaxHistValue( hist, &min_value, &max_value, &min_idx, &max_idx);
//scale the bin values so that they will fit in the image representation
cvScale( hist->bins, hist->bins, ((double)histImage->height)/max_value, 0 );
//set all histogram values to 255
cvSet( histImage, cvScalarAll(255), 0 );
//create a factor for scaling along the width
bin_w = cvRound((double)histImage->width/hist_size);
for( i = 0; i < hist_size; i++ ) {
//draw the histogram data onto the histogram image
cvRectangle( histImage, cvPoint(i*bin_w, histImage->height),
cvPoint((i+1)*bin_w,
histImage->height - cvRound(cvGetReal1D(hist->bins,i))),
cvScalarAll(0), -1, 8, 0 );
//get the value at the current histogram bucket
float* bins = cvGetHistValue_1D(hist,i);
//increment the mean value
mean += bins[0];
}
//finish mean calculation
mean /= hist_size;
//go back through now that mean has been calculated in order to calculate variance
for( i = 0; i < hist_size; i++ ) {
float* bins = cvGetHistValue_1D(hist,i);
variance += pow((bins[0] - mean),2);
}
//finish variance calculation
variance /= hist_size;
cvShowImage("histogram",histImage);
IplImage* out_sobel = cvCreateImage( cvSize(img->width, img->height), IPL_DEPTH_16S, 1);
cvSobel(in_gray, out_sobel, 1,1,7);
cvShowImage("Sobel", out_sobel);
//create an image to hold the histogram
IplImage* histImage_Sobel = cvCreateImage(cvSize(300,400), 8, 1);
//create a histogram to store the information from the image
CvHistogram* histSobel = cvCreateHist(1, &hist_size, CV_HIST_ARRAY, ranges, 1);
//calculate the histogram and apply to hist
cvCalcHist( &histImage_Sobel, histSobel, 0, NULL );
//grab the min and max values and their indeces
cvGetMinMaxHistValue( histSobel, &min_value, &max_value, &min_idx, &max_idx);
//scale the bin values so that they will fit in the image representation
cvScale( histSobel->bins, histSobel->bins, ((double)histImage_Sobel->height)/max_value, 0 );
//set all histogram values to 255
cvSet( histImage_Sobel, cvScalarAll(255), 0 );
//create a factor for scaling along the width
bin_w = cvRound((double)histImage_Sobel->width/hist_size);
for( i = 0; i < hist_size; i++ ) {
//draw the histogram data onto the histogram image
cvRectangle( histImage_Sobel, cvPoint(i*bin_w, histImage_Sobel->height),
cvPoint((i+1)*bin_w,
histImage_Sobel->height - cvRound(cvGetReal1D(histSobel->bins,i))),
cvScalarAll(0), -1, 8, 0 );
//get the value at the current histogram bucket
float* bins = cvGetHistValue_1D(histSobel,i);
//increment the mean value
mean += bins[0];
}
cvShowImage("Hist Sobel",histImage_Sobel);
// Create Laplacian and the histogram for it
IplImage *output=cvCreateImage( cvSize(img->width, img->height), IPL_DEPTH_16S, 1);
cvLaplace(in_gray, output, 7);
cvShowImage("Laplacian", output);
//create an image to hold the histogram
IplImage* histImage_Laplace = cvCreateImage(cvSize(300,400), 8, 1);
//create a histogram to store the information from the image
CvHistogram* histLaplace = cvCreateHist(1, &hist_size, CV_HIST_ARRAY, ranges, 1);
//calculate the histogram and apply to hist
cvCalcHist( &histImage_Laplace, histLaplace, 0, NULL );
//grab the min and max values and their indeces
cvGetMinMaxHistValue( histLaplace, &min_value, &max_value, &min_idx, &max_idx);
//scale the bin values so that they will fit in the image representation
cvScale( histLaplace->bins, histLaplace->bins, ((double)histImage_Laplace->height)/max_value, 0 );
//set all histogram values to 255
cvSet( histImage_Laplace, cvScalarAll(255), 0 );
//create a factor for scaling along the width
bin_w = cvRound((double)histImage_Laplace->width/hist_size);
for( i = 0; i < hist_size; i++ ) {
//draw the histogram data onto the histogram image
cvRectangle( histImage_Laplace, cvPoint(i*bin_w, histImage_Laplace->height),
cvPoint((i+1)*bin_w,
histImage_Laplace->height - cvRound(cvGetReal1D(histLaplace->bins,i))),
cvScalarAll(0), -1, 8, 0 );
//get the value at the current histogram bucket
float* bins = cvGetHistValue_1D(histLaplace,i);
//increment the mean value
mean += bins[0];
}
cvShowImage("Hist Laplace",histImage_Laplace);
cvWaitKey(0);
printf("Mean= %f\n",mean);
printf("variance=%f\n",variance);
//clean up images
cvReleaseImage(&histImage_Laplace);
cvReleaseImage(&histImage_Sobel);
cvReleaseImage(&histImage);
cvReleaseImage(&in_gray);
cvReleaseImage(&img);
//remove windows
cvDestroyWindow("Original");
cvDestroyWindow("histogram");
}
Code for Canny & Contours
#include "cv.h"
#include "highgui.h"
#define CVX_RED CV_RGB(0xff,0x00,0x00)
#define CVX_GREEN CV_RGB(0x00,0xff,0x00)
#define CVX_BLUE CV_RGB(0x00,0x00,0xff)
int main(int argc, char* argv[])
{
CvSeq* c;
int i;
cvNamedWindow("Original", 1 );
cvNamedWindow("Canny_Edge", 1 );
cvNamedWindow("Contours", 1 );
IplImage* img_8uc1 = cvLoadImage( argv[1], CV_LOAD_IMAGE_GRAYSCALE );
IplImage* img_edge = cvCreateImage( cvGetSize(img_8uc1), 8, 1 );
IplImage* img_8uc3 = cvCreateImage( cvGetSize(img_8uc1), 8, 3 );
cvThreshold( img_8uc1, img_edge, 128, 255, CV_THRESH_BINARY );
CvMemStorage* storage =cvCreateMemStorage(0);
CvSeq* first_contour = NULL;
int Nc;
int n=0;
cvShowImage("Original", img_8uc1);
IplImage *out_canny=cvCreateImage( cvSize(img_8uc1->width, img_8uc1->height), IPL_DEPTH_8U, 1);
cvCanny(img_8uc1, out_canny, 50.0 ,100.0, 3);
cvShowImage("Canny_Edge", out_canny);
/* Nc = cvFindContours(
img_edge,
storage,
&first_contour,
sizeof(CvContour),
CV_RETR_LIST,
CV_CHAIN_APPROX_SIMPLE,
cvPoint(0,0)// Try all four values and see what happens
);*/
Nc = cvFindContours(
out_canny,
storage,
&first_contour,
sizeof(CvContour),
CV_RETR_TREE,
CV_CHAIN_APPROX_SIMPLE,
cvPoint(0,0)// Try all four values and see what happens
);
printf("Total contours detected: %d\n",Nc);
for(c=first_contour; c!=NULL; c=c->h_next )
{
cvCvtColor( img_8uc1, img_8uc3, CV_GRAY2BGR );
cvDrawContours(
img_8uc3,
c,
CVX_RED,
CVX_BLUE,
1, // Try different values of max_level, and see what happens
2,
8,
cvPoint(0,0));
printf("Contour #%d\n",n);
cvShowImage("Contours", img_8uc3 );
printf(" %d elements: \n",c->total);
for(i=0; i<c->total; ++i ) {
CvPoint* p = CV_GET_SEQ_ELEM( CvPoint, c, i );
printf("(%d,%d)\n",p->x,p->y);
}
cvWaitKey(0);
n++;
}
printf("Finished all contours\n");
cvCvtColor( img_8uc1, img_8uc3, CV_GRAY2BGR );
cvShowImage( argv[0], img_8uc3 );
cvWaitKey(0);
cvDestroyWindow( argv[0] );
cvReleaseImage( &img_8uc1 );
cvReleaseImage( &img_8uc3 );
cvReleaseImage( &img_edge );
return 0;
}
Giga thoughts ... 