OpenCV: Fun with filters and convolution

My initial encounter with filters, convolution and correlation in OpenCV made me play around with the filters for Gaussian smooth, erosion and dilation operations on random image files. However I found the experience rather unsatisfactory and I wanted to get a real feel for the working of these operations. Suddenly a thought struck me. Could I restore an old family photograph of my parents? The photograph has areas of white patches that had to be removed.

So I started to dig a little more into the filters, convolution and correlation matrices to get a better understanding.

This is the original photograph.

My Mom & late Dad
As can be seen there are large patches in several places in the photograph. So I decided to use cvFloodFill to fill these areas.

a)      cvFloodFill: Since I had to identify the spots where these patches occurred I took a dump of the cvMat of the image which I resized to about 29 x 42. By inspecting the data I could see that the patches typically corresponded to intensity values that were greater than 170. So I decided that the cvFloodFill should happen with the seed around these parts.   So the code checks the intensity values > 170 and calls cvFloodFill. After much tweaking I could see that the white patches were now filled with gray (newval intensity= 150.0) So I was able to get rid of the white patches.

b)      cvSmooth: The next step that I took was to perform a Gaussian smooth of the picture. This smoothed out the filled parts

c)       cvErode & cvDilate: I followed this with cvErode to smooth out the dark areas  and cvDilate to smooth out the bright areas.

d)      cvFilter2D:  I wanted to now sharpen the image. I did a lot of experiments with different kernel values but I found this to be extremely difficult to work with. After much trial and error I came with a kernel values of
double a[9]={-1,20,1,-1,20,1,-1,20,1};
The sharpening was reasonable but there are areas where there are white streaks. I still need to figure out a kernel that can sharpen images. For this also to understand what was happening I tried to dump the values of the image to get a feel of where the values lay.

e)      cvSmooth: Finally I performed a cvSmooth of the filtered output.

While I have had fair success there is still a lot more left to be desired from the final version.
The complete process flow

The code is included below

#include “cv.h”
#include “highgui.h”
#include “stdio.h”
int main(int argc, char** argv)
{
IplImage* img = cvLoadImage(“dad_mom.jpg”,0);
IplImage* dst;
IplImage* dst1;
IplImage* dst2;
IplImage* dst3;
IplImage* dst4;
int i,j,k;
int height,width,step,channels;
uchar* data;
uchar* data1;
uchar* outdata;
CvScalar s;
CvScalar lodiff,highdiff,newval;
// get the image data
height    = img->height;
width     = img->width;
step      = img->widthStep;
channels  = img->nChannels;
data      = (uchar *)img->imageData;
double a[9]={-1,20,1,-1,20,1,-1,20,1};
//double a[9]={1/16,1/8,1/16,1/8,1/4,1/8,1/16,1/8,1/16};
double values[9]={1/16,0,-1/16,2/16,0,-2/16,1/16,0,-1/16};
CvPoint seed;
CvMat kernel= cvMat(3,3,CV_32FC1,a);
printf(“Processing a %d x %d image with %d channels\n”,height,width,channels);
// Create windows
cvNamedWindow(“Original”,CV_WINDOW_AUTOSIZE);
cvNamedWindow(“Flood Fill”,CV_WINDOW_AUTOSIZE);
cvNamedWindow(“Smooth”,CV_WINDOW_AUTOSIZE);
cvNamedWindow(“Erode”,CV_WINDOW_AUTOSIZE);
cvNamedWindow(“Dilate”,CV_WINDOW_AUTOSIZE);
cvNamedWindow(“Filter”,CV_WINDOW_AUTOSIZE);
cvNamedWindow(“Smooth1”,CV_WINDOW_AUTOSIZE);
// Original image
cvShowImage(“Original”, img);
/* Flood fill in white patches intensity > 170.0 */
highdiff=cvRealScalar(5.0);
lodiff=cvRealScalar(5.0);
newval=cvRealScalar(150.0);
for(i=0;i<height;i++) {
for(j=0;j<width;j++) {
for(k=0;k<channels;k++) {
//printf(“Data = %d \n”,data[i*step+j*channels+k]);
if((data[i*step+j*channels+k]) > 170.0)
{
seed=cvPoint(j,i);
//printf(“data=%dFlood
seed=%d,%d\n”,data[i*step+j*channels+k],i,j);
cvFloodFill(img,seed,newval,lodiff,highdiff, NULL,CV_FLOODFILL_FIXED_RANGE,NULL);                                                                              }
else
{
;
}
}
}
//printf(“\n”);
}
cvShowImage(“Flood Fill”,img);
// Gaussian smooth
dst = cvCloneImage(img);
cvSmooth( img, dst, CV_GAUSSIAN, 3, 3, 0, 0 );
cvShowImage(“Smooth”,dst);
// Erode the image
dst1 = cvCloneImage(img);
IplConvKernel* kern = cvCreateStructuringElementEx(3,3,1,1,CV_SHAPE_RECT,values);
cvErode(dst,dst1,kern,1);
cvShowImage(“Erode”,dst1);
// Perform dilation operation
dst2 = cvCloneImage(img);
cvDilate(dst1,dst2,kern,1);
cvShowImage(“Dilate”,dst2);
// Filter the image with convolution kernel. Sharpen the image
dst3 = cvCloneImage(img);
printf(“reached here\n”);
cvFilter2D(dst2,dst3,&kernel,cvPoint(-1,-1));
cvShowImage(“Filter”,dst3);
// Smoothen the image
dst4 = cvCloneImage(img);
cvSmooth( dst3, dst4, CV_MEDIAN, 3, 0, 0, 0 );
cvShowImage(“Smooth1”,dst4);
// Cleanup
cvWaitKey(0);
cvReleaseImage(&img);
cvReleaseImage(&dst);
cvDestroyWindow(“Original”);
vDestroyWindow(“Restore”);

}

Checkout my book ‘Deep Learning from first principles Second Edition- In vectorized Python, R and Octave’.  My book is available on Amazon  as paperback ($18.99) and in kindle version($9.99/Rs449).

You may also like my companion book “Practical Machine Learning with R and Python:Second Edition- Machine Learning in stereo” available in Amazon in paperback($12.99) and Kindle($9.99/Rs449) versions.