OpenCV part-3 Blog

The beauty of OpenCV Vol-3.

In this final article, i would like to discuss some advance operation which we can perform using OpenCV. Here in this article, we are going to discuss Grabcut foreground extraction, Feature matching, Haar cascade object, MOG background. In my last two articles, i have discussed some basic operations which can be done by using OpenCV like reading an image, loading video source, Drawing and writing on images, thresholding, Canny Edge Detection etc, these were just basic operations which are basically in order to understand how things would look like when you are doing computer vision projects likewise, how does your own model would print the type of object at on a single frame, how do the bounding boxes print on a frame, how do we show the frames while running our model, how do we test our results on a video frame and many more. If you don't get this, i will write a blog in the future on how the computer vision projects work and what are the backend parameters and many more. So without wasting any time, let's get started.

Here are the links of the last two articles:

Vol-1 ->https://tdbtech.blogspot.com/2019/02/opencv-part-1-blog.html

Vol-2 ->https://tdbtech.blogspot.com/2019/02/opencv-part-2-blog.html

***An important thing:-I am writing every single code on the jupyter notebook, if you would like to use any other python IDE then feel free to use it, else i would always prefer to use the jupyter notebook. I will write a blog on how to install and use the jupyter notebook.

GrabCut Foreground Extraction:

The idea of GrabCut Foreground Extraction is to Extract the background and find the Foreground. So what happens to the background?. The user draws a rectangle and everything outside that rectangle will surely be cut out. The computer does initial labeling given on the data we provide and it labels the foreground and background pixels. Now a Gaussian mixture model(GMM) is used to model the foreground and background and depends upon the data we gave, GMM learns and creates new pixel distribution. A graph is built from these pixel distributions. Nodes are nothing but pixels, Additionally, we will add two nodes, Source node and Sink node. Every foreground node is connected to the source node and the background is connected to the sink node. The weights of edges connecting those pixels are depending upon the probability of the pixel being foreground/background. Then a minicut algorithm is used to segment the graph. It cuts the graph into two separating nodes, i.e the source node and the sink node, then finally foreground would be the pixels on the source node and background on the sink node.

Here's the code:

import numpy as np

import cv2

from matplotlib import pyplot as plt

img=cv2.imread('#your image path')

mask=np.zeros(img.shape[:2],np.uint8)

bgdModel=np.zeros((1,65),np.float64)

fgdModel=np.zeros((1,65),np.float64)

rect=(161,70,150,150)#(start_x,start_y,width,height) adn you can take your own

cv2.grabCut(img,mask,bgdModel,fgdModel,rect,5,cv2.GC_INIT_WITH_RECT)

mask2=np.where((mask==2|mask==0),0,1).astype('uint8')#here mask is changed so that all 0to 2 pixels changed to background where 1 & 3 on the foreground.

img=img*mask2[:,:,np.newaxis]

plt.imshow(img)
plt.colorbar()

plt.show()

On the code above written, cv2.grabCut takes a few parameters, an input image, mask, a rectangle of a new object, background model, foreground model, amount of iterations to run, the model you are using.

Here's the result:

***Gaussian mixture model-> a mixture model is a probabilistic model for representing the presence of subpopulations within an overall population, without requiring that an observed data set should identify the sub-population to which an individual observation belongs.

Corner Detection using OpenCV:

The idea behind using corner detection is for motion detection, image registration, video tracking, 3D modeling and recognizing object shapes, size, and characters. Corner detection is an approach used within computer vision systems in order to extract certain features and infer contents of an image.

Here our goal is to find the corners in a given image. And a lot of corners can be found with the help of OpenCV, by using OpenCV, the hardest part is almost done as usual.

Here's the code:

import numpy as np

import cv2

img=cv2.imread('#Your image path')

gray=cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)

gray=np.float32(gray)

corners=cv2.goodFeaturesToTrack(gray,100,0.01,10)

corners=np.int0(corners)

for corner in corners:

       x,y=corner.ravel()

       cv2.circle(img,(x,y),3,255,-1)

cv2.imshow('corner',img)

First, we have imported some important libraries and read the image as usual. Then converted our image into grayscale, then to float32. Then we have used the function goodFeaturesToTrack which contain few parameters, img, maximum corners to detect, quality and maximum distance between the corners. Then we iterate through each corner and making the circle at each point.

Here's the result:

Feature matching(Homography):

The idea of feature matching is to find identical regions of an image that matches a template we provide, by giving a certain threshold. Here a perfect or closely perfect match is required. We start with an image or say template that we are hoping to find in our main image. The template image is quite smaller as compared to the main image. The main advantage here is that both images need not be of the same size, angle, rotation. As we just need to match the same features.

Now we are going to use the form of "Brute Force" matching. Here first we will find all the features in both the images and will match them all, You can find as many matches as you want but make sure you don't get the False positives. There is a number of chances of getting False positives when you are using a large number of matches. So here we go with the code:

import numpy as np

import cv2

import matplotlib.pyplot as plt
img1=cv2.imread('#your main image')

img2=cv2.imread('#your template image')

orb=cv2.ORB_create()

kp1,des1=orb.detectAndCompute(img1,None)

kp2,des2=orb.detectAndCompute(img2,None)

bf=cv2.BFMatcher(cv2.NORM_HAMMING,crossCheck=True)

matches=bf.match(des1,des2)

matches=sorted(matches,key=lambda x:x.distance)

img3=cv2.drawMatches(img1,kp1,img2,kp2,matches[:10],None,flags=2)

plt.imshow(img3)

plt.show()

First, we have imported all the important libraries, as usual, then we have read two images, i.e main and the template one. orb is the detector which we are going to use for features. Then using BFMatcher object we will find key points and their features. Then we have sorted them based on their distances. In drawMatches function, the 5th parameter takes how many matches exactly do we want. Here i have taken 10, you can take whatever you want...

Here's the result:

MOG background reduction using OpenCV:

Here i am going to discuss how we can reduce the background of images, by detecting motions. This is going to require us to revisit the use of video. one is the absence of people and others with their presence. The idea here is to extract the moving foreground from the static background. You can also use this to compare two similar images, and immediately extract the differences between them.

The code is quite simple as compared to some previous ones. Here's the code:

import numpy as np

import cv2

cap=cv2.VideoCapture('#path to your video file')

fgbg=cv2.createBackgroundSubtractorMOG2()

while(1):

       ret,frame=cap.read()

       fgmask=fgbg.apply(frame)

       cv2.imshow('original',frame)

       cv2.imshow('Mog',fgmask)

       k=cv2.waitKey(30) & 0xFF

       if k==27:

              break

cv2.destroyAllWindows()

cap.release()

Here's the result:

Haar cascade object detection:

Now we are going to talk about object detection using Haar cascade. We will do face and eye recognition using cascade files. Well for that, you will need cascade files. As thanks to Github, we are having various cascade files to detect faces, eyes, cars, number plates etc.
We will use Face_cascade and eye_cascade files.
Now you just have to download these files and we are good to go.
Here's the code:
import numpy as np
import cv2
img=cv2.imread('#your image path')
gray=cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
face_cascade=cv2.CascadeClassifier('#your face cascade path')
eye_cascade=cv2.CascadeClassifier('#your eye cascade path')
faces=face_cacade.detectMultiScale(gray,1.3,5)
for (x,y,w,h) in faces:
          cv2.rectangle(img,(x,y),(x+h,y+w),(255,0,0),2)
          roi_gray=gray[y:y+h,x:x+w]
          roi_color=img[y:y+h,x:x+w]
          eyes=eye_cascade.detectMultiScale(roi_gray)
          for (ex,ey,eh,ew) in eyes:
                 cv2.rectangle(img,(ex,ey),(ex+eh,ey+ew),(0,255,0),2)
cv2.imshow('img',img)
cv2.waitKey(0)
cv2.destroyAllWindows()

First, we have imported some important libraries, then we read the cascade files. Basically, it finds faces! We also want to find eyes, but, in a world of false positives, wouldn't it be prudent to logically make it so that we only find eyes in faces? Let's hope we're not looking for eyes that aren't in faces! In all seriousness, "eye detection" probably wouldn't find an eyeball laying around. Most eye detection uses the surrounding skin, eyelids, eyelashes, and eyebrows to also make the detection. Thus, our next step is to break down the faces first, before getting to the eyes.

Here's the result:

Now you can see how we detect human face and eye.

That's all for OpenCV blogs. In the future, i will tell you that how you can make your own cascade and test results depending upon your chosen object. Hope you like my tutorial for OpenCV. 
Thank you......