Feeds:
Posts

## Face Recognition/Authentication Using Support Vector Machines

This post is part of a series on face recognition, I have been posting on face recognition for a while. There would be at least 7-8 more posts in the near future on the topic. Though I can not promise a time frame within which all would be up.

Previous Related Posts:

3. A Huge Collection of Datasets (Post links to a number of face image databases)

This post would reference two of my posts. One on SVMs and the other on Face Recognition using Eigenfaces.

Note: This post focuses on the idea behind using SVMs for face recognition and authentication. In future posts I will cover the various packages that can be used to implement SVMs and how to go about using them, and specifically for face recognition. The same can be easily extended to other similar problems such as content based retrieval systems, speech recognition, character or signature verification systems as well.

_____

Difference between Face Authentication (Verification) and Face Recognition (also called identification):

This might seem like a silly thing to start with. But for the sake of completeness, It is a good point to start with.

Face Authentication can be considered a subset of face recognition. Though due to the small difference there are a few non-concurrent parts in both the systems.

Face Authentication (also called verification) involves a one to one check that compares an input image (also called a query image, probe image or simply probe) with only the image (or class) that the user claims to be. In simple words, if you stand in front of a face authentication system and claim to be a certain user, the system will ONLY check if you are that user or not.

Face Recognition (or Identification) is another thing, though ofcourse related. It involves a one to many comparison of the input image (or probe or query image) with a template library. In simple words, in a face recognition system the input image will be compared with ALL the classes and then a decision will be made so as to identify to WHO the the input image belongs to. Or if it does not belong to the database at all.

Like I just said before, though both Authentication and Recognition are related there are some differences in the method involved, which are obvious due to the different nature of both.

_____

A Touch-Up of Support Vector Machines:

A few posts ago I wrote a post on why Support Vector Machines had this rather “seemingly” un-intuitive name. It had a brief introduction to SVMs as well. For those completely new to Support Vector Machines this post should help. I’ll still add a little for this post.

Support Vector Machine is a binary classification method that finds the optimal linear decision surface between two classes. The decision surface is nothing but a weighted combination of the support vectors. In other words, the support vectors decide the nature of the boundary between the two classes. Take a look at the image below:

The SVM takes in labeled training examples $\{\; x_i, y_i \}$, where $x_i$ represents the features and $y_i$ the class label, that could be either 1 or -1.  On training we obtain a set of Support Vectors $m$, multipliers $\alpha_i$, $y_i$and the term $b$. To understand what $b$ does, look at the above figure. It is somewhat like the intercept term $c$ in the equation of a straight line, $y = mx + c$. The terms $w$ and $x$ determine the orientation of the hyperplane while $b$ determines the actual position of the hyperplane.

As is indicated in the diagram, the linear decision surface is :

$w\star x + b = 0 \qquad(1)$

where $\displaystyle w = \sum_{i=1}^m \alpha_i y_i s_i$

where $s_i$ are the support vectors.

The above holds when the data (classes) is linearly separable. Sometimes however, that’s not the case. Take the following example:

The two classes are indicated by the two different colors. The data is clearly not LINEARLY separable.

However when mapped onto two dimensions, a linear decision surface between them can be made with ease.

Take another example. In this example the data is not linearly separable in 2-D, so they are mapped onto three dimensions where a linear decision surface between the classes can be made.

By Cover’s Theorem it is more likely that a data-set not linearly separable in some dimension would be linearly separable  in a higher dimension. The above two examples are simple, sometimes the data might be linearly separable at very high dimensions, maybe at infinite dimensions.

But how do we realize it? This done by employing the beautiful Kernel Trick. In place of the inner products we use a suitable Mercer Kernel. I don’t believe it is a good idea to discuss kernels here, or it will be a needless digression from face recognition. I promise to discuss it some time later.

Thus the non-linear decision surface changes from $\qquad(1)$ to:

$\displaystyle w = \sum_{i=1}^m \alpha_i y_i K(s_i, x) +b = 0 \qquad(2)$

Where $K$ represents a Kernel. It could be a Radial Basis (Gaussian) Kernel, A linear Kernel, A polynomial Kernel or a custom Kernel. :)

_____

Face Authentication is a two class problem. As I have mentioned earlier, here the system is presented with a claimed identity and it has to make a decision whether the claimant is really that person or not. The SVM in such applications will have to be fed with the images of one person, which will constitute one class and the other class will consist of images of other people other than that person. The SVM will then generate a linear decision surface.

For a input/probe image $p$, the identity is accepted if:

$w \star p + b < 0$

Or it is rejected. We can parameterize the decision surface by modifying the above as:

$w \star x + b = \Delta$

Then, a claim will be accepted if for a probe, $p$

$w \star p + b < \Delta$

_____

Now face recognition is a $\mathcal{K}$ class problem. Where $\mathcal{K}$ is the number of classes (or individuals).  Whereas the traditional Support Vector Machine is a binary classifier. So we’ll make a few changes to the way we are representing the faces to suit our classifier. I will come back to this in a while.

Feature Extraction: The faces will have to be represented by some appropriate features, these could be weights obtained using the Eigenfaces method, or using gabor features or anything else. I have written a post earlier that talked of a face recognition system based on Eigenfaces. I would direct the reader to check face representation using Eigenfaces there.

Using Eigenfaces, each probe $\Phi$could be represented as a vector of weights:

$\Omega = \begin{bmatrix}w_1\\w_2\\ \vdots\\w_M \end{bmatrix}$

After obtaining such a weight vector for the input or probe image and for all the other images stored in the library, we were simply finding the Euclidean or the Mahalanobis distance of the weight vector of the probe image with those of the images in the template library.  And then were recognizing the probe as a face that gave the minimum score provided it was below a certain threshold. I have discussed this is much detail there. And since I have, I would not discuss this again here.

_____

Representation in Difference Space:

SVMs are binary classifiers, that is – they give the class which might be 1 or -1, so we would have to modify the representation of faces a little bit than what we were doing in that previous post to make it somewhat more desirable. In the previous approach that is “a view based or face space approach”, each image was encoded separately. Here, we would change the representation and encode faces into a difference space. The difference space takes into account the dissimilarities between faces.

In the difference space there can be two different classes.

1. The class that encodes the dissimilarities between different images of the same person,

2. The other class encodes the dissimilarities between images of other people. These two classes are then given to a SVM which then generates a decision surface.

As  I wrote earlier, Face recognition traditionally can be thought of as a $\mathcal{K}$ class problem and face authentication can be thought of as a $\mathcal{K}$ instances two class problem. To reduce it to a two class problem we formulate the problem into a difference space as I have already mentioned.

Now consider a training set $\mathcal{T} = \{ \;t_1, \ldots, t_M\}$ having ${M}$ training images belonging to $\mathcal{K}$ individuals. Each individual can have more than one image, that means $M > \mathcal{K}$ ofcourse. It is from $\mathcal{T}$ that we generate the two classes I mentioned above.

1. The within class differences set. This set takes into account the differences in the images of the same class or individual. In more formal terms:

$\mathcal{C}_1 = \{ \; t_i - t_j | t_i \backsim t_j \}$

Where $t_i$ and $t_j$ are images and $t_i \backsim t_j$ indicates that they belong to the same person.

This set contains the differences not just for one individual but for all $\mathcal{K}$ individuals.

2. The between class differences set. This set gives the dissimilarities of different images of different individually. In more formal terms:

$\mathcal{C}_2 = \{ \; t_i - t_j | t_i \nsim t_j\}$

Where $t_i$ and $t_j$ are images and $t_i \nsim t_j$ indicates that they do not belong to the same person.

_____

Face Authentication:

For Authentication the incoming probe $p$ and a claimed identity $i$ is presented.

Using this, we first find out the similarity score:

$\delta = \displaystyle \sum_{i=1}^m \alpha_i y_i K(s_i, ClaimedID - p) +b$

We then accept this claim if it lies below a certain threshold $\Delta$ or else reject it. I have discussed the need for a threshold at the end of this post, please have a look. $\Delta$ is to be found heuristically.

_____

Face Recognition:

Consider a set of images $\mathcal{T} = \{ \;t_1, \ldots, t_M\}$, and a probe $p$ which is to be indentified.

We take $p$ and score it with every image in the set $t_i$:

$\delta = \displaystyle \sum_{i=1}^m \alpha_i y_i K(s_i, t_i - p) + b$

The image with the lowest score but below a threshold is recognized. I have written at the end of this post explaining why this threshold is important. This threshold is mostly chose heuristically.

_____

References and Important Papers

1. Face Recognition Using Eigenfaces, Matthew A. Turk and Alex P. Pentland, MIT Vision and Modeling Lab, CVPR ‘91.

2. Eigenfaces Versus Fischerfaces : Recognition using Class Specific Linear Projection, Belhumeur, Hespanha, Kreigman, PAMI ‘97.

3. Eigenfaces for Recognition, Matthew A. Turk and Alex P. Pentland, Journal of Cognitive Neuroscience ‘91.

4. Support Vector Machines Applied to Face Recognition, P. J. Phillips, Neural Information Processing Systems ’99.

5. The Nature of Statistical Learning Theory (Book), Vladimir Vapnik, Springer ’99.

6. A Tutorial on Support Vector Machines for Pattern Recognition, Christopher J. C. Burges, Data Mining and Knowledge Discovery, ’99

_____

Onionesque Reality Home >>

### 39 Responses

1. Hello

I’m doing my telecomunications thesis about SVM face recognition. I’m developming a matlab-interface for this task. I’m based on the work you have published here (Support Vector Machines Applied to Face Recognition – P.Jonathon Phillips).

I was implementing my own code for this but finally I opted to use de STPRtoolbox for Matlab (http://cmp.felk.cvut.cz/cmp/software/stprtool/) and I have some problems with this.

I would like to ask some help in some questions I have and I can’t resolve.

Some of them are the use of large 1D data input (it carries me some problems).

I have a struct called data for the input for my gallery (7 grayscale images (16×16) of each of the 15 individuals,) . So, my input has:

A vector data.X formed by disordered data of the 2 classes (C1: 7 x 15 vectoriced image differences of the same person, C2: 7 x 15 vectoriced image differences of different persons).

A vector data.y formed by the label of each input value (-1 or +1).

A vector data that express the dimension of the data input (in my case dimesion 1).

My question is, I’m creating the data input in the correct way? Do you know one matlab toolbox that could help me for this?

I’d appreciate if you could help me with this because I have lots of time with my thesis and I’m still lost and getting crazy with this.

Best regards,
Manuel P.

2. Manuel!
I am extremely extremely sorry! Your comment was in the spam folder till now and I have just seen it.

Is it too late for my input?

3. Hello,

Is not too late because I’m still in development with this issue :) . Finally I opted to implement the SVM face recognition system with libSVM instead STPRtoolbox for Matlab (this toolbox carried me many problems). Although I have new problems related with libSVM (I always find difficulties for everything).

I would like to know if have you worked with the libSVM toolbox, if have you used it with related tasks with SVM classification (I think maybe you could help me) or something about wich strategies has followed to implement the SVM face classification.

Anyway, anything related with this issue could help me.

Regards

Manuel

4. Plz Give the code of MatLab

where Face is recognized by Support Vector Mechines Method

if mail me, my mail id:

5. Hi,
I see that you do a big research about Face recognition, and it is amazing, really!
So if I can help you with your research, I just find one great book about Face recognition which inspire some future ideas by identifying potential research directions.
The book consists of 28 chapters, each focusing on a certain aspect of the problem.
This book will serve as a handbook for students, researchers and practitioners in the area of automatic (computer) face recognition.
This is the link : http://www.intechopen.com/books/show/title/face_recognition
Hope you will enjoy it.

6. i m making the project based on face reconignation of distorted image using support vector machine plz give the detail about this project…..as soon as possible

7. plz give the matlab code for face recongination……bcuz i m making major project of Face Recognition From a Distorted Image Using Support Vector Machine”…………
my id -bhavesh.jh@gmail.com

• I am glad you are doing this project!

8. any body give the idea about my project Face Recognition From a Distorted Image Using Support Vector Machine.
my id -bhavesh.jh@gmail.com

9. hello, i am a student in master of science, I really need code of this project ( face recognition with SVM ) please help me.

10. ahhhhhhhhhhhhhhhhh ,plzzzzzzzzzz send source code of face recognition with SVM , i have a really really really bad problem :(((

11. on June 18, 2011 at 11:04 am | Reply jhilmil mathur

hi .. plz send me the code for face detection using svm and wavelet (feature extraction)…
plz help me .. i m making this project
jhilmilmathur@gmail.om

12. pls send me the code for face detection and recognition using any algorithm except PCA and ada boost algorithm pls………

13. I am now doing my final project and I really need the face reocgnition with SVM code. Can you please send it to me? It would be realy realy helpful..

meraviran@gmail.com

14. if it’s posible, plz send the matlab code for face recongination with SVM.
thank you very much.

15. if it’s posible, plz send the matlab code for face recongination with SVM.
thank you very much.
tofighi_alireza@yahoo.com

16. please can you send me the code for face recognition system using mathlab its for my project ill be thankful

17. can u send the SVM code for face recognition..please

18. hello …

i ‘m a school student…m really sorry if i am spamming ur blog….but i guess ur blog is among the best blogs on education till date..so …u’ll definately understand me …

please watch the following videos on you-tube,
kindly share your views about my work and also vote for me and ask all your near n dear ones to do the same.

If you face any problem in doing so can please go through the bullets below.

You just need to create an account on you-tube using your gmail account
Go to you-tube.com ( in another tab)
Click ‘Create Account’ {your account would be created}
Click the ‘Like’ button

My work is done!
Please do take the pains of liking the videos and ask as many people as you can, it would help me a lot.Please don’t forget to ask other to do the same.
Thank You

hoping for ur reponse…

• Good job Archit!

19. Hi, i am working on the protein databases at Munich University, Germany. I have to classify the the database in interacting and non-interacting proteins. I am developing a tool for this approach and i need SVM source code in C sharp language. I’ll be very thankful

usmanmirzapk@yahoo.com

• Didn’t see this earlier. Sorry I don’t know about C# packages.

20. This Blog is only copy and paste i don’t think author know how to code also. i have not find the author giving idea about writing the code. all the mathematics and literature is present all over net. what is the big deal here.

sorry but i am not impressed. Totally time wastage.

• Dear rdtytuiop,

I must point out three things:
1. I had actually put the code earlier, but I realized that people only want it as a shortcut to do their BE/BTech projects (I use BE because I know that’s what you call that). In that case putting up code will not help people but harm their understanding.

2. For SVM, I assume one would be using routines from something like SVM-light. I think that basically takes 30 minutes of work. I don’t call that writing code. Why don’t you anyway write it up and link me up? I would be glad to link in the post.

3. The post was a synthesis of atleast a dozen papers. My attempt was to try and simplify the steps. Sorry it didn’t help you.

21. on June 1, 2013 at 3:32 pm | Reply Gia Su Thanh

I am now doing my final project and I really need the face reocgnition with SVM code. Can you please send it to me? It would be realy realy helpful..
mail: giasuthanh@gmail.com

22. Hello,
I am a Ph.D. student and I am going to implement an ensemble of SVMs for video-based face recognition. I am quit new in this topic, but I am supposed to implement a mini project in this topic as a start. I am familiar with a number of classifiers but it is my first time to work with SVM. Any how I am studying about it and I am sure I will get it soon. BUT, my focus is on the classification part and I do not want to spend too much time on face detection and feature extraction part. Could you please suggest some feature extraction techniques for face that are suitable for SVM? I need to prepare a proposal for this mini-project soon. So, I just need a name of some feature extraction techniques for face that could be easily found in internet (since the main focus of mini project is on the classification part).

23. @ Elham
Are you sure that you are PhD student and you don’t know about SVM! this is undergraduate! You would also like to work on ensemble learning?!

O.K. look at below link may be useful for you:
http://www.csie.ntu.edu.tw/~cjlin/libsvm/

http://www.mojsazan.com/forum/index.php

Best
Raha

• Dear Raha!

First of all, I would like to thank you for your reply! The links are useful, actually I am using the LibSVM now!

There are a lot of things in science that I don’t know and I am sure that there are a lot of things that you don’t know.
I don’t know where you are studying and in which level, BUT, I do know that in the professional research groups people learn to ask questions freely and get their answers with respect and without being criticised. In fact, sharing knowledge with respect is a primary potential feature that successful researchers have.
I hope that you learn it someday or at least try to keep it in mind.

Regards,
Elham

24. Dear Shubhendu Trivedi,

Thanks for the detailed descriptions on Face recognition using svm. When I try to implement I had the following obstacles.

1. Within class and between class. :

I have taken gabor feature as the feature vector and for dimensionality reduction – PCA was done (30 dimension). My output vector was 112*30*40.
which was reshaped to 1D – (134400*1).

In total I have 45 images with 3 class (each 15 images.)

Finding the withinclass and between class –

I have subtracted within each class (15 images) images. Obtained withinclass matrix of size 134400*3. (134400 for each class)

I have randomly chosen 3 images from each class and subtracted to get the betweenclass matrix of size 134400*1.

Please let me know whether the above steps are correct ? If not so kindly suggest on the modifications
And also kindly let me know how to proceed with SVM for the obtained output matrices.

25. plz give me this code a badly need this code.. my email is madihasaeed7712@gmail.com

26. on June 20, 2016 at 9:07 am | Reply himel rahman

I am an undergrad student currently working on facial expression recognition using svm. Can you send me the code of face recognition using svm so that i can work further at himel101rahman@gmail.com

27. on June 20, 2016 at 9:12 am | Reply himel rahman

I am an undergrad student, currently working on facial expression recognition using svm. Can you help me by providing face recognition using svm so that i can work further

Respected, can you send me SVM code for face recognition, my email id fahad_but@yahoo.com

29. hello.. i m doing my FYP on face recognition .. using matlab.. i need SVM code for thatt.. can u please help mee with thattt??

30. sir i’m doing a project same as this for my university final year project can you share your ode with me please.
rizmir22@gmail.com

31. please send me the matlab code

32. hello sir… please send me the matlab code on emaaaaan9232@gmail.com

33. sir can you send me the MATLAB code for svm face recognition..very urgent
manisep28@gmail.com

34. sir can you send me the MATLAB code for svm face recognition..very urgent
manisep28@gmail.com