Creepy.

And a sign that this isn’t going to work so well.  I think at this phase, you’re supposed to see something that, well, kinda looks more like a face.

It’s 4 PM and this is all I have so far.  This sucks.

For those playing the home game, here’s some of the sites with examples that I’ve been using:

OpenCV Face Detection: http://nashruddin.com/OpenCV_Face_Detection  This brief post shows how to use the Face Detection stuff in OpenCV.  It’s a bit clearer than the example on the OpenCV site.

Implementing Eigenface:  http://www.cognotics.com/opencv/servo_2007_series/part_5/index.html  This is a detailed article on recognizing faces using OpenCV, including explaining how some of the algorithms work.  Some of the information here is what makes me think that my initial stab at recognition is doomed to failure.

And, of course:

OpenCV: http://opencv.willowgarage.com/wiki/Welcome

OpenCVSharp: http://code.google.com/p/opencvsharp/

I think I may have run into a problem on my quest to remove Wesley Crusher from Star Trek before I’ve even started writing the algorithm to do so.  The algorithm I was planning on using appears to require highly normalized data.  As in every face needs to be looking in exactly the same direction and be the same size in the frame.  As the images I pulled are random grabs from an episode, they’re looking left or right or straight ahead or are blurry or have varying degrees of Face vs. Hair, etc.  I’m concerned that I’ll just get a completely muddled recognizer that is looking for one totally random blob of pixels vs. another.

I’ll try it anyway, though.  Even if it doesn’t work, I’m sure it will come up with some interesting results…

Today definitely involved more work than yesterday.  At the end of the day, I’ve got a decent face detector written, with massive help from OpenCV.  More importantly, I’ve got over 1500 facial images extracted from an Star Trek episode that I can use for training the face recognizer.  That’ll be tomorrow.

It should be noted that when you’re randomly grabbing frames of people’s faces from a video, you end up with a lot of goofy looking facial expressions.

Anyway, if you want to see the rest of the facial images, they’re in SVN, located here:  https://mathpirate.net/svn/Projects/AlternativeInputDevices/Data/TNGFaces/

It finished processing the first episode and extracted 1829 things it thinks are faces.  The vast majority of them are good quality faces.  The remainder are poor quality or blurry faces of people in the background, various body parts that aren’t faces, and, of course, random pieces of the ship.  Still, this should work quite well for training data.

Thing is, I have to manually classify this first bunch…

The face detector doesn’t seem to be noticing Geordi or Worf.  That’s understandable, since Geordi has the VISOR on and Worf’s got the ridged forehead.  However, it IS finding a Cardassian, who is completely covered in makeup.  Oh well.  Since the main use of this won’t be for Starfleet Security, I think I can get away without fixing that particular problem…

As I said in the beginning, detecting the faces is only part of the equation.  Just knowing that something in a face isn’t all that useful for me.  I need to know whose face it is.  In order to write that part of the code, I’m going to need a whole bunch of faces to use as training data.  For this to be useful, I’m going to need a large number of views of the same person, as well as sets for different people.  You can’t rightly have something you call “facial recognition” if all it can do is recognize your own face.  So, that means that the webcam is ruled out for this phase.  There’s only so many faces I have around my apartment, and it just won’t do.

There are, of course, other options.  I could try to raid someplace like Flickr and steal someone’s family photos, or scour gossip sites for celebrity images.  However, what I have in mind is much easier and, as you’ll see, has a practical benefit, as well.

Now, I just wrote a face detector.  It’s able to detect faces in video streams. So, why not put it to work on a video stream, detecting faces for me to use as my training set?  I’ll only get faces that I know the system can detect and they’ll already be cropped to the location of the face for me.

Obviously, the best source material to use is an episode of Star Trek: The Next Generation.  This has two main benefits.

First, once I get all the faces detected and classified and the recognizer written, I’ll be able to run it against a different episode of TNG as a test.  Many of the people will be the same between episodes, so it should be able to recognize them, but the system should not recognize members of the supporting cast and guest stars, since they’re not in the source episode.  This is important, because I need to make sure it handles people it doesn’t know about correctly.

The second and far more important benefit is that if I am successful, I will have written a program that is able to watch an episode of Star Trek: The Next Generation and let me know if Wesley Crusher is in it, so that I can skip those scenes.  That sort of technology has money-making potential.