Random header image... Refresh for more!

Admirial Nechayev Will Eat Your Soul



And a sign that this isn’t going to work so well.  I think at this phase, you’re supposed to see something that, well, kinda looks more like a face.

It’s 4 PM and this is all I have so far.  This sucks.

November 28, 2009   No Comments

A Little Light Reading

For those playing the home game, here’s some of the sites with examples that I’ve been using:

OpenCV Face Detection: http://nashruddin.com/OpenCV_Face_Detection  This brief post shows how to use the Face Detection stuff in OpenCV.  It’s a bit clearer than the example on the OpenCV site.

Implementing Eigenface:  http://www.cognotics.com/opencv/servo_2007_series/part_5/index.html  This is a detailed article on recognizing faces using OpenCV, including explaining how some of the algorithms work.  Some of the information here is what makes me think that my initial stab at recognition is doomed to failure.

And, of course:

OpenCV: http://opencv.willowgarage.com/wiki/Welcome

OpenCVSharp: http://code.google.com/p/opencvsharp/

November 28, 2009   No Comments

Spatial Anomalies

I think I may have run into a problem on my quest to remove Wesley Crusher from Star Trek before I’ve even started writing the algorithm to do so.  The algorithm I was planning on using appears to require highly normalized data.  As in every face needs to be looking in exactly the same direction and be the same size in the frame.  As the images I pulled are random grabs from an episode, they’re looking left or right or straight ahead or are blurry or have varying degrees of Face vs. Hair, etc.  I’m concerned that I’ll just get a completely muddled recognizer that is looking for one totally random blob of pixels vs. another.

I’ll try it anyway, though.  Even if it doesn’t work, I’m sure it will come up with some interesting results…

November 27, 2009   No Comments

End of Day 2.

Today definitely involved more work than yesterday.  At the end of the day, I’ve got a decent face detector written, with massive help from OpenCV.  More importantly, I’ve got over 1500 facial images extracted from an Star Trek episode that I can use for training the face recognizer.  That’ll be tomorrow.

It should be noted that when you’re randomly grabbing frames of people’s faces from a video, you end up with a lot of goofy looking facial expressions.


Anyway, if you want to see the rest of the facial images, they’re in SVN, located here:  https://mathpirate.net/svn/Projects/AlternativeInputDevices/Data/TNGFaces/

November 26, 2009   No Comments

Images Acquired

It finished processing the first episode and extracted 1829 things it thinks are faces.  The vast majority of them are good quality faces.  The remainder are poor quality or blurry faces of people in the background, various body parts that aren’t faces, and, of course, random pieces of the ship.  Still, this should work quite well for training data.

Thing is, I have to manually classify this first bunch…

November 26, 2009   No Comments

Cloak Engaged

The face detector doesn’t seem to be noticing Geordi or Worf.  That’s understandable, since Geordi has the VISOR on and Worf’s got the ridged forehead.  However, it IS finding a Cardassian, who is completely covered in makeup.  Oh well.  Since the main use of this won’t be for Starfleet Security, I think I can get away without fixing that particular problem…

November 26, 2009   No Comments

Boldly Going…

As I said in the beginning, detecting the faces is only part of the equation.  Just knowing that something in a face isn’t all that useful for me.  I need to know whose face it is.  In order to write that part of the code, I’m going to need a whole bunch of faces to use as training data.  For this to be useful, I’m going to need a large number of views of the same person, as well as sets for different people.  You can’t rightly have something you call “facial recognition” if all it can do is recognize your own face.  So, that means that the webcam is ruled out for this phase.  There’s only so many faces I have around my apartment, and it just won’t do.

There are, of course, other options.  I could try to raid someplace like Flickr and steal someone’s family photos, or scour gossip sites for celebrity images.  However, what I have in mind is much easier and, as you’ll see, has a practical benefit, as well.

Now, I just wrote a face detector.  It’s able to detect faces in video streams. So, why not put it to work on a video stream, detecting faces for me to use as my training set?  I’ll only get faces that I know the system can detect and they’ll already be cropped to the location of the face for me.


Obviously, the best source material to use is an episode of Star Trek: The Next Generation.  This has two main benefits.

First, once I get all the faces detected and classified and the recognizer written, I’ll be able to run it against a different episode of TNG as a test.  Many of the people will be the same between episodes, so it should be able to recognize them, but the system should not recognize members of the supporting cast and guest stars, since they’re not in the source episode.  This is important, because I need to make sure it handles people it doesn’t know about correctly.

The second and far more important benefit is that if I am successful, I will have written a program that is able to watch an episode of Star Trek: The Next Generation and let me know if Wesley Crusher is in it, so that I can skip those scenes.  That sort of technology has money-making potential.

November 26, 2009   No Comments

Speed +5, Accuracy +3

As I hoped, tweaking the parameters to the function greatly sped up the processing of the face detection.  It’s now acceptably fast.  Not 120 fps fast, but it’s unlikely that faces in video will change that fast.  It’s running probably ten frames a second, which is good enough for what I wanted.  Additionally, the false positives have been cut down quite a bit.  So, here’s what I did:

The key function in face detection is called CvHaarDetectObjects.  It has something to do with Haar Classification and Cascades and all sorts of other technical mumbo jumbo that I don’t really care about at the moment because someone else has already figured it out and written it all for me.  All I know is that I have to call it with some parameters and the function returns to be a list of faces.  (And things that it thinks are faces because it’s delusional and psychotic…)  It’s not strictly a face detector, it’s a generalized object detector, it just happens to be used for detecting faces.

The first few parameters are straightforward, the image and storage, etc.  Then you get into less apparent parameters, like “Scale Factor” and “Min Neighbors”.

Scale Factor is how much to increase the size of the scanning window between passes.  I think that translates into something like “Higher numbers run faster wil less accuracy.”

Min Neighbors has something to do with overlaps and multiple detected objects or something like that.  Basically, the higher the number, the fewer false positives you get, but if you set it too high, you risk getting fewer true positives, too.

Then there’s a flag parameter which alters how the operation works.  I used the “Do Canny Pruning” value because that’s what someone else said to do.  Supposedly that value make things run faster and result in fewer false positives.

Finally, there’s Min Size.  That’s the size of the scan window to use, which translates roughly to the smallest object you expect to find.  The smaller the window the more things you’ll pick up, but at the expense of speed.

Right now, I’m calling the following and it seems to be doing a pretty good job:

CvSeq sequence = _classifierCascade.HaarDetectObjects(image, _memStorage, 1.2, 30, HaarDetectionType.DoCannyPruning, new CvSize(40, 40));

Now, it’s on to acquiring the training images…

November 26, 2009   No Comments

Next steps…

Now, I need to do three things:

  1. Tweak the face detection settings so it’s not lame-ass slow.
  2. Tweak the face detection settings so that it doesn’t think walls and boxes look like faces.
  3. Get a whole bunch of identifiable faces for detection training.

Steps 1 and 2 might turn out to be difficult and error prone.  I know there were settings like minimum face size and tricks like grayscaling images and performing noise reduction that I skipped which may potentially help those problems. 

Step 3, though…  That I’ve already figured out…

November 26, 2009   No Comments

Putting People In Boxes

So, after fixing the crashing issue, I started up the detection algorithm and let it roll.


Despite the fact that I’m in a poorly lit room and wasn’t even looking at the camera, it found me.  I like that it’s sensitive enough for that, since my planned application of this can’t rely on well-lit clear images all the time.

Of course, the sensitivity has a downside…


I turned on the lights and it still found me, which was good.  However, it also believes that there are faces on the wall, on the ceiling, a big face made up of shelving and boxes, as well as three separate faces on the Commodore Plus/4 box.

It’s confused, of course, because it’s actually the Tomy Tutor box that has all the faces on it.


So, in summary:

  • Multiple faces: good.
  • Faces where they don’t exist: psychotic and delusional.

Must fix the psychotic and delusional part before I let this loose on the world.

There’s one other slight problem…  The facial detection is pegging my CPU and processing about two frames a second with a two second delay.  That’s not going to be acceptable, either…

November 26, 2009   No Comments