When attempting to test out a robot creation, it really helps to turn on the robotic creation first.

A Crazy Project Weekend is when I take an extended weekend and dedicate my time to a AAA project:  One that is Achievable, Awesome, and slightly Abnormal.  There are a couple of rules, made up on the spot this instant, guiding the Crazy Project:

• Work must be done within the limited weekend time frame.  You cannot begin any concrete work prior to the time window, and if you do not complete by the end of the time, you have failed.  You may do some preparation ahead of time, such as feasibility research or acquiring necessary materials, however, nothing should be built and there should be no written plans.  The point is to see what can be done in five days, not what can be done in five days and a couple of hours an evening for three weeks prior to those five days. 
• It must not be something you would otherwise normally do.  Setting up a website with a blog and a bunch of pictures of the dog doesn’t count.  Cleaning the garage doesn’t count.  It must not be something that anyone would normally do.
• You have to learn something.  If you know exactly what you’re doing going in, then it’s no fun.  One of the central pieces of the project must involve something you’ve never worked with before.  There must be several moments where you have no idea what in the hell you’re doing and wonder what you’ve gotten yourself into.
• You must post regular progress updates throughout the weekend, detailing what you’re doing and what you’ve done.  Viewers must be able to get a glimpse of your thought process and understand what you’re going through.  You should talk about initial goals and milestones, obstacles you see on the path to those milestones, and the general approach you plan to take.
• Reaction from outsiders to your project must be a mix of “Why did you do that?” and “Oh man, that is AWESOME”.
• It’s fine to have a mental plan going in, to make sure that you’ve appropriately scoped the project so you have a reasonable chance of success, no matter how unreasonable the project itself may be.
• Continuing the effort from a previous Crazy Project Weekend is acceptable, even though it violates some of the previous rules.

The first Crazy Weekend Project was over Labor Day Weekend, in September 2009.  I decided that it would be a good use of my time to build a robot out of Lego Mindstorms that could play a game of Pong on an unmodified Atari 2600 and win.  Initially, I had planned to make it play a perfect game of Pong, but I didn’t get there.  Full details here:  https://mathpirate.net/log/category/crazy-weekend-project-1-pong-robot/

The second Crazy Weekend Project was over Thanksgiving 2009.  It was limited, in that I only dedicated about half the day to the project (The other half being dedicated to XBox 360…).  There were two goals for this project:  Put together a speech recognition system capable of recognizing and responding to a set series of commands, as well as write a system that could identify faces.  Speech recognition came together very quickly, so the bulk of the time was spent trying to make Wesley Crusher disappear from episodes of Star Trek: The Next Generation.  Full details here:  https://mathpirate.net/log/category/crazy-weekend-project-2/

This will be my third Crazy Weekend Project.

Here’s the video:

[mediaplayer src=’https://mathpirate.net/hold/LCARS1.wmv’ ]

https://mathpirate.net/hold/LCARS1.wmv

This is a scene from the Next Generation episode Journey’s End.  The first run is simply the result with Wesley Crusher blacked out.  The second part displays the faces that are recognized.  There are occasional blips (Data gets blacked out in several frames, Wesley’s ear gets “recognized” a few times), but overall, not too shabby for something thrown together in a few days with no virtually no tuning of the system being done.  The reason he’s not blacked out at the end is likely due to the decision to expand the minimum face size to 50×50.  If the face size were smaller, it likely would have detected and blocked that part, as well.

I tried running it on a different episode and the results were…  A bit different.

I guess the high success rate was due to finding the training images again.  Oh well, at least it did that right.

And this wasn’t a complete loss, JeanLucPicard was identified correctly most of the time!  And some of the other characters were properly identified some of the time.

I also took a look at the distances for each match.  You see, the match is actually a 2500 dimensional distance calculation between two images.  I was hoping to use this distance to weed out false matches, unfortunately, there wasn’t really that much of a separation between correct and incorrect matches.

At any rate, all that is just tweaking the system.

With that, I’ve actually reached the point I was hoping to reach for this project.  I set out to write something that would detect and identify faces from a video stream and that’s what I’ve done.  However, there’s obviously another step that needs to be done, which is the automatic removal of Wesley Crusher.  It knows who he is, so it’s halfway there.

Of course, on the technical side, there are a few things left to do, like figure out why I’m getting Access Violations randomly when calling EigenDecomposite() or trying to make it not be so horribly slow or figuring out if there’s some way I can return nobody when it’s not confident enough about its identification.

And, of course, I also have to run it against a different episode in order to make sure that the recognition isn’t so accurate because it’s simply recognizing the original faces that it trained on…

But that’s what tomorrow is for.

Images in action:

Target Acquired

Multiple hits.  Note that it doesn’t see Geordi…  I think the face detector relies heavily on the eyes, so his VISOR is confusing it.

It can even detect redshirts!

IT WORKS!

HOLY CRAP IT ACTUALLY WORKS!

I trained it on a set of about half of the images, because it takes a long time to train.  Then I ran it on the video and it was actually returning the right person most of the time.  I’m actually shocked that it’s working so well, especially seeing that this is the first time I’ve had the recognition part actually compiling and running.  No real tweaking necessary.  The problems I’ve been having have all been with calling the functions and simply using the library, it’s not that the software was doing what I told it to do and it all came out wrong.

Looking at this trace, you can even get a sense of the cuts in the scene:

Recognized: JeanLucPicard
Recognized: DeannaTroi
Recognized: JeanLucPicard
Recognized: DeannaTroi
Recognized: DeannaTroi
Recognized: JeanLucPicard
Recognized: DeannaTroi
Recognized: Anthwarta
Recognized: Anthwarta
Recognized: Anthwarta
Recognized: Anthwarta
Recognized: Anthwarta
Recognized: Anthwarta
Recognized: Anthwarta
Recognized: DeannaTroi
Recognized: DeannaTroi
Recognized: DeannaTroi
Recognized: JeanLucPicard
Recognized: JeanLucPicard
Recognized: DeannaTroi
Recognized: JeanLucPicard
Recognized: DeannaTroi
Recognized: DeannaTroi
Recognized: JeanLucPicard
Recognized: DeannaTroi
Recognized: DeannaTroi
Recognized: DeannaTroi
Recognized: DeannaTroi
Recognized: JeanLucPicard
Recognized: DeannaTroi
Recognized: JeanLucPicard
Recognized: Anthwarta
Recognized: Anthwarta
Recognized: Anthwarta
Recognized: Anthwarta

Of course, right now, it’s only working on the positive cases.  When it knows someone, it knows them.  When it doesn’t know someone, it randomly guesses about who it is.

Now, something interesting about that…  I haven’t trained the system on Wesley yet.  He’s down in the Ws and the images I trained with were alphabetical.  However, in the scenes with him, he’s fairly regularly identified as “JeanLucPicard”.

Gotta wonder if the Doctor and the Captain have something they need to talk to him about…

Now I just have to label the faces on the screen, since screenshots are better than random text.

If your facial recognition system is only capable of recognizing a single face, you will have 100% accuracy on that face when that face is in the scene.

Looks like it’s time to try processing the full set of images…

I was expecting the facial recognition system to give me odd results when I first ran it.

I was not expecting the results to be quite this odd:

As I’ve been implementing this and actually figuring out how some of the pieces work together, it appears that variability of the data might not be as big of an issue as I thought.  I had originally assumed that the recognition algorithm would produce a single blob of data for each person you want to recognize, and that you’d be matching against that.  Now that I look at it, that doesn’t seem to be the case.  Instead, it looks like it chops each training image down to some constituent particles, then you run through these particles to determine which one is the closest to the image you want to match.  Which means that hopefully variability won’t be a huge problem, since I’m matching to an individual image.

Then again, I’m talking about the 2500-dimension eigenvalue PCA subspace or something like that, so I really have absolutely no idea what I’m talking about.