Posts from — November 2009

Here’s the video:

[mediaplayer src=’https://mathpirate.net/hold/LCARS1.wmv’ ]

https://mathpirate.net/hold/LCARS1.wmv

This is a scene from the Next Generation episode Journey’s End.  The first run is simply the result with Wesley Crusher blacked out.  The second part displays the faces that are recognized.  There are occasional blips (Data gets blacked out in several frames, Wesley’s ear gets “recognized” a few times), but overall, not too shabby for something thrown together in a few days with no virtually no tuning of the system being done.  The reason he’s not blacked out at the end is likely due to the decision to expand the minimum face size to 50×50.  If the face size were smaller, it likely would have detected and blocked that part, as well.

With this, I tried a technique called “Inpainting”, where you give it a mask of the damaged area, and the software will remove the damage.

I tried running it on a different episode and the results were…  A bit different.

I guess the high success rate was due to finding the training images again.  Oh well, at least it did that right.

And this wasn’t a complete loss, JeanLucPicard was identified correctly most of the time!  And some of the other characters were properly identified some of the time.

I also took a look at the distances for each match.  You see, the match is actually a 2500 dimensional distance calculation between two images.  I was hoping to use this distance to weed out false matches, unfortunately, there wasn’t really that much of a separation between correct and incorrect matches.

At any rate, all that is just tweaking the system.

With that, I’ve actually reached the point I was hoping to reach for this project.  I set out to write something that would detect and identify faces from a video stream and that’s what I’ve done.  However, there’s obviously another step that needs to be done, which is the automatic removal of Wesley Crusher.  It knows who he is, so it’s halfway there.

Of course, on the technical side, there are a few things left to do, like figure out why I’m getting Access Violations randomly when calling EigenDecomposite() or trying to make it not be so horribly slow or figuring out if there’s some way I can return nobody when it’s not confident enough about its identification.

And, of course, I also have to run it against a different episode in order to make sure that the recognition isn’t so accurate because it’s simply recognizing the original faces that it trained on…

But that’s what tomorrow is for.

Images in action:

Target Acquired

Multiple hits.  Note that it doesn’t see Geordi…  I think the face detector relies heavily on the eyes, so his VISOR is confusing it.

It can even detect redshirts!

IT WORKS!

HOLY CRAP IT ACTUALLY WORKS!

I trained it on a set of about half of the images, because it takes a long time to train.  Then I ran it on the video and it was actually returning the right person most of the time.  I’m actually shocked that it’s working so well, especially seeing that this is the first time I’ve had the recognition part actually compiling and running.  No real tweaking necessary.  The problems I’ve been having have all been with calling the functions and simply using the library, it’s not that the software was doing what I told it to do and it all came out wrong.

Looking at this trace, you can even get a sense of the cuts in the scene:

Recognized: JeanLucPicard
Recognized: DeannaTroi
Recognized: JeanLucPicard
Recognized: DeannaTroi
Recognized: DeannaTroi
Recognized: JeanLucPicard
Recognized: DeannaTroi
Recognized: Anthwarta
Recognized: Anthwarta
Recognized: Anthwarta
Recognized: Anthwarta
Recognized: Anthwarta
Recognized: Anthwarta
Recognized: Anthwarta
Recognized: DeannaTroi
Recognized: DeannaTroi
Recognized: DeannaTroi
Recognized: JeanLucPicard
Recognized: JeanLucPicard
Recognized: DeannaTroi
Recognized: JeanLucPicard
Recognized: DeannaTroi
Recognized: DeannaTroi
Recognized: JeanLucPicard
Recognized: DeannaTroi
Recognized: DeannaTroi
Recognized: DeannaTroi
Recognized: DeannaTroi
Recognized: JeanLucPicard
Recognized: DeannaTroi
Recognized: JeanLucPicard
Recognized: Anthwarta
Recognized: Anthwarta
Recognized: Anthwarta
Recognized: Anthwarta

Of course, right now, it’s only working on the positive cases.  When it knows someone, it knows them.  When it doesn’t know someone, it randomly guesses about who it is.

Now, something interesting about that…  I haven’t trained the system on Wesley yet.  He’s down in the Ws and the images I trained with were alphabetical.  However, in the scenes with him, he’s fairly regularly identified as “JeanLucPicard”.

Gotta wonder if the Doctor and the Captain have something they need to talk to him about…

Now I just have to label the faces on the screen, since screenshots are better than random text.

If your facial recognition system is only capable of recognizing a single face, you will have 100% accuracy on that face when that face is in the scene.

Looks like it’s time to try processing the full set of images…

I was expecting the facial recognition system to give me odd results when I first ran it.

I was not expecting the results to be quite this odd:

OpenCV has some convienent data storage functions for all of its crazy datatypes.  I’m thankful that I don’t have to worry about reading and writing CvMat, IplImage, and so on.

However, the file size leaves something to be desired.  It writes everything as text in XML.  As a result, a float increases in size by 400%.  AWESOME!  And that’s not even counting all of the XML overhead.

I’m thinking I might have to tweak this at some point…

So many things are wrong with this that I’m not even going to mention any of them.