Category — Crazy Weekend Project 1: Pong Robot

And with that, I’m declaring Crazy Project Weekend complete.  All in all, it was a success, although there were some things that didn’t work out.  I started on Friday morning with nothing but an idea and ended up successfully building a robot that could win a game of Pong, despite never having touched most of the major technologies (OpenCV, Mindstorms, Bluetooth) prior to this weekend.

I’m definitely going to have to do this sort of thing more often.

Just a reminder, the source code, if you’re interested, is available in SVN:  https://mathpirate.net/svn/

And now…  Sleep.

Full Video

I’m working on a YouTube friendly condensed version at the moment.

[Edit: Here’s the YouTube version.]

21-9!  Take that and dance, 30+ year old technology!  I’ve got two words for you:  Inyo and Face!

Video coming soon.

I am really starting to hate those robotic motors.  Sometimes they won’t move at all, and other times they’ll overcompensate for all the times they don’t move and end up flying across the screen when the ball is two pixels away.  They just do not want to be controlled.  The battery change has given them new life.  Actions that were once just right are now wildly off the mark.

Stupid real devices.  Does anyone know how to attach a debugger to the laws of physics so I can see what’s going wrong?

Now I just need to make the robot recharge its own batteries, perhaps even make it run off solar power…

I’ve watched the robot play a bit more and I’ve noticed a few tendencies that are preventing it from winning more often.

• Predictions are sometimes skewed by initial guesses.  The farther away the ball is, the worse the predictions tend to be.  Sometimes there will be a batch of outliers that will pull the prediction just far enough up or down so the paddle just barely misses the ball.
  • Possible solution:  Use a rolling window of the last N predictions for the average.
  • Downside:  Prediction will be more susceptible to outliers within the window.
• The paddle can’t get across the screen fast enough.  When the paddle misses the ball at the bottom of the screen, the serve will usually occur toward the top of the screen, and even at its fastest, the paddle doesn’t get there in time.
  • Possible solution:  Make it move faster.  Possibly even take into account the ball’s distance from the target point when determining the paddle speed.
  • Downside:  The faster it moves, the less control it has.  It might be overshooting the target more often than it hits it.
• Prediction history will sometimes carry over after a point is scored.  When this happens, you have all the points where the ball was going, which are heavily weighing down the points of where the ball is going.  It usually ends up with the prediction point somewhere in the middle of the screen and nowhere near where the ball actually is.  And, of course, the robot tracks the predicted point and misses the ball entirely.  There’s already logic to throw out the history after a point is scored, but it’s based on the number of “unrecognized frames”, and it’s not always accurate.
  • Possible solution:  The history window will help this problem immensely, but won’t make it go away.  The history should also be thrown out if the position of the ball jumps more than 25% of the screen width in a short period of time.
  • Downside:  This kind of calculation can run into a number of false positives.  The recognition is occasionally flaky, and the ball might become the paddle briefly and trigger this solution based on that.

Additionally, I’ve noticed that when the robot misses the ball once, it usually misses the ball several more times.  These second and third misses are usually the direct result of one of the problems listed above.  If I can correct those issues, then missing one ball won’t turn into missing three balls, which means that the robot stands a much better chance of winning.  A score of 21-20 that was decided on the last ball will turn into a score of 21-10, which is a much nicer victory to see.

Sadly, I don’t think the whole TV thing is going to work out.  I got nowhere on the calibration.  cvFindChessboardCorners will find the corners on a real chessboard, but didn’t accept the Atari screen at all.

So, at this point, that’s a dead end and it’s not worth chasing.  I’m sure there’s a way to make it work out, but I don’t really feel like spending time trying to sort it out.

I think I’ll go back and try to tweak the projection logic and the robot a bit more.  It’s only won a single game so far, and that was far too close.  If I can’t make it play on a real TV, at least I can make it play better on the direct video feed.

Well, okay, not quite 24 hours, but whatever.

So, I’ve made no progress on the camera calibration.  (Unless you want to consider failure progress because I now know more about what doesn’t work, in which case I’ve made tons and tons of progress…)  The cvCalibrateCamera2 function seems to have requirements that aren’t fully explained anywhere, except perhaps in error codes that are occasionally in Japanese.  I’m know I’m calling it wrong, but I have no idea how to call it right…   I think I’ll keep trying for a little bit longer and see if I can figure out what I’m missing.

If that doesn’t work out, then there is a slight change in strategy I’ve been considering.  You see, OpenCV has a function called “cvFindChessboardCorners”, and it is used to find the corners of a chessboard.  The corners of a chessboard are then used for calibration.  How is that going to help me, when I’m trying to find the edges of a TV and not the corners of a chessboard pattern?

Well, it’s not like Pong is the only Atari game I own or anything…