Random header image... Refresh for more!

In The Future, There Will Be Robots

So, I’m going to be building a robot that can play Pong over the next five days.

Why am I going to spend five days building a robot that can play Pong?

Well, first of all, it’ll be totally cool to have a Pong playing robot.  You simply cannot deny the awesome factor of that.

Second, it will be fun to do.  I’ve wanted to learn about computer vision for a while, and I’ve wanted to tinker with robotics for a while, and I love video games.  So why not combine all of those into one big crazy project?

And finally, ATARI PLAYING ROBOT!  Do you really need any other reason?

I chose Pong as the first game to attempt because it has uncomplicated graphics and clearly defined rules that can be described using fairly simple mathematical models.  The ball will remain in motion at a constant speed in a straight line until it hits something else.  (Newton’s First Law of Motion, hence all that “Newtonian Motion Simulator” nonsense earlier.)  There’s no need for any kind of pathfinder or decision making AI or similar kind of complicated logic for playing the game.  If I can detect the position of the ball in two subsequent frames, and I know where the walls are, then I can predict exactly where my paddle needs to be placed in order to hit the ball back.  Computers are very good at calculating that sort of thing.  Pong is also a paddle controlled game, which means the robot only needs to be able to rotate clockwise or counterclockwise.  No complicated multi-axis positioning that a joystick would require.

(It should be noted that the fundamentals needed to play Pong are the same things that are needed to play many other games.  If you get Pong, then you get Breakout almost for free, because Breakout is Pong rotated 90 degrees.  Similar to Breakout is the frantic bomb-catching game of Kaboom!.  And, although they’re visually very different and utilize very different controls, games like Rock Band or Guitar Hero are essentially variants of Kaboom!.  Which means that I can go from building something that can play Pong and have a clear path to a 5-star performance on Green Grass and High Tides.)

I’ve seen other people make game-playing contraptions, but none of them have tried actually watching the screen and reacting to what’s there.  I’ve seen a Wii Bowling Robot that can bowl a perfect game, but it’s taking advantage of fact that if you roll perfectly straight, you get a strike.  I’ve seen several Rock Band/GH bots, but they’re hacked controllers that play back a recording.  And I’ve seen a program that can play a kick-ass game of Super Mario World, but it’s an AI built directly into the game code.  I’ve never seen anyone who has made a robot that uses a camera to play the game using the normal TV display and normal controllers.  That’s my goal here.  To build a robot that can play games on a completely unmodified Atari 2600.

For the robot, I’m going to use Lego Mindstorms.  That seemed like the easiest way to go, especially since I have no electrical engineering experience.  If this goes well, I might step it up a notch and try a raw microprocessor and motors and risk of electric shock and fire type robot, but for now, the Legos seem good.  I bought the kit about a year ago and it’s been hanging around on a shelf ever since then.  It was actually around that time that I first started thinking of building an Atari controlling robot.

Of course, all of this plan is extremely difficult if I can’t see the game.  In order to do that, I plan to use OpenCV.  It’s an open source computer vision library (with an available .Net binding), that seems to have all manner of image processing and recognition functionality in a package that looks disturbingly easy to use.  Whether or not it is, I’ll find out soon enough.  It’s the same software that’s used in some of those DARPA challenge rally cars that drive themselves across the desert, so I figure it oughta be able to distinguish a couple of big blocky rectangles moving around on a TV screen.  (Unless the raster scan on my CRT TV throws it off…)

So, the end-to-end plan is this:  I point a camera at the TV, showing a game of Atari 2600 Pong.  The camera feeds into an OpenCV based processor running on my computer.  The computer calculates where the paddle needs to be in order to hit the ball, and uses Bluetooth to tell the Mindstorms robot to rotate the Atari paddle left or right.  The paddle hits the ball back.  Because the AI on the Video Olympics version of Pong is simple “follow” logic that doesn’t move fast enough to reach the ball in time when the ball reaches a certain speed, my robot is basically guaranteed to be able to beat the AI.

Seems possible, eh?

I plan to tackle the computer vision stuff first, then move onto the robotics.


There are no comments yet...

Kick things off by filling out the form below.

Leave a Comment