So, I’m going to be building a robot that can play Pong over the next five days.<\/p>\n
Why am I going to spend five days building a robot that can play Pong?<\/p>\n
Well, first of all, it’ll be totally cool to have a Pong playing robot.\u00c2\u00a0 You simply cannot deny the awesome factor of that.<\/p>\n
Second, it will be fun to do.\u00c2\u00a0 I’ve wanted to learn about computer vision for a while, and I’ve wanted to tinker with robotics for a while, and I love video games.\u00c2\u00a0 So why not combine all of those into one big crazy project?<\/p>\n
And finally, ATARI PLAYING ROBOT<\/strong><\/em>!\u00c2\u00a0 Do you really\u00c2\u00a0need any other\u00c2\u00a0reason?<\/p>\n I chose Pong as the first game to attempt because it has uncomplicated graphics and clearly defined rules that can be described using fairly simple mathematical models.\u00c2\u00a0 The ball will remain in motion at a constant speed in a straight line until it hits something else.\u00c2\u00a0 (Newton’s First Law of Motion, hence all that “Newtonian Motion Simulator” nonsense earlier.)\u00c2\u00a0 There’s no\u00c2\u00a0need for\u00c2\u00a0any kind of pathfinder or decision making AI or similar kind of complicated logic for playing the game. \u00c2\u00a0If I can detect the position of the ball in two subsequent frames, and I know where the walls are, then I can predict exactly where my paddle needs to be placed in order to hit the ball back.\u00c2\u00a0 Computers are very good at calculating that sort of thing.\u00c2\u00a0 Pong is also a paddle controlled game, which means the robot only needs to be able to rotate clockwise or counterclockwise.\u00c2\u00a0 No complicated multi-axis positioning that a joystick would require.<\/p>\n (It should be noted that the fundamentals needed to play Pong are the same things that are needed to play many other games.\u00c2\u00a0 If you get Pong, then you get Breakout almost for free, because Breakout is Pong rotated 90 degrees.\u00c2\u00a0 Similar to Breakout is the frantic\u00c2\u00a0bomb-catching game of Kaboom!.\u00c2\u00a0 And, although they’re visually very different and utilize very different controls, games like Rock Band or Guitar Hero are essentially variants of Kaboom!.\u00c2\u00a0 Which means that I can go from building something that can play Pong and have a clear path to a 5-star performance on Green Grass and High Tides.)<\/p>\n I’ve seen other people make game-playing contraptions, but none of them have tried actually watching the screen and reacting to what’s there.\u00c2\u00a0 I’ve seen a Wii Bowling Robot that can bowl a perfect game, but it’s taking advantage of fact that if you roll perfectly straight, you get a strike.\u00c2\u00a0 I’ve seen several Rock Band\/GH bots, but they’re hacked controllers that play back a recording.\u00c2\u00a0 And I’ve seen a program that can play a kick-ass game of Super Mario World, but it’s an AI built directly into the game code.\u00c2\u00a0 I’ve never seen anyone who has made a robot that uses a camera to play the game using the normal TV display and normal controllers.\u00c2\u00a0 That’s my goal here.\u00c2\u00a0 To build a robot that can play games on a completely unmodified Atari 2600.<\/p>\n For the robot, I’m going to use Lego Mindstorms.\u00c2\u00a0 That seemed like the easiest way to go, especially since I have no electrical engineering experience.\u00c2\u00a0 If this goes well, I might step it up a notch and try a raw microprocessor and motors and risk of electric shock and fire type robot, but for now, the Legos seem good.\u00c2\u00a0 I bought the kit about a year ago and it’s been hanging around on a shelf ever since then.\u00c2\u00a0 It was actually around that time that I first started thinking of building an Atari controlling robot.<\/p>\n Of course, all of this plan is extremely difficult if I can’t see the game.\u00c2\u00a0 In order to do that, I plan to use OpenCV.\u00c2\u00a0 It’s an open source computer vision library (with an available .Net binding), that seems to have all manner of image processing and recognition functionality in a package that looks disturbingly easy to use.\u00c2\u00a0 Whether or not it is, I’ll find out soon enough.\u00c2\u00a0 It’s the same software that’s used in some of those DARPA challenge rally cars that drive themselves across the desert, so I figure it oughta be able to distinguish a couple of big blocky rectangles moving around on a TV screen.\u00c2\u00a0 (Unless the raster scan on my CRT TV throws it off…)<\/p>\n So, the end-to-end plan is this:\u00c2\u00a0 I point a camera at the TV, showing a game of Atari 2600 Pong.\u00c2\u00a0 The camera feeds into an OpenCV based processor running on my computer.\u00c2\u00a0 The computer calculates where the paddle needs to be in order to hit the ball, and uses Bluetooth to tell the Mindstorms robot to rotate the Atari paddle left or right.\u00c2\u00a0 The paddle hits the ball back.\u00c2\u00a0 Because the AI on the Video Olympics version of Pong is simple “follow” logic that doesn’t move fast enough to reach the ball in time when the ball reaches a certain speed, my robot is basically guaranteed to be able to beat the AI.<\/p>\n Seems possible, eh?<\/p>\n I plan to tackle the computer vision stuff first, then move onto the robotics.<\/p>\n","protected":false},"excerpt":{"rendered":" So, I’m going to be building a robot that can play Pong over the next five days. Why am I going to spend five days building a robot that can play Pong? Well, first of all, it’ll be totally cool to have a Pong playing robot.\u00c2\u00a0 You simply cannot deny the awesome factor of that. […]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[24],"tags":[17,21,15,20,19,22,25,238],"_links":{"self":[{"href":"https:\/\/mathpirate.net\/log\/wp-json\/wp\/v2\/posts\/57"}],"collection":[{"href":"https:\/\/mathpirate.net\/log\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/mathpirate.net\/log\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/mathpirate.net\/log\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/mathpirate.net\/log\/wp-json\/wp\/v2\/comments?post=57"}],"version-history":[{"count":5,"href":"https:\/\/mathpirate.net\/log\/wp-json\/wp\/v2\/posts\/57\/revisions"}],"predecessor-version":[{"id":486,"href":"https:\/\/mathpirate.net\/log\/wp-json\/wp\/v2\/posts\/57\/revisions\/486"}],"wp:attachment":[{"href":"https:\/\/mathpirate.net\/log\/wp-json\/wp\/v2\/media?parent=57"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/mathpirate.net\/log\/wp-json\/wp\/v2\/categories?post=57"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/mathpirate.net\/log\/wp-json\/wp\/v2\/tags?post=57"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}