I spent some time last night diving back into the world of Lego Mindstorms, trying to solve one of the outstanding problems from the Pong Robot that I built a few weeks ago.  If you recall, one of the biggest problems I had was with precision movement of the servo that was rotating the paddle knob.  It would consistently overshoot the mark or simply not move at all.  This was very frustrating, and I tried several different movement techniques before coming across one that worked well enough…  Until I recharged the batteries.

One of the problems I had was that I couldn’t tell the motor how far to rotate.  I had to tell it to turn on, then, sometime later, tell it to stop.  This meant that processing delay could have been the cause of most of the overshoots.  By the time the stop message had gotten to the motor, the paddle had gone too far, leading the movement processor to send a command telling the paddle to move the other direction, where it promptly overshot the mark heading in the other direction.  What I needed was to tell the motor exactly how many degrees to turn.

Now, I tried the degree method before, sort of.  Unfortunately, I was limited to fixed angles that were set up in the Lego Graphical Program that I wrote (If you can call drag and drop programming “writing”…), so it didn’t end up working that well.  In general, it overshot worse using these angles.  And movements got stacked up in the queue, so it kept moving long after it should have stopped.  All in all, a failure.

I eventually bypassed the program altogether and moved to using the Mindstorms Direct Command API, which lets you send commands to the NXT over Bluetooth.  Using Direct Commands, I had slightly more control over when to start and stop, although the primary benefit was turnaround time.  All of my logic was centralized in the same C# application, rather than being spread between C# and the Lego Graphical Program.  I could tweak a setting and run and see what it did, rather than drag a couple of program bricks around, redeploy, run the program on the NXT, then run the desktop app, only to find out it didn’t work at all.  However, what I didn’t find was any way to provide a movement angle at all.  There were all sorts of settings, like power, braking mode, even some used for twin motor locomotion.  But no “rotation angle”.  So, I made do with simply adjusting the power.  The “success” of the robot was less a factor of going where it was supposed to go than luck in having it correct from its mistakes fast enough to hit the ball.

Obviously, luck wasn’t going to cut it.  The robot would perform reasonably at the standard speed, but as soon as you bumped up the speed a notch, it didn’t stand a chance.  I could tell that it would have problems if I wanted to expand to games like Breakout, where the speed is variable in normal play, and there was no way that the current movement would work for games like Kaboom!, where you have to have pinpoint movement control and be thinking five moves ahead to have any kind of hope of survival.

Yesterday, I decided to look at the problem again.  In reading what other people had written about the Direct Command API, I discovered that the semi-obscurely named “Tacho Limit” parameter that I’d been ignoring was actually the number of degrees to rotate with the movement command.  Great!  That’s precisely what I needed.  Except for the minor fact that this Tacho Limit was obeyed like someone on a crotch rocket obeys the speed limit.  Somewhere around the Tacho Limit the motor would sort of decide to stop moving, but would then coast for another random length of time.  At full power, this random length of time tended to mean somewhere in the neighborhood of 720 degrees.

Two full revolutions past the mark is not my idea of precise.

And then, on top of that, there’s a counter in the motor that keeps track of how far you’ve wanted to go and how far it’s actually gone and won’t move again until you’ve told it to move past where it landed.  Which means, if you tell it to rotate 360 degrees, and it goes 1080, the motor won’t go anywhere at all until you’ve told it to move more than the difference of 720 degrees.  And then, it only travels the difference over the error.  So, needless to say, quite frequently, I was telling to move and it refused to go anywhere, and when it finally did decide to move, it didn’t move anywhere near the distance I wanted it to go.

Basically, I would have gotten more precise movement if I’d handed the paddle to a three-year-old that I’d pre-loaded with PixyStix.

I don’t understand it.  I’ve seen the motor move fast and stop with such force that the entire assembly shudders.  Why is it just gradually coasting to a stop now when it should be moving precisely 30 degrees.  Not 34, not 37.  30.

I’m back in the graphical environment now, trying to drag and drop a program that’ll give me more control over the rotation, but which will hopefully cause the motor to actually stop when it’s supposed to stop.  I’m debugging using a happy face icon and a bunch of sleepy ZZzzzs.  This is all I will ever be able to think about the next time someone tries to show me a UML diagram.

And speaking of stopping, I really need to stop now, given that I happen to have something that could be described as “work” in the morning.   Just like the motor overshoot, this could easily continue for another hour or two if I let it.   When I’m half asleep at the morning meeting, I don’t think the excuse “I was playing with Legos all night” is gonna fly…

Sadly, I don’t think the whole TV thing is going to work out.  I got nowhere on the calibration.  cvFindChessboardCorners will find the corners on a real chessboard, but didn’t accept the Atari screen at all.

So, at this point, that’s a dead end and it’s not worth chasing.  I’m sure there’s a way to make it work out, but I don’t really feel like spending time trying to sort it out.

I think I’ll go back and try to tweak the projection logic and the robot a bit more.  It’s only won a single game so far, and that was far too close.  If I can’t make it play on a real TV, at least I can make it play better on the direct video feed.

So, I’ve made no progress on the camera calibration.  (Unless you want to consider failure progress because I now know more about what doesn’t work, in which case I’ve made tons and tons of progress…)  The cvCalibrateCamera2 function seems to have requirements that aren’t fully explained anywhere, except perhaps in error codes that are occasionally in Japanese.  I’m know I’m calling it wrong, but I have no idea how to call it right…   I think I’ll keep trying for a little bit longer and see if I can figure out what I’m missing.

If that doesn’t work out, then there is a slight change in strategy I’ve been considering.  You see, OpenCV has a function called “cvFindChessboardCorners”, and it is used to find the corners of a chessboard.  The corners of a chessboard are then used for calibration.  How is that going to help me, when I’m trying to find the edges of a TV and not the corners of a chessboard pattern?

Well, it’s not like Pong is the only Atari game I own or anything…

I tried the gears, but it ended up seeming highly unstable.  The motor gear didn’t want to lock with the paddle gear.  So…  Back to the drawing board.

It looks like WordPress is deciding that the Silverlight Video Player is obviously a Flash object, because it’s using an object tag. So, it’s being helpful and converting it to a Flash object for me.

Thank you very much!

However, it looks like there’s trouble with that video player even when I’m not getting second-guessed about its type.  Something seems to make it not want to appear when there’s more than one on the page, or in certain cases when there’s more than one on the page.  Suck.  Unfortunately, I’m building a robot that can play Atari this weekend, I’m not debugging Silverlight Video Player Embedding, so I can’t spend time on that right now to sort out these issues.  No more embedded videos, I guess, except for the YouTubery ones.  Sorry.

Computer Vision Processing Game Screen? Check.
Game Play Logic Predicting Ball Trajectory? Check.
Mindstorm Robot Attached To Paddle? Check.
Bluetooth Communication To Robot? Check.
Linking Trajectory Prediction to Robot Communication? Check.

Okay! Let’s go!

Maiden Voyage

Complex Systems Interacting To Produce Failure? Check.

That high pitched noise at the end after it jams is coming from the motor. Motors are not supposed to give off high pitched noises. Additionally, prior to jamming, it forced the potentiometer inside the paddle to skip past the end of its range a couple of times (The loud clicks that you heard). That can’t be a good thing to have it doing…

It’s clicking.  My robot is clicking.

Well, okay, it only clicks once, then it stops responding altogether.

But still, that click means it’s doing something, right?

Right?

I mean, it should be rotating the paddle spinner left or right until I press stop, and it’s obviously not doing that, but a click means that the command is getting to the NXT over Bluetooth, so at least something is working right.  That is progress, isn’t it?

I guess I should actually be reading the return message.  And maybe I shouldn’t be closing the serial port every time.

Whatever.  I’ll look at it in the morning.

I forgot that the Mindstorms NXT base unit takes 6 AA batteries.  Now I’ve got to charge up a whole boatload of batteries.  Looks like it can’t be USB powered, either.  I knew I should have bought the power kit/battery pack.  I wonder if it’s worth  trek to the Lego store tomorrow to see if they have it.  Is the Lego store even there?

At any rate, while that’s going on, might as well start looking at Bluetooth programming.

I’ve been working at the robot for the better part of an hour and a half, and I still don’t have anything that looks like it’ll work.  The main problem is that I have to work with Lego parts, so it’s hard to get anything that will fit the paddle.  They’re also the crazy Mindstorm/Technic pieces that are more like a plastic Erector Set than Lego pieces, so they’re hard to connect in the way you want them to go together.  I almost got something, but it doesn’t want to hold on to the paddle.

Back to the drawing board…