{"id":2002,"date":"2011-03-03T23:05:21","date_gmt":"2011-03-04T07:05:21","guid":{"rendered":"https:\/\/mathpirate.net\/log\/?p=2002"},"modified":"2011-03-03T23:12:43","modified_gmt":"2011-03-04T07:12:43","slug":"time-is-relative-anyway","status":"publish","type":"post","link":"https:\/\/mathpirate.net\/log\/2011\/03\/03\/time-is-relative-anyway\/","title":{"rendered":"Time Is An Illusion, Anyway"},"content":{"rendered":"

This week has been rough.<\/p>\n

I’ve been working through a new strategy that promises to make our company’s deployments to production at least ten times faster than our current process.\u00c2\u00a0 While it’s going to make things easier for us in the end, getting there is anything but easy.<\/p>\n

I’ve gotten\u00c2\u00a0permission errors.<\/p>\n

I’ve hit ACL problems.<\/p>\n

I’ve come across registry access violations.<\/p>\n

I’ve had writeable UNC shares turn read-only.<\/p>\n

I’ve seen application pools die and IIS refuse to start.<\/p>\n

I watched installers fail to run remotely,\u00c2\u00a0successfully\u00c2\u00a0run locally, then fail to run locally under the exact same circumstances two minutes later.<\/p>\n

I even had UAC end up locking my account<\/em> because it decided that I’m not trustworthy enough to enter a password for a installation tool from an unelevated command prompt.<\/p>\n

It’s been a never ending stream of failure and frustration.<\/p>\n

I thought I’d seen it all.\u00c2\u00a0 Then today happened.<\/p>\n

I left the office last night feeling really good about where things were at.\u00c2\u00a0 I finally worked through all of the permission problems, all of the path access issues.\u00c2\u00a0 Everything was running under the right accounts, and stuff was flowing through the system and installing flawlessly.<\/p>\n

Put a checkmark on the list, I’m done!<\/p>\n

I planned on coming in this morning, updating some release instructions, then moving on to the second piece of the process.\u00c2\u00a0 The second piece is very similar to the first, so I figured I’d breeze through the day and have everything up and running in time to leave work during rush hour.\u00c2\u00a0 I want to preface the release instructions with a link to the progress I made, so I go to the site that installed flawlessly last night and try to pull it up.<\/p>\n

IIS Yellow Screen Of Death<\/em><\/p>\n

Ah crap.\u00c2\u00a0 That sucks.\u00c2\u00a0 I sort of expected something like that, though.\u00c2\u00a0 We’re doing things that are wild and new for our company, and most of the problems we’re seeing are the result of inexperience with the technology.\u00c2\u00a0 I figured the site would have died sometime in the middle of the night for some reason or another.\u00c2\u00a0 I figured that it would be something we’d just overlooked, something easy to fix.<\/p>\n

I didn’t figure on getting an error message that was last seen in 1997.<\/p>\n

An error occurred loading a configuration file: Failed to start monitoring changes to ‘[filename]’ because the network BIOS command limit has been reached.<\/em><\/p><\/blockquote>\n

…<\/p>\n

The network BIOS command limit has been reached…?\u00c2\u00a0 WTF?<\/p>\n

Am I running low on “System Resources” now, too?<\/p>\n

What in the hell does that mean?\u00c2\u00a0 Network BIOS command limit…<\/p>\n

I do a search<\/a> and immediately find many other people with the same problem, all of whom are saying that the KB article the error message points at is completely useless.\u00c2\u00a0 Instead, they say you have to pull a registry setting out of thin air, and that’ll solve your problems.\u00c2\u00a0 ((Under HKLM\\Software\\Microsoft\\ASP.NET, create a new value called “FCNMode” and set it to 2 to change the way IIS monitors file change notifications, or set it to 1 to disable the notifications altogether.\u00c2\u00a0 Other sites suggest the same value, but under the HKEY_LOCAL_MACHINE\\SOFTWARE\\Wow6432Node\\Microsoft\\ASP.NET key instead.\u00c2\u00a0 Try both.\u00c2\u00a0 Then reboot.))\u00c2\u00a0 Turns out that\u00c2\u00a0the way IIS monitors file system changes on UNC shares can\u00c2\u00a0clog the tubes, and that magical reg key tries to unclog them.\u00c2\u00a0 Eventually, I get it to work and carry on with the tasks of the day.<\/p>\n

It all goes fairly well and smoothly.\u00c2\u00a0 As I had hoped, the experience from setting up the first piece of the deployment process translated very closely to setting up the second piece.\u00c2\u00a0 It only took two hours to get through the same amount of the process that had taken three days the first go around.\u00c2\u00a0 And most of that two hours was spent keeping detailed instructions so that next time only takes half an hour and the time after that can be fully automated.<\/p>\n

All is well.<\/p>\n

In the very last part of the setup, I had to install a Windows component, then reboot the box.\u00c2\u00a0 ((Why they haven’t figured out the whole rebooting thing for the case where I just installed a minor application, I don’t know, but whatever…))\u00c2\u00a0 I give the box a minute, then try to\u00c2\u00a0remote back in.\u00c2\u00a0 Terminal Services Client blinks at me, but does nothing.\u00c2\u00a0 I try again and again it blinks.\u00c2\u00a0 A third time, and I get a failed to connect message.\u00c2\u00a0 Fine, maybe it’s still rebooting.\u00c2\u00a0 I give it another minute and try again.<\/p>\n

\"\"<\/a><\/p>\n

Remote Desktop cannot connect to the remote computer because the authentication certificate received from the remote computer is expired or invalid.\u00c2\u00a0 In some cases, this error might also be caused by a large time discrepancy between the client and server computers.<\/em><\/p><\/blockquote>\n

Uh, okay…\u00c2\u00a0 Never seen that error before.\u00c2\u00a0 Did the network flip out, is the server confused?\u00c2\u00a0 Maybe I just have to give it a bit more time before trying to connect again.<\/p>\n

So I give it another minute.\u00c2\u00a0 It gives me the error again.<\/p>\n

The second sentence intrigues me.\u00c2\u00a0 A large time discrepancy, eh?\u00c2\u00a0 Hmmm…<\/p>\n

Last year, during a brief foray into temporal mechanics<\/a>, I learned all about how Windows deals with time and time synchronization.\u00c2\u00a0 One of the things I found was a command line tool in Windows called “w32tm”.\u00c2\u00a0 That program has a command line switch, “\/stripchart”, which can show you how far off your system’s clock is from that of another system.\u00c2\u00a0 Normally, when dealing with a pair of computers on a corporate network, tied to a domain, you’ll find that the clocks will only differ by a few seconds at most.\u00c2\u00a0 You can\u00c2\u00a0run w32tm \/stripchart and watch as the two machines drift around in time, speeding up, slowing down, and dancing around the time synchronization point.\u00c2\u00a0 Try, for instance, “w32tm \/stripchart \/computer:time.windows.com”, and see where you are in comparison to the Windows time server.<\/p>\n

Anyway, w32tm’s stripchart seemed like the perfect tool to investigate this potential “large time discrepancy”.\u00c2\u00a0 What would it be, an hour, two hours, maybe even a day?<\/p>\n

The current time is 3\/3\/2011 4:52:43 PM.\r\n16:52:43 d:+00.0020000s o:-31607977.8967224s\u00c2\u00a0 [@\u00c2\u00a0\u00c2\u00a0\u00c2\u00a0\u00c2\u00a0\u00c2\u00a0\u00c2\u00a0\u00c2\u00a0\u00c2\u00a0\u00c2\u00a0\u00c2\u00a0\u00c2\u00a0\u00c2\u00a0\u00c2\u00a0\u00c2\u00a0\u00c2\u00a0\u00c2\u00a0\u00c2\u00a0\u00c2\u00a0\u00c2\u00a0\u00c2\u00a0\u00c2\u00a0\u00c2\u00a0\u00c2\u00a0\u00c2\u00a0\u00c2\u00a0 |\u00c2\u00a0\u00c2\u00a0\u00c2\u00a0\u00c2\u00a0\u00c2\u00a0\u00c2\u00a0\u00c2\u00a0\u00c2\u00a0\u00c2\u00a0\u00c2\u00a0\u00c2\u00a0\u00c2\u00a0\u00c2\u00a0\u00c2\u00a0\u00c2\u00a0\u00c2\u00a0\u00c2\u00a0\u00c2\u00a0\u00c2\u00a0\u00c2\u00a0\u00c2\u00a0\u00c2\u00a0\u00c2\u00a0\u00c2\u00a0\u00c2\u00a0]<\/em><\/pre>\n
The current time is correct.\u00c2\u00a0 I ran it at 4:52 PM on March 3rd, 2011.\u00c2\u00a0 “d:” is, I believe,\u00c2\u00a0the “delay”, the round trip time between the servers.\u00c2\u00a0 2 ms response time.\u00c2\u00a0 Not bad.\u00c2\u00a0 And “o:” is the time offset, in seconds, between your computer and the remote computer.<\/p>\n
o:, in this case, is -31.6 million seconds.<\/p>\n
In case you don’t feel like doing the math, 31.6 million seconds is one year, 19 hours, 59 minutes, and 37 seconds.<\/p>\n
The computer rebooted and came back one year, 19 hours, 59 minutes, and 37 seconds in the past<\/strong><\/em>.\u00c2\u00a0 I think I’d agree with the error’s\u00c2\u00a0assessment of a “large time discrepancy”.<\/p>\n
The computer now believes that it is March 2nd, 2010, at around 8:50 PM.\u00c2\u00a0 And I can’t convince it otherwise.\u00c2\u00a0 If I go on the box and try to change the time manually, it immediately corrects itself to March 2nd, 2010.<\/p>\n
Now, ordinarily, I’d say that such an error was the result of time synchronization service gone awry, or maybe some failing system hardware.\u00c2\u00a0 However, given the events of this past week and all of the strange behavior that I’ve seen, I believe it is equally likely that that particular computer has actually travelled back in time and is running in our datacenter exactly one year, 19 hours, 59 minutes and 37 seconds ago.<\/p>\n
Now, if you’ll excuse me, I have a hole in time to exploit and profit from.<\/p>\n","protected":false},"excerpt":{"rendered":"
This week has been rough. I’ve been working through a new strategy that promises to make our company’s deployments to production at least ten times faster than our current process.\u00c2\u00a0 While it’s going to make things easier for us in the end, getting there is anything but easy. I’ve gotten\u00c2\u00a0permission errors. I’ve hit ACL problems. […]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[9,1],"tags":[14,186,185,131,26],"_links":{"self":[{"href":"https:\/\/mathpirate.net\/log\/wp-json\/wp\/v2\/posts\/2002"}],"collection":[{"href":"https:\/\/mathpirate.net\/log\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/mathpirate.net\/log\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/mathpirate.net\/log\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/mathpirate.net\/log\/wp-json\/wp\/v2\/comments?post=2002"}],"version-history":[{"count":6,"href":"https:\/\/mathpirate.net\/log\/wp-json\/wp\/v2\/posts\/2002\/revisions"}],"predecessor-version":[{"id":2010,"href":"https:\/\/mathpirate.net\/log\/wp-json\/wp\/v2\/posts\/2002\/revisions\/2010"}],"wp:attachment":[{"href":"https:\/\/mathpirate.net\/log\/wp-json\/wp\/v2\/media?parent=2002"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/mathpirate.net\/log\/wp-json\/wp\/v2\/categories?post=2002"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/mathpirate.net\/log\/wp-json\/wp\/v2\/tags?post=2002"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}