I’ll probably post later today about the actual Las Vegas trip itself, but right now I’m still recovering from two late nights in a row working on server issues.
It started Sunday night, right before the folks and I were to go to Fremont Street. I had pulled the Powerbook out and logged on to see how things were going, and Chan_Ming and Crow (fellow MK Online staffers) mentioned there was a problem with the site. I logged into the server, and about the worst thing that COULD have happened HAD happened: a physical fault had developed on the hard drive, and the partition that held the site was corrupted. Unfortunately, I couldn’t back up any of the data that was on that partition, as that directory was dying. So, I copied off what configuration data I could, shut the webserver down, and called CCShadow to have him contact our hosting provider and get the drive replaced.
Well, Monday night, after I got home, I managed to log into the server after the hard drive was replaced… and to say I wasn’t pleased with the job they did was an understatement. They did a VERY minimal install; they didn’t even install the web server software! Fortunately I had enough information where I could get an idea of what did need to be installed, and I had backups of the site from just after the migration to a more powerful server. Moreover, the databases themselves were backed up nightly, so I had a backup from Sunday morning. I ended up staying up late Monday night getting it worked out, and then finished getting everything working yesterday evening. I’ll still need to talk to CCS when he gets back into the country, but for now everything’s pretty much working.
If the MK Online stuff wasn’t bad enough… on Monday morning, I was working on the main D’Arque Cathedral server when it stopped responding at 9 AM CST. Of course, there was nothing I could do until I got home, but I knew the house at least was still standing and had power, as my DNS server was still online and responding. When I got home that night, I noticed the server had turned itself off. I shrugged, switched it back on, and got back to work on the MK Online server. Less than twenty minutes later, I noticed it had shut off again. I did some investigating and tried a couple of things (including replacing the power supply), only to come to a determination that the machine was overheating. The CPU fan wasn’t going as fast as usual, and the case wasn’t well-ventilated as it was. *sigh*
So, on my way home from work yesterday, I stopped in Fry’s and picked up a new case with a 420 watt power supply, fans on the side and top, and an LCD temperature gauge, plus a new CPU/heatsink combination. When I got home, I proceeded to disassemble the server and throw everything into the new case. It took quite a bit of doing (especially trying to get the new heatsink/fan onto the CPU in the case), but I finally got it up and running late last night. The new case has blue LED lights on the fans, and six of them on the front of the case. Dad took one look at it this morning and said, “Those will have to be disconnected.” My response: “Uhm… whenever you’re in this office working, your back is to the servers. How will it bother you? In addition… I don’t want to screw around with a production server.”
So, both servers are now up and running for the most part. I’ll probably end up going to bed early tonight as a result, just to make sure that I can catch up on my sleep. As I said before, I’ll post a Las Vegas wrap-up a bit later.