So, we were down for approximately 18 hours. The downtime was due to our hosting provider migrating the site over to a newer, faster server. The site should now actually be more responsive and you should hopefully see less “server is busy” errors.
A few years back, we had an extended downtime where we were down for about a week. During that downtime, we had been moved to another server by our hosting provider, due to the fact that the hard drives had failed on the old one. Shortly after the upgrade, I noticed that there was a problem with the motherboard. It wasn’t a show-stopper, but fixing it required some jury-rigging. The jury-rigging held up pretty well.
However, with the release of the new date coming closer and closer, I started realizing how much traffic we were getting and that we could do with slimming down system usage. For that, I wanted to get rid of the jury-rigging. So, we asked our hosting provider for a replacement motherboard. They responded yesterday by offering to move us to a new server.
I agreed, thinking they were going to take the hard drives from the old server and put them in the new server. Instead, they took the old drives and made them slaves to the existing drives.
It also didn’t help that when I logged into the server, I realized they put a completely different distribution of Linux than what we were using before. I called them and griped, because at this time I really don’t have time to learn a new distribution of Linux and configure it for our environment. They agreed to install our normal distribution, and did so.
… and when I logged in, I found they installed NONE of the software we needed. There wasn’t a web server, a mail server, or anything. By this point I was tired of depending on the hosting company and decided to fix it myself. To give a comparison, it was the equivalent of someone handing me a machine with Windows 7 Starter Edition and saying it was ready for use, without any of the necessary software. It took several hours, and by the time I went to bed at 1 AM CDT, I had everything but web services working.
Fortunately, when I got up this morning, I realized where I went wrong on the web services and got those fixed too.
Right now, I realize there are a couple of bugs in the site. For example, the online list isn’t working. However, I will be working with CCShadow to try and get them fixed as soon as we can.
Thank you for your patience while we get everything back to normal. :-)